docker: Fix target mount cache paths; relax locks on cargo home.
docker: Persist lychee cache. Signed-off-by: Jason Volk <jason@zemos.net>
This commit is contained in:
@@ -55,7 +55,7 @@ so please be patient.
|
||||
2. Some unsavory options are required for some targets. It might be possible to omit these if
|
||||
you're not building the full tree. Otherwise I've included them in the create command below.
|
||||
|
||||
- To run the complement compliance suite we need the `--allow-insecure-entitlement netwok.host`.
|
||||
- To run the complement compliance suite we need the `--allow-insecure-entitlement network.host`.
|
||||
This requirement is probably a defect in Complement.
|
||||
|
||||
Finally create
|
||||
@@ -84,3 +84,48 @@ cues from the primary user of this system, the [GitHub CI](https://github.com/ma
|
||||
building the targets for `complement-tester` and `complement-testee` using `bake.sh` and then
|
||||
invoking `complement.sh`. You can take cues again from another user of this in the
|
||||
[GitHub CI](https://github.com/matrix-construct/tuwunel/blob/main/.github/workflows/test.yml#L79).
|
||||
|
||||
## Notes
|
||||
|
||||
- For CI our builders are more persistent rather than being created and destroyed for
|
||||
each invocation. The builder is meant to be safely reused across operations, but
|
||||
for concurrent operations this is tricky, see the next point.
|
||||
|
||||
- For CI our builders are isolated only by actor/user. This will probably change to
|
||||
`actor/repo/branch` as it's easy to cause issues with concurrent builds right now.
|
||||
However we can't accept destroying the builder after each use, so we'll likely
|
||||
choose `actor/repo/branch` with the expectation of one build at a time under those
|
||||
constraints. Some external caching might need to be contrived between builders for
|
||||
deduplication but with care such that malicious actors cannot poison data used
|
||||
by other actors, otherwise it defeats the purpose of builder isolation.
|
||||
|
||||
#### On Target Caches
|
||||
|
||||
The challenge here is to get all aspects of the target directory perfectly optimal
|
||||
within the many constraints including cargo issues and our goals. This is highly complex
|
||||
because we have to provide each image being built with a safe environment yet share as
|
||||
much as possible between builds. This includes maximum reuse of prior builds but without
|
||||
unnecessary dirtying or more serious unexpected conflicts.
|
||||
|
||||
We first create a hausdorff space based on builds which could never benefit from sharing
|
||||
and would always be unsound (even silently) if they came into contact; so all cache id's
|
||||
are prefixed by the matrix components:
|
||||
`${sys_name}/${sys_version}/${rust_target}/${rust_toolchain}/${cargo_profile}`. This is
|
||||
important because we can simplify the mount path inside the image which is important for
|
||||
the absolute paths generated by fingerprints and dependency files. We need to keep those
|
||||
the same if any builds expect to share them.
|
||||
|
||||
The top-level of target directory is immediately partitioned by cargo into different
|
||||
profiles. Note that we already imposed separation based on profile but we still have to
|
||||
deal with that subdirectory in the mount, which has special-cases for the dev, test, and
|
||||
bench profiles. These directories at their top-level are the final artifact area which is
|
||||
not concurrency safe and there are open issues in cargo for explicit artifact directories.
|
||||
Within these unsafe directories are sub-directories which contain hash-sharded components
|
||||
making them safe again for shared caching, so long as the path to them remains consistent
|
||||
for all images mounting.
|
||||
|
||||
As you can see this is already getting very complicated. If this is done wrong lots of
|
||||
different bad things can happen such as broken builds from bad conflicts, constant
|
||||
rebuilds from modest conflicts, or over-use of resources from too much separation.
|
||||
|
||||
TODO
|
||||
|
||||
Reference in New Issue
Block a user