| Commit message (Collapse) | Author | Age |
|---|
| |
|
|
| |
PiperOrigin-RevId: 199732415
|
| |
|
|
|
|
| |
(minor) ActionFS now implements MetadataProvider.getInput
PiperOrigin-RevId: 199575194
|
| |
|
|
|
|
| |
Adds a helper class, ActionInputMap to do this with minimal wrapping overhead.
PiperOrigin-RevId: 199391251
|
| |
|
|
|
|
| |
Actual class to be removed in a later change.
PiperOrigin-RevId: 198937695
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Local execution has an inherent race condition: if a user modifies a file while an action is executed, then it is impossible for Bazel to tell which version of the file was actually read during action execution. The file may have been modified before or after the tool has read it, or, in the worst case, the tool may have read both the original and the modified version. In addition, the file may be changed back to the original state before Bazel can check the file, so computing the digest before / after may not be sufficient.
This is a concern for both local and remote caches, although the cost of poisoning a shared remote cache is significantly higher, and is what has triggered this work.
Fixes #3360.
We solve this by keeping a reference to the FileContentsProxy, and using that to check for modificaitons before storing the cache entry. We output a warning if this check fails.
This change does not increase memory consumption; Java objects are always allocated in multiples of 8 bytes, we use compressed oops, and the FileArtifactValue currently has 12 bytes worth of fields (excl. object overhead), so adding another pointer is effectively free.
As a possible performance optimization on purely local builds, we could also consider not computing digests at all, and only use the FileContentsProxy for caching.
PiperOrigin-RevId: 182510358
|
| |
|
|
|
|
| |
cache if we're doing input discovery: input discovery may find new artifacts we don't yet know about. Similarly, in SingleBuildFileCache, use the Artifact's path if it's available, rather than the poor man's route of the execRoot.
PiperOrigin-RevId: 171635892
|
| |
|
|
|
|
|
|
|
| |
Add a single getMetadata method (matching MetadataHandler), and rewrite
everything in those terms.
This is in preparation for merging ActionInputFileCache and MetadataHandler.
PiperOrigin-RevId: 161053535
|
| |
|
|
|
|
|
|
|
|
|
|
| |
'create' method.
This paves the way for changing PathFragment to e.g. an abstract class with multiple subclasses. This way we can split out the windows-specific stuff into one of these concrete classes, making the code more readable and also saving memory (since the shallow heap size of the NonWindowsPathFragment subclass will hopefully be smaller than that of the current PathFragment).
This also lets us pursue gc churn optimizations. We can now do interning in PathFragment#create and can also get rid of unnecessary intermediate PathFragment allocations.
RELNOTES: None
PiperOrigin-RevId: 152145768
|
| |
|
|
|
|
|
|
|
| |
Fixes: https://github.com/bazelbuild/bazel/issues/2520
Fixes one of the bugs causing https://github.com/bazelbuild/bazel/issues/2473
--
PiperOrigin-RevId: 147335357
MOS_MIGRATED_REVID=147335357
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
refactoring: enabling potential fast digest computation of more than one digest function type.
Usage: bazel --host_jvm_args="-Dbazel.DigestFunction=SHA1" build ...
Ugliness: using a system property (a static non-final variable), because the better way to do it (a flag) would result in a much, much larger refactoring.
More ugliness: I have updated the minimal amount of tests. A lot of tests are still relying on the default value of MD5. Ideally, they need to be updated as well.
--
MOS_MIGRATED_REVID=139490836
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
ActionInputFileCache:
Change getDigest() to return the underlying byte[16] owned by each
FileArtifactValue.
Remove throws clause from getInputFromDigest(); this should be an
in-memory operation, and no implementation actually throws.
PerActionFileCache:
Invert mapping from artifact to digest only if needed. Remove interner,
as it was used only for the reverse map keys, not the returned values.
This should be a significant cpu savings as eagerly constructing the
reverse maps was a noticeable hotspot.
--
MOS_MIGRATED_REVID=127972359
|
| |
|
|
|
|
|
| |
Reduces garbage.
--
MOS_MIGRATED_REVID=109914243
|
| |
|
|
|
|
|
|
|
|
|
| |
The headers were modified with
`find . -type f -exec 'sed' '-Ei' 's|Copyright 201([45]) Google|Copyright 201\1 The Bazel Authors|' '{}' ';'`
And manual edit for not Google owned copyright. Because of the nature of ijar, I did not modified the header of file owned by Alan Donovan.
The list of authors were extracted from the git log. It is missing older Google contributors that can be added on-demand.
--
MOS_MIGRATED_REVID=103938715
|
| |
|
|
|
| |
--
MOS_MIGRATED_REVID=94318260
|
| |
|
|
|
|
|
|
|
|
|
|
| |
Instead of updating the file size cache as a side effect of updating
the digest cache we combine them into one cache to make it harder for
them to fall out of sync (mainly by way of programmer error). This
should also have a smaller memory footprint.
Also fixes a bug around cached exceptions.
--
MOS_MIGRATED_REVID=92928216
|
| |
|
|
|
|
|
|
|
|
|
| |
ActionInputFileCache when given a digest to do a reverse lookup so that metadata can be more easily retrieved.
Discovered inputs are no longer ever put into SingleBuildFileCache, so it is safe to roll this forward now
Because of code changes (meant to ensure the safety of this change), there were conflicts that had to be manually resolved.
--
MOS_MIGRATED_REVID=90425783
|
| |
|
|
|
|
|
| |
insert ActionInputs into SingleBuildFileCache.
--
MOS_MIGRATED_REVID=89032980
|
| |
|
|
|
|
|
| |
to do a reverse lookup so that metadata can be more easily retrieved.
--
MOS_MIGRATED_REVID=88733565
|
|
|
--
MOE_MIGRATED_REVID=85702957
|
shahan
ulfjack
janakr
nharmata
Laszlo Csomor
Ola Rozenfeld
Googler
Mark Schaller
Damien Martin-Guillerez
Michajlo Matijkiw
Han-Wen Nienhuys