| Commit message (Collapse) | Author | Age |
|---|
| |
|
|
| |
PiperOrigin-RevId: 187243942
|
| |
|
|
|
|
|
|
| |
If two SkyValue objects are equal, then Skyframe caches and returns the old one, instead of the new one. That's a problem for the detection of file changes, which uses the FileContentsProxy, which wasn't part of the equals check before this change.
Progress on #3360.
PiperOrigin-RevId: 183834897
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Local execution has an inherent race condition: if a user modifies a file while an action is executed, then it is impossible for Bazel to tell which version of the file was actually read during action execution. The file may have been modified before or after the tool has read it, or, in the worst case, the tool may have read both the original and the modified version. In addition, the file may be changed back to the original state before Bazel can check the file, so computing the digest before / after may not be sufficient.
This is a concern for both local and remote caches, although the cost of poisoning a shared remote cache is significantly higher, and is what has triggered this work.
Fixes #3360.
We solve this by keeping a reference to the FileContentsProxy, and using that to check for modificaitons before storing the cache entry. We output a warning if this check fails.
This change does not increase memory consumption; Java objects are always allocated in multiples of 8 bytes, we use compressed oops, and the FileArtifactValue currently has 12 bytes worth of fields (excl. object overhead), so adding another pointer is effectively free.
As a possible performance optimization on purely local builds, we could also consider not computing digests at all, and only use the FileContentsProxy for caching.
PiperOrigin-RevId: 182510358
|
| |
|
|
|
|
|
|
| |
After some consideration, I think it makes sense to always allow a getDigest call, instead of specifying it as disallowed based on type. This is a follow-up CL for a previous CL introducing the getType method, which increased the complexity of the specification.
I have a follow-up CL, which is related, namely unknown commit. After that CL, Metadata instances for directories (but not Filesets) also have digests rather than using mtime, which is compatible with the documentation changes made here. Said CL is solving a correctness issue with directory dependencies, which I think we want, and using the digest in the Metadata is a natural way to get correct action cache lookups.
PiperOrigin-RevId: 181440548
|
| |
|
|
|
|
|
|
| |
These tests don't require a full Skyframe instance, so we might as well move
them to a lighter-weight test class. Also, it turns out that we have duplicate
tests for equality and hashing - this is now explicit.
PiperOrigin-RevId: 181285144
|
| |
|
|
|
|
|
|
| |
This is in preparation for merging FileArtifactValue and FileStateValue.
Progress on #3360.
PiperOrigin-RevId: 179832948
|
| |
|
|
|
|
|
|
| |
Blaze had its own class to avoid GC from varargs array creation for the precondition happy path. Guava now (mostly) implements these, making it unnecessary to maintain our own.
This change was almost entirely automated by search-and-replace. A few BUILD files needed fixing up since I removed an export of preconditions from lib:util, which was all done by add_deps. There was one incorrect usage of Preconditions that was caught by error prone (which checks Guava's version of Preconditions) that I had to change manually.
PiperOrigin-RevId: 175033526
|
| |
|
|
|
|
|
|
|
|
| |
Replace all previous uses of Metadata with FileArtifactValue (or a simple inner
class in the case of ActionCacheChecker.CONSTANT_METADATA).
Care was taken to make the equals method obey the equals contract, even in the
presence of multiple implementations.
PiperOrigin-RevId: 160115080
|
| |
|
|
|
|
|
|
|
|
|
|
| |
an ordinary Artifact in ArtifactFunction.
If the data available in the FileValue is not sufficient to recreate the FileArtifactValue without disk access, then ActionMetadataHandler already stores the FileArtifactValue separately.
This is approximately step -3 in a grand plan to save a bunch more memory when building with --batch --discard_analysis_cache --keep_going.
--
PiperOrigin-RevId: 150646007
MOS_MIGRATED_REVID=150646007
|
| |
|
|
|
|
|
| |
digest are mutually exclusive.
--
MOS_MIGRATED_REVID=128843642
|
| |
|
|
|
|
|
| |
always digested.
--
MOS_MIGRATED_REVID=128768429
|
| |
|
|
|
|
|
|
|
| |
Since we no longer stored mtime for empty files, this bug meant that we always compared empty files equal (which is good). But we shouldn't be using Metadata based on mtime for them.
A follow-up change will do a refactor to make this impossible.
--
MOS_MIGRATED_REVID=128742054
|
| |
|
|
|
|
|
| |
RELNOTES: Bazel no longer regards an empty file as changed if its mtime has changed.
--
MOS_MIGRATED_REVID=127328552
|
| |
|
|
|
|
|
| |
ArtifactFunction.
--
MOS_MIGRATED_REVID=114174899
|
| |
|
|
|
|
|
| |
Reduces garbage.
--
MOS_MIGRATED_REVID=109914243
|
| |
|
|
|
|
|
|
|
|
|
| |
The headers were modified with
`find . -type f -exec 'sed' '-Ei' 's|Copyright 201([45]) Google|Copyright 201\1 The Bazel Authors|' '{}' ';'`
And manual edit for not Google owned copyright. Because of the nature of ijar, I did not modified the header of file owned by Alan Donovan.
The list of authors were extracted from the git log. It is missing older Google contributors that can be added on-demand.
--
MOS_MIGRATED_REVID=103938715
|
| |
|
|
|
|
|
| |
MoreObjects#toStringHelper
--
MOS_MIGRATED_REVID=98383075
|
| |
|
|
|
|
|
| |
alternative include scanning implementations possible.
--
MOS_MIGRATED_REVID=96337469
|
| |
|
|
|
| |
--
MOS_MIGRATED_REVID=94318260
|
| |
|
|
|
|
|
| |
hash code implementation from super class.
--
MOS_MIGRATED_REVID=94297315
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
We define unused artifacts as those that aren't consumed by any
action. This can be because an action produced more outputs than
a dependent action needed, or because it's a top level artifact
and we don't care about its contents, just that it was built
without issue. Actions may prevent outputs from being discarded
by declaring them as mandatory. This is particularly useful for
test outputs. The motivation behind this change is to reduce
storage overhead for things we can do without.
It is worth noting this change doesn't cover all cases. In particular
it has difficulty identifying *_binary artifacts as orphaned. This
is due to the insertion of a virtual runfiles artifact which depends
upon the rule's outputs.
--
MOS_MIGRATED_REVID=88467504
|
|
|
--
MOE_MIGRATED_REVID=85702957
|
janakr
ulfjack
tomlu
Michael Thvedt
Mark Schaller
Damien Martin-Guillerez
Googler
Michajlo Matijkiw
Han-Wen Nienhuys