aboutsummaryrefslogtreecommitdiffhomepage
path: root/src/main/java/com/google/devtools/build/lib/actions/ActionCacheChecker.java
Commit message (Collapse)AuthorAge
* Refactoring: makes the code simpler by deleting Metadata and InputArtifactData.Gravatar shahan2018-06-06
| | | | | | (minor) ActionFS now implements MetadataProvider.getInput PiperOrigin-RevId: 199575194
* Clean up code that directly imports nested classes like Builder, Entry, etc.Gravatar jcater2018-05-01
| | | | PiperOrigin-RevId: 194985157
* Adjust semantics of Metadata interface, remove isFile (use getType)Gravatar ulfjack2018-01-10
| | | | | | | | After some consideration, I think it makes sense to always allow a getDigest call, instead of specifying it as disallowed based on type. This is a follow-up CL for a previous CL introducing the getType method, which increased the complexity of the specification. I have a follow-up CL, which is related, namely unknown commit. After that CL, Metadata instances for directories (but not Filesets) also have digests rather than using mtime, which is compatible with the documentation changes made here. Said CL is solving a correctness issue with directory dependencies, which I think we want, and using the digest in the Metadata is a natural way to get correct action cache lookups. PiperOrigin-RevId: 181440548
* Move FileStateValue.Type to Metadata; add Metadata.getType()Gravatar ulfjack2017-12-21
| | | | | | | | This is in preparation for merging FileArtifactValue and FileStateValue. Progress on #3360. PiperOrigin-RevId: 179832948
* Add ActionKeyContext to Action#getKey.Gravatar tomlu2017-11-29
| | | | | | | This key context can be used by actions to share partial key computations, for instance when computing MD5s for nested sets. RELNOTES: None PiperOrigin-RevId: 177359607
* Replace all usages of Blaze's Preconditions class with guava.Gravatar tomlu2017-11-09
| | | | | | | | Blaze had its own class to avoid GC from varargs array creation for the precondition happy path. Guava now (mostly) implements these, making it unnecessary to maintain our own. This change was almost entirely automated by search-and-replace. A few BUILD files needed fixing up since I removed an export of preconditions from lib:util, which was all done by add_deps. There was one incorrect usage of Preconditions that was caught by error prone (which checks Guava's version of Preconditions) that I had to change manually. PiperOrigin-RevId: 175033526
* Collect action cache hits, misses, and reasons for the misses.Gravatar jmmv2017-09-27
| | | | | | | | As a bonus, this brings in a bunch of new unit tests for the ActionCacheChecker. RELNOTES: None. PiperOrigin-RevId: 170059577
* Make Metadata an interface for FileArtifactValueGravatar ulfjack2017-06-26
| | | | | | | | | | Replace all previous uses of Metadata with FileArtifactValue (or a simple inner class in the case of ActionCacheChecker.CONSTANT_METADATA). Care was taken to make the equals method obey the equals contract, even in the presence of multiple implementations. PiperOrigin-RevId: 160115080
* ActionMetadataHandler: proper metadata even for the volatile workspace statusGravatar ulfjack2017-06-21
| | | | | | | | | | | | | | This is part of cleaning up the ActionInputFileCache / MetadataHandler split. Both classes generally store and return identical information, except the MetadataHandler lies about constant metadata artifacts. Instead of lying here, update the ActionCacheChecker, which is the only place where we actually want constant metadata. Additionally, remove getMetadataMaybe; again, the only caller that needs special casing is the ActionCacheChecker, so we move the special casing there instead of having it in the MetadataHandler. PiperOrigin-RevId: 159665345
* Rewrite repository lookup to return a failed value rather than throwGravatar ulfjack2017-04-20
| | | | | | | | | | | | | | | | | | | | | | | We need to lookup repositories as part of converting exec paths to artifacts, which in turn is needed for action cache lookups. These lookups should not cause a Skyframe exit, so we must not throw an exception here, unless the error makes it impossible to continue. Instead, we need to leave the decision whether to error out or not to the caller. Note that we may unnecessarily fetch a remote repository in order to do the action cache lookup, even if the action no longer depends on the input file, although this should only be possible for C++ compile actions. It's possible that there's another bug in the C++ compile action key computation that also contributes. This change also makes it so that the post-resolution action cache code ignores any errors wrt. repository lookup rather than throwing. If any of the paths could not be found, then the action cache lookup fails and we re-execute the corresponding action, which is exactly what should happen. Fixes #2759. PiperOrigin-RevId: 153696243
* Refactor all ctor callsites of PathFragment to instead call a static ↵Gravatar nharmata2017-04-05
| | | | | | | | | | | | 'create' method. This paves the way for changing PathFragment to e.g. an abstract class with multiple subclasses. This way we can split out the windows-specific stuff into one of these concrete classes, making the code more readable and also saving memory (since the shallow heap size of the NonWindowsPathFragment subclass will hopefully be smaller than that of the current PathFragment). This also lets us pursue gc churn optimizations. We can now do interning in PathFragment#create and can also get rid of unnecessary intermediate PathFragment allocations. RELNOTES: None PiperOrigin-RevId: 152145768
* Convert --use_action_cache to a regular optionGravatar Googler2017-03-01
| | | | | | | | RELNOTES: Convert --use_action_cache to a regular option -- PiperOrigin-RevId: 148804881 MOS_MIGRATED_REVID=148804881
* Clean up the semantics of input discovering actions a bit by making ↵Gravatar Lukacs Berki2017-02-28
| | | | | | | | | | | | updateInputs() and inputsKnown() non-overridable and removing setInputs(). This comes at the cost of adding a flag to every action instance that's not used for non-input-discovering actions, but I think that's a deal. Simpler APIs are good, mmmmkay? Also fixed a few pre-existing issues in TestAction and ObjcCompileAction. -- PiperOrigin-RevId: 148749734 MOS_MIGRATED_REVID=148749734
* Skip computing digests when --use_action_cache=false.Gravatar Googler2017-02-16
| | | | | | | | | | | | | Does this by delegating responsibility for constructing ActionCache.Entry instances to the ActionCache, and having the StubActionCache return null. Tests show a 1-2% elapsed-time reduction for clean builds: [] Doesn't use interface default methods, because we still need JDK7 for Mac. -- PiperOrigin-RevId: 147722062 MOS_MIGRATED_REVID=147722062
* Simplify the Action interface by asking it a set of allowed inputs instead ↵Gravatar Lukacs Berki2017-02-02
| | | | | | | | | | of to resolve exec paths found in the action cache. The resolution algorithm was the same in all cases where it was implemented. -- PiperOrigin-RevId: 146344672 MOS_MIGRATED_REVID=146344672
* Track client environment in SkyframeGravatar Klaus Aehlig2016-09-14
| | | | | | | | | | | | | ...to determine which actions have to be recomputed based on changes to the client environment. Note that this change does it the simple way and reconsideres all actions on a changed client environment, while still only reexecuting those, where the part that was inherited from the environment actually did change. -- Change-Id: Ie1116d094642165e5e959447a6fcf49d19b37d6e Reviewed-on: https://bazel-review.googlesource.com/#/c/5431 MOS_MIGRATED_REVID=133010705
* Make Digest (renamed Md5Digest) a little more multi-purpose.Gravatar Shreya Bhattarai2016-08-23
| | | | | -- MOS_MIGRATED_REVID=130986194
* Allow Skyframe graph lookups and value retrievals to throw InterruptedException.Gravatar Janak Ramakrishnan2016-08-16
| | | | | | | The only place we now don't handle InterruptedException is in the action graph created after analysis, since I'm not sure that will be around for that much longer. -- MOS_MIGRATED_REVID=130327770
* Drive-by cleanup to remove unnecessary function from the ActionCache interface.Gravatar Shreya Bhattarai2016-08-10
| | | | | -- MOS_MIGRATED_REVID=129753109
* Split ActionMetadata into ActionAnalysisMetadata and ActionExecutionMetadata.Gravatar Rumou Duan2016-04-26
| | | | | | | Except in action execution logic (ActionExecutionFunction, SkyframeActionExecutor, etc.), switch Action interface references to either ActionAnalysisMetadata if possible or ActionExecutionMetadata. -- MOS_MIGRATED_REVID=120723431
* Refactor the Event class; always construct through static methods.Gravatar Ulf Adams2016-04-21
| | | | | -- MOS_MIGRATED_REVID=120418505
* Improve space efficiency of Blaze action cache: For actions that don't ↵Gravatar Eric Fellheimer2016-01-22
| | | | | | | perform input discovery, there is no need to store the full set of edges in the action cache. This data is only used to formulate the set of input files for an action prior to checking the validity of a cache entry. For non-input-discovering actions, the set of input files is known statically and the action cache data is not used. -- MOS_MIGRATED_REVID=112704382
* Use Bazel Preconditions variant which avoids varargs array creationGravatar Mark Schaller2015-12-10
| | | | | | | Reduces garbage. -- MOS_MIGRATED_REVID=109914243
* Rationalize copyright headersGravatar Damien Martin-Guillerez2015-09-25
| | | | | | | | | | | The headers were modified with `find . -type f -exec 'sed' '-Ei' 's|Copyright 201([45]) Google|Copyright 201\1 The Bazel Authors|' '{}' ';'` And manual edit for not Google owned copyright. Because of the nature of ijar, I did not modified the header of file owned by Alan Donovan. The list of authors were extracted from the git log. It is missing older Google contributors that can be added on-demand. -- MOS_MIGRATED_REVID=103938715
* Fix a bug in the ActionCacheChecker where the code that intended to remove ↵Gravatar Philipp Wollermann2015-06-15
| | | | | | | outdated keys from the cache actually removed the correct one, so the cache never got cleaned up. -- MOS_MIGRATED_REVID=95986001
* Handle exceptions encountered resolving packages during the execution phaseGravatar Michajlo Matijkiw2015-06-10
| | | | | | | | | | | | | | Currently we may do lookups of not-already-cached packages during the execution phase for actions that discover inputs. Exceptions encountered during this would go unhandled and result in a crash. Here we introduce PackageRootResolutionException which wraps these exceptions and triggers an ActionExecutionException which is cleanly handled in the exec phase. As part of this change SkyframeActionExecutor#getArtifactRoots(...) will fail properly on errors getting package roots. -- MOS_MIGRATED_REVID=95578891
* Belated cleanup of MetadataHandler#discardMetadata. Since we have per-action ↵Gravatar Janak Ramakrishnan2015-04-14
| | | | | | | metadata handlers, we can just unconditionally discard all output metadata with this call. -- MOS_MIGRATED_REVID=91008004
* Resolve inputs that were stored in action cache ahead of time so metadata ↵Gravatar Janak Ramakrishnan2015-03-30
| | | | | | | | | | | for them can be retrieved with other inputs, avoiding a restart. This also allows us to delete the UndeclaredInputsHandler, since we eagerly cache metadata before entering any ActionCacheChecker methods. This should also allow us to split FileAndMetadataCache into two separate classes, as well as asserting that SingleBuildFileCache only sees non-artifact ActionInputs. To be done in follow-ups. -- MOS_MIGRATED_REVID=89718712
* Move action-cache updating to inside ActionExecutionFunction, in preparation ↵Gravatar Janak Ramakrishnan2015-03-30
| | | | | | | | | for allowing it to be restarted in case of missing deps. Note that this means that action-cache writing is no longer part of the ACTION_COMPLETE profiling unit. -- MOS_MIGRATED_REVID=89702039
* Delay updating inputs of an action when processing the action cache until it ↵Gravatar Janak Ramakrishnan2015-03-20
| | | | | | | | | | | is known that the action is a cache hit. This adds momentary memory overhead when checking the action cache, but should prevent a host of potential errors. Note that this cl assumes that an action that discovers its inputs does *not* take the names of its inputs into account when calculating its key, which is already stated as part of the javadoc of Action#getKey. -- MOS_MIGRATED_REVID=88971626
* add baseline functionality for not saving unused artifactsGravatar Michajlo Matijkiw2015-03-13
| | | | | | | | | | | | | | | | | | | We define unused artifacts as those that aren't consumed by any action. This can be because an action produced more outputs than a dependent action needed, or because it's a top level artifact and we don't care about its contents, just that it was built without issue. Actions may prevent outputs from being discarded by declaring them as mandatory. This is particularly useful for test outputs. The motivation behind this change is to reduce storage overhead for things we can do without. It is worth noting this change doesn't cover all cases. In particular it has difficulty identifying *_binary artifacts as orphaned. This is due to the insertion of a virtual runfiles artifact which depends upon the rule's outputs. -- MOS_MIGRATED_REVID=88467504
* Introduce a PackageBoundaryMode enum to FilesetTraversalParams.Gravatar Laszlo Csomor2015-02-12
| | | | | | | | | | | | | | This enables the introduction of a third behavior mode when RecursiveFilesystemTraversalFunction encounters a package boundary: on top of the existing cross/no-cross behavior, it can now bail out with an error. This is preparatory work to support Skyframe-native Filesets. The CL also contains a couple of prettifying cleanups (in ActionCacheChecker and in FilesetManifestAction). -- MOS_MIGRATED_REVID=86185105
* Replace some calls to ArtifactFactory.resolveSourceArtifact(PathFragment ↵Gravatar Marian Lobur2015-02-11
| | | | | | | execPath), with a skyframe native implementation ArtifactFactory.resolveSourceArtifact(Iterable<PathFragment> execPaths, PackageRootResolver resolver). -- MOS_MIGRATED_REVID=86062289
* Update from Google.Gravatar Han-Wen Nienhuys2015-02-25
-- MOE_MIGRATED_REVID=85702957
generated by cgit v1.2.3 (git 2.39.1) at 2024-06-27 14:43:39 +0000