aboutsummaryrefslogtreecommitdiffhomepage
path: root/src/main/java/com/google/devtools/build/lib/actions/AbstractAction.java
Commit message (Collapse)AuthorAge
* If --batch, --keep_going, --discard_analysis_cache, and the new ↵Gravatar janakr2017-03-31
| | | | | | | | | | | | | | | --noexperimental_enable_critical_path_profiling flags are all specified, then Bazel will delete Actions from ActionLookupValues as they are executed in order to save memory. Because an already-run action may output an artifact that is only requested later in the build, we need to maintain a way for the artifact to look up the action. But in most cases we don't need to keep the action itself, just its output metadata. Some actions unfortunately are needed post-execution, and so we special-case them. Also includes dependency change with description: Move action out of key. This keeps action references from polluting the graph -- actions are just stored in one SkyValue, instead of being present in SkyKeys. This does mean additional memory used: we have a separate ActionLookupData object per Action executed. That may reach ~24M for million-action builds. PiperOrigin-RevId: 151756383
* A partial, manual rollback of commit 7af14dfdbd6addb779226c0a103b2a8dc72c16b1.Gravatar Lukacs Berki2017-03-14
| | | | | | | | This became necessary because extra actions for C++ compile actions require .h files, but the compiler only returns the .pcm files in the .d file for headers that it reads from the .pcm file. This is not a problem for correctness because the .pcm files depend on the headers, but that doesn't help the extra actions that would then only get the .pcm files. -- PiperOrigin-RevId: 150052839 MOS_MIGRATED_REVID=150052839
* Clean up the semantics of input discovering actions a bit by making ↵Gravatar Lukacs Berki2017-02-28
| | | | | | | | | | | | updateInputs() and inputsKnown() non-overridable and removing setInputs(). This comes at the cost of adding a flag to every action instance that's not used for non-input-discovering actions, but I think that's a deal. Simpler APIs are good, mmmmkay? Also fixed a few pre-existing issues in TestAction and ObjcCompileAction. -- PiperOrigin-RevId: 148749734 MOS_MIGRATED_REVID=148749734
* Remove AbtractAction#getInputsForExtraAction() (the third time).Gravatar Lukacs Berki2017-02-27
| | | | | | | | | | This time, if the action discovers inputs, the extra action is made to depend on the outputs of the action so that by the time the extra action runs, the inputs of the original action are discovered. This avoids us having to think about the state the shadowed action may keep. Yes, actions should not keep state, but they do. Such is life. -- PiperOrigin-RevId: 148627715 MOS_MIGRATED_REVID=148627715
* Rollback of commit e716ae46f359dc1361574f44569811ff80a758ac.Gravatar Ulf Adams2017-02-24
| | | | | | -- PiperOrigin-RevId: 148445872 MOS_MIGRATED_REVID=148445872
* Encore of commit 41c2a26eef89167e807cbc9f33487dc66bb757d3 that removed ↵Gravatar Lukacs Berki2017-02-24
| | | | | | | | | | | | | | | | | | | AbstractAction#getInputFilesForExtraAction(). Turns out, we didn't add *mandatory* inputs of the shadowed action to the extra action, thus, breakage. Original description: Remove AbstractAction#getInputFilesForExtraAction(). This method was used to return the discovered inputs for extra actions, but it turns out that we can use #discoverInputs() just as well. Note that this makes it possible for #discoverInputs() to be called more than once per action instance (once for the action and once for each extra action), but this appears to work. A followup change may be able to dispense with that, but let's take baby steps for now. Also note that this introduces synchronization between an action and its associated extra action. -- PiperOrigin-RevId: 148429641 MOS_MIGRATED_REVID=148429641
* Rollback of commit 41c2a26eef89167e807cbc9f33487dc66bb757d3.Gravatar Carmi Grushko2017-02-20
| | | | | | | | | | This is not a clean rollback: I had to change CppCompileAction on line 1278 and pass cppSemantics.getIncludeProcessing(). Please flag this if it doesn't make sense :) -- PiperOrigin-RevId: 147868235 MOS_MIGRATED_REVID=147868235
* Remove AbstractAction#getInputFilesForExtraAction().Gravatar Lukacs Berki2017-02-03
| | | | | | | | | | | | This method was used to return the discovered inputs for extra actions, but it turns out that we can use #discoverInputs() just as well. Note that this makes it possible for #discoverInputs() to be called more than once per action instance (once for the action and once for each extra action), but this appears to work. A followup change may be able to dispense with that, but let's take baby steps for now. Also note that this introduces synchronization between an action and its associated extra action. -- PiperOrigin-RevId: 146450132 MOS_MIGRATED_REVID=146450132
* Simplify the Action interface by asking it a set of allowed inputs instead ↵Gravatar Lukacs Berki2017-02-02
| | | | | | | | | | of to resolve exec paths found in the action cache. The resolution algorithm was the same in all cases where it was implemented. -- PiperOrigin-RevId: 146344672 MOS_MIGRATED_REVID=146344672
* Simplify action input discovery by removing ↵Gravatar Lukacs Berki2017-01-31
| | | | | | | | | | Action#getInputsWhenSkippingInputDiscovery(). This has the side effect of always allowing new inputs to be discovered when Action#discoversInputs() return true, but since those actions are very few and have mostly to do with C++, I think it's an acceptable tradeoff for th ecode being that much simpler. -- PiperOrigin-RevId: 146098150 MOS_MIGRATED_REVID=146098150
* Names of extra-action protos now take into account all aspect names.Gravatar Dmitry Lomov2016-12-16
| | | | | | | | | | | | | If an Aspect registered an action that an extra-action is shadowing, its name is used when creating the extra-action's ID and name. Since recently, an aspect can see other aspects applied to the same target. This CL record the names of other aspects applied to the target as well, disambiguating the action owners. -- PiperOrigin-RevId: 142264153 MOS_MIGRATED_REVID=142264153
* Prune modules when building modules themselves to reduce build times shortenGravatar Googler2016-12-02
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | critical paths. When the inputs of a module M are reduced to a set S, then that same set S also needs to be supplied to compile something that uses M. To do this, input discovery is divided into two stages. For each CppCompileAction, the first stage discovers the necessary modules (M above). These are then added as inputs to ensure that they are built. The second stage then finds all the modules (S above) that are required to use those and also adds them as inputs. For now, the new behavior is guarded by a new flag --experimental_prune_more_modules. This is currently implemented by reading the .d files of used modules add adding all their module dependencies. There are two noteworthy alternatives: 1. Hack up input discovery to understand modules, e.g. if a modular header is hit, continue scanning from all it's headers. However, this seems very brittle and a lot of additional information would have to be passed to the input discovery. 2. Directly pass the results from input discovery of one CppCompileAction to another one. However, this seems to tightly couple the execution of different CppCompileActions and might lead to a mess of different states, more memory consumption, etc. With the current implementation, there is a bit of runtime overhead of reading the .d files (many times). This could potentially be improved by caching the results. However, even without this caching, the runtime overhead is limited (<10%) for all builds I have tried (I have tried with builds where all the compile results are already in the executor's cache. -- MOS_MIGRATED_REVID=140793217
* Properly handle bazel's output deletion within a write-protected directory.Gravatar Kush Chakraborty2016-11-23
| | | | | | | | Currently, we fail in deleting outputs from a previous invocation in a write-protected directory. This change retries deleting the outputs, after making the parent write-protected directory writable again. -- MOS_MIGRATED_REVID=140035023
* Fix up documentation for Actions APIGravatar Jon Brandvein2016-11-18
| | | | | -- MOS_MIGRATED_REVID=139569372
* Expose aspect-related information in the extra-action proto that Bazel hands ↵Gravatar Carmi Grushko2016-11-11
| | | | | | | | | to action_listener() rules. RELNOTES: Extra actions now contain aspect-related information. -- MOS_MIGRATED_REVID=138832922
* Add experimental flag to stop requiring all transitive modules as inputs.Gravatar Googler2016-10-31
| | | | | | | | | Requiring all transitive modules to always be available can lead to long critical paths and even unnecessary compiles in combination with the prune_header_modules feature. -- MOS_MIGRATED_REVID=137696794
* [Roll-forward] Rollback of commit dd299dbc61be51112534a4698f7cf2deee43053b.Gravatar Jon Brandvein2016-10-12
| | | | | | | | | | | | | | | | | | | | | | | *** Reason for rollback *** Fixed underlying broken CL that was depended on *** Original change description *** Automated [] rollback of commit e025939e71b179ae0f6bd09ef3af474f49b853a2. *** Reason for rollback *** Depends on commit 9c25afe750a937b2152c21a93effc8b9ba82c27b, which needs to be rolled back. *** Original change description *** Add API for individual actions This exposes action inputs, outputs, argv, content, and substitutions (if applicable). -- MOS_MIGRATED_REVID=135821603
* Rollback of commit e025939e71b179ae0f6bd09ef3af474f49b853a2.Gravatar Ulf Adams2016-10-11
| | | | | | | | | | | | | | | *** Reason for rollback *** Depends on commit 9c25afe750a937b2152c21a93effc8b9ba82c27b, which needs to be rolled back. *** Original change description *** Add API for individual actions This exposes action inputs, outputs, argv, content, and substitutions (if applicable). -- MOS_MIGRATED_REVID=135783964
* Add API for individual actionsGravatar Jon Brandvein2016-10-05
| | | | | | | This exposes action inputs, outputs, argv, content, and substitutions (if applicable). -- MOS_MIGRATED_REVID=135226123
* Update ActionMetadata -> ActionExecutionMetadata references.Gravatar Greg Estren2016-09-28
| | | | | -- MOS_MIGRATED_REVID=134537085
* Action#discoverInputs returns an Iterable<Artifact> instead of aGravatar Cal Peyser2016-08-25
| | | | | | | Collection<Artifact> -- MOS_MIGRATED_REVID=131285541
* Extend Action interface by client variablesGravatar Klaus Aehlig2016-08-24
| | | | | | | | | | | | | | | | As per our design on [Specifying environment variables](http://bazel.io/designs/2016/06/21/environment.html), actions may depend, in a controlled way, on the environment in which the Bazel client is invoked. Those environment variables are considered essential for the action, in the sense that it was to be repeated if either of them changes their value (note that other variables in client environment may well change without invalidating actions). Therefore, make the variables that need to be taken from the client environment part of the meta data for actions. -- Change-Id: I2ff6cf40b4ce8e0fea5c7e464f5f3b3e693025ac Reviewed-on: https://bazel-review.googlesource.com/#/c/5390 MOS_MIGRATED_REVID=131150211
* Allow Skyframe graph lookups and value retrievals to throw InterruptedException.Gravatar Janak Ramakrishnan2016-08-16
| | | | | | | The only place we now don't handle InterruptedException is in the action graph created after analysis, since I'm not sure that will be around for that much longer. -- MOS_MIGRATED_REVID=130327770
* Split ActionMetadata into ActionAnalysisMetadata and ActionExecutionMetadata.Gravatar Rumou Duan2016-04-26
| | | | | | | Except in action execution logic (ActionExecutionFunction, SkyframeActionExecutor, etc.), switch Action interface references to either ActionAnalysisMetadata if possible or ActionExecutionMetadata. -- MOS_MIGRATED_REVID=120723431
* Refactor the Event class; always construct through static methods.Gravatar Ulf Adams2016-04-21
| | | | | -- MOS_MIGRATED_REVID=120418505
* Add javadoc to AbstractAction.computeKey()Gravatar Alex Humesky2016-04-21
| | | | | -- MOS_MIGRATED_REVID=120416303
* Update the documentation for the Action class.Gravatar Ulf Adams2016-04-18
| | | | | -- MOS_MIGRATED_REVID=120117310
* Allow actions to specify if extra actions can attach to them.Gravatar Janak Ramakrishnan2016-04-07
| | | | | -- MOS_MIGRATED_REVID=119203499
* Make ActionOwner a final class, since all non-test implementations were ↵Gravatar Janak Ramakrishnan2016-03-21
| | | | | | | | | basically doing the same thing with it. This simplifies the code and avoids unnecessary re-wrapping, which saves memory. -- MOS_MIGRATED_REVID=117693050
* Little refactorings on resource management stuff:Gravatar Philipp Wollermann2016-02-28
| | | | | | | | | | - Remove unneeded abstract override AbstractAction#getResourceConsumption() - Remove comments justifying returning ResourceSet.ZERO from getResourceConsumption() - instead document it once on the base method. - Replace ResourceSet.create(0, 0, 0) with ResourceSet.ZERO. - Make sure no one ever returns null from getResourceConsumption() and then simplify the code in SkyframeActionExecutor to not check for it. -- MOS_MIGRATED_REVID=115739250
* Move annotation classes used for referencing framework Java classes from ↵Gravatar John Field2015-12-16
| | | | | | | | | | | | | | Skylark into their own package. This allows, e.g., classes in the syntax package to access classes in the cmdline package without creating circular dependencies. While we're here: - Removed a couple of unused BUILD deps flagged in []. - Updated SkylarkRuleImplementationFunctionsTest to remove non-ASCII characters and clarify the intent of the test. -- MOS_MIGRATED_REVID=110360763
* Use Bazel Preconditions variant which avoids varargs array creationGravatar Mark Schaller2015-12-10
| | | | | | | Reduces garbage. -- MOS_MIGRATED_REVID=109914243
* workers: Restart worker processes when their binary has changed since they ↵Gravatar Philipp Wollermann2015-11-05
| | | | | | | were launched. -- MOS_MIGRATED_REVID=107050157
* Rationalize copyright headersGravatar Damien Martin-Guillerez2015-09-25
| | | | | | | | | | | The headers were modified with `find . -type f -exec 'sed' '-Ei' 's|Copyright 201([45]) Google|Copyright 201\1 The Bazel Authors|' '{}' ';'` And manual edit for not Google owned copyright. Because of the nature of ijar, I did not modified the header of file owned by Alan Donovan. The list of authors were extracted from the git log. It is missing older Google contributors that can be added on-demand. -- MOS_MIGRATED_REVID=103938715
* Move Label from the lib.syntax to the lib.cmdline package so that:Gravatar Lukacs Berki2015-09-21
| | | | | | | | | | - Label parsing can be simplified - lib.syntax is only contains the code for Skylark and is reasonably independent from the problem domain of building things This change is mostly only changes to imports declarations. The rest is reversing the dependency between :cmdline and :syntax and moving a tiny amount of code between Printer and FilesetEntry and the addition of SkylarkPrintableValue that I couldn't be bothered to separate out into its own change. -- MOS_MIGRATED_REVID=103527877
* Cleanup Skylark types some moreGravatar Francois-Rene Rideau2015-09-17
| | | | | | | | | | | Clarify the criterion for being a valid Skylark value; stop claiming immutability is "the" criterion when Skylark now has mutable values; stop relying on a reflection with a magic list (this also fixes the SkylarkShell build). Clarify the criterion for determining immutable types when making a SkylarkNestedSet. Clarify and use the criterion for being a valid Skylark dict key. -- MOS_MIGRATED_REVID=103313934
* Eliminate BuildConfiguration.getShortName().Gravatar Lukacs Berki2015-06-29
| | | | | | | This was only used in error reporting, and the interface of BuildConfiguration could use some more simplicity. The comment on BuildConfiguration.getShortName() was confusing: that field is *not* actually used to compute the output directory name, which took me a few minutes to realize. -- MOS_MIGRATED_REVID=97128287
* Remove BuildConfiguration.cacheKey() and replace .shortCacheKey() with a ↵Gravatar Lukacs Berki2015-06-29
| | | | | | | | | | | .checksum() method that is computed from the checksum of the build options. Also remove Fragment.cacheKey() and Fragment.getName(). The reasoning is that within a single build, if two BuildConfigurations share the same BuildOptions, they must be the same because BuildConfiguration can only be a function of BuildOptions, specific targets (which stay the same during a build) and BlazeDirectories (which stay the same during the lifetime of a server). Between different builds if the configurations are recreated, they will always be considered different because Java reference equality (==) is used for comparing BuildConfigurations. Also remove "Serializable" tags from configuration-related things. -- MOS_MIGRATED_REVID=97107881
* Handle exceptions encountered resolving packages during the execution phaseGravatar Michajlo Matijkiw2015-06-10
| | | | | | | | | | | | | | Currently we may do lookups of not-already-cached packages during the execution phase for actions that discover inputs. Exceptions encountered during this would go unhandled and result in a crash. Here we introduce PackageRootResolutionException which wraps these exceptions and triggers an ActionExecutionException which is cleanly handled in the exec phase. As part of this change SkyframeActionExecutor#getArtifactRoots(...) will fail properly on errors getting package roots. -- MOS_MIGRATED_REVID=95578891
* Improve an error messageGravatar Francois-Rene Rideau2015-06-03
| | | | | -- MOS_MIGRATED_REVID=95006895
* Add runfiles to metadata caches and make them isFile awareGravatar Michajlo Matijkiw2015-05-26
| | | | | -- MOS_MIGRATED_REVID=94318260
* remove ActionMetadata#getInputCountGravatar Michajlo Matijkiw2015-03-23
| | | | | | | | | | | | Removing ActionMetadata#getInputCount instead of relying on all implementations of ActionMetadata keeping getInputCount and getInputs in sync. getInputCount is only used by a few tests and has no applications in the main code at the moment, so it is trivial to remove before we may come to rely on it more or its semantics get more complicated. -- MOS_MIGRATED_REVID=89130009
* Delay updating inputs of an action when processing the action cache until it ↵Gravatar Janak Ramakrishnan2015-03-20
| | | | | | | | | | | is known that the action is a cache hit. This adds momentary memory overhead when checking the action cache, but should prevent a host of potential errors. Note that this cl assumes that an action that discovers its inputs does *not* take the names of its inputs into account when calculating its key, which is already stated as part of the javadoc of Action#getKey. -- MOS_MIGRATED_REVID=88971626
* add baseline functionality for not saving unused artifactsGravatar Michajlo Matijkiw2015-03-13
| | | | | | | | | | | | | | | | | | | We define unused artifacts as those that aren't consumed by any action. This can be because an action produced more outputs than a dependent action needed, or because it's a top level artifact and we don't care about its contents, just that it was built without issue. Actions may prevent outputs from being discarded by declaring them as mandatory. This is particularly useful for test outputs. The motivation behind this change is to reduce storage overhead for things we can do without. It is worth noting this change doesn't cover all cases. In particular it has difficulty identifying *_binary artifacts as orphaned. This is due to the insertion of a virtual runfiles artifact which depends upon the rule's outputs. -- MOS_MIGRATED_REVID=88467504
* Return the set of files found when discovering inputs. Skyframe will ↵Gravatar Janak Ramakrishnan2015-03-11
| | | | | | | | | probably want to store these and may declare dependencies in advance so all metadata is present in the graph. Long-term, we may be able to use the returned set to make input-discovering actions immutable, which would be nice. -- MOS_MIGRATED_REVID=88348068
* Convert ResourceSet ctor calls to factory methodGravatar Mark Schaller2015-02-20
| | | | | | | | | | Most current invocations of the ResourceSet constructor shouldn't care about any new dimensions we add to restrict test execution. This change allows us to make those changes, keep the ResourceSet ctor symmetrical, and avoid contaminating those ctor call sites with noise. -- MOS_MIGRATED_REVID=86826112
* Replace some calls to ArtifactFactory.resolveSourceArtifact(PathFragment ↵Gravatar Marian Lobur2015-02-11
| | | | | | | execPath), with a skyframe native implementation ArtifactFactory.resolveSourceArtifact(Iterable<PathFragment> execPaths, PackageRootResolver resolver). -- MOS_MIGRATED_REVID=86062289
* Update from Google.Gravatar Han-Wen Nienhuys2015-02-25
-- MOE_MIGRATED_REVID=85702957
generated by cgit v1.2.3 (git 2.39.1) at 2024-07-07 20:06:29 +0000