aboutsummaryrefslogtreecommitdiffhomepage
path: root/src/main/java/com/google/devtools/build/lib/actions
Commit message (Collapse)AuthorAge
* Remove AbstractAction#getInputFilesForExtraAction().Gravatar Lukacs Berki2017-02-03
| | | | | | | | | | | | This method was used to return the discovered inputs for extra actions, but it turns out that we can use #discoverInputs() just as well. Note that this makes it possible for #discoverInputs() to be called more than once per action instance (once for the action and once for each extra action), but this appears to work. A followup change may be able to dispense with that, but let's take baby steps for now. Also note that this introduces synchronization between an action and its associated extra action. -- PiperOrigin-RevId: 146450132 MOS_MIGRATED_REVID=146450132
* ForgeClient accepts Spawn, update some callersGravatar Ulf Adams2017-02-03
| | | | | | | | | | | | | | | This is one step towards changing ForgeClient to require Spawn in all cases. This is changing some of the easy ones. Also add a new Spawn implementation class - for some reason BaseSpawn is trying to be helpful and changing the environment around, as well as requiring that inputs and outputs are inhertied from the owning action, which isn't always the case. Ideally, BaseSpawn will go away in favor of the new implementation, but that requires updating the callers that require the special BaseSpawn functionality.. -- PiperOrigin-RevId: 146370998 MOS_MIGRATED_REVID=146370998
* Simplify the Action interface by asking it a set of allowed inputs instead ↵Gravatar Lukacs Berki2017-02-02
| | | | | | | | | | of to resolve exec paths found in the action cache. The resolution algorithm was the same in all cases where it was implemented. -- PiperOrigin-RevId: 146344672 MOS_MIGRATED_REVID=146344672
* Simplify the Spawn interface, update Executor interfaceGravatar Ulf Adams2017-02-01
| | | | | | | | | | | | | | | | | | | | Drop the Spawn.getOwner() method, which was duplicating functionality already available through Spawn.getResourceOwner(), in favor of the latter. In order to do that, change Executor.reportSubcommand to take a Spawn instance instead, which in turn requires updating all call sites. This is part of establishing Spawn as an abstraction for lower-level local and remote execution in all cases instead of passing args, env, input/output files and possibly other metadata as individual method parameters to the underlying implementations. In order for that to be maintainable, Spawn should be light-weight and easy to construct correctly, and not contain too much unnecessary data. -- PiperOrigin-RevId: 146224914 MOS_MIGRATED_REVID=146224914
* Simplify the Spawn interfaceGravatar Ulf Adams2017-02-01
| | | | | | | | | | | | | | | | | | | | | | | | Remove getSpawnInfo and asShellCommand, which are never really overridden in a useful way. asShellCommand moves to the Spawns class, and getSpawnInfo is only ever called by SpawnAction, and the implementation moves there. I'm considering using Spawn as the general lower-level abstraction for both local and remote execution. It sort of is that already, except it's not used consistently - we often pass a tuple of (args, env) plus possibly input and output files through parameter-heavy method call hierarchies instead of using this existing abstraction. However, I'm concerned about the amount of baggage it's carrying as well as the number of implementations for what is supposed to be a simple interface (or possibly even a simple value class), and this is an attempt to slim it down a bit. This should have no visible effects on builds. -- PiperOrigin-RevId: 146109838 MOS_MIGRATED_REVID=146109838
* Simplify action input discovery by removing ↵Gravatar Lukacs Berki2017-01-31
| | | | | | | | | | Action#getInputsWhenSkippingInputDiscovery(). This has the side effect of always allowing new inputs to be discovered when Action#discoversInputs() return true, but since those actions are very few and have mostly to do with C++, I think it's an acceptable tradeoff for th ecode being that much simpler. -- PiperOrigin-RevId: 146098150 MOS_MIGRATED_REVID=146098150
* Switch to RunfilesSuppliers for communicating runfilesGravatar Michajlo Matijkiw2017-01-30
| | | | | | | | | | | | | | | | | | | | | | | ActionSpawn/SpawnAction now deal exclusively in RunfilesSuppliers, manifests maps are no more. There is some lingering awkwardness, in particular: - Manifests still need to be tracked in some places, we can work out if this is still necessary on a case by case basis. - Skylark makes actions' runfiles available via 'resolve_command' where they are consumed by 'action'. I've updated the documentation, though the name isn't entirely accurate anymore. That being said these interfaces _are_ marked as experimental, so we _should_ be able to be flexible here. Overall, I think the benefits consolidating runfiles into suppliers, from both code cleanliness and performance perspectives (no longer needing to parse manifests), outweights the awkwardnesses. RELNOTES: resolve_command/action's input_manifest return/parameter is now list -- PiperOrigin-RevId: 145817429 MOS_MIGRATED_REVID=145817429
* Support mapping of Paths to URIsGravatar Klaus Aehlig2017-01-18
| | | | | | | | | | | Bazel-created files (like log files of test runs) are internally reported as Paths. However, this is not always the most useful representation of the location of that artifact for a consumer of build events. Therefore, support a mapping of paths to more useful URIs. -- PiperOrigin-RevId: 144843525 MOS_MIGRATED_REVID=144843525
* Description redacted.Gravatar Adam Michael2017-01-18
| | | | | | -- PiperOrigin-RevId: 144739068 MOS_MIGRATED_REVID=144739068
* Tell Python binaries the location of their runfiles tree using the ↵Gravatar Lukacs Berki2017-01-13
| | | | | | | | | | | | RUNFILES_DIR environment variable. Note that this is slightly different from the Java launcher (where the same is JAVA_RUNFILES) Fixes #2339. -- PiperOrigin-RevId: 144439111 MOS_MIGRATED_REVID=144439111
* Add a small helper method to ActionInputHelperGravatar Ulf Adams2017-01-11
| | | | | | | | | | This is part of a series of changes to clean up the test strategies; this specific change is a Google-internal refactoring. I'm planning to also update the external test strategies subsequently. -- PiperOrigin-RevId: 144184752 MOS_MIGRATED_REVID=144184752
* Move ExecutionInfoSpecifier to lib.actionsGravatar Ulf Adams2017-01-09
| | | | | | | | | We were previously jumping through hoops to avoid cyclic dependencies. It's unclear why the class was here in the first place rather than in lib.actions. -- PiperOrigin-RevId: 143948959 MOS_MIGRATED_REVID=143948959
* Move ProcMeminfoParser to lib.unix, where it belongsGravatar Ulf Adams2017-01-09
| | | | | | -- PiperOrigin-RevId: 143939410 MOS_MIGRATED_REVID=143939410
* Some little changes to prep for rolling forward the execroot changeGravatar Kristina Chodorow2017-01-05
| | | | | | | | | | This are random little nits that aren't dependent on changing the directory structure, so it makes the (rather large) CL that is coming cleaner. -- PiperOrigin-RevId: 143690681 MOS_MIGRATED_REVID=143690681
* Manually add periods to documentation strings where needed.Gravatar John Cater2017-01-03
| | | | | | | | -- Change-Id: I9ec3209a69ba5a51943b334f278ba93d67d4f9f4 Reviewed-on: https://cr.bazel.build/8090 PiperOrigin-RevId: 143470915 MOS_MIGRATED_REVID=143470915
* Move some classes to lib.exec in preparation for layering enforcementGravatar Ulf Adams2016-12-21
| | | | | | | | | | | | This change is part of the mu-bazel effort, which aims to build a minimally useful Bazel binary with most extraneous functionality removed. As part of that, we want to enforce layering of packages. In particular, lib.actions must not depend on lib.rules or lib.exec. lib.rules must not depend on lib.exec. Moving these classes is a necessary step to enforce that layering. -- PiperOrigin-RevId: 142668172 MOS_MIGRATED_REVID=142668172
* Names of extra-action protos now take into account all aspect names.Gravatar Dmitry Lomov2016-12-16
| | | | | | | | | | | | | If an Aspect registered an action that an extra-action is shadowing, its name is used when creating the extra-action's ID and name. Since recently, an aspect can see other aspects applied to the same target. This CL record the names of other aspects applied to the target as well, disambiguating the action owners. -- PiperOrigin-RevId: 142264153 MOS_MIGRATED_REVID=142264153
* Streamline Fingerprint implementationGravatar Michajlo Matijkiw2016-12-15
| | | | | | | | | | | | Thread all updates through a CodedOutputStream. This has the benefit of potentially hashing less data, as many int values can be represented more compactly, and reducing the churn of hashing strings (generally), since CodedOutputStream is already heavily optimized for this. While the buffer size is a little generous, it winds up paying off. -- PiperOrigin-RevId: 142151062 MOS_MIGRATED_REVID=142151062
* Prune modules when building modules themselves to reduce build times shortenGravatar Googler2016-12-02
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | critical paths. When the inputs of a module M are reduced to a set S, then that same set S also needs to be supplied to compile something that uses M. To do this, input discovery is divided into two stages. For each CppCompileAction, the first stage discovers the necessary modules (M above). These are then added as inputs to ensure that they are built. The second stage then finds all the modules (S above) that are required to use those and also adds them as inputs. For now, the new behavior is guarded by a new flag --experimental_prune_more_modules. This is currently implemented by reading the .d files of used modules add adding all their module dependencies. There are two noteworthy alternatives: 1. Hack up input discovery to understand modules, e.g. if a modular header is hit, continue scanning from all it's headers. However, this seems very brittle and a lot of additional information would have to be passed to the input discovery. 2. Directly pass the results from input discovery of one CppCompileAction to another one. However, this seems to tightly couple the execution of different CppCompileActions and might lead to a mess of different states, more memory consumption, etc. With the current implementation, there is a bit of runtime overhead of reading the .d files (many times). This could potentially be improved by caching the results. However, even without this caching, the runtime overhead is limited (<10%) for all builds I have tried (I have tried with builds where all the compile results are already in the executor's cache. -- MOS_MIGRATED_REVID=140793217
* Properly handle bazel's output deletion within a write-protected directory.Gravatar Kush Chakraborty2016-11-23
| | | | | | | | Currently, we fail in deleting outputs from a previous invocation in a write-protected directory. This change retries deleting the outputs, after making the parent write-protected directory writable again. -- MOS_MIGRATED_REVID=140035023
* Fix up documentation for Actions APIGravatar Jon Brandvein2016-11-18
| | | | | -- MOS_MIGRATED_REVID=139569372
* Adding an option to set the digest function that everything uses. Minor ↵Gravatar Ola Rozenfeld2016-11-18
| | | | | | | | | | | | | refactoring: enabling potential fast digest computation of more than one digest function type. Usage: bazel --host_jvm_args="-Dbazel.DigestFunction=SHA1" build ... Ugliness: using a system property (a static non-final variable), because the better way to do it (a flag) would result in a much, much larger refactoring. More ugliness: I have updated the minimal amount of tests. A lot of tests are still relying on the default value of MD5. Ideally, they need to be updated as well. -- MOS_MIGRATED_REVID=139490836
* Expose aspect-related information in the extra-action proto that Bazel hands ↵Gravatar Carmi Grushko2016-11-11
| | | | | | | | | to action_listener() rules. RELNOTES: Extra actions now contain aspect-related information. -- MOS_MIGRATED_REVID=138832922
* Migrate ActionOwner to @AutoValue.Gravatar Carmi Grushko2016-11-10
| | | | | -- MOS_MIGRATED_REVID=138680612
* Expose Artifact#isTreeArtifact to Skylark.Gravatar Rumou Duan2016-11-04
| | | | | -- MOS_MIGRATED_REVID=138112581
* Move createActionCache to ExecutorBuilder.Gravatar Ulf Adams2016-11-02
| | | | | -- MOS_MIGRATED_REVID=137936478
* Refactor the module API to use the builder pattern for executor creation.Gravatar Ulf Adams2016-11-02
| | | | | | | This significantly simplifies several of our modules. -- MOS_MIGRATED_REVID=137713119
* Add experimental flag to stop requiring all transitive modules as inputs.Gravatar Googler2016-10-31
| | | | | | | | | Requiring all transitive modules to always be available can lead to long critical paths and even unnecessary compiles in combination with the prune_header_modules feature. -- MOS_MIGRATED_REVID=137696794
* Report completion of a target together with failed actionsGravatar Klaus Aehlig2016-10-27
| | | | | | | | | | | | | | | | Report the completion of all targets together with the root causes on the build event stream. To do so, have TargetCompleteEvent and ActionExecutedEvent be instances of BuildEvent; however, ignore an ActionExecutedEvent in the BuildEventStreamer if the execution was successful. By this change we get, for the first time, a build event stream that is naturally closed, i.e., without Aborted events closing up lose ends. Add a test asserting this property. -- Change-Id: Ie90dd52ee80deb0fdabdce1da551935522880a1a Reviewed-on: https://bazel-review.googlesource.com/#/c/6279 MOS_MIGRATED_REVID=137273002
* Add a new concept of failure causesGravatar Klaus Aehlig2016-10-25
| | | | | | | | | | | | | | | | | | | | Not all possible reasons for failure are uniquely identified by a label. Therefore, add a new data type describing possible root causes of failures and use it. The new type is added in causes/*.java and coresponds to Haskell's one-line definition data Cause = LabelCause Label | ActionCause Path Label deriving Show With future clean up of other failure causes inadequately described by a label, we expect that type to be extended by new constructors (i.e., new classes implementing Cause). -- Change-Id: I6fec74c78cec6abb9c10e32743b05a792888fead Reviewed-on: https://bazel-review.googlesource.com/#/c/6617 MOS_MIGRATED_REVID=137156390
* Proper action output checks for TreeArtifacts. Instead of crashing Bazel, we ↵Gravatar Rumou Duan2016-10-19
| | | | | | | now handle failed TreeArtifact output checks gracefully. -- MOS_MIGRATED_REVID=136627086
* Rollback of commit e6191f7209ebf3f67480bc7aa9db5ac58beb2156.Gravatar Damien Martin-Guillerez2016-10-19
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | *** Reason for rollback *** Roll-forward with fix. Tested with `bazel build src:srcs //src/test/...` using a bootstrapped bazel. Fixes #1923. RELNOTES[INC]: Non test-only targets can no longer depends on test-only targets. *** Original change description *** Automated [] rollback of commit a9f20b0d6459d395444c45cf5e94a899f3443633. *** Reason for rollback *** Broke Bazel CI: ERROR: /home/ci/workspace/Bazel/JAVA_VERSION/1.8/PLATFORM_NAME/linux-x86_64/src/BUILD:284:1: in filegroup rule //src:srcs: non-test target '//src:srcs' depends on testonly target '//src/java_tools/junitrunner/java/com/google/testing/junit/junit4:srcs' and doesn't have testonly attribute set. ERROR: Analysis of target '//scripts/packages:packages' failed; build aborted. http://ci.bazel.io/view/Bazel%20b... *** -- MOS_MIGRATED_REVID=136589937
* Add file extension property to SkylarkGravatar Paul Roberts2016-10-18
| | | | | | | RELNOTES[NEW]: Files now have an "extension" property in Skylark. -- MOS_MIGRATED_REVID=136425934
* Fix a bug in which Bazel reports action conflicts between ActionTemplates ↵Gravatar Rumou Duan2016-10-17
| | | | | | | and associated expanded actions. -- MOS_MIGRATED_REVID=136212908
* [Roll-forward] Rollback of commit dd299dbc61be51112534a4698f7cf2deee43053b.Gravatar Jon Brandvein2016-10-12
| | | | | | | | | | | | | | | | | | | | | | | *** Reason for rollback *** Fixed underlying broken CL that was depended on *** Original change description *** Automated [] rollback of commit e025939e71b179ae0f6bd09ef3af474f49b853a2. *** Reason for rollback *** Depends on commit 9c25afe750a937b2152c21a93effc8b9ba82c27b, which needs to be rolled back. *** Original change description *** Add API for individual actions This exposes action inputs, outputs, argv, content, and substitutions (if applicable). -- MOS_MIGRATED_REVID=135821603
* Rollback of commit e025939e71b179ae0f6bd09ef3af474f49b853a2.Gravatar Ulf Adams2016-10-11
| | | | | | | | | | | | | | | *** Reason for rollback *** Depends on commit 9c25afe750a937b2152c21a93effc8b9ba82c27b, which needs to be rolled back. *** Original change description *** Add API for individual actions This exposes action inputs, outputs, argv, content, and substitutions (if applicable). -- MOS_MIGRATED_REVID=135783964
* Add a method in MetadataHandler to retrieve contained artifacts from ↵Gravatar Rumou Duan2016-10-07
| | | | | | | TreeArtifacts. -- MOS_MIGRATED_REVID=135485914
* Remove duplicated "Can be None". The allowReturnNones = true generates "MayGravatar Adam Michael2016-10-07
| | | | | | | | | return `None`.". Fixes https://github.com/bazelbuild/bazel/issues/1893 -- MOS_MIGRATED_REVID=135419555
* JS, Localization: Adds support for localization with closure managed rules.Gravatar Pedro Liberal Fernandez2016-10-05
| | | | | | | RELNOTES: JS: Add support for localization with closure managed rules. -- MOS_MIGRATED_REVID=135260827
* Add API for individual actionsGravatar Jon Brandvein2016-10-05
| | | | | | | This exposes action inputs, outputs, argv, content, and substitutions (if applicable). -- MOS_MIGRATED_REVID=135226123
* Update ActionMetadata -> ActionExecutionMetadata references.Gravatar Greg Estren2016-09-28
| | | | | -- MOS_MIGRATED_REVID=134537085
* Add a SandboxedSpawnActionContext interface and implementations for our ↵Gravatar Philipp Wollermann2016-09-23
| | | | | | | sandboxed execution strategies. -- MOS_MIGRATED_REVID=134054610
* Rollback of commit 82d43279f93d95e4c41b4bc598a3cc05ddd1ae1a.Gravatar Laszlo Csomor2016-09-21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | *** Reason for rollback *** Breaks TensorFlow and other Bazel jobs on ci.bazel.io *** Original change description *** Change execution root for external repositories to be ../repo Some of the important aspect of this change: * Remote repos in the execution root are under output_base/execroot/repo_name, so the prefix is ../repo_name (to escape the local workspace name). * Package roots for external repos were previously "output_base/", they are now output_base/external/repo_name (which means source artifacts always have a relative path from their repository). * Outputs are under bazel-bin/external/repo_name/ (or similarly under genfiles). Note that this is a bit of a change from how this was implemented in the previous cl. Fixes #1262. RELNOTES[INC]: Previously, an external repository would be symlinked into the execution root at execroot/local_repo/external/remote_repo. This changes it to be at execroot/remote_repo. This may break genrules/Skylark actions that hardcode execution root paths. If this causes breakages for you, ensure that genrules are using $(location :target) to access files and Skylark rules are using http://bazel.io/docs/skylark/lib/File.html's path, dirname, etc. functions. Roll forward of bdfd58a. -- MOS_MIGRATED_REVID=133709658
* Add flag --nosandbox/--no_sandbox/--no-sandbox to disable sandbox.Gravatar Yue Gan2016-09-21
| | | | | -- MOS_MIGRATED_REVID=133697962
* Change execution root for external repositories to be ../repoGravatar Kristina Chodorow2016-09-20
| | | | | | | | | | | | | | | | | | | | | | | Some of the important aspect of this change: * Remote repos in the execution root are under output_base/execroot/repo_name, so the prefix is ../repo_name (to escape the local workspace name). * Package roots for external repos were previously "output_base/", they are now output_base/external/repo_name (which means source artifacts always have a relative path from their repository). * Outputs are under bazel-bin/external/repo_name/ (or similarly under genfiles). Note that this is a bit of a change from how this was implemented in the previous cl. Fixes #1262. RELNOTES[INC]: Previously, an external repository would be symlinked into the execution root at execroot/local_repo/external/remote_repo. This changes it to be at execroot/remote_repo. This may break genrules/Skylark actions that hardcode execution root paths. If this causes breakages for you, ensure that genrules are using $(location :target) to access files and Skylark rules are using http://bazel.io/docs/skylark/lib/File.html's path, dirname, etc. functions. Roll forward of bdfd58a. -- MOS_MIGRATED_REVID=133606309
* Remove some more occurences of deprecated getExecRoot() callsGravatar Kristina Chodorow2016-09-15
| | | | | -- MOS_MIGRATED_REVID=133257532
* Track client environment in SkyframeGravatar Klaus Aehlig2016-09-14
| | | | | | | | | | | | | ...to determine which actions have to be recomputed based on changes to the client environment. Note that this change does it the simple way and reconsideres all actions on a changed client environment, while still only reexecuting those, where the part that was inherited from the environment actually did change. -- Change-Id: Ie1116d094642165e5e959447a6fcf49d19b37d6e Reviewed-on: https://bazel-review.googlesource.com/#/c/5431 MOS_MIGRATED_REVID=133010705
* Add repository parameter to source artifact resolverGravatar Kristina Chodorow2016-09-09
| | | | | | | Needed for #1262. Doesn't do anything, yet, other than make the CL smaller. -- MOS_MIGRATED_REVID=132671036
* Add an "is main repo" field to RootsGravatar Kristina Chodorow2016-09-09
| | | | | | | This is part of prepping for #1262. -- MOS_MIGRATED_REVID=132553178
* Refactoring getTimeoutSeconds of a Spawn into Spaws.java, becaused it is usedGravatar Googler2016-09-08
| | | | | | | in places other than the sandbox code. -- MOS_MIGRATED_REVID=132436150
generated by cgit v1.2.3 (git 2.39.1) at 2024-08-14 12:27:59 +0000