aboutsummaryrefslogtreecommitdiffhomepage
path: root/src/main/java/com/google/devtools/build/lib/actions
Commit message (Collapse)AuthorAge
* Automated rollback of commit 3bace1b937934fb2cea6260067ecc1cdbe526847.Gravatar felly2018-08-14
| | | | | | | | | | | | | *** Reason for rollback *** b/112583337 RELNOTES: None *** Original change description *** Track Fileset in artifact expansion. PiperOrigin-RevId: 208658921
* Fixes broken javadoc FilesetTraversalParams (getNestedTraversal should beGravatar shahan2018-08-13
| | | | | | getNestedArtifact). PiperOrigin-RevId: 208565138
* Track Fileset in artifact expansion.Gravatar felly2018-08-13
| | | | PiperOrigin-RevId: 208549418
* ActionAnalysisMetadata exposes whether loose headers are present. ↵Gravatar shahan2018-08-12
| | | | | | TransitiveVersionTable uses futures. PiperOrigin-RevId: 208402465
* Automated rollback of commit 9374ecf94ce44e8bc56e68678cb512abf4cf9ce2.Gravatar felly2018-08-10
| | | | | | | | | | | | | | | | | | | | | | *** Reason for rollback *** Rollforward with deduplication of traversals and regression test. RELNOTES: None *** Original change description *** Automated rollback of commit 39974a43abdd32e3a1acbc7da945b08da9983e4e. *** Reason for rollback *** b/112458627 *** Original change description *** Allow skyframe-aware actions to pass partial results through ActionExecutionContext. Remove FilesetProvider. PiperOrigin-RevId: 208231719
* Automated rollback of commit 39974a43abdd32e3a1acbc7da945b08da9983e4e.Gravatar felly2018-08-10
| | | | | | | | | | | | | *** Reason for rollback *** b/112458627 *** Original change description *** Allow skyframe-aware actions to pass partial results through ActionExecutionContext. Remove FilesetProvider. PiperOrigin-RevId: 208213955
* Allow skyframe-aware actions to pass partial results through ↵Gravatar felly2018-08-09
| | | | | | | | ActionExecutionContext. Remove FilesetProvider. PiperOrigin-RevId: 208111251
* Rename a param to better reflect its intent.Gravatar nharmata2018-08-06
| | | | | RELNOTES: None PiperOrigin-RevId: 207550943
* When no new configured targets have been analyzed, only check for artifact ↵Gravatar janakr2018-08-03
| | | | | | | | | | conflicts if the current set of configured targets is not a subset of the largest set of configured targets that have been checked for conflicts. Also rework the flow between SkyframeBuildView and SkyframeActionExecutor to remove the SkyframeExecutor middleman. Also reword the error message in case of an ArtifactPrefixConflictException, since a clean should no longer be necessary. PiperOrigin-RevId: 207322139
* Thread a path resolver through target/aspect complete event.Gravatar tomlu2018-08-02
| | | | | | | At the moment, an identity path resolver is passed. This will later be replaced by a contextual path resolver. RELNOTES: None PiperOrigin-RevId: 207138772
* Automated rollback of commit a6255612e4892729d3758775c76085b26b9bc584.Gravatar twerth2018-08-02
| | | | | | | | | | | | | | | | | | | | | *** Reason for rollback *** Breaks FDO builds, see http://b/112112125 *** Original change description *** Remove PerActionFileCache Instead, make ActionMetadataHandler implement the MetadataProvider interface. This fixes an issue where an action that runs two spawns where one depends on an output of the other was unable to get the metadata for the intermediate output. We don't currently have actions that do this, but we will have in a future change (which will also implicitly act as a regression test). PiperOrigin-RevId: 207084179
* Pass a path to ActionExecutedEvent that is resolved via ActionExecutionContext.Gravatar tomlu2018-08-01
| | | | | | | If the output artifact is backed by a different file system then we need to use that file system later. RELNOTES: None PiperOrigin-RevId: 206981369
* Remove the need for discoverInputsStage2(). Much like every other SkyframeGravatar Googler2018-08-01
| | | | | | | | function, discoverInputs() can be implemented to return null upon encountering missing ActionExecutionValues. RELNOTES: None. PiperOrigin-RevId: 206913969
* Open source SpawnMetricsGravatar ulfjack2018-08-01
| | | | PiperOrigin-RevId: 206893284
* Remove source index from SourceFileArtifactValue.Gravatar felly2018-07-30
| | | | PiperOrigin-RevId: 206586252
* Modify --subcommands to allow pretty printing the arguments of a subcommandGravatar ahumesky2018-07-26
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | as a list, rather than as a single line (i.e., newline delimited rather than space delimited). Before: SUBCOMMAND: # //src/main/java/com/google/devtools/build/lib:string_util [action 'Building src/main/java/com/google/devtools/build/lib/libstring_util.jar (2 source files) [for host]'] (cd /tmp/devbazel_output_base/execroot/io_bazel && \ exec env - \ LC_CTYPE=en_US.UTF-8 \ external/embedded_jdk/bin/java -XX:+UseParallelOldGC -XX:-CompactStrings '--add-exports=jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED' '--add-exports=jdk.compiler/com.sun.tools.javac.code=ALL-UNNAMED' '--add-exports=jdk.compiler/com.sun.tools.javac.comp=ALL-UNNAMED' '--add-exports=jdk.compiler/com.sun.tools.javac.file=ALL-UNNAMED' '--add-exports=jdk.compiler/com.sun.tools.javac.main=ALL-UNNAMED' '--add-exports=jdk.compiler/com.sun.tools.javac.tree=ALL-UNNAMED' '--add-exports=jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED' '--add-opens=jdk.compiler/com.sun.tools.javac.file=ALL-UNNAMED' '--patch-module=java.compiler=external/bazel_tools/third_party/java/jdk/langtools/java_compiler.jar' '--patch-module=jdk.compiler=external/bazel_tools/third_party/java/jdk/langtools/jdk_compiler.jar' '--add-opens=java.base/java.nio=ALL-UNNAMED' -jar external/bazel_tools/tools/jdk/JavaBuilder_deploy.jar @bazel-out/host/bin/src/main/java/com/google/devtools/build/lib/libstring_util.jar-2.params) After: SUBCOMMAND: # //src/main/java/com/google/devtools/build/lib:string_util [action 'Building src/main/java/com/google/devtools/build/lib/libstring_util.jar (2 source files) [for host]'] (cd /tmp/devbazel_output_base/execroot/io_bazel && \ exec env - \ LC_CTYPE=en_US.UTF-8 \ external/embedded_jdk/bin/java \ -XX:+UseParallelOldGC \ -XX:-CompactStrings \ '--add-exports=jdk.compiler/com.sun.tools.javac.api=ALL-UNNAMED' \ '--add-exports=jdk.compiler/com.sun.tools.javac.code=ALL-UNNAMED' \ '--add-exports=jdk.compiler/com.sun.tools.javac.comp=ALL-UNNAMED' \ '--add-exports=jdk.compiler/com.sun.tools.javac.file=ALL-UNNAMED' \ '--add-exports=jdk.compiler/com.sun.tools.javac.main=ALL-UNNAMED' \ '--add-exports=jdk.compiler/com.sun.tools.javac.tree=ALL-UNNAMED' \ '--add-exports=jdk.compiler/com.sun.tools.javac.util=ALL-UNNAMED' \ '--add-opens=jdk.compiler/com.sun.tools.javac.file=ALL-UNNAMED' \ '--patch-module=java.compiler=external/bazel_tools/third_party/java/jdk/langtools/java_compiler.jar' \ '--patch-module=jdk.compiler=external/bazel_tools/third_party/java/jdk/langtools/jdk_compiler.jar' \ '--add-opens=java.base/java.nio=ALL-UNNAMED' \ -jar \ external/bazel_tools/tools/jdk/JavaBuilder_deploy.jar \ @bazel-out/host/bin/src/main/java/com/google/devtools/build/lib/libstring_util.jar-2.params) RELNOTES: --subcommands can now take a "pretty_print" value ("--subcommands=pretty_print") to print the arguments of subcommands as a list for easier reading. PiperOrigin-RevId: 206213009
* Use the path resolver when dealing with tree artifacts.Gravatar felly2018-07-26
| | | | | RELNOTES: None PiperOrigin-RevId: 206157591
* Remove unused method "hasActionFilesystem()".Gravatar felly2018-07-25
| | | | | | RELNOTES: None PiperOrigin-RevId: 206021118
* Use the single-string arg formatter for param file format.Gravatar tomlu2018-07-25
| | | | | | | This avoids bazel crashes for illegally formatted strings. Previously the code would assume that a correct string was passed with only minimal validation. RELNOTES: None PiperOrigin-RevId: 206012819
* Automated rollback of commit f309ad3be36363070e87eef0ee04b12f4956d601.Gravatar janakr2018-07-24
| | | | | | | | | | *** Reason for rollback *** Fixed duplicate derived inputs bug. Test is in diffbase. RELNOTES[INC]: If the same artifact is generated by two distinct but identical actions, and a downstream action has both those actions' outputs in its inputs, the artifact will now appear twice in the downstream action's inputs. If this causes problems in Skylark actions, you can use the uniquify=True argument in Args.add_args. PiperOrigin-RevId: 205863806
* Open source SpawnExecutedEventGravatar ulfjack2018-07-24
| | | | | | | This will be used to compute the critical path using Spawns instead of Actions, which should be more accurate. PiperOrigin-RevId: 205817400
* ArtifactFactory.getSourceArtifact returns SourceArtifact.Gravatar shahan2018-07-23
| | | | PiperOrigin-RevId: 205751282
* Remove PerActionFileCacheGravatar ulfjack2018-07-23
| | | | | | | | | | | | | Instead, make ActionMetadataHandler implement the MetadataProvider interface. This fixes an issue where an action that runs two spawns where one depends on an output of the other was unable to get the metadata for the intermediate output. We don't currently have actions that do this, but we will have in a future change (which will also implicitly act as a regression test). PiperOrigin-RevId: 205629237
* Add an execution requirement key to trigger prefetching of remote outputsGravatar ulfjack2018-07-20
| | | | PiperOrigin-RevId: 205361498
* Require the ActionExecutionMetadata to be passed to IncludeScannerGravatar ulfjack2018-07-19
| | | | | | | Also add an execution requirement that allows disabling reporting to the CLI; this will be used in a future change. PiperOrigin-RevId: 205216096
* Introduce --strict_fileset_output which treats all output Artifacts ↵Gravatar felly2018-07-18
| | | | | | encountered in a Fileset as a regular file. PiperOrigin-RevId: 205152271
* Automated rollback of commit d491bf10f42e213292382c98a1dc439537f00f43.Gravatar janakr2018-07-17
| | | | | | | | | | | | | | | | | | *** Reason for rollback *** Still bugs lurking. See linked bug. *** Original change description *** Automated rollback of commit eb587075b0d6ffab1cf9e69ede1b7e547905e547. *** Reason for rollback *** Depot has been fixed. RELNOTES[INC]: If the same artifact is generated by two distinct but identical actions, and a downstream action has both those actions' outputs in its inputs, the artifact will now appear twice in the downstream action's inputs. If this causes problems in Skylark actions, you can use the uniquify=True argument in Args.add_args. PiperOrigin-RevId: 204997569
* Automated rollback of commit eb587075b0d6ffab1cf9e69ede1b7e547905e547.Gravatar janakr2018-07-16
| | | | | | | | | | *** Reason for rollback *** Depot has been fixed. RELNOTES[INC]: If the same artifact is generated by two distinct but identical actions, and a downstream action has both those actions' outputs in its inputs, the artifact will now appear twice in the downstream action's inputs. If this causes problems in Skylark actions, you can use the uniquify=True argument in Args.add_args. PiperOrigin-RevId: 204827477
* Fix misspelling in ↵Gravatar kush2018-07-13
| | | | | | | https://github.com/bazelbuild/bazel/commit/b39c69394c5e7991ca8d04efac2142c22947a7c5 RELNOTES: None PiperOrigin-RevId: 204522278
* Stop requiring the creation and storage of FileStateValues within ↵Gravatar kush2018-07-12
| | | | | | | | | ResolvedFile objects. While this does not eliminate the need for stat operation yet, it gets rid of one usage of the stat result (and is also a self contained change) RELNOTES: None PiperOrigin-RevId: 204417559
* Store the hashCode as an Int instead of a string in FilesetOutputSymlink, andGravatar kush2018-07-11
| | | | | | | | delay the string conversion till we actually write the manifest file. This might get some memory savings after adding some in unknown commit. RELNOTES: None PiperOrigin-RevId: 204216582
* Set the version of a computed node to the max of its child versions rather ↵Gravatar janakr2018-07-11
| | | | | | | | | | | | | | | | | than the graph version when that is feasible. * It's not feasible when the computation accesses outside state, i.e. is non-hermetic, so see below. * It's also more complicated (and not worth the trouble) when the computation is taking place just for the error status. Have SkyFunctionName declare whether the function it corresponds to is hermetic or non-hermetic. Only non-hermetically-generated SkyValues can be directly marked changed, and non-hermetic SkyFunctions have their values saved at the graph version, not the max of the child versions. All SkyFunctions are hermetic except for the ones that can be explicitly dirtied. A marked-hermetic SkyFunction that has a transient error due to filesystem access can be re-evaluated and get the correct version: if it throws an IOException at version 1 and then, when re-evaluated at version 2 with unchanged dependencies, has a value, the version will be version 1. All Skyframe unit tests that were doing non-hermetic things to nodes need to declare that those nodes are non-hermetic. I tried to make the minimal set of changes there, so that we had good incidental coverage of hermetic+non-hermetic nodes. Also did some drive-by clean-ups around that code. Artifacts are a weird case, since they're doing untracked filesystem access (for source directories). Using max(child versions) for them gives rise to the following correctness bug: 1. do a build at v1 that creates a FileStateValue for dir/ at v1. Then at v2, add a file to dir/ and do a build that consumes dir/ as a source artifact. Now the artifact for dir/ will (incorrectly) have v1. Then at v1, do that build again. We'll consume the "artifact from the future". However, this can only have an effect when using the local action cache, since the incorrect value of the artifact (the mtime) is only consumed by the action cache. Bazel is already broken in this way (incremental builds don't invalidate directories), so this change doesn't make things worse. PiperOrigin-RevId: 204210719
* Make the FileArtifactvalue of generated targets available via the ↵Gravatar kush2018-07-11
| | | | | | | | | | | | | | FilesetOutputSymlink whenever available. In this change I'm simply plumbing the FileArtifactValue we requested within RecursiveFilesystemTraversalFunction to the FilesetOutputSymlink. This does not work when the targets are output directories (or symlink to output dirs). The main scenarios this happens is when: 1. Fileset depends on the output dir created by a genrule. 2. Fileset depends on a GoAppengineBinary which creates an output dir. 3. Fileset depends on another Fileset in the non-recommended way (Fileset.entry.files = [<another_fileset>]) instead of the recommended way (FilesetEntry.srcdir = <another_fileset>). RELNOTES: None PiperOrigin-RevId: 204209612
* Omit parts of BEP events (like stdout, stderr) when the associated artifact ↵Gravatar tomlu2018-07-11
| | | | | | | uploader cannot upload a particular file. RELNOTES: None PiperOrigin-RevId: 204167372
* Remove the "slow read" warning when using multi-threaded digesting.Gravatar philwo2018-07-11
| | | | | | | | | | Calculating the throughput of a digest operation and using it to assess whether I/O is slow or not only makes sense when we're doing sequential I/O. With parallel hashing, it's expected that individual operations are slow due to how the scheduler works, but it doesn't mean that the overall progress is slow. RELNOTES: None. PiperOrigin-RevId: 204115311
* Fix typo in javadoc for FileValue#realRootedPath.Gravatar shreyax2018-07-09
| | | | PiperOrigin-RevId: 203813658
* PiperOrigin-RevId: 203687257Gravatar Googler2018-07-08
|
* Replace instances of Blaze with Bazel.Gravatar Dan Fabulich2018-07-05
| | | | | | | PiperOrigin-RevId: 203300374 Change-Id: Iaa47f870ab2e0cad40a202aad2c7f9430f73c856 PiperOrigin-RevId: 203407392
* Include more information about build events' referenced local files.Gravatar tomlu2018-07-04
| | | | | | | Instead of just a path, events now include information about the type of file (output, source file, stdout/stderr, test logs, etc.). This information can be used by the uploaders to determine a) whether to upload, b) what kind of lease to give the files. RELNOTES: None PiperOrigin-RevId: 203285549
* Relax the threshold for calling getDigestInExclusiveMode().Gravatar Googler2018-07-02
| | | | | | | | | If the policy goal is to minimize disk seeks, this should be equally good. Second attempt, with test update. RELNOTES: None. PiperOrigin-RevId: 202907857
* Eliminates unnecessary comparisons in ActionInputMap.resize.Gravatar shahan2018-06-29
| | | | PiperOrigin-RevId: 202644128
* Add an interner for non source artifacts that are deserialized.Gravatar mjhalupka2018-06-27
| | | | PiperOrigin-RevId: 202311773
* Instead of depending on mutable inputs of upstream CppCompileAction instances,Gravatar shahan2018-06-26
| | | | | | | | | CppCompileAction.discoverInputsStage2 retrieves values of discovered modules from ActionExecutionValue. This addresses a possible a correctness issue. PiperOrigin-RevId: 202162180
* Add some debugging info to ArtifactFactory, and allow tests to access the ↵Gravatar janakr2018-06-22
| | | | | | ArtifactResolverSupplier in SkyframeExecutor. PiperOrigin-RevId: 201705857
* Add a method in FileArtifactValue to create a new instance given an existing ↵Gravatar felly2018-06-22
| | | | | | FileStatus. PiperOrigin-RevId: 201683773
* Automated rollback of commit 45b308a62f42c2c0bcfe79dcd4046c4025a31059.Gravatar janakr2018-06-20
| | | | | | | | | | | | | | | | | | | | | | | | *** Reason for rollback *** Breaks Javascript compilation. There's currently no way to iterate over a depset of Artifacts and deduplicate by identical paths in Skylark. This means that actions that want to do something once per Artifact in a depset (add a flag to the command line with the path of the Artifact for instance) will have duplicate entries if there are multiple Artifacts with the same path. This is not a true automated rollback, since I tried to make this as minimal as possible, to avoid merge conflicts and keep some of the benefits of the rolled back CL. In particular, the tests that were changed in the original CL to give artifacts their correct owners did not need to be changed back, since the owners are still correct. Moreover, this effectively contains rollbacks of unknown commit and https://github.com/bazelbuild/bazel/commit/39d6f89644107a8f7c080c4f4aec8527c1a73954, but keeps test coverage from those CLs as well. Comments added to CL thread where not a clean rollback (there are plenty of files not changed at all: ActionArtifactCycleReporter is the main wart, since it can still handle SkyKeys with Artifact as the argument, but Artifact is no longer the argument to any SkyKey). RELNOTES: None *** Original change description *** Make Artifact#equals take the owner into account for derived artifacts. Derived artifacts' owners are important because they are used to determine the artifact's generating action. Source artifacts' owners are not used in this way, so I left them alone. This allows us to get rid of most uses of ArtifactSkyKey. We may be able to delete it entirely in a follow-up. PiperOrigin-RevId: 201464780
* Stop holding on to the Action and ActionKeyContext in ↵Gravatar janakr2018-06-19
| | | | | | | | ActionConflictException. We can construct the exception message eagerly, since it is going to be thrown with very high likelihood in any case. The complex objects inside it made it hard to serialize. PiperOrigin-RevId: 201130522
* Remove support for --discard_actions_after_execution.Gravatar tomlu2018-06-18
| | | | | | | The memory savings from this flag are not worth the complexity, and it interferes with action restarting. RELNOTES: Remove support for --discard_actions_after_execution. PiperOrigin-RevId: 201077905
* Refactoring: uses OutputService for ActionFileSystem injectionGravatar shahan2018-06-18
| | | | | | Moves more Action-oriented from skyframe package, which has become very unwieldy, to action package. This is needed to avoid circular dependencies caused when build-base is needed for skyframe. PiperOrigin-RevId: 200996982
* Add a mechanism for build event protocol events to upload filesGravatar ulfjack2018-06-15
| | | | | | | | This should be a no-op, mostly replacing PathConverter with BuildEventArtifactUploader, since none of the implementations perform any upload yet. PiperOrigin-RevId: 200685325
generated by cgit v1.2.3 (git 2.39.1) at 2024-06-12 19:43:54 +0000