aboutsummaryrefslogtreecommitdiffhomepage
path: root/src/main/java/com/google/devtools/build/lib/skyframe
Commit message (Collapse)AuthorAge
* Use ConcurrentHashMap instead of Cache where we aren't using any of theGravatar Googler2018-08-15
| | | | | | | | LoadingCache's capabilities. According to profiles, using ConcurrentHashMap is about 3x more efficient. RELNOTES: None PiperOrigin-RevId: 208874176
* PiperOrigin-RevId: 208861891Gravatar shahan2018-08-15
|
* Support tree artifacts in target complete.Gravatar tomlu2018-08-15
| | | | | RELNOTES: None PiperOrigin-RevId: 208861642
* Filter out events from analysis when constructing execution-phase values in ↵Gravatar janakr2018-08-14
| | | | | | | | | | Skyframe. This avoids some unnecessary iteration over already-emitted events that can show up in profiles, and allows us to store execution-phase values a bit more compactly, since we don't need to carry around wrapper objects and nested sets everywhere. This crucially depends on the fact that we can't build a target in the execution phase without first having analyzed it in a separate Skyframe call. Skyframe normally propagates all events/posts up the graph because it must be able to emit them if a user requests a node that only transitively depends on the node that emitted an event. However, because we do analysis in a separate Skyframe call, any warnings/posts associated with the analysis nodes will be emitted then, and we don't need to propagate them into execution. PiperOrigin-RevId: 208767078
* Automated rollback of commit 37bd5f665aa614c6dc640c9d19852dd8d5efb0d8.Gravatar felly2018-08-14
| | | | | | | | | | | | | | | | | | | | | *** Reason for rollback *** Rollforward with fix: Don't use Immutable map builder to agggregate Filesets. We may reach the same one via direct and indirect runfiles, so we need to allow for duplicate entries. *** Original change description *** Automated rollback of commit 3bace1b937934fb2cea6260067ecc1cdbe526847. *** Reason for rollback *** b/112583337 RELNOTES: None *** Original change description *** Track Fileset in artifact expansion. PiperOrigin-RevId: 208748163
* Automated rollback of commit 3bace1b937934fb2cea6260067ecc1cdbe526847.Gravatar felly2018-08-14
| | | | | | | | | | | | | *** Reason for rollback *** b/112583337 RELNOTES: None *** Original change description *** Track Fileset in artifact expansion. PiperOrigin-RevId: 208658921
* Blacklists some classes for serialization.Gravatar shahan2018-08-13
| | | | PiperOrigin-RevId: 208554171
* Track Fileset in artifact expansion.Gravatar felly2018-08-13
| | | | PiperOrigin-RevId: 208549418
* Add easy access to MessageDigests in DigestHashFunctionGravatar ccalvarin2018-08-13
| | | | | | | Move the message-digest cloning to DigestHashFunction and out of Fingerprint, to make it possible to configure Fingerprint to use different hash functions. We keep the default MD5 for now, we'd like it to use the global default but want to isolate the configuration change from any change adding potential contention. RELNOTES: None. PiperOrigin-RevId: 208502993
* ActionAnalysisMetadata exposes whether loose headers are present. ↵Gravatar shahan2018-08-12
| | | | | | TransitiveVersionTable uses futures. PiperOrigin-RevId: 208402465
* Use NestedSets to store topLevelModules and discoveredModules. Also improveGravatar Googler2018-08-11
| | | | | | | | computeTransitivelyUsedModules to return early on missing values instead of starting to create datastructures. RELNOTES: None. PiperOrigin-RevId: 208351193
* PiperOrigin-RevId: 208245798Gravatar mjhalupka2018-08-10
|
* Automated rollback of commit 9374ecf94ce44e8bc56e68678cb512abf4cf9ce2.Gravatar felly2018-08-10
| | | | | | | | | | | | | | | | | | | | | | *** Reason for rollback *** Rollforward with deduplication of traversals and regression test. RELNOTES: None *** Original change description *** Automated rollback of commit 39974a43abdd32e3a1acbc7da945b08da9983e4e. *** Reason for rollback *** b/112458627 *** Original change description *** Allow skyframe-aware actions to pass partial results through ActionExecutionContext. Remove FilesetProvider. PiperOrigin-RevId: 208231719
* Combine prefix event and payload stdout/stderrGravatar Klaus Aehlig2018-08-10
| | | | | | | | | ...in a single event instead of relying on the synchronisation on the reporter. In this way, it will be possible to focus on output of errors, and also to deduplicate warnings. Change-Id: I9669e8497ca6e22a01f3e285ec9ee42c1ecf3fb5 PiperOrigin-RevId: 208223882
* Persist mappings from original command line patterns to expanded labels in ↵Gravatar Googler2018-08-10
| | | | | | TargetParsingCompleteEvent. PiperOrigin-RevId: 208217102
* Automated rollback of commit 39974a43abdd32e3a1acbc7da945b08da9983e4e.Gravatar felly2018-08-10
| | | | | | | | | | | | | *** Reason for rollback *** b/112458627 *** Original change description *** Allow skyframe-aware actions to pass partial results through ActionExecutionContext. Remove FilesetProvider. PiperOrigin-RevId: 208213955
* Use getDoneValues instead of getGraphMap for dump/stat functions.Gravatar twerth2018-08-09
| | | | | RELNOTES: None PiperOrigin-RevId: 208166562
* Batch all DependencyResolver#getTarget calls. This leads to some duplicate ↵Gravatar janakr2018-08-09
| | | | | | iteration, but that should be cheap, while requesting packages sequentially can hurt... PiperOrigin-RevId: 208126130
* Allow skyframe-aware actions to pass partial results through ↵Gravatar felly2018-08-09
| | | | | | | | ActionExecutionContext. Remove FilesetProvider. PiperOrigin-RevId: 208111251
* Pass Fileset information to the Action filesystem.Gravatar felly2018-08-09
| | | | PiperOrigin-RevId: 208096548
* Support a .bazelignore fileGravatar Klaus Aehlig2018-08-09
| | | | | | | | | | | | | | | | | | | Add suport for a file .bazelignore, allowing to specify directories that should be ignored. So far, even for the "ignored" directories, the directory structure, including following symlinks, still has to be well founded. We plan to lift that restriction in the future, but already with the restriction, it is an improvement over the status quo. RELNOTES: bazel now supports a .bazelignore file specifying directories to be ignored; however, these directories still have to be well founded and, in particular, may not contain symlink cycles. Improves on #4888. Change-Id: Icfe0b7f5c35eceea16e28944ca7711431b71271a PiperOrigin-RevId: 208050372
* Split up getTargetsToBuild() into multiple methods so compute() can ↵Gravatar Googler2018-08-09
| | | | | | | | determine which pattern expanded to which targets. Plan is to modify either TargetParsingCompleteEvent or LoadingPhaseCompleteEvent (which are both posted to the event handler) so that targets can be traced back to original command line patterns. PiperOrigin-RevId: 208031588
* Order tree artifacts.Gravatar felly2018-08-06
| | | | | | | Fixes #5686. RELNOTES: None PiperOrigin-RevId: 207559658
* Use fact that the subgraph of ActionLookupValues has the same transitive ↵Gravatar janakr2018-08-05
| | | | | | dependency relations as the full graph to avoid descending into non-ActionLookupValue parts of the graph when collecting ActionLookupValues. PiperOrigin-RevId: 207472612
* When no new configured targets have been analyzed, only check for artifact ↵Gravatar janakr2018-08-03
| | | | | | | | | | conflicts if the current set of configured targets is not a subset of the largest set of configured targets that have been checked for conflicts. Also rework the flow between SkyframeBuildView and SkyframeActionExecutor to remove the SkyframeExecutor middleman. Also reword the error message in case of an ArtifactPrefixConflictException, since a clean should no longer be necessary. PiperOrigin-RevId: 207322139
* Add support to completion function to create a path resolver from its ↵Gravatar tomlu2018-08-03
| | | | | | | looked-up artifact values. RELNOTES: None PiperOrigin-RevId: 207295716
* Set target/aspect completion functions as unsharable.Gravatar tomlu2018-08-02
| | | | | RELNOTES: None PiperOrigin-RevId: 207176404
* Disables ActionFS sandboxing of discovered derived inputs forGravatar shahan2018-08-02
| | | | | | performance reasons. PiperOrigin-RevId: 207171305
* Thread a path resolver through target/aspect complete event.Gravatar tomlu2018-08-02
| | | | | | | At the moment, an identity path resolver is passed. This will later be replaced by a contextual path resolver. RELNOTES: None PiperOrigin-RevId: 207138772
* remote: add support for directory inputs in runfilesGravatar buchgr2018-08-02
| | | | | | | | | | | | | | | | Add support for tree artifacts (ctx.action.declare_directory(...)) in runfiles. Before this change we would throw away the information about the files inside a tree artifact before executing an action. That's fine for local execution where the sandbox just copies/symlinks a directory and doesn't care much what's inside. However, in remote execution we actually need to upload each individual file and so we need to be aware of all individual files not just directories. This change makes it so that this information is made available to a SpawnRunner via the SpawnInputExpander. RELNOTES: None PiperOrigin-RevId: 207091668
* Automated rollback of commit a6255612e4892729d3758775c76085b26b9bc584.Gravatar twerth2018-08-02
| | | | | | | | | | | | | | | | | | | | | *** Reason for rollback *** Breaks FDO builds, see http://b/112112125 *** Original change description *** Remove PerActionFileCache Instead, make ActionMetadataHandler implement the MetadataProvider interface. This fixes an issue where an action that runs two spawns where one depends on an output of the other was unable to get the metadata for the intermediate output. We don't currently have actions that do this, but we will have in a future change (which will also implicitly act as a regression test). PiperOrigin-RevId: 207084179
* Add a flag to evaluate the top level transitions in SkyframeGravatar ulfjack2018-08-02
| | | | | | | | | | | | | | | | | | | | | | | | This adds a new PrepareAnalysisPhaseFunction, which started out as a copy of some existing code from SkyframeExecutor, BuildView, AnalysisPhaseRunner, AnalysisUtils, and ConfigurationResolver, which was then modified to work inside Skyframe. Most of our tests already work with the new code, except for some of the tests related to configuration trimming in combination with dependency cycles. The reason for this is that we can only recover from dependency cycles at the end of a Skyframe invocation, but never inside a Skyframe invocation. The new code therefore cannot return partial results like the old code. This seems to make null builds a bit faster. In my testing, I saw null build times for a single test target go from ~50ms to ~40ms. This is probably due to slightly better caching - it seems that computing the configuration transitions and top-level targets is non-negligible, even if there's only a single top-level configuration for a single top-level target. This might be an even bigger win if there are a lot of top-level targets and configurations. PiperOrigin-RevId: 207083192
* Pass a path to ActionExecutedEvent that is resolved via ActionExecutionContext.Gravatar tomlu2018-08-01
| | | | | | | If the output artifact is backed by a different file system then we need to use that file system later. RELNOTES: None PiperOrigin-RevId: 206981369
* Only detect artifact conflicts between targets in current build.Gravatar tomlu2018-08-01
| | | | | | | Previously, two successive builds of targets A and B would fail if the two sets had conflicting actions. Now, only fail such builds if the targets from the current build has action conflicts. RELNOTES: None PiperOrigin-RevId: 206974011
* Refactor setupPackageCache to get all BuildConfiguration.Options references ↵Gravatar juliexxia2018-08-01
| | | | | | | | out of it. Setting up the package cache is necessary for proper loading. In the future, BuildConfiguration.Option options will be targets that need to be loaded in order to be parsed. Thus, we will need to be able to do loading before BuildConfiguration.Options are parsed. PiperOrigin-RevId: 206964473
* Remove the need for discoverInputsStage2(). Much like every other SkyframeGravatar Googler2018-08-01
| | | | | | | | function, discoverInputs() can be implemented to return null upon encountering missing ActionExecutionValues. RELNOTES: None. PiperOrigin-RevId: 206913969
* Allows subclasses of SkyframeExecutor to enable analysis during execution.Gravatar shahan2018-07-30
| | | | PiperOrigin-RevId: 206568465
* Blacklist PackageFactory$BuiltInRuleFunction from serialization.Gravatar janakr2018-07-27
| | | | PiperOrigin-RevId: 206389059
* Automated rollback of commit 7e87730de985b7099b9b683571d58efdaab70890.Gravatar ccalvarin2018-07-27
| | | | | | | | | | | | | | | | | *** Reason for rollback *** Go back to the default constructor - instead of requiring everywhere to know the correct hash function, we'll have the default rely on global state. It will make transition easier, even if it makes the origin of the hash less obvious. *** Original change description *** Remove default MD5 in most of Bazel's virtual filesystems. This forces the ex-default to be explicit in a lot of tests, but I'd rather that than have the risk of implicit md5-use in production code. To keep this CL smaller, do not remove the default from UnixFS quite yet. RELNOTES: None. PiperOrigin-RevId: 206358838
* Automated rollback of commit db01c6f926bcb4774d901797c59f51dd54c05624.Gravatar janakr2018-07-27
| | | | | | | | | | *** Reason for rollback *** Rolling forward with fixes. *** Original change description *** PiperOrigin-RevId: 206339696
* Simplify tree artifact directory check, and avoid an extra system call.Gravatar felly2018-07-27
| | | | | | RELNOTES: None PiperOrigin-RevId: 206328416
* Remove default MD5 in most of Bazel's virtual filesystems.Gravatar ccalvarin2018-07-26
| | | | | | | | | This forces the ex-default to be explicit in a lot of tests, but I'd rather that than have the risk of implicit md5-use in production code. To keep this CL smaller, do not remove the default from UnixFS quite yet. RELNOTES: None. PiperOrigin-RevId: 206223521
* PiperOrigin-RevId: 206181674Gravatar shahan2018-07-26
|
* Target pattern eval: use the same number of threads as loadingGravatar ulfjack2018-07-26
| | | | | | | | Instead of using the default thread pool size of 200, use the number set for the loading phase. This is in preparation for interleaving the loading and target pattern eval phases. PiperOrigin-RevId: 206172915
* Use the path resolver when dealing with tree artifacts.Gravatar felly2018-07-26
| | | | | RELNOTES: None PiperOrigin-RevId: 206157591
* Support optional repository verificationGravatar Klaus Aehlig2018-07-25
| | | | | | | | | | | | | | | | Add an option to provide a file with a resolved value, that will be used to verify that the repositories mentioned in this file produce a correct directory tree. RELNOTES: newly added options --experimental_repository_hash_file and --experimental_verify_repository_rules allow to verify for repositories the directory generated against pre-recorded hashes. See documentation for those options. Work towards #5660. Change-Id: I2d8becb188d0fa51e890fb8f6139f321cca14b7b PiperOrigin-RevId: 206016792
* Remove LoadingResultGravatar ulfjack2018-07-25
| | | | | | | | | | | Instead, refactor the code to use TargetPatternPhaseValue exclusively. This removes the need to convert from TargetPatternPhaseValue to LoadingResult, and prepares for interleaving. It also reduces the number of Skyframe calls which may speed up null builds a bit, as a followup for https://github.com/bazelbuild/bazel/commit/1067310e18cb9ac203110726de0be53bdc403cea. PiperOrigin-RevId: 205989338
* Configured targets register created source artifacts with Skyframe.Gravatar shahan2018-07-24
| | | | PiperOrigin-RevId: 205876673
* Automated rollback of commit f309ad3be36363070e87eef0ee04b12f4956d601.Gravatar janakr2018-07-24
| | | | | | | | | | *** Reason for rollback *** Fixed duplicate derived inputs bug. Test is in diffbase. RELNOTES[INC]: If the same artifact is generated by two distinct but identical actions, and a downstream action has both those actions' outputs in its inputs, the artifact will now appear twice in the downstream action's inputs. If this causes problems in Skylark actions, you can use the uniquify=True argument in Args.add_args. PiperOrigin-RevId: 205863806
* Move LoadingPhaseCompleteEvent posting to TargetPatternPhaseFunctionGravatar ulfjack2018-07-24
| | | | | | | | | | | Also simplify LoadingPhaseCompleteEvent, and SkyframeExecutor, and remove LoadingCallback, which is unnecessary now that we only have a single implementation (previously LoadingPhaseRunner). This also removes some of the excessive Skyframe calls introduced by https://github.com/bazelbuild/bazel/commit/1067310e18cb9ac203110726de0be53bdc403cea, and prepares for interleaving target pattern eval and loading. PiperOrigin-RevId: 205813197
generated by cgit v1.2.3 (git 2.39.1) at 2024-06-19 04:44:14 +0000