aboutsummaryrefslogtreecommitdiffhomepage
path: root/src/main/java/com/google/devtools/build/lib/skyframe/SkyframeExecutor.java
Commit message (Collapse)AuthorAge
* Automated rollback of commit 9321316b34767b06c3071b2cf2a4de189874fcce.Gravatar ccalvarin2017-12-11
| | | | | | | | | | | | | | | | | | | | *** Reason for rollback *** Design change, 2 boolean flags instead of 1 enum flag *** Original change description *** Add --incremental_state_retention_strategy This option is intended to replace some of the uses of --batch. It lets users specify that builds should not be incremental, and how eagerly to discard the state that is kept around for incrementality. Note that for both values discard_eargerly and keep_for_life_of_build, the build graph is kept around until the next build. This may change. Will add tests for keep_for_life_of_build in a later change, for now it will warn that that feature is experimen... *** ROLLBACK_OF=178661777 RELNOTES: None. PiperOrigin-RevId: 178681472
* Add --incremental_state_retention_strategyGravatar ccalvarin2017-12-11
| | | | | | | | | This option is intended to replace some of the uses of --batch. It lets users specify that builds should not be incremental, and how eagerly to discard the state that is kept around for incrementality. Note that for both values discard_eargerly and keep_for_life_of_build, the build graph is kept around until the next build. This may change. Will add tests for keep_for_life_of_build in a later change, for now it will warn that that feature is experimental. RELNOTES: --[no]keep_incrementality_data is gone, replaced by the enum-valued --incremental_state_retention_strategy PiperOrigin-RevId: 178661777
* Automated rollback of commit d0f06a6e4b61adc39bf5e1cfbae39501a89dc8e3.Gravatar nharmata2017-12-05
| | | | | RELNOTES: None PiperOrigin-RevId: 177965330
* Fix getBuildFiles to not assume BUILD is the name of the build file.Gravatar John Cater2017-12-04
| | | | | | | Fixes #4056. Change-Id: Ia7425c2146f15e9293605ee3da53007805e82275 PiperOrigin-RevId: 177813070
* Add ActionKeyContext to Action#getKey.Gravatar tomlu2017-11-29
| | | | | | | This key context can be used by actions to share partial key computations, for instance when computing MD5s for nested sets. RELNOTES: None PiperOrigin-RevId: 177359607
* Remove the concept of a mandatory input which is allowed to be missing.Gravatar Googler2017-11-29
| | | | | RELNOTES: None. PiperOrigin-RevId: 177332323
* Update PathPackageLocator to take a list of potential build file names,Gravatar John Cater2017-11-28
| | | | | | | | | | | | | | instead of assuming BUILD. - Default the list to the same value as PackageLookupFunction: BUILD.bazel, BUILD. - Move BuildFileNames to the packages package, so it is more generally available. Part of #4056. Change-Id: Ie12512b492cd7d47a9e56ec3bc209f829feaf4b5 PiperOrigin-RevId: 177261295
* Automated rollback of commit b64119807b014d9f3b99fb8a02e22daf1a8299b6.Gravatar nharmata2017-11-28
| | | | | | | | | | | | | *** Reason for rollback *** Tickles the shell integration test version of b/35042288 *** Original change description *** Change BlacklistedPackagesPrefixesFunction to take a pair of a hardcoded set of directories and a file path containing more directories to blacklist. The current usage of PrecomputedValue#BLACKLISTED_PACKAGE_PREFIXES_FILE is overly general and is only meaningfully used by unit tests; in practice, the blacklist file path can never change over the lifetime of the Bazel server. Perform a minor simplifying refactor as a result of this. RELNOTES: None. PiperOrigin-RevId: 177176068
* Change BlacklistedPackagesPrefixesFunction to take a pair of a hardcoded set ↵Gravatar nharmata2017-11-28
| | | | | | | of directories and a file path containing more directories to blacklist. The current usage of PrecomputedValue#BLACKLISTED_PACKAGE_PREFIXES_FILE is overly general and is only meaningfully used by unit tests; in practice, the blacklist file path can never change over the lifetime of the Bazel server. Perform a minor simplifying refactor as a result of this. RELNOTES: None. PiperOrigin-RevId: 177164057
* RELNOTES: --keep_incrementality_data flag allows Bazel servers to be run in ↵Gravatar janakr2017-11-12
| | | | | | | | | | | | | | memory-saving non-incremental mode independent of --batch and --discard_analysis_cache. A command run with --nokeep_incrementality_data will discard data that would be needed for incremental builds. Subsequent commands can be sent to the same server, but they will not get the benefit of incrementality from this command. However, if --keep_incrementality_data is specified on a subsequent command, the commands after that will get the benefits of incrementality. There are two benefits to not being dependent on --batch. First, this allows Bazel servers to be run in extreme memory-saving mode without the startup penalties (JVM startup, JITting) that --batch execution imposes. Second, this allows Bazel developers to inspect the state of a Bazel server after an extreme memory-saving build. In order to avoid discarding data unnecessarily (for instance, on a "bazel info used-heap-size-after-gc" or "bazel dump --skyframe=summary") the actual resetting of the graph is done lazily, right before its use in SequencedSkyframeExecutor#sync. This is morally a partial rollback of https://github.com/bazelbuild/bazel/commit/98cd82cbdcac7c48164a611c5a9aa8fc2f1720ef. For now, our tests specify all of the flags. After this change sticks, I plan to get rid of the --batch flag from these tests, which should allow for some clean-ups. Eventually --batch and --discard_analysis_cache may not imply that we don't keep incremental state: we can require that it be specified explicitly. PiperOrigin-RevId: 175335075
* Replace all usages of Blaze's Preconditions class with guava.Gravatar tomlu2017-11-09
| | | | | | | | Blaze had its own class to avoid GC from varargs array creation for the precondition happy path. Guava now (mostly) implements these, making it unnecessary to maintain our own. This change was almost entirely automated by search-and-replace. A few BUILD files needed fixing up since I removed an export of preconditions from lib:util, which was all done by add_deps. There was one incorrect usage of Preconditions that was caught by error prone (which checks Guava's version of Preconditions) that I had to change manually. PiperOrigin-RevId: 175033526
* Move TransitiveTargetKey to a dedicated top-level classGravatar ulfjack2017-10-30
| | | | PiperOrigin-RevId: 173873310
* Thread FileSystem through to a single Path#delete call site.Gravatar tomlu2017-10-30
| | | | | | This requires a fairly large amount of changes to fundamental objects like BlazeRuntime, Executor, and so on, as well as changing a lot of test code to thread the file system through. I expect future CLs to be much smaller. PiperOrigin-RevId: 173678144
* Reset evaluator eagerly when incremental state changes. And add a regression ↵Gravatar janakr2017-10-24
| | | | | | test to make sure we are using the expected type of node entries when discarding/keeping graph edges. PiperOrigin-RevId: 173131307
* Internal changeGravatar twerth2017-10-18
| | | | PiperOrigin-RevId: 172569624
* Add memory profiler.Gravatar tomlu2017-10-18
| | | | | | | | | | | This adds two dump command, bazel dump --rules and bazel dump --skylark_memory. dump --rules outputs a summary of the count, action count, and memory consumption of each rule and aspect class. dump --skylark_memory outputs a pprof-compatible file with all Skylark analysis allocations. Users can then use pprof as per normal to analyse their builds. RELNOTES: Add memory profiler. PiperOrigin-RevId: 172558600
* Expose ActionLogBufferPathGenerator in SkyframeExecutor.Gravatar janakr2017-10-12
| | | | PiperOrigin-RevId: 171906076
* Small refactoring of AnalysisTestCase to allow alternate SkyframeExecutor ↵Gravatar janakr2017-10-11
| | | | | | implementations. PiperOrigin-RevId: 171730718
* Pull clearing of legacy configured target data up to SkyframeExecutor and ↵Gravatar janakr2017-10-10
| | | | | | rename method to better reflect what it is primarily doing. PiperOrigin-RevId: 171621356
* Allow NodeEntry implementations to keep just deps, as opposed to all edges ↵Gravatar janakr2017-10-10
| | | | | | or no edges. Also add option to disable checks in MemoizingEvaluatorTest that don't make sense for implementations that don't keep track of dirty nodes. Also extract RecordingDifferencer to an interface. And add a test for the situation that a node changes during a build that it's not requested, and which fails, necessitating cleanup. PiperOrigin-RevId: 171616817
* Move SkylarkSemanticsOptions to the packages/ directory, alongside ↵Gravatar brandjon2017-10-07
| | | | | | | | | SkylarkSemanticsCodec Note that the syntax package and its test package still depend indirectly on the options parser via other Bazel-specific packages. RELNOTES: None PiperOrigin-RevId: 171342823
* Pass SkylarkSemantics through Skyframe instead of the options classGravatar brandjon2017-10-06
| | | | | | | Also remove the use of the @UsesOnlyCoreTypes annotation on SkylarkSemanticsOptions. It was only there to help mark that the options class was safe to put in Skyframe. RELNOTES: None PiperOrigin-RevId: 171248504
* Don't symlink into the execroot if possible in SymlinkAction: instead, ↵Gravatar janakr2017-10-02
| | | | | | symlink directly to the target artifact. Also offer the option to not provide the package roots to create the execroot: we would like to avoid the execroot if possible. PiperOrigin-RevId: 170515263
* More narrowly specify SkyframeExecutor#prepareExecution as ↵Gravatar janakr2017-09-28
| | | | | | #detectModifiedOutputFiles and move implementation to SequencedSkyframeExecutor. PiperOrigin-RevId: 170230031
* Stop injecting embedded artifacts. They're never used by anyone in Skyframe, ↵Gravatar janakr2017-09-26
| | | | | | and error-checking for their existence is already done by the client. PiperOrigin-RevId: 169966701
* Adjust visibility of SkyframeExecutor#findArtifactConflicts.Gravatar janakr2017-09-26
| | | | PiperOrigin-RevId: 169955807
* Stop injecting WorkspaceStatusAction into the Skyframe graph as a ↵Gravatar janakr2017-09-26
| | | | | | | | | | | | precomputed value. Instead, manually check if the value has changed, and if it has, invalidate its consuming WorkspaceStatusValue node, forcing its re-evaluation, where it will pick up the new value. This seems more awkward than the original code, but it is more correct in spirit: injecting a precomputed value which can change even while the source state does not is a smell. Long-term, the key for the WorkspaceStatusValue should incorporate a hash of the action, and that hash should be in the configuration, just as other configuration flags are. That isn't possible right now just because we don't have configuration trimming, and we drop all nodes on configuration changes, so putting workspace status options into the configuration would lose change pruning whenever we changed workspace status options. If/when those problems are fixed, we can extend this change to have WorkspaceStatusFunction continue to request the action out-of-band, but keyed by the hash. Then we can stop invalidating stale nodes. See also https://github.com/bazelbuild/bazel/issues/3785. PiperOrigin-RevId: 169947071
* Remove unused "extraPrecomputedValues" from WorkspaceBuilder, and therefore ↵Gravatar janakr2017-09-25
| | | | | | | | remove all "precomputed values for analysis" from SkyframeExecutor. We use SkyframeExecutor#injectExtraPrecomputedValues these days. This simplifies logic around when to inject precomputed values, and makes the graph more consistent. PiperOrigin-RevId: 169733304
* Remove BuildInfoFactory precomputed value.Gravatar janakr2017-09-25
| | | | PiperOrigin-RevId: 169728184
* Remove product name precomputed value.Gravatar janakr2017-09-25
| | | | PiperOrigin-RevId: 169723724
* Remove PrecomputedValue for BlazeDirectories. All consumers have the value ↵Gravatar janakr2017-09-25
| | | | | | | | injected directly, which makes sense, because it's immutable over the lifetime of the server. Step 3. PiperOrigin-RevId: 169717587
* Move ConfiguredTargetFunction dynamic config selection to its own class.Gravatar gregce2017-09-20
| | | | | | | | Also clarify the interfaces *TransitionResolver* - which determines what transition to apply to an input configuration and *ConfigurationResolver* - which determines the output configuration from that transition. PiperOrigin-RevId: 169311986
* Inform SkyframeExecutor when a command starts.Gravatar janakr2017-09-19
| | | | PiperOrigin-RevId: 169179218
* Project reorg: move *ConfiguredTarget to new configuredtargets/ pathGravatar gregce2017-09-18
| | | | | | | Exempt RuleConfiguredTarget in this change because that's liable to touch a billion files. PiperOrigin-RevId: 168929827
* Lots more cleanup of "dynamic configurations" comments and test code.Gravatar gregce2017-09-14
| | | | PiperOrigin-RevId: 168607439
* Cleanup todo from static config removalGravatar gregce2017-09-14
| | | | PiperOrigin-RevId: 168583913
* Static config removal TODO cleanup.Gravatar gregce2017-09-14
| | | | PiperOrigin-RevId: 168583577
* Remove outdated references to static vs. dynamic configurations.Gravatar gregce2017-09-13
| | | | PiperOrigin-RevId: 168452997
* Remove unused class after removal of BazelConfigurationCollection.Gravatar gregce2017-09-13
| | | | | | Also pipe keepGoing back into initial configuration creation. PiperOrigin-RevId: 168412512
* Add a new toolchain type for c++. In order to do this, ↵Gravatar cpeyser2017-09-08
| | | | | | | | PlatformConfiguration is made a legal configuration fragment for every rule class. Add a default "dummy" c++ toolchain to prevent resolution errors when legacy toolchain selection logic is used. Add toolchain mocks to java and shell tests. PiperOrigin-RevId: 167901210
* Rule classes cannot request duplicate toolchains.Gravatar cpeyser2017-09-08
| | | | PiperOrigin-RevId: 167861778
* RuleContext instances obtained through BuildViewTestCase#getRuleContext have aGravatar cpeyser2017-09-07
| | | | | | ToolchainContext. PiperOrigin-RevId: 167729868
* Rename all logger instances to "logger" (instead "LOG" or "log").Gravatar lberki2017-09-05
| | | | | RELNOTES: None. PiperOrigin-RevId: 167505493
* Error reporting for invalid dynamic configurationsGravatar cparsons2017-08-31
| | | | | | | also, AppleConfiguration no longer throws NPE with invalid cpu. RELNOTES: None. PiperOrigin-RevId: 167013760
* Automated rollback of commit bb0aba3e91b603bf96d4e214a3886d1f47c3d90a.Gravatar janakr2017-08-24
| | | | | | | | | | | | | | | | | | | | *** Reason for rollback *** Publishing duplicate build events for tests. *** Original change description *** Request test artifacts to be built in parallel with running the test. In cases where not all test artifacts are needed to run the test, this allows for greater parallelism in the build. We need to inject the request for the test to be run into the TargetCompletion function, so that it can properly process any errors. Thanks to nharmata@ for suggesting this. MEMORY: One additional Skyframe node and 2 edges for each test to be run (sharded tests still only count as one). So with 500 test target... *** PiperOrigin-RevId: 166307568
* Request test artifacts to be built in parallel with running the test.Gravatar janakr2017-08-24
| | | | | | | | | In cases where not all test artifacts are needed to run the test, this allows for greater parallelism in the build. We need to inject the request for the test to be run into the TargetCompletion function, so that it can properly process any errors. Thanks to nharmata@ for suggesting this. MEMORY: One additional Skyframe node and 2 edges for each test to be run (sharded tests still only count as one). So with 500 test targets, 100K is a conservative estimate of memory usage. PiperOrigin-RevId: 166256545
* Remove BuildConfiguration.useDynamicConfigurations.Gravatar gregce2017-08-18
| | | | | | | | This is always true. Part of the static config cleanup effort. PiperOrigin-RevId: 165628823
* Remove ConfigurationCollection{Function,Value}, ↵Gravatar gregce2017-08-18
| | | | | | | | BuildConfigurationCollection.Transitions. Part of the static config cleanup effort. PiperOrigin-RevId: 165607492
* Change WalkableGraphFactory#prepareAndGet to take multiple SkyKeys as graph ↵Gravatar Googler2017-08-17
| | | | | | | | | | roots It also changes a few accessors of utility methods in Skyframe library. It refactors the QueryExpressionMapper to use a general QueryExpressionVisitor. RELNOTES: None PiperOrigin-RevId: 165534908
* Remove ConfigurationFactory as part of the static configuration cleanup.Gravatar gregce2017-08-17
| | | | PiperOrigin-RevId: 165478994
generated by cgit v1.2.3 (git 2.39.1) at 2024-07-07 02:07:19 +0000