| Commit message (Collapse) | Author | Age |
|
|
|
|
| |
RELNOTES: None
PiperOrigin-RevId: 208166562
|
|
|
|
|
|
|
| |
The memory savings from this flag are not worth the complexity, and it interferes with action restarting.
RELNOTES: Remove support for --discard_actions_after_execution.
PiperOrigin-RevId: 201077905
|
|
|
|
|
|
| |
Moves more Action-oriented from skyframe package, which has become very unwieldy, to action package. This is needed to avoid circular dependencies caused when build-base is needed for skyframe.
PiperOrigin-RevId: 200996982
|
|
|
|
|
|
| |
(minor) ActionFS now implements MetadataProvider.getInput
PiperOrigin-RevId: 199575194
|
|
|
|
|
|
| |
Also pass the GraphInconsistencyReciever into SkyframeExecutor as a parameter.
PiperOrigin-RevId: 196716642
|
|
|
|
|
|
| |
CommonCommandOptions to better facilitate certain kinds of benchmarking.
PiperOrigin-RevId: 196695342
|
|
|
|
|
|
|
|
| |
it off after a release.
Discarding actions hasn't been shown to have a significant positive effect on heap memory usage, after careful research by mschaller@. It's holding back other projects (threading Fileset metadata through) and adding complexity, so it's time to kill it. Out of an abundance of caution, I'll keep actions in memory via a flag flip, then, if it sticks, I'll change the default, and then I'll unwire everything.
PiperOrigin-RevId: 196682768
|
|
|
|
| |
PiperOrigin-RevId: 195973862
|
|
|
|
|
|
| |
This should reduce memory consumption in NestedSet deserialization, which currently does not recycle Artifact instances.
PiperOrigin-RevId: 194083901
|
|
|
|
|
|
|
|
| |
the ones in SkyframeExecutor, called once for each Root in the package path list. This ensures there is a single canonical ArtifactRoot for each source root.
It is still the case that every Package loaded has its own Root (https://source.bazel.build/bazel/+/master:src/main/java/com/google/devtools/build/lib/packages/Package.java;l=287?q=Package.java), which is a waste of memory, but because of the map inside ArtifactFactory introduced in this change, all Artifacts with the same logical source root share a single ArtifactRoot instance.
PiperOrigin-RevId: 192513819
|
|
|
|
|
| |
RELNOTES: None
PiperOrigin-RevId: 192445971
|
|
|
|
|
|
| |
requests if SkyframeExecutor has reason to believe that those requests may not be CPU-bound.
PiperOrigin-RevId: 190844728
|
|
|
|
|
|
| |
If inlining is off, fixes a bug where null return values resulted in an error.
PiperOrigin-RevId: 190255818
|
|
|
|
|
|
| |
from the provided defaults in Options classes. These are used to create BuildOptionsDiffForReconstruction, which lets us store only the diffs in our BuildConfigurationValue.Keys.
PiperOrigin-RevId: 190117455
|
|
|
|
|
|
|
| |
Unconditionally inject the new values, and let skyframe change pruning avoid over-invalidation.
Change-Id: I3f478ea756121bde1078e9e79ddcfbeb54951cbb
PiperOrigin-RevId: 189156698
|
|
|
|
|
| |
RELNOTES: None
PiperOrigin-RevId: 188164754
|
|
|
|
|
| |
RELNOTES: None
PiperOrigin-RevId: 187997144
|
|
|
|
|
| |
RELNOTES: None
PiperOrigin-RevId: 187865083
|
|
|
|
|
| |
RELNOTES: None
PiperOrigin-RevId: 187852219
|
|
|
|
|
|
|
|
| |
doesn't save memory in the 32-bit case, but makes it easier for people to see how many SkyKeys we have.
There's some unnecessary interning in tests, but it was easier to copypasta and doesn't harm anything, I think.
PiperOrigin-RevId: 187694309
|
|
|
|
|
| |
RELNOTES: Use bazel dump --action_graph=/path/to/action.proto --action_graph:targets://foo:bar,//foo:foo to filter for certain targets in the action graph dump.
PiperOrigin-RevId: 187608321
|
|
|
|
|
|
|
|
| |
These may be reused by configured query, so rather move it early before
consumers starts depending on the old name.
RELNOTES: None
PiperOrigin-RevId: 186633754
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Note that this dumps the current state in skyframe (which may contain more nodes than you're interested in):
- bazel build --nobuild //interesting:targets
- bazel dump --action_graph=/path/to/file
- printproto --proto2 --raw_protocol_buffer --message=action_graph.ActionGraphContainer --multiline --proto=third_party/bazel/src/main/protobuf/action_graph.proto /path/to/file
We'll add filtering options in a later CL.
RELNOTES[NEW]: Add option to dump the action graph to a file: 'bazel dump --action_graph=/path/to/file'.
PiperOrigin-RevId: 186597930
|
|
|
|
|
|
| |
are referenced by TopLevelTargets when we discard the analysis cache.
PiperOrigin-RevId: 185574670
|
|
|
|
|
|
|
| |
ExtendedEventHandler so we can get the target via the package manager during
rule dumps.
PiperOrigin-RevId: 185042522
|
|
|
|
|
|
|
| |
This will serve as an alternative to --batch, leaving behind a server without state from the previous build.
RELNOTES: Introduces --[no]keep_state_after_build
PiperOrigin-RevId: 182778500
|
|
|
|
|
|
|
|
|
|
|
| |
This class represents a root (such as a package path or an output root) used for file lookups and artifacts. It is meant to be as opaque as possible in order to hide the user's environment from sky keys and sky functions.
Roots are used by RootedPaths and ArtifactRoots.
This CL attempts to make the minimum number of modifications necessary to change RootedPath and ArtifactRoot to use these fields. Deprecated methods and invasive accessors are permitted to minimise the risk of any observable changes.
RELNOTES: None
PiperOrigin-RevId: 182271759
|
|
|
|
|
|
| |
removing the layer of indirection of getting SkyKey out of ActionLookupKey, which uses more memory for no reason.
PiperOrigin-RevId: 181658615
|
|
|
|
|
| |
RELNOTES: None
PiperOrigin-RevId: 179944277
|
|
|
|
|
|
|
|
|
| |
New name clears the namespace a 2nd flag that will wipe the build graph after the build. The old name would be confusing as it could easily apply to that, and so needs to be more specifically just about tracking state in the first place. The new flag can be clearly separate and about keeping state after the build.
Partial roll forward of https://github.com/bazelbuild/bazel/commit/9321316b34767b06c3071b2cf2a4de189874fcce, with fixes to documentation that are still relevant.
RELNOTES: Rename --keep_incrementality_data to --track_incremental_state
PiperOrigin-RevId: 179078292
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
*** Reason for rollback ***
Design change, 2 boolean flags instead of 1 enum flag
*** Original change description ***
Add --incremental_state_retention_strategy
This option is intended to replace some of the uses of --batch. It lets users specify that builds should not be incremental, and how eagerly to discard the state that is kept around for incrementality. Note that for both values discard_eargerly and keep_for_life_of_build, the build graph is kept around until the next build. This may change.
Will add tests for keep_for_life_of_build in a later change, for now it will warn that that feature is experimen...
***
ROLLBACK_OF=178661777
RELNOTES: None.
PiperOrigin-RevId: 178681472
|
|
|
|
|
|
|
|
|
| |
This option is intended to replace some of the uses of --batch. It lets users specify that builds should not be incremental, and how eagerly to discard the state that is kept around for incrementality. Note that for both values discard_eargerly and keep_for_life_of_build, the build graph is kept around until the next build. This may change.
Will add tests for keep_for_life_of_build in a later change, for now it will warn that that feature is experimental.
RELNOTES: --[no]keep_incrementality_data is gone, replaced by the enum-valued --incremental_state_retention_strategy
PiperOrigin-RevId: 178661777
|
|
|
|
|
| |
RELNOTES: None
PiperOrigin-RevId: 177965330
|
|
|
|
|
|
|
| |
Fixes #4056.
Change-Id: Ia7425c2146f15e9293605ee3da53007805e82275
PiperOrigin-RevId: 177813070
|
|
|
|
|
|
|
| |
This key context can be used by actions to share partial key computations, for instance when computing MD5s for nested sets.
RELNOTES: None
PiperOrigin-RevId: 177359607
|
|
|
|
|
| |
RELNOTES: None.
PiperOrigin-RevId: 177332323
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
instead of assuming BUILD.
- Default the list to the same value as PackageLookupFunction:
BUILD.bazel, BUILD.
- Move BuildFileNames to the packages package, so it is more generally
available.
Part of #4056.
Change-Id: Ie12512b492cd7d47a9e56ec3bc209f829feaf4b5
PiperOrigin-RevId: 177261295
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
*** Reason for rollback ***
Tickles the shell integration test version of b/35042288
*** Original change description ***
Change BlacklistedPackagesPrefixesFunction to take a pair of a hardcoded set of directories and a file path containing more directories to blacklist. The current usage of PrecomputedValue#BLACKLISTED_PACKAGE_PREFIXES_FILE is overly general and is only meaningfully used by unit tests; in practice, the blacklist file path can never change over the lifetime of the Bazel server. Perform a minor simplifying refactor as a result of this.
RELNOTES: None.
PiperOrigin-RevId: 177176068
|
|
|
|
|
|
|
| |
of directories and a file path containing more directories to blacklist. The current usage of PrecomputedValue#BLACKLISTED_PACKAGE_PREFIXES_FILE is overly general and is only meaningfully used by unit tests; in practice, the blacklist file path can never change over the lifetime of the Bazel server. Perform a minor simplifying refactor as a result of this.
RELNOTES: None.
PiperOrigin-RevId: 177164057
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
memory-saving non-incremental mode independent of --batch and --discard_analysis_cache.
A command run with --nokeep_incrementality_data will discard data that would be needed for incremental builds. Subsequent commands can be sent to the same server, but they will not get the benefit of incrementality from this command. However, if --keep_incrementality_data is specified on a subsequent command, the commands after that will get the benefits of incrementality.
There are two benefits to not being dependent on --batch. First, this allows Bazel servers to be run in extreme memory-saving mode without the startup penalties (JVM startup, JITting) that --batch execution imposes. Second, this allows Bazel developers to inspect the state of a Bazel server after an extreme memory-saving build.
In order to avoid discarding data unnecessarily (for instance, on a "bazel info used-heap-size-after-gc" or "bazel dump --skyframe=summary") the actual resetting of the graph is done lazily, right before its use in SequencedSkyframeExecutor#sync. This is morally a partial rollback of https://github.com/bazelbuild/bazel/commit/98cd82cbdcac7c48164a611c5a9aa8fc2f1720ef.
For now, our tests specify all of the flags. After this change sticks, I plan to get rid of the --batch flag from these tests, which should allow for some clean-ups. Eventually --batch and --discard_analysis_cache may not imply that we don't keep incremental state: we can require that it be specified explicitly.
PiperOrigin-RevId: 175335075
|
|
|
|
|
|
|
|
| |
Blaze had its own class to avoid GC from varargs array creation for the precondition happy path. Guava now (mostly) implements these, making it unnecessary to maintain our own.
This change was almost entirely automated by search-and-replace. A few BUILD files needed fixing up since I removed an export of preconditions from lib:util, which was all done by add_deps. There was one incorrect usage of Preconditions that was caught by error prone (which checks Guava's version of Preconditions) that I had to change manually.
PiperOrigin-RevId: 175033526
|
|
|
|
| |
PiperOrigin-RevId: 174202685
|
|
|
|
|
|
| |
This requires a fairly large amount of changes to fundamental objects like BlazeRuntime, Executor, and so on, as well as changing a lot of test code to thread the file system through. I expect future CLs to be much smaller.
PiperOrigin-RevId: 173678144
|
|
|
|
|
|
| |
test to make sure we are using the expected type of node entries when discarding/keeping graph edges.
PiperOrigin-RevId: 173131307
|
|
|
|
|
|
|
|
|
|
|
| |
This adds two dump command, bazel dump --rules and bazel dump --skylark_memory.
dump --rules outputs a summary of the count, action count, and memory consumption of each rule and aspect class.
dump --skylark_memory outputs a pprof-compatible file with all Skylark analysis allocations. Users can then use pprof as per normal to analyse their builds.
RELNOTES: Add memory profiler.
PiperOrigin-RevId: 172558600
|
|
|
|
|
|
| |
rename method to better reflect what it is primarily doing.
PiperOrigin-RevId: 171621356
|
|
|
|
|
|
| |
or no edges. Also add option to disable checks in MemoizingEvaluatorTest that don't make sense for implementations that don't keep track of dirty nodes. Also extract RecordingDifferencer to an interface. And add a test for the situation that a node changes during a build that it's not requested, and which fails, necessitating cleanup.
PiperOrigin-RevId: 171616817
|
|
|
|
|
|
|
|
|
| |
SkylarkSemanticsCodec
Note that the syntax package and its test package still depend indirectly on the options parser via other Bazel-specific packages.
RELNOTES: None
PiperOrigin-RevId: 171342823
|
|
|
|
|
|
| |
#detectModifiedOutputFiles and move implementation to SequencedSkyframeExecutor.
PiperOrigin-RevId: 170230031
|
|
|
|
|
|
| |
and error-checking for their existence is already done by the client.
PiperOrigin-RevId: 169966701
|