| Commit message (Collapse) | Author | Age |
|
|
|
|
|
| |
INIT_ENV, and COMMIT.
PiperOrigin-RevId: 193261989
|
|
|
|
|
|
| |
getValueOrUntypedExceptions.
PiperOrigin-RevId: 192329649
|
|
|
|
|
|
| |
when all we want to do is register an edge and don't require the value.
PiperOrigin-RevId: 191966203
|
|
|
|
|
|
| |
Profiling can hold onto objects for the duration of the build, and some of those objects may be temporary that should not be persisted. In particular, UnixGlob and its inner classes should not outlive loading and analysis. For the most part, care was taken in this CL to only use strings that required no additional construction, mainly to minimize garbage (retaining references to newly created strings is not as great a concern since only the strings corresponding to the slowest K tasks are retained, for some relatively small values of K). Action descriptions for actually executing actions are eagerly expanded because that work is minimal compared to the work of actually executing an action.
PiperOrigin-RevId: 191251488
|
|
|
|
| |
PiperOrigin-RevId: 190656902
|
|
|
|
|
|
|
|
| |
Split registering the unique new deps of a node between those where we're enqueueing a known dependency from a prior build and one where we're adding a new dependency.
Replace prefetchBatch with getBatchAsync and add createIfAbsentBatchAsync.
PiperOrigin-RevId: 190471980
|
|
|
|
|
|
| |
always be the case.
PiperOrigin-RevId: 189792299
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Fix skylark caching to properly include transitive dependencies when there is a diamond-like dependency in the loaded bzl files.
Also add guards to make sure we're not attempting to cache skylark files that transitively request a dependency that is in error.
*** Reason for rollback ***
Looking for source of non-determinism
*** Original change description ***
Automated rollback of commit 7ba939dfd5df48903929e9c14ebd0449656403e4.
*** Reason for rollback ***
Likely cause for non-determinism in skyframe
*** Original change description ***
Cache SkylarkLookupImportValues in memory so that we don't recompute them multiple times.
PiperOrigin-RevId: 189686604
|
|
|
|
|
|
|
|
|
|
|
|
| |
*** Reason for rollback ***
Likely cause for non-determinism in skyframe
*** Original change description ***
Cache SkylarkLookupImportValues in memory so that we don't recompute them multiple times.
PiperOrigin-RevId: 188729929
|
|
|
|
|
|
| |
multiple times.
PiperOrigin-RevId: 187941859
|
|
|
|
|
|
|
|
| |
doesn't save memory in the 32-bit case, but makes it easier for people to see how many SkyKeys we have.
There's some unnecessary interning in tests, but it was easier to copypasta and doesn't harm anything, I think.
PiperOrigin-RevId: 187694309
|
|
|
|
|
|
| |
don't care about the group structure, and simplify the logic for prefetching old deps.
PiperOrigin-RevId: 187681887
|
|
|
|
|
|
| |
This also gets rid of some static initialization cycles which we should try very hard to avoid in the future.
PiperOrigin-RevId: 187334087
|
|
|
|
| |
PiperOrigin-RevId: 187209783
|
|
|
|
|
|
| |
evaluators that do not store errors alongside values, but which still support error transience.
PiperOrigin-RevId: 187058808
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
#doMutatingEvaluation
When multiple keys need to be evaluated, we may schedule keys before informing
progress on done nodes, which can throw InterruptExceptions. If the main thread
is interrupted during #informProgressReceiverThatValueIsDone, then we may not
properly track evaluations which are already scheduled. It could cause threads
continue to run after the main loop is closed.
RELNOTES: None
PiperOrigin-RevId: 186801930
|
|
|
|
|
|
|
|
|
|
|
|
| |
*** Reason for rollback ***
Fixed bug due to TransitiveTargetFunction requesting multiple Package dependencies when computing its aspect deps by only applying the optimization to TransitiveTraversalFunction.
*** Original change description ***
Automated rollback of commit cce164aed44aba1de244f0d764cd33a5cc6980b2.
PiperOrigin-RevId: 186766812
|
|
|
|
|
| |
RELNOTES: None.
PiperOrigin-RevId: 186658512
|
|
|
|
| |
PiperOrigin-RevId: 186617412
|
|
|
|
|
|
| |
ErrorInfoManager.
PiperOrigin-RevId: 186524034
|
|
|
|
| |
PiperOrigin-RevId: 186211672
|
|
|
|
|
|
| |
if we have them instead of re-computing them each time on a skyframe restart.
PiperOrigin-RevId: 186017079
|
|
|
|
|
|
|
|
| |
Remove WalkableGraph#isUpToDate and BuildDriver#alreadyEvaluated and delegate the
work to implementation.
RELNOTES: None
PiperOrigin-RevId: 185562370
|
|
|
|
|
|
|
| |
Also clarifies a comment on preventNewEvaluations.
RELNOTES: None.
PiperOrigin-RevId: 184198568
|
|
|
|
|
|
| |
null in AbstractPackageLoader and tests).
PiperOrigin-RevId: 183896826
|
|
|
|
|
| |
RELNOTES: None
PiperOrigin-RevId: 182389876
|
|
|
|
|
|
| |
and #toValue. I saw significant contention from this method in some experiments, and the synchronization isn't needed, since we only call these methods on done nodes (as determined by #isDone), and a node that is observably done cannot racily change its value.
PiperOrigin-RevId: 181507133
|
|
|
|
|
|
|
|
| |
Remove WalkableGraph#isUpToDate and BuildDriver#alreadyEvaluated and delegate the
work to implementation.
RELNOTES: None
PiperOrigin-RevId: 179815374
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
In the case that a node is already done when evaluation starts, we now report
events and postables early, rather than waiting until the end of evaluation.
This makes reporting more timely, and ensures reporting even if the evaluation
is interrupted.
This caused a problem with moving the TargetCompleteEvent into Skyframe
(unknown commit). I added two unit tests at the Skyframe level to cover the
guarantees that we need for that.
Note that the replay call in constructResult can duplicate the events from
a cache hit - this is not a problem since the replaying visitor automatically
removes duplicates (and it wasn't obvious which keys correspond to cache
hits).
PiperOrigin-RevId: 179788157
|
|
|
|
|
|
|
|
|
|
|
|
| |
Don't make copies of Events on replay. The same events may be replayed a lot,
so it's better to copy before storing the events. Also avoid a copy if the
tag doesn't actually change.
The intent of this change is to reduce gc churn on incremental builds. When
I wrote this change (~a year ago), this was a noticable source of gc churn in
some benchmarks I ran at the time.
PiperOrigin-RevId: 177450696
|
|
|
|
|
|
| |
deps from a compressed GroupedList without uncompressing it. Also some minor GC improvements.
PiperOrigin-RevId: 177338852
|
|
|
|
|
|
| |
store bare.
PiperOrigin-RevId: 176505963
|
|
|
|
|
|
|
|
| |
came from an AbstractParallelEvaluator evaluation. It's against the standard Java contract to throw but still have the thread's interrupted bit set.
Also get rid of some unnecessary initializeTester() calls in MemoizingEvaluatorTest: we already call it via a @Before annotation.
PiperOrigin-RevId: 176496034
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
memory-saving non-incremental mode independent of --batch and --discard_analysis_cache.
A command run with --nokeep_incrementality_data will discard data that would be needed for incremental builds. Subsequent commands can be sent to the same server, but they will not get the benefit of incrementality from this command. However, if --keep_incrementality_data is specified on a subsequent command, the commands after that will get the benefits of incrementality.
There are two benefits to not being dependent on --batch. First, this allows Bazel servers to be run in extreme memory-saving mode without the startup penalties (JVM startup, JITting) that --batch execution imposes. Second, this allows Bazel developers to inspect the state of a Bazel server after an extreme memory-saving build.
In order to avoid discarding data unnecessarily (for instance, on a "bazel info used-heap-size-after-gc" or "bazel dump --skyframe=summary") the actual resetting of the graph is done lazily, right before its use in SequencedSkyframeExecutor#sync. This is morally a partial rollback of https://github.com/bazelbuild/bazel/commit/98cd82cbdcac7c48164a611c5a9aa8fc2f1720ef.
For now, our tests specify all of the flags. After this change sticks, I plan to get rid of the --batch flag from these tests, which should allow for some clean-ups. Eventually --batch and --discard_analysis_cache may not imply that we don't keep incremental state: we can require that it be specified explicitly.
PiperOrigin-RevId: 175335075
|
|
|
|
|
|
|
|
| |
Blaze had its own class to avoid GC from varargs array creation for the precondition happy path. Guava now (mostly) implements these, making it unnecessary to maintain our own.
This change was almost entirely automated by search-and-replace. A few BUILD files needed fixing up since I removed an export of preconditions from lib:util, which was all done by add_deps. There was one incorrect usage of Preconditions that was caught by error prone (which checks Guava's version of Preconditions) that I had to change manually.
PiperOrigin-RevId: 175033526
|
|
|
|
|
|
| |
This change prepares a move to Guava's preconditions. Guava only has vararg-avoiding overloads up to 4 args.
PiperOrigin-RevId: 175015502
|
|
|
|
| |
PiperOrigin-RevId: 174508154
|
|
|
|
| |
PiperOrigin-RevId: 174202685
|
|
|
|
|
| |
RELNOTES: None
PiperOrigin-RevId: 173950304
|
|
|
|
|
|
| |
general interface.
PiperOrigin-RevId: 172606623
|
|
|
|
|
|
| |
or no edges. Also add option to disable checks in MemoizingEvaluatorTest that don't make sense for implementations that don't keep track of dirty nodes. Also extract RecordingDifferencer to an interface. And add a test for the situation that a node changes during a build that it's not requested, and which fails, necessitating cleanup.
PiperOrigin-RevId: 171616817
|
|
|
|
|
|
|
|
| |
catastrophes. Our stricter behavior in the face of errors means that it is no longer possible for a done node to depend on a not-done node in this build.
This opens up the possibility to discard graph edges on all --batch builds, or at least those with --discard_analysis_cache.
PiperOrigin-RevId: 171375405
|
|
|
|
|
|
|
|
|
|
|
|
| |
precomputed value. Instead, manually check if the value has changed, and if it has, invalidate its consuming WorkspaceStatusValue node, forcing its re-evaluation, where it will pick up the new value.
This seems more awkward than the original code, but it is more correct in spirit: injecting a precomputed value which can change even while the source state does not is a smell. Long-term, the key for the WorkspaceStatusValue should incorporate a hash of the action, and that hash should be in the configuration, just as other configuration flags are. That isn't possible right now just because we don't have configuration trimming, and we drop all nodes on configuration changes, so putting workspace status options into the configuration would lose change pruning whenever we changed workspace status options.
If/when those problems are fixed, we can extend this change to have WorkspaceStatusFunction continue to request the action out-of-band, but keyed by the hash. Then we can stop invalidating stale nodes.
See also https://github.com/bazelbuild/bazel/issues/3785.
PiperOrigin-RevId: 169947071
|
|
|
|
| |
PiperOrigin-RevId: 169278760
|
|
|
|
| |
PiperOrigin-RevId: 168802886
|
|
|
|
|
|
|
|
|
| |
Split collect, concurrent, vfs, windows into package-level BUILD files.
Move clock classes out of "util", into their own Java package.
Move CompactHashSet into its own Java package to break a dependency cycle.
Give nestedset and inmemoryfs their own package-level BUILD files.
PiperOrigin-RevId: 167702127
|
|
|
|
|
|
|
|
|
|
| |
- Move ProfilerInfo into a subpackage (it's not necessary for profiling, just for analyzing a profile).
- Make some fields in Profiler public for ProfileInfo.
- Mark Profiler as ThreadSafe; there's no cyclic dependency here.
This is based on ulfjack's microbazel patch series: https://github.com/ulfjack/bazel/commit/44553fcac0fc876784d8f48c2e577d8c999712de
PiperOrigin-RevId: 167121952
|
|
|
|
|
|
|
|
|
|
| |
roots
It also changes a few accessors of utility methods in Skyframe library. It
refactors the QueryExpressionMapper to use a general QueryExpressionVisitor.
RELNOTES: None
PiperOrigin-RevId: 165534908
|
|
|
|
|
|
|
|
|
|
| |
This change enables alternate evaluation strategies.
Drive-by fix to GraphTester, making TestFunction static because it has
no dependencies on its outer class, and adding #unsetConstantValue.
RELNOTES: None.
PiperOrigin-RevId: 164327055
|
|
|
|
|
|
|
| |
evaluation of the graph with a single constant version is expected
RELNOTES: None
PiperOrigin-RevId: 164059941
|