| Commit message (Collapse) | Author | Age |
|---|
| |
|
|
|
|
|
|
| |
node having the same priority, later enqueueings having higher priority, re-enqueued nodes having highest priority, and new root nodes having lowest priority. Experimentally, this can save significant RAM (1.4G in some builds!) while not affecting speed.
Also do a semi-drive-by deleting ExecutorFactory parameter to AbstractQueueVisitor, since it was always AbstractQueueVisitor.EXECUTOR_FACTORY.
PiperOrigin-RevId: 208560889
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
than the graph version when that is feasible.
* It's not feasible when the computation accesses outside state, i.e. is non-hermetic, so see below.
* It's also more complicated (and not worth the trouble) when the computation is taking place just for the error status.
Have SkyFunctionName declare whether the function it corresponds to is hermetic or non-hermetic. Only non-hermetically-generated SkyValues can be directly marked changed, and non-hermetic SkyFunctions have their values saved at the graph version, not the max of the child versions. All SkyFunctions are hermetic except for the ones that can be explicitly dirtied.
A marked-hermetic SkyFunction that has a transient error due to filesystem access can be re-evaluated and get the correct version: if it throws an IOException at version 1 and then, when re-evaluated at version 2 with unchanged dependencies, has a value, the version will be version 1.
All Skyframe unit tests that were doing non-hermetic things to nodes need to declare that those nodes are non-hermetic. I tried to make the minimal set of changes there, so that we had good incidental coverage of hermetic+non-hermetic nodes. Also did some drive-by clean-ups around that code.
Artifacts are a weird case, since they're doing untracked filesystem access (for source directories). Using max(child versions) for them gives rise to the following correctness bug: 1. do a build at v1 that creates a FileStateValue for dir/ at v1. Then at v2, add a file to dir/ and do a build that consumes dir/ as a source artifact. Now the artifact for dir/ will (incorrectly) have v1. Then at v1, do that build again. We'll consume the "artifact from the future". However, this can only have an effect when using the local action cache, since the incorrect value of the artifact (the mtime) is only consumed by the action cache. Bazel is already broken in this way (incremental builds don't invalidate directories), so this change doesn't make things worse.
PiperOrigin-RevId: 204210719
|
| |
|
|
|
|
|
|
|
|
| |
node's in-progress data should be forgotten, and its evaluation should be restarted from scratch, as if it were freshly created/dirtied. To guard against this happening unexpectedly, any such events are passed to a GraphInconsistencyReceiver, which can verify that the SkyFunction is behaving properly.
This is the first change in a series to permit action rewinding when it is discovered that a previously generated input file is no longer available. When an action detects that one of its inputs is unusable, it can return this sentinel value, causing it to be re-evaluated from scratch. Follow-up changes will make the node corresponding to the input, and the node corresponding to the action that generated the input, dirty when this happens, causing the upstream action to be re-run, regenerating the desired input.
Currently works for builds that do not keep edges, although follow-ups may make this possible for all builds.
PiperOrigin-RevId: 194863097
|
| |
|
|
|
|
|
|
|
|
|
|
| |
Don't make copies of Events on replay. The same events may be replayed a lot,
so it's better to copy before storing the events. Also avoid a copy if the
tag doesn't actually change.
The intent of this change is to reduce gc churn on incremental builds. When
I wrote this change (~a year ago), this was a noticable source of gc churn in
some benchmarks I ran at the time.
PiperOrigin-RevId: 177450696
|
| |
|
|
|
|
|
|
| |
Blaze had its own class to avoid GC from varargs array creation for the precondition happy path. Guava now (mostly) implements these, making it unnecessary to maintain our own.
This change was almost entirely automated by search-and-replace. A few BUILD files needed fixing up since I removed an export of preconditions from lib:util, which was all done by add_deps. There was one incorrect usage of Preconditions that was caught by error prone (which checks Guava's version of Preconditions) that I had to change manually.
PiperOrigin-RevId: 175033526
|
| |
|
|
|
|
|
| |
We were previously duplicate-posting Postable events posted to the
Skyframe environment.
PiperOrigin-RevId: 162323598
|
| |
|
|
|
|
|
|
|
|
| |
-Have SkylarkImportLookupFunction include causes in the SkyFunctionExceptions it throws.
-Better transitive skyframe error declarations in ASTFileLookupFunction.
-Have ErrorInfo differentiate between direct and transitive transience.
-Introduce ErrorInfoManager and have ParallelEvaluator/ParallelEvaluatorContext use it.
RELNOTES: None
PiperOrigin-RevId: 159163186
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
With more specific information to be reported by Skyfunctions, e.g.,
to inform the build-event protocol on missing files, the EventHandler
interface is no longer enough. Therefore, provide an enriched context
for reporting events.
--
Change-Id: I2d06166fe4d5b9054e24ad8c752fafc039e3f9f8
Reviewed-on: https://cr.bazel.build/8794
PiperOrigin-RevId: 148463437
MOS_MIGRATED_REVID=148463437
|
| |
|
|
|
| |
--
MOS_MIGRATED_REVID=136431274
|
| |
|
|
|
|
|
|
|
|
| |
Remove DirtyKeyTracker and inflight node tracking as independent
progress tracking bits of code, and instead use an internal-facing
DirtyTrackingProgressReceiver wrapper of the user's
EvaluationProgressReceiver.
--
MOS_MIGRATED_REVID=135291924
|
| |
|
|
|
|
|
| |
can use different cycle detection algorithms if they wish.
--
MOS_MIGRATED_REVID=131347160
|
| |
|
|
|
| |
--
MOS_MIGRATED_REVID=131344134
|
|
|
respect to cycle checking.
Reducing the size of ParallelEvaluator.java is also probably long overdue.
I believe this change stands on its own, but if you don't think the third change is worth it, and this isn't worth it on its own, feel free to push back.
--
MOS_MIGRATED_REVID=131340165
|
janakr
ulfjack
tomlu
nharmata
Klaus Aehlig
Liam Miller-Cushon
Chloe Calvarin