| Commit message (Collapse) | Author | Age |
|
|
|
|
|
|
|
|
|
|
| |
respect to cycle checking.
Reducing the size of ParallelEvaluator.java is also probably long overdue.
I believe this change stands on its own, but if you don't think the third change is worth it, and this isn't worth it on its own, feel free to push back.
--
MOS_MIGRATED_REVID=131340165
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Instead, just put them directly into a map. This avoids the memory churn and CPU cost of the set.
As a result, we have to use HashMaps instead of ImmutableMap.Builders, which I hope is ok (especially since we're not keeping them around), and due to that, we have some nice nondeterminism in the returned order, which matters for some cycle-checking tests.
Also, don't use a map at all when we don't need to (when building events).
Note that, since we have to deduplicate at some point, this means that changing the return type of SkyFunction.Environment#getValues to not be a random-access map is probably not worth it. Changing the return type of ProcessableGraph#getBatch to not be a random access map might still be worthwhile, although it might require some funny operations.
--
MOS_MIGRATED_REVID=131070418
|
|
|
|
|
|
|
| |
The only place we now don't handle InterruptedException is in the action graph created after analysis, since I'm not sure that will be around for that much longer.
--
MOS_MIGRATED_REVID=130327770
|
|
|
|
|
|
|
|
|
| |
Make sure that ParallelEvaluator treats SchedulerExceptions as less severe than other RuntimeExceptions (it already did, but add a comment emphasizing this).
The combination of the above means that we don't ignore hypothetical crashes in ParallelEvaluator worker threads _after_ a SchedulerException (e.g. due to an error in nokeep_going evaluation).
--
MOS_MIGRATED_REVID=130044230
|
|
|
|
|
|
|
|
|
| |
nokeep_going build because it finished building in between the time it was first requested and when we checked it for done-ness after the SkyFunction evaluation.
This results in an IllegalStateException that gets ignored (and, importantly, not propagated) by AbstractQueueVisitor because AbstractQueueVisitor only records and propagates the first Throwable encountered. For nokeep_going evaluations, this will be the SchedulerException that we use for control flow. The IllegalStateException in question is benign in this case because it's merely from a Preconditions failure and doesn't leave anything in a bad state. It's possible, though, that we have other bugs that are being masked in this way.
--
MOS_MIGRATED_REVID=129919336
|
|
|
|
|
| |
--
MOS_MIGRATED_REVID=129895423
|
|
|
|
|
|
|
| |
QueryableGraph.Reason which conveys more information. Add a few more Reason enum values to make this refactor benign.
--
MOS_MIGRATED_REVID=129118462
|
|
|
|
|
|
|
| |
parameters conveying the requesting node (if any), the requested node(s), as well as a reason for the skyframe graph lookup. Alternate graph implementations may be interested in this information.
--
MOS_MIGRATED_REVID=128496089
|
|
|
|
|
|
|
| |
SkyFunction if it throws an exception but still has missing deps. Instead, pretend it didn't throw, and restart it when its known deps are all done, presumably to throw the same exception. This removes a basic source of non-determinism.
--
MOS_MIGRATED_REVID=127129202
|
|
|
|
|
|
|
| |
alternate graph implementations to optimize how they construct node entries.
--
MOS_MIGRATED_REVID=126932020
|
|
|
|
|
|
|
| |
interfaces, in preparation for further changes.
--
MOS_MIGRATED_REVID=126924789
|
|
|
|
|
|
|
|
|
| |
Collapse the "evaluating" boolean into the "signaledDeps" int field, since signaledDeps is always 0 if evaluating is false, so we can use the sentinel value -1 to indicate that evaluation has not yet started. This leads to a slightly less tolerant node entry: it must "start evaluating" before you can do things like set its value. Places that wasn't being done have been fixed, at least as far as we have test coverage for.
Also, factor the "dirty" parts of BuildingState out into a subclass. It would probably be cleaner to use composition here, but I don't want to pay the price of another object.
--
MOS_MIGRATED_REVID=126729331
|
|
|
|
|
|
|
| |
The biggest savings here is that we were not eagerly discarding the InMemoryNodeEntry#directDeps field after an entry was marked dirty, even though we would never read its value again. But rather than just fix that, by getting rid of the field in BuildingState, we can potentially save memory with smaller BuildingState objects as well.
--
MOS_MIGRATED_REVID=126709632
|
|
|
|
|
|
|
| |
last build's deps. This is only an issue with cycles, since there we try to maintain the invariant that a parent is not done before its children by removing its deps on its children before we construct its cycle value.
--
MOS_MIGRATED_REVID=126494009
|
|
|
|
|
| |
--
MOS_MIGRATED_REVID=126344189
|
|
|
|
|
|
|
| |
InvalidatedNodeEntry.
--
MOS_MIGRATED_REVID=126139179
|
|
|
|
|
|
|
| |
lastEvaluated/lastChanged version fields, we lost memory alignment, so this boolean was costing us 8 bytes per instance.
--
MOS_MIGRATED_REVID=125998857
|
|
|
|
|
| |
--
MOS_MIGRATED_REVID=125376554
|
|
|
|
|
| |
--
MOS_MIGRATED_REVID=125362963
|
|
|
|
|
|
|
|
|
|
|
| |
nodes, where there is no work to do anyway.
This triggered some non-determinism that we explicitly workaround in the unit tests.
Also add a comment about a potential but unrelated optimization.
--
MOS_MIGRATED_REVID=125355303
|
|
|
|
|
| |
--
MOS_MIGRATED_REVID=124874002
|
|
|
|
|
|
|
| |
functionality was only used in tests.
--
MOS_MIGRATED_REVID=124841573
|
|
|
|
|
| |
--
MOS_MIGRATED_REVID=124747935
|
|
|
|
|
| |
--
MOS_MIGRATED_REVID=123347295
|
|
|
|
|
|
|
|
|
|
|
| |
Now that we batch-prefetch the nodes, there is no reason to delay this
check until the async Runnable runs, since we have more debugging
information this way.
--
Change-Id: Ic73d99ed8de184ba1e29f0dee5375f5d45b5379d
Reviewed-on: https://bazel-review.googlesource.com/3680
MOS_MIGRATED_REVID=123018542
|
|
|
|
|
|
|
| |
where we weren't checking to see if a reverse dep already existed when we declared a reverse dep.
--
MOS_MIGRATED_REVID=122581019
|
|
|
|
|
|
|
|
|
|
|
| |
discover that it no longer has certain deps.
In the common case, where a node's deps do not change in the end, this reduces lock contention and CPU.
The downside of this is that we now create a set of the previous reverse deps during each evaluation of a node. We don't store this set in order to conserve memory, so we pay for it in CPU. We will probably only construct it two or three times (most SkyFunctions don't have so many groups), so the cost shouldn't be so high, but we can try to mitigate if it shows up in profiling.
--
MOS_MIGRATED_REVID=122566267
|
|
|
|
|
|
|
| |
conversion is unnecessary and wasteful. In the remaining cases, the set conversion can be explicit.
--
MOS_MIGRATED_REVID=122294939
|
|
|
|
|
|
|
| |
helpers to enforce concurrency synchronization points and determinism even if they are not using an InMemoryGraph-backed evaluator.
--
MOS_MIGRATED_REVID=121977783
|
|
|
|
|
|
|
| |
allowing for future work that may want to access a particular group in the GroupedList without advancing the iterator.
--
MOS_MIGRATED_REVID=120933039
|
|
|
|
|
| |
--
MOS_MIGRATED_REVID=118623130
|
|
|
|
|
|
|
| |
non-keep-going build. Setting errorDepKey was only mostly harmless, to mix memes. (Actually, it was quite harmful.)
--
MOS_MIGRATED_REVID=118410594
|
|
|
|
|
| |
--
MOS_MIGRATED_REVID=117894693
|
|
|
|
|
|
|
| |
it. The copying showed up as a source of memory spikiness.
--
MOS_MIGRATED_REVID=117741939
|
|
|
|
|
|
|
| |
SkyQueryEnvironment. QueryEnvironmentFactory, recently introduced by unknown commit, is a much more general purpose mechanism.
--
MOS_MIGRATED_REVID=117590252
|
|
|
|
|
|
|
| |
copying it. The copying showed up as a source of memory spikiness in an OOM.
--
MOS_MIGRATED_REVID=117477133
|
|
|
|
|
|
|
| |
...so the docs show up on hover in an IDE.
--
MOS_MIGRATED_REVID=116986129
|
|
|
|
|
|
|
| |
are created, as opposed to when they are requested from the ParallelEvaluator. That delay can lead to large memory spikes and churn.
--
MOS_MIGRATED_REVID=116224565
|
|
|
|
|
|
|
| |
SkyFunction
--
MOS_MIGRATED_REVID=115671161
|
|
|
|
|
|
|
| |
Also delete some code that's been dead for a while, now that we eagerly shut down evaluation when we come across a child in error during a fail-fast evaluation.
--
MOS_MIGRATED_REVID=115272603
|
|
|
|
|
| |
--
MOS_MIGRATED_REVID=114488935
|
|
|
|
|
|
|
| |
recover from errors. In the case of a single keep_going build, with no subsequent nokeep_going builds, storing the errors is unnecessary.
--
MOS_MIGRATED_REVID=114355846
|
|
|
|
|
|
|
| |
tests. Replace it in tests with direct Skyframe graph lookups.
--
MOS_MIGRATED_REVID=114335937
|
|
|
|
|
|
|
| |
ArtifactFunction.
--
MOS_MIGRATED_REVID=114174899
|
|
|
|
|
|
|
| |
node don't have access to its dependencies ("grandparents don't know grandchildren", or vice versa, depending on your point of view), changes to a node's dependencies can't affect downstream nodes.
--
MOS_MIGRATED_REVID=114143894
|
|
|
|
|
| |
--
MOS_MIGRATED_REVID=113313675
|
|
|
|
|
|
|
| |
type-safety, we now must pass in the exception type of the callback.
--
MOS_MIGRATED_REVID=113313312
|
|
|
|
|
| |
--
MOS_MIGRATED_REVID=113197641
|
|
|
|
|
|
|
|
|
|
|
| |
EvaluationResult to return true for hasError() iff errorMap is non-empty or there is a catastrophe.
There was no good reason for the previous behavior of saying hasError even if there was a transitive recovered-from error, since callers shouldn't care. This was a latent bug that was only benign since none of the consumers of hasError were invoking Skyframe with recoverable SkyFunctions.
Also add an EvaluationResultSubject so that tests can more fluently assert things about EvaluationResult objects going forward.
--
MOS_MIGRATED_REVID=113192415
|
|
|
|
|
|
|
| |
all so that custom implementations can have custom options passed around.
--
MOS_MIGRATED_REVID=112502778
|