aboutsummaryrefslogtreecommitdiffhomepage
path: root/src/main/java/com/google/devtools/build/skyframe/ParallelEvaluator.java
Commit message (Collapse)AuthorAge
...
* Prevent new evaluations from starting if a done child's error is discovered. ↵Gravatar Janak Ramakrishnan2016-02-23
| | | | | | | Also delete some code that's been dead for a while, now that we eagerly shut down evaluation when we come across a child in error during a fail-fast evaluation. -- MOS_MIGRATED_REVID=115272603
* Allow ParallelEvaluator to not store errors alongside values if nodes ↵Gravatar Janak Ramakrishnan2016-02-11
| | | | | | | recover from errors. In the case of a single keep_going build, with no subsequent nokeep_going builds, storing the errors is unnecessary. -- MOS_MIGRATED_REVID=114355846
* Support for handling TreeArtifact metadata and returning TreeArtifacts from ↵Gravatar Michael Thvedt2016-02-09
| | | | | | | ArtifactFunction. -- MOS_MIGRATED_REVID=114174899
* Fix SkyframeBuilder's error handling, and change the contract of ↵Gravatar Janak Ramakrishnan2016-01-28
| | | | | | | | | | | EvaluationResult to return true for hasError() iff errorMap is non-empty or there is a catastrophe. There was no good reason for the previous behavior of saying hasError even if there was a transitive recovered-from error, since callers shouldn't care. This was a latent bug that was only benign since none of the consumers of hasError were invoking Skyframe with recoverable SkyFunctions. Also add an EvaluationResultSubject so that tests can more fluently assert things about EvaluationResult objects going forward. -- MOS_MIGRATED_REVID=113192415
* Skyfunctions can now catch up to five exceptions.Gravatar John Field2016-01-15
| | | | | | | This is a prerequisite to making some changes to PackageFunction that require handling another exception. -- MOS_MIGRATED_REVID=112172100
* Add catastrophe field to EvaluationResult so that callers can identify the ↵Gravatar Janak Ramakrishnan2016-01-04
| | | | | | | | | cause of a catastrophic failure (this is distinct from a crash). Also clean up catastrophe logic in ParallelEvaluator -- the catastrophic nature of an exception is important only if the build is keep_going, and only if the exception is catastrophic can we have an exception in the first place. -- MOS_MIGRATED_REVID=111293164
* Clarify the message of the exception thrown when a .getValue() calls returns ↵Gravatar Lukacs Berki2015-12-13
| | | | | | | null but the SkyFunction that called it returns non-null. -- MOS_MIGRATED_REVID=110009359
* Use Bazel Preconditions variant which avoids varargs array creationGravatar Mark Schaller2015-12-10
| | | | | | | Reduces garbage. -- MOS_MIGRATED_REVID=109914243
* Repurpose the not-meaningfully-used ErrorInfo#isTransient to mean "is ↵Gravatar Nathan Harmata2015-12-10
| | | | | | | | | transitively transient". Some followup changes will use this method. Previously, ErrorInfo#isTransient was only used internally in ParallelEvaluator; I think this method was originally added to ErrorInfo solely for the sake of convenience. -- MOS_MIGRATED_REVID=109840031
* Avoid list copy in BuildingState.getNextDirtyDirectDepsGravatar Mark Schaller2015-12-09
| | | | | | | | | Also, in GroupedList, short-circuit expensive group equality check with a reference check, saving time and garbage when groups are the same object. -- MOS_MIGRATED_REVID=109795332
* Tolerate independent cycle underneath main cycle during a re-evaluation.Gravatar Janak Ramakrishnan2015-11-24
| | | | | | | With the new-ish behavior of change pruning, where nodes transition from NEEDS_REBUILDING to REBUILDING, we need to make that transition unconditionally for dirty nodes -- being ready after unfinished deps were removed is irrelevant. -- MOS_MIGRATED_REVID=108508979
* More batching of graph lookups during evaluationGravatar Mark Schaller2015-11-19
| | | | | -- MOS_MIGRATED_REVID=108186573
* Batch (r)deps lookups during evaluationGravatar Mark Schaller2015-11-19
| | | | | -- MOS_MIGRATED_REVID=108174404
* Put NodeEntry in Supplier<SkyValue> passed to EvaluationProgressReceiver so ↵Gravatar Janak Ramakrishnan2015-11-13
| | | | | | | it's readily accessible in the program. -- MOS_MIGRATED_REVID=107698784
* Record Skyframe timing metrics even if Profiling is disabled.Gravatar Eric Fellheimer2015-11-06
| | | | | -- MOS_MIGRATED_REVID=107228414
* Add sanity check in bubbleErrorUp that the parentEntry is non-null iff the ↵Gravatar Nathan Harmata2015-11-05
| | | | | | | parent is non-null. -- MOS_MIGRATED_REVID=107062267
* Introduce ForkJoinQuiescingExecutor, permit its use in evaluationGravatar Mark Schaller2015-11-02
| | | | | | | | | | | | | | | | | | This CL introduces a QuiescingExecutor implementation specialized for ForkJoinPools with the same interrupt handling, error propagation, and task completion semantics as AbstractQueueVisitor. Currently it does this by largely sharing its implementation with AQV. Future refactoring could let it rely more on ForkJoinPool's own awaitQuiescence implementation to avoid the overhead of AQV's remainingTasks counter maintenance. Subtasks spawned by tasks executing in ForkJoinQuiescingExecutor will rely on ForkJoinPool's thread-local task deques for low contention and (mostly) LIFO ordering. -- MOS_MIGRATED_REVID=106864395
* Cleanup ValueVisitor (and dirty QuiescingExecutor)Gravatar Mark Schaller2015-11-02
| | | | | | | | | | | | | Raises the level of abstraction of ValueVisitor's dependence on AbstractQueueVisitor. Except for the "ForTestingOnly" methods now available on the QuiescingExecutor interface, ValueVisitor is agnostic to the implementation of its executor. This also cleans up the full spectrum of visibility modifiers on ValueVisitor methods, all of which ought to be private. -- MOS_MIGRATED_REVID=106847453
* Change ValueVisitor to use delegationGravatar Mark Schaller2015-11-02
| | | | | | | | This simplifies a future change that introduces new variance to ValueVisitor's use of AbstractQueueVisitor. -- MOS_MIGRATED_REVID=106846210
* Introduce ErrorClassifierGravatar Mark Schaller2015-11-02
| | | | | | | | | | Changes the AbstractQueueVisitor strategy for varying its response to unhandled exceptions from inheritance to composition. This will help with a forthcoming switch from inheritance to delegation for ValueVisitor's use of AbstractQueueVisitor. -- MOS_MIGRATED_REVID=106730708
* Introduce QuiescingExecutorGravatar Mark Schaller2015-11-02
| | | | | | | | | | | | This interface (mostly) encapsulates what the ValueVisitor expects from the AbstractQueueVisitor class it currently inherits from. This makes it easier for a future CL to change ValueVisitor's strategy of code reuse from inheritance to composition. RELNOTES: -- MOS_MIGRATED_REVID=106728863
* Share SkyKey interning between evaluation and deserialization.Gravatar Eric Fellheimer2015-10-27
| | | | | -- MOS_MIGRATED_REVID=106323413
* Singleton-ify ErrorTransienceValueGravatar Michajlo Matijkiw2015-10-22
| | | | | | | | | | | | | | We rely on ErrorTransienceValue being different across evaluations by way of java reference inequality, instead make it a singleton and make it so that ErrorTransienceValue compares equal to nothing, including itself. Alternatively we could attach the corresponding Version to ErrorTransienceValue and use that to force inequality across evaluations. However this gets into the different possible Version implementations and their varying semantics, so go with the simplest approach first and see how it works out. -- MOS_MIGRATED_REVID=106001826
* Remove wildcard of SkyFunctionName, as it is finalGravatar Mark Schaller2015-10-22
| | | | | | | Nothing can extend SkyFunctionName. It's final. -- MOS_MIGRATED_REVID=105989976
* In SkyQueryEnvironment, check TransitiveTraversalValues for errorsGravatar Mark Schaller2015-10-22
| | | | | | | | | | | | | | | In SkyQueryEnvironment, the buildTransitiveClosure method ignored the error messages in successfully evaluated TransitiveTraversalValues, under the belief that any such error messages are also obtainable by calling getMissingAndExceptions. This happens to be true right now, but may not be true in the future. If, in particular, the TransitiveTraversal skyfunction inlines its call to the TargetMarker skyfunction, those error messages won't be available through getMissingAndExceptions. -- MOS_MIGRATED_REVID=105980430
* Keep track of any RuntimeExceptions thrown during evaluation. Previously, ↵Gravatar Janak Ramakrishnan2015-10-21
| | | | | | | RuntimeExceptions that were thrown after an InterruptedException was thrown would be swallowed. -- MOS_MIGRATED_REVID=105902192
* Don't log SchedulerExceptions.Gravatar Nathan Harmata2015-10-20
| | | | | | | RELNOTES: None -- MOS_MIGRATED_REVID=105787681
* Move to using "well-known" SkyFunctionNames in tests exercising graphGravatar Michajlo Matijkiw2015-10-16
| | | | | -- MOS_MIGRATED_REVID=105539869
* Allow other ExecutorService implementations in AbstractQueueVisitorGravatar Mark Schaller2015-10-13
| | | | | | | Previously, only ThreadPoolExecutor implementations were allowed. -- MOS_MIGRATED_REVID=105340237
* Don't iterate over children when building events if events won't be stored ↵Gravatar Janak Ramakrishnan2015-10-12
| | | | | | | anyway. Also batch retrievals of children when building events. -- MOS_MIGRATED_REVID=105174435
* Clear interrupt bit before processing any error thrown by SkyFunction during ↵Gravatar Janak Ramakrishnan2015-10-12
| | | | | | | error bubbling. -- MOS_MIGRATED_REVID=105102597
* Replace EvaluableGraph#createIfAbsent with the potentially more efficient ↵Gravatar Nathan Harmata2015-10-05
| | | | | | | EvaluableGraph#createIfAbsentBatch. -- MOS_MIGRATED_REVID=104534858
* Pre-populate the SkyFunctionEnvironment with the node's previously declared ↵Gravatar Nathan Harmata2015-10-05
| | | | | | | deps. -- MOS_MIGRATED_REVID=104531377
* Add debugging info for crash during maybeMarkRebuilding.Gravatar Janak Ramakrishnan2015-09-30
| | | | | -- MOS_MIGRATED_REVID=104316416
* Refactor ErrorInfo creation to share single constructorGravatar Michajlo Matijkiw2015-09-30
| | | | | | | | | | | | Single constructor allows us to enforce/document high-level constraints in a single place. Move previous constructors to static methods which do their custom transformations but ultimately funnel their final calculated fields through the common constructor. Also added some tests to demonstrate expected behavior of static methods. -- MOS_MIGRATED_REVID=104142909
* Rationalize copyright headersGravatar Damien Martin-Guillerez2015-09-25
| | | | | | | | | | | The headers were modified with `find . -type f -exec 'sed' '-Ei' 's|Copyright 201([45]) Google|Copyright 201\1 The Bazel Authors|' '{}' ';'` And manual edit for not Google owned copyright. Because of the nature of ijar, I did not modified the header of file owned by Alan Donovan. The list of authors were extracted from the git log. It is missing older Google contributors that can be added on-demand. -- MOS_MIGRATED_REVID=103938715
* Prefetch and assert done on direct deps. Also print a more informative error ↵Gravatar Janak Ramakrishnan2015-09-25
| | | | | | | message if a previously known direct dep isn't done. -- MOS_MIGRATED_REVID=103880990
* Don't remove reverse deps until node is known to be changed. This helps ↵Gravatar Janak Ramakrishnan2015-09-23
| | | | | | | avoid mutating the deps of nodes that are still going to be deps after evaluation is finished. -- MOS_MIGRATED_REVID=103659429
* Tiny typo fixGravatar Mark Schaller2015-09-21
| | | | | -- MOS_MIGRATED_REVID=103387106
* Fix Bazel breakageGravatar Damien Martin-Guillerez2015-09-11
| | | | | -- MOS_MIGRATED_REVID=102835622
* Delay cleaning of in-flight nodes until the following build. This allows us ↵Gravatar Janak Ramakrishnan2015-09-11
| | | | | | | | | to interrupt evaluation in constant time. Some ParallelEvaluator tests that implicitly relied on cleaning happening before the next evaluation were moved into MemoizingEvaluatorTest as a result. -- MOS_MIGRATED_REVID=102696653
* Add a method to EvaluationProgressReceiver for getting information on ↵Gravatar Nathan Harmata2015-09-03
| | | | | | | SkyFunction#compute calls. -- MOS_MIGRATED_REVID=102268773
* Don't pass temporary direct deps to environment when bubbling up.Gravatar Janak Ramakrishnan2015-09-01
| | | | | | | Normally, any known direct deps are guaranteed to be done. That's not true when bubbling. Since knowing no direct deps is a plausible state of affairs, just don't pass any. This may aid future code changes. -- MOS_MIGRATED_REVID=101980323
* Prefetch children of node being checked for cycles, in case graph ↵Gravatar Janak Ramakrishnan2015-08-31
| | | | | | | implementation caches batched results. -- MOS_MIGRATED_REVID=101961270
* Remove useless graph.get call, since we already have entry.Gravatar Janak Ramakrishnan2015-08-31
| | | | | -- MOS_MIGRATED_REVID=101960993
* Presize some collections at the core of ParallelEvaluatorGravatar Michajlo Matijkiw2015-08-31
| | | | | | | | | | We know what size these will be ahead of time, and they can get large, presize since there's no reason not to. Also change arg type for related methods to be Set since it's all we deal in and we know it will have better performance on size than generic Iterable. -- MOS_MIGRATED_REVID=101806154
* Allow ErrorTransienceValue to be added back as a dep during change pruning ↵Gravatar Janak Ramakrishnan2015-08-26
| | | | | | | | | if it isn't newer than the current node. ErrorTransienceValue has to be treated specially during pruning because normally deps added during pruning remain if the node is actually rebuilt. The whole point of ErrorTransienceValue is that on a re-evaluation, the node may not depend on it anymore. So we have special logic to force a rebuild if the ErrorTransienceValue has changed. If it hasn't, however, it is ok for pruning to add it back as a dep -- we'll end up marking this value clean, since ErrorTransienceValue is always the last dep. -- MOS_MIGRATED_REVID=101522197
* Return raw SkyValue instead of wrapped ValueWithMetadata.Gravatar Janak Ramakrishnan2015-08-25
| | | | | -- MOS_MIGRATED_REVID=101377320
* Fix some nits and allow alternate evaluators to use MemoizingEvaluatorTest.Gravatar Janak Ramakrishnan2015-08-21
| | | | | -- MOS_MIGRATED_REVID=101150250
* Convert evaluated tracking to take a lazy SkyValueGravatar Mark Schaller2015-08-21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The EvaluationProgressReceiver#evaluated method took a SkyValue, but that parameter was only used by some of its implementations, and only under some conditions. Outside of tests, the main users are SkyframeBuildView's ConfiguredTargetValueInvalidationReceiver and SkyframeBuilder's ExecutionProgressReceiver. The former builds up a set of built ConfiguredTarget keys when the SkyValue is non-null and the EvaluationState is BUILT, and so its nullity check can live behind those two conditions. The latter cares about builting up a set of ConfiguredTargets, and raising events on the eventBus when a TARGET_COMPLETION or ASPECT_COMPLETION value is evaluated and is non-null. The extraction of these values can live behind the conditions that check the type of the SkyKey. By making the SkyValue parameter lazy, this change enforces that it's only accessed under these conditions. This CL introduces a semantic change that should be small in effect. The SkyframeBuildView keeps track of a set, dirtiedConfiguredTargetKeys, and ConfiguredTarget keys evaluated as CLEAN were removed from it if they had a non-null value. With this change, ConfiguredTarget keys evaluated as CLEAN get removed regardless of whether their values are null or non-null. The set is used to determine whether artifact conflict search has to be rerun, and the extra removals that result from this CL could cause fewer artifact conflict searches to run, but the only affected searches would be those that were caused by dirtied configured target values in error all of which were subsequently marked as clean, which is probably rare. -- MOS_MIGRATED_REVID=101144655
generated by cgit v1.2.3 (git 2.39.1) at 2024-07-04 07:55:46 +0000