aboutsummaryrefslogtreecommitdiffhomepage
path: root/src/test/java/com/google/devtools/build/skyframe
Commit message (Collapse)AuthorAge
* Make MemoizingEvaluatorTest#nodeInvalidatedThenDoubleCycle a bit more complete.Gravatar Janak Ramakrishnan2016-09-01
| | | | | -- MOS_MIGRATED_REVID=131944129
* Allow cycle detector to be injected into ParallelEvaluator, so that callers ↵Gravatar Janak Ramakrishnan2016-08-26
| | | | | | | can use different cycle detection algorithms if they wish. -- MOS_MIGRATED_REVID=131347160
* Refactor ParallelEvaluator in preparation for making it more modular with ↵Gravatar Janak Ramakrishnan2016-08-26
| | | | | | | | | | | respect to cycle checking. Reducing the size of ParallelEvaluator.java is also probably long overdue. I believe this change stands on its own, but if you don't think the third change is worth it, and this isn't worth it on its own, feel free to push back. -- MOS_MIGRATED_REVID=131340165
* Avoid copying SkyKeys into a set and then into a map when retrieving them. ↵Gravatar Janak Ramakrishnan2016-08-23
| | | | | | | | | | | | | Instead, just put them directly into a map. This avoids the memory churn and CPU cost of the set. As a result, we have to use HashMaps instead of ImmutableMap.Builders, which I hope is ok (especially since we're not keeping them around), and due to that, we have some nice nondeterminism in the returned order, which matters for some cycle-checking tests. Also, don't use a map at all when we don't need to (when building events). Note that, since we have to deduplicate at some point, this means that changing the return type of SkyFunction.Environment#getValues to not be a random-access map is probably not worth it. Changing the return type of ProcessableGraph#getBatch to not be a random access map might still be worthwhile, although it might require some funny operations. -- MOS_MIGRATED_REVID=131070418
* Allow Skyframe graph lookups and value retrievals to throw InterruptedException.Gravatar Janak Ramakrishnan2016-08-16
| | | | | | | The only place we now don't handle InterruptedException is in the action graph created after analysis, since I'm not sure that will be around for that much longer. -- MOS_MIGRATED_REVID=130327770
* Fix bug in ParallelEvaluator when an error dep was not newly requested in a ↵Gravatar Janak Ramakrishnan2016-08-11
| | | | | | | | | nokeep_going build because it finished building in between the time it was first requested and when we checked it for done-ness after the SkyFunction evaluation. This results in an IllegalStateException that gets ignored (and, importantly, not propagated) by AbstractQueueVisitor because AbstractQueueVisitor only records and propagates the first Throwable encountered. For nokeep_going evaluations, this will be the SchedulerException that we use for control flow. The IllegalStateException in question is benign in this case because it's merely from a Preconditions failure and doesn't leave anything in a bad state. It's possible, though, that we have other bugs that are being masked in this way. -- MOS_MIGRATED_REVID=129919336
* Get rid of InvalidatableGraph. This explicit concept is no longer needed.Gravatar Nathan Harmata2016-08-11
| | | | | -- MOS_MIGRATED_REVID=129895423
* Global cleanup change.Gravatar Googler2016-08-03
| | | | | -- MOS_MIGRATED_REVID=129226221
* Delete NodeEntryField since it's now superfluous in the presence of the new ↵Gravatar Nathan Harmata2016-08-03
| | | | | | | QueryableGraph.Reason which conveys more information. Add a few more Reason enum values to make this refactor benign. -- MOS_MIGRATED_REVID=129118462
* Augment the QueryableGraph#get[BatchWithFieldHints] method to take in ↵Gravatar Nathan Harmata2016-07-27
| | | | | | | parameters conveying the requesting node (if any), the requested node(s), as well as a reason for the skyframe graph lookup. Alternate graph implementations may be interested in this information. -- MOS_MIGRATED_REVID=128496089
* In keep-going mode, don't fail-fast on the evaluation of an individual ↵Gravatar Janak Ramakrishnan2016-07-12
| | | | | | | SkyFunction if it throws an exception but still has missing deps. Instead, pretend it didn't throw, and restart it when its known deps are all done, presumably to throw the same exception. This removes a basic source of non-determinism. -- MOS_MIGRATED_REVID=127129202
* Replace QueryableGraph#getBatch with #getBatchWithFieldHints. This allows ↵Gravatar Janak Ramakrishnan2016-07-11
| | | | | | | alternate graph implementations to optimize how they construct node entries. -- MOS_MIGRATED_REVID=126932020
* Refactor QueryableGraph and ThinNodeQueryableGraph to be independent ↵Gravatar Janak Ramakrishnan2016-07-11
| | | | | | | interfaces, in preparation for further changes. -- MOS_MIGRATED_REVID=126924789
* Refactor BuildingState to save memory.Gravatar Janak Ramakrishnan2016-07-07
| | | | | | | | | Collapse the "evaluating" boolean into the "signaledDeps" int field, since signaledDeps is always 0 if evaluating is false, so we can use the sentinel value -1 to indicate that evaluation has not yet started. This leads to a slightly less tolerant node entry: it must "start evaluating" before you can do things like set its value. Places that wasn't being done have been fixed, at least as far as we have test coverage for. Also, factor the "dirty" parts of BuildingState out into a subclass. It would probably be cleaner to use composition here, but I don't want to pay the price of another object. -- MOS_MIGRATED_REVID=126729331
* Filter out already-removed deps when removing a node from the rdeps of its ↵Gravatar Janak Ramakrishnan2016-07-04
| | | | | | | last build's deps. This is only an issue with cycles, since there we try to maintain the invariant that a parent is not done before its children by removing its deps on its children before we construct its cycle value. -- MOS_MIGRATED_REVID=126494009
* Add all the sources to //:srcs filegroup and add a check to detectGravatar Damien Martin-Guillerez2016-07-01
| | | | | | | | | | | missing file to it. We need to activate this check on presubmits -- Change-Id: Ia95e92d3816ce92bb69bc0e2cf56e9c60b68d970 Reviewed-on: https://bazel-review.googlesource.com/#/c/3949/ MOS_MIGRATED_REVID=126404792
* Get rid of boolean field in InMemoryNodeEntry. After adding the ↵Gravatar Janak Ramakrishnan2016-06-28
| | | | | | | lastEvaluated/lastChanged version fields, we lost memory alignment, so this boolean was costing us 8 bytes per instance. -- MOS_MIGRATED_REVID=125998857
* Add test in GraphConcurrencyTest for concurrent createIfAbsent and get calls.Gravatar Janak Ramakrishnan2016-06-24
| | | | | -- MOS_MIGRATED_REVID=125703258
* Minor change to add QueryableGraphBackedSkyFunctionEnvironment.Gravatar Janak Ramakrishnan2016-06-21
| | | | | -- MOS_MIGRATED_REVID=125362963
* Minor improvement to cycle detection algorithm: Do not recurse into done ↵Gravatar Eric Fellheimer2016-06-21
| | | | | | | | | | | nodes, where there is no work to do anyway. This triggered some non-determinism that we explicitly workaround in the unit tests. Also add a comment about a potential but unrelated optimization. -- MOS_MIGRATED_REVID=125355303
* Add toString for NotifyingNodeEntry -- we lost this in commit ↵Gravatar Janak Ramakrishnan2016-06-16
| | | | | | | 8be7fd0bb7cc8f0819f23ef4bbf6328472110db7 when NotifyingNodeEntry stopped inheriting from InMemoryNodeEntry. -- MOS_MIGRATED_REVID=124960351
* Add EvaluationResultSubject#hasReverseDepsInGraphThat method.Gravatar Janak Ramakrishnan2016-05-27
| | | | | -- MOS_MIGRATED_REVID=123251531
* Fix bug in lazy removal of reverse deps -- there was one remaining case ↵Gravatar Janak Ramakrishnan2016-05-18
| | | | | | | where we weren't checking to see if a reverse dep already existed when we declared a reverse dep. -- MOS_MIGRATED_REVID=122581019
* Remove reverse deps lazily, only when the node has finished building and we ↵Gravatar Janak Ramakrishnan2016-05-17
| | | | | | | | | | | discover that it no longer has certain deps. In the common case, where a node's deps do not change in the end, this reduces lock contention and CPU. The downside of this is that we now create a set of the previous reverse deps during each evaluation of a node. We don't store this set in order to conserve memory, so we pay for it in CPU. We will probably only construct it two or three times (most SkyFunctions don't have so many groups), so the cost shouldn't be so high, but we can try to mitigate if it shows up in profiling. -- MOS_MIGRATED_REVID=122566267
* Stop converting temporary direct deps to a set. In almost all cases, this ↵Gravatar Janak Ramakrishnan2016-05-16
| | | | | | | conversion is unnecessary and wasteful. In the remaining cases, the set conversion can be explicit. -- MOS_MIGRATED_REVID=122294939
* Allow Skyframe tests to use the NotifyingGraph and DeterministicGraph ↵Gravatar Janak Ramakrishnan2016-05-11
| | | | | | | helpers to enforce concurrency synchronization points and determinism even if they are not using an InMemoryGraph-backed evaluator. -- MOS_MIGRATED_REVID=121977783
* Add MemoizingEvaluatorTest#breakCycle to check that we have correct behavior ↵Gravatar Janak Ramakrishnan2016-05-11
| | | | | | | in this situation. -- MOS_MIGRATED_REVID=121971860
* Catch package lookup exceptions during AST lookupGravatar Kristina Chodorow2016-05-06
| | | | | | | | | Noticed this while debugging #1228. Referencing a non-existent (BuildFileNotFoundException) package would print "Loading failed; build aborted" with no error message about what had actually gone wrong. -- MOS_MIGRATED_REVID=121602225
* Fix keep-going build with an existing cycle by only setting errorDepKey in a ↵Gravatar Janak Ramakrishnan2016-03-29
| | | | | | | non-keep-going build. Setting errorDepKey was only mostly harmless, to mix memes. (Actually, it was quite harmful.) -- MOS_MIGRATED_REVID=118410594
* Return ErrorInfoSubject for EvaluationResultSubject#hasErrorEntryForKeyThat ↵Gravatar Janak Ramakrishnan2016-03-28
| | | | | | | instead of DefaultSubject. -- MOS_MIGRATED_REVID=118261285
* Clean up some testing code: @Before methods' call order is unspecified, so ↵Gravatar Janak Ramakrishnan2016-03-28
| | | | | | | we could get an NPE if one initializes a data structure the other uses. Also delete an unused field and do a small visibility drive-by. -- MOS_MIGRATED_REVID=118248373
* Add unsetComputedValue method to GraphTesterGravatar Mark Schaller2016-03-08
| | | | | | | Useful for manipulating computed value nodes in tests. -- MOS_MIGRATED_REVID=116555900
* Clean up some testing code: don't have following tests fail from the ↵Gravatar Janak Ramakrishnan2016-03-08
| | | | | | | TrackingAwaiter if one test fails, and make the Preconditions check in GraphTester a bit more informative. -- MOS_MIGRATED_REVID=116391607
* Use static creation method for SkyKey. This allows interning SkyKeys as they ↵Gravatar Janak Ramakrishnan2016-03-03
| | | | | | | are created, as opposed to when they are requested from the ParallelEvaluator. That delay can lead to large memory spikes and churn. -- MOS_MIGRATED_REVID=116224565
* Prevent new evaluations from starting if a done child's error is discovered. ↵Gravatar Janak Ramakrishnan2016-02-23
| | | | | | | Also delete some code that's been dead for a while, now that we eagerly shut down evaluation when we come across a child in error during a fail-fast evaluation. -- MOS_MIGRATED_REVID=115272603
* Allow for more flexibility in choices of Version for the ↵Gravatar Shreya Bhattarai2016-02-11
| | | | | | | GraphConcurrencyTest by allowing us to set an initial version, as well as a method to return the version's successor. -- MOS_MIGRATED_REVID=114367424
* Allow ParallelEvaluator to not store errors alongside values if nodes ↵Gravatar Janak Ramakrishnan2016-02-11
| | | | | | | recover from errors. In the case of a single keep_going build, with no subsequent nokeep_going builds, storing the errors is unnecessary. -- MOS_MIGRATED_REVID=114355846
* Don't check direct deps when doing change pruning. Since dependents of a ↵Gravatar Janak Ramakrishnan2016-02-09
| | | | | | | node don't have access to its dependencies ("grandparents don't know grandchildren", or vice versa, depending on your point of view), changes to a node's dependencies can't affect downstream nodes. -- MOS_MIGRATED_REVID=114143894
* Add Truth subject for ErrorInfo and improve one for EvaluationResult.Gravatar Janak Ramakrishnan2016-02-05
| | | | | -- MOS_MIGRATED_REVID=113911470
* Intern IntVersions.Gravatar Janak Ramakrishnan2016-01-28
| | | | | -- MOS_MIGRATED_REVID=113197641
* Fix SkyframeBuilder's error handling, and change the contract of ↵Gravatar Janak Ramakrishnan2016-01-28
| | | | | | | | | | | EvaluationResult to return true for hasError() iff errorMap is non-empty or there is a catastrophe. There was no good reason for the previous behavior of saying hasError even if there was a transitive recovered-from error, since callers shouldn't care. This was a latent bug that was only benign since none of the consumers of hasError were invoking Skyframe with recoverable SkyFunctions. Also add an EvaluationResultSubject so that tests can more fluently assert things about EvaluationResult objects going forward. -- MOS_MIGRATED_REVID=113192415
* Set test_class for java_test test target where it is neededGravatar Damien Martin-Guillerez2016-01-19
| | | | | | | | | This make those test target compatible with --nolegacy_bazel_java_test. -- Change-Id: I2316c9aa53327b417ecce5fd5dab95ec726da11d Reviewed-on: https://bazel-review.googlesource.com/#/c/2690 MOS_MIGRATED_REVID=112446514
* Add catastrophe field to EvaluationResult so that callers can identify the ↵Gravatar Janak Ramakrishnan2016-01-04
| | | | | | | | | cause of a catastrophic failure (this is distinct from a crash). Also clean up catastrophe logic in ParallelEvaluator -- the catastrophic nature of an exception is important only if the build is keep_going, and only if the exception is catastrophic can we have an exception in the first place. -- MOS_MIGRATED_REVID=111293164
* Use Bazel Preconditions variant which avoids varargs array creationGravatar Mark Schaller2015-12-10
| | | | | | | Reduces garbage. -- MOS_MIGRATED_REVID=109914243
* Repurpose the not-meaningfully-used ErrorInfo#isTransient to mean "is ↵Gravatar Nathan Harmata2015-12-10
| | | | | | | | | transitively transient". Some followup changes will use this method. Previously, ErrorInfo#isTransient was only used internally in ParallelEvaluator; I think this method was originally added to ErrorInfo solely for the sake of convenience. -- MOS_MIGRATED_REVID=109840031
* Extract ReverseDepsUtil interface so that InMemoryNodeEntry can be partially ↵Gravatar Janak Ramakrishnan2015-11-24
| | | | | | | isolated from implementation details. -- MOS_MIGRATED_REVID=108523104
* Tolerate independent cycle underneath main cycle during a re-evaluation.Gravatar Janak Ramakrishnan2015-11-24
| | | | | | | With the new-ish behavior of change pruning, where nodes transition from NEEDS_REBUILDING to REBUILDING, we need to make that transition unconditionally for dirty nodes -- being ready after unfinished deps were removed is irrelevant. -- MOS_MIGRATED_REVID=108508979
* Allow SkyValues to be marked not "comparable". Such values are not compared ↵Gravatar Eric Fellheimer2015-11-19
| | | | | | | for the purpose of change pruning. -- MOS_MIGRATED_REVID=108203369
* Avoid re-evaluating a parent node when a child is found to be unchanged from ↵Gravatar Janak Ramakrishnan2015-11-19
| | | | | | | | | | | an earlier version at which the child changed but the parent did not. Concrete scenario: Parent depends on Child. We first evaluate at version v1, Child has value A1, Parent has value B1. We then evaluate at version v2, which changes a dependency of Child. Child has value A2, and Child.getVersion() returns v2. Parent re-evaluates to B1, so is unchanged. Parent.getVersion() returns v1. Now evaluate at version v3, which also changes a dependency of Child. Child re-evaluates to A2, so Child.getVersion() returns v2. If we signal Parent with v2 and Parent only knows that it is at version v1, then Parent must unnecessarily re-evaluate. To fix this, we store an additional version in the entry -- the version at which the node was last evaluated, even if the evaluation did not result in a new value. Parent can then compare that version to its children's versions. If that version is at least as recent as their versions, it knows that the result of evaluating will be the same as it was at that last evaluated version, which is its current value. An alternative solution might be to just signal the parent with a boolean, saying whether or not the child was changed on this evaluation. However, this would be incorrect in the scenario above, with the modification that in the second evaluation, the user just requests the value of Child -- Parent is not updated. In that case, during the third evaluation, Child would report that it was not changed during this evaluation, but we must still re-evaluate Parent since it has not yet picked up the value of Child from the earlier build. -- MOS_MIGRATED_REVID=108163443
* Compare GroupedLists without regard to the order within a given group. Also ↵Gravatar Janak Ramakrishnan2015-11-19
| | | | | | | make sure that we don't store duplicate elements within a given group (although that is currently taken care of by the callers). -- MOS_MIGRATED_REVID=108155105
generated by cgit v1.2.3 (git 2.39.1) at 2024-08-01 14:07:32 +0000