aboutsummaryrefslogtreecommitdiffhomepage
path: root/src/main/java/com/google/devtools/build/skyframe
Commit message (Collapse)AuthorAge
* Rollback of commit e6191f7209ebf3f67480bc7aa9db5ac58beb2156.Gravatar Damien Martin-Guillerez2016-10-19
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | *** Reason for rollback *** Roll-forward with fix. Tested with `bazel build src:srcs //src/test/...` using a bootstrapped bazel. Fixes #1923. RELNOTES[INC]: Non test-only targets can no longer depends on test-only targets. *** Original change description *** Automated [] rollback of commit a9f20b0d6459d395444c45cf5e94a899f3443633. *** Reason for rollback *** Broke Bazel CI: ERROR: /home/ci/workspace/Bazel/JAVA_VERSION/1.8/PLATFORM_NAME/linux-x86_64/src/BUILD:284:1: in filegroup rule //src:srcs: non-test target '//src:srcs' depends on testonly target '//src/java_tools/junitrunner/java/com/google/testing/junit/junit4:srcs' and doesn't have testonly attribute set. ERROR: Analysis of target '//scripts/packages:packages' failed; build aborted. http://ci.bazel.io/view/Bazel%20b... *** -- MOS_MIGRATED_REVID=136589937
* Fix unnecessary static imports of typesGravatar Liam Miller-Cushon2016-10-18
| | | | | -- MOS_MIGRATED_REVID=136431274
* Stop storing a set in GroupedListHelper to deduplicate SkyKey dep requests. ↵Gravatar Janak Ramakrishnan2016-10-14
| | | | | | | Instead, deduplicate when the helper is actually added to a GroupedList. -- MOS_MIGRATED_REVID=136145321
* EvaluationProgressReceiver absorbs other progress trackers.Gravatar Chloe Calvarin2016-10-06
| | | | | | | | | | Remove DirtyKeyTracker and inflight node tracking as independent progress tracking bits of code, and instead use an internal-facing DirtyTrackingProgressReceiver wrapper of the user's EvaluationProgressReceiver. -- MOS_MIGRATED_REVID=135291924
* Fixed EvaluationProgressReceiver object namesGravatar Googler2016-09-27
| | | | | | | | | | EvaluationProgressReceiver objects have two common naming schemes currently, and calling them invalidationReceiver is misleading, so to make the naming convention standard, all object names are based on "progressReceiver." -- MOS_MIGRATED_REVID=134411011
* Introduce a Builder for ForkJoinQuiescingExecutor.Gravatar Nathan Harmata2016-09-21
| | | | | -- MOS_MIGRATED_REVID=133714902
* Allow reverse dependency lookups on Skyframe graph to throw InteruptedExceptionGravatar Googler2016-09-14
| | | | | -- MOS_MIGRATED_REVID=132999234
* Allow cycle detector to be injected into ParallelEvaluator, so that callers ↵Gravatar Janak Ramakrishnan2016-08-26
| | | | | | | can use different cycle detection algorithms if they wish. -- MOS_MIGRATED_REVID=131347160
* Factor out cycle detection.Gravatar Janak Ramakrishnan2016-08-26
| | | | | -- MOS_MIGRATED_REVID=131344134
* Refactor ParallelEvaluator in preparation for making it more modular with ↵Gravatar Janak Ramakrishnan2016-08-26
| | | | | | | | | | | respect to cycle checking. Reducing the size of ParallelEvaluator.java is also probably long overdue. I believe this change stands on its own, but if you don't think the third change is worth it, and this isn't worth it on its own, feel free to push back. -- MOS_MIGRATED_REVID=131340165
* Avoid copying SkyKeys into a set and then into a map when retrieving them. ↵Gravatar Janak Ramakrishnan2016-08-23
| | | | | | | | | | | | | Instead, just put them directly into a map. This avoids the memory churn and CPU cost of the set. As a result, we have to use HashMaps instead of ImmutableMap.Builders, which I hope is ok (especially since we're not keeping them around), and due to that, we have some nice nondeterminism in the returned order, which matters for some cycle-checking tests. Also, don't use a map at all when we don't need to (when building events). Note that, since we have to deduplicate at some point, this means that changing the return type of SkyFunction.Environment#getValues to not be a random-access map is probably not worth it. Changing the return type of ProcessableGraph#getBatch to not be a random access map might still be worthwhile, although it might require some funny operations. -- MOS_MIGRATED_REVID=131070418
* Allow Skyframe graph lookups and value retrievals to throw InterruptedException.Gravatar Janak Ramakrishnan2016-08-16
| | | | | | | The only place we now don't handle InterruptedException is in the action graph created after analysis, since I'm not sure that will be around for that much longer. -- MOS_MIGRATED_REVID=130327770
* Have AQV propagate the most severe error encountered by any of its tasks. ↵Gravatar Nathan Harmata2016-08-12
| | | | | | | | | Make sure that ParallelEvaluator treats SchedulerExceptions as less severe than other RuntimeExceptions (it already did, but add a comment emphasizing this). The combination of the above means that we don't ignore hypothetical crashes in ParallelEvaluator worker threads _after_ a SchedulerException (e.g. due to an error in nokeep_going evaluation). -- MOS_MIGRATED_REVID=130044230
* Fix bug in ParallelEvaluator when an error dep was not newly requested in a ↵Gravatar Janak Ramakrishnan2016-08-11
| | | | | | | | | nokeep_going build because it finished building in between the time it was first requested and when we checked it for done-ness after the SkyFunction evaluation. This results in an IllegalStateException that gets ignored (and, importantly, not propagated) by AbstractQueueVisitor because AbstractQueueVisitor only records and propagates the first Throwable encountered. For nokeep_going evaluations, this will be the SchedulerException that we use for control flow. The IllegalStateException in question is benign in this case because it's merely from a Preconditions failure and doesn't leave anything in a bad state. It's possible, though, that we have other bugs that are being masked in this way. -- MOS_MIGRATED_REVID=129919336
* Get rid of InvalidatableGraph. This explicit concept is no longer needed.Gravatar Nathan Harmata2016-08-11
| | | | | -- MOS_MIGRATED_REVID=129895423
* Delete NodeEntryField since it's now superfluous in the presence of the new ↵Gravatar Nathan Harmata2016-08-03
| | | | | | | QueryableGraph.Reason which conveys more information. Add a few more Reason enum values to make this refactor benign. -- MOS_MIGRATED_REVID=129118462
* Augment the QueryableGraph#get[BatchWithFieldHints] method to take in ↵Gravatar Nathan Harmata2016-07-27
| | | | | | | parameters conveying the requesting node (if any), the requested node(s), as well as a reason for the skyframe graph lookup. Alternate graph implementations may be interested in this information. -- MOS_MIGRATED_REVID=128496089
* In keep-going mode, don't fail-fast on the evaluation of an individual ↵Gravatar Janak Ramakrishnan2016-07-12
| | | | | | | SkyFunction if it throws an exception but still has missing deps. Instead, pretend it didn't throw, and restart it when its known deps are all done, presumably to throw the same exception. This removes a basic source of non-determinism. -- MOS_MIGRATED_REVID=127129202
* Replace QueryableGraph#getBatch with #getBatchWithFieldHints. This allows ↵Gravatar Janak Ramakrishnan2016-07-11
| | | | | | | alternate graph implementations to optimize how they construct node entries. -- MOS_MIGRATED_REVID=126932020
* Refactor QueryableGraph and ThinNodeQueryableGraph to be independent ↵Gravatar Janak Ramakrishnan2016-07-11
| | | | | | | interfaces, in preparation for further changes. -- MOS_MIGRATED_REVID=126924789
* Refactor BuildingState to save memory.Gravatar Janak Ramakrishnan2016-07-07
| | | | | | | | | Collapse the "evaluating" boolean into the "signaledDeps" int field, since signaledDeps is always 0 if evaluating is false, so we can use the sentinel value -1 to indicate that evaluation has not yet started. This leads to a slightly less tolerant node entry: it must "start evaluating" before you can do things like set its value. Places that wasn't being done have been fixed, at least as far as we have test coverage for. Also, factor the "dirty" parts of BuildingState out into a subclass. It would probably be cleaner to use composition here, but I don't want to pay the price of another object. -- MOS_MIGRATED_REVID=126729331
* Re-use the InMemoryNodeEntry#directDeps field for temporary direct deps.Gravatar Janak Ramakrishnan2016-07-07
| | | | | | | The biggest savings here is that we were not eagerly discarding the InMemoryNodeEntry#directDeps field after an entry was marked dirty, even though we would never read its value again. But rather than just fix that, by getting rid of the field in BuildingState, we can potentially save memory with smaller BuildingState objects as well. -- MOS_MIGRATED_REVID=126709632
* Filter out already-removed deps when removing a node from the rdeps of its ↵Gravatar Janak Ramakrishnan2016-07-04
| | | | | | | last build's deps. This is only an issue with cycles, since there we try to maintain the invariant that a parent is not done before its children by removing its deps on its children before we construct its cycle value. -- MOS_MIGRATED_REVID=126494009
* Batch prefetch now also retrieves the deps from the node's previous run.Gravatar Eric Fellheimer2016-07-01
| | | | | -- MOS_MIGRATED_REVID=126344189
* Minor documentation and visibility tweaks to InMemoryNodeEntry and ↵Gravatar Janak Ramakrishnan2016-06-29
| | | | | | | InvalidatedNodeEntry. -- MOS_MIGRATED_REVID=126139179
* Get rid of boolean field in InMemoryNodeEntry. After adding the ↵Gravatar Janak Ramakrishnan2016-06-28
| | | | | | | lastEvaluated/lastChanged version fields, we lost memory alignment, so this boolean was costing us 8 bytes per instance. -- MOS_MIGRATED_REVID=125998857
* Batchify child node lookups in Skyframe cycle detection.Gravatar Eric Fellheimer2016-06-21
| | | | | -- MOS_MIGRATED_REVID=125376554
* Minor change to add QueryableGraphBackedSkyFunctionEnvironment.Gravatar Janak Ramakrishnan2016-06-21
| | | | | -- MOS_MIGRATED_REVID=125362963
* Minor improvement to cycle detection algorithm: Do not recurse into done ↵Gravatar Eric Fellheimer2016-06-21
| | | | | | | | | | | nodes, where there is no work to do anyway. This triggered some non-determinism that we explicitly workaround in the unit tests. Also add a comment about a potential but unrelated optimization. -- MOS_MIGRATED_REVID=125355303
* Log how long Skyframe cycle checking takes.Gravatar Eric Fellheimer2016-06-15
| | | | | -- MOS_MIGRATED_REVID=124874002
* Remove ability of AbstractQueueVisitor to continue after an interrupt. That ↵Gravatar Janak Ramakrishnan2016-06-15
| | | | | | | functionality was only used in tests. -- MOS_MIGRATED_REVID=124841573
* Use the precomputed skykey hash to short circuit equality comparison.Gravatar Shreya Bhattarai2016-06-14
| | | | | -- MOS_MIGRATED_REVID=124747935
* Allow AQV users to inject arbitrary handling of classified errors.Gravatar Nathan Harmata2016-05-27
| | | | | -- MOS_MIGRATED_REVID=123347295
* Assert batch existence of nodes in DirtyingNodeVisitor#visit.Gravatar Janak Ramakrishnan2016-05-24
| | | | | | | | | | | Now that we batch-prefetch the nodes, there is no reason to delay this check until the async Runnable runs, since we have more debugging information this way. -- Change-Id: Ic73d99ed8de184ba1e29f0dee5375f5d45b5379d Reviewed-on: https://bazel-review.googlesource.com/3680 MOS_MIGRATED_REVID=123018542
* Fix bug in lazy removal of reverse deps -- there was one remaining case ↵Gravatar Janak Ramakrishnan2016-05-18
| | | | | | | where we weren't checking to see if a reverse dep already existed when we declared a reverse dep. -- MOS_MIGRATED_REVID=122581019
* Remove reverse deps lazily, only when the node has finished building and we ↵Gravatar Janak Ramakrishnan2016-05-17
| | | | | | | | | | | discover that it no longer has certain deps. In the common case, where a node's deps do not change in the end, this reduces lock contention and CPU. The downside of this is that we now create a set of the previous reverse deps during each evaluation of a node. We don't store this set in order to conserve memory, so we pay for it in CPU. We will probably only construct it two or three times (most SkyFunctions don't have so many groups), so the cost shouldn't be so high, but we can try to mitigate if it shows up in profiling. -- MOS_MIGRATED_REVID=122566267
* Stop converting temporary direct deps to a set. In almost all cases, this ↵Gravatar Janak Ramakrishnan2016-05-16
| | | | | | | conversion is unnecessary and wasteful. In the remaining cases, the set conversion can be explicit. -- MOS_MIGRATED_REVID=122294939
* Allow Skyframe tests to use the NotifyingGraph and DeterministicGraph ↵Gravatar Janak Ramakrishnan2016-05-11
| | | | | | | helpers to enforce concurrency synchronization points and determinism even if they are not using an InMemoryGraph-backed evaluator. -- MOS_MIGRATED_REVID=121977783
* Add random-access indexing into GroupedList. Intended to be a no-op, ↵Gravatar Janak Ramakrishnan2016-04-28
| | | | | | | allowing for future work that may want to access a particular group in the GroupedList without advancing the iterator. -- MOS_MIGRATED_REVID=120933039
* Make MinimalVersion package-private, since it has no outside consumers.Gravatar Janak Ramakrishnan2016-03-31
| | | | | -- MOS_MIGRATED_REVID=118623130
* Fix keep-going build with an existing cycle by only setting errorDepKey in a ↵Gravatar Janak Ramakrishnan2016-03-29
| | | | | | | non-keep-going build. Setting errorDepKey was only mostly harmless, to mix memes. (Actually, it was quite harmful.) -- MOS_MIGRATED_REVID=118410594
* Fix debugging info when crashing with missing SkyFunction.Gravatar Janak Ramakrishnan2016-03-23
| | | | | -- MOS_MIGRATED_REVID=117894693
* Transform the getBatch result in SkyFunctionEnvironment instead of copying ↵Gravatar Janak Ramakrishnan2016-03-22
| | | | | | | it. The copying showed up as a source of memory spikiness. -- MOS_MIGRATED_REVID=117741939
* Remove the WalkableGraphFactory#afterUse hook, which was only called in ↵Gravatar Nathan Harmata2016-03-21
| | | | | | | SkyQueryEnvironment. QueryEnvironmentFactory, recently introduced by unknown commit, is a much more general purpose mechanism. -- MOS_MIGRATED_REVID=117590252
* Transform the passed-in map in AbstractSkyFunctionEnvironment instead of ↵Gravatar Janak Ramakrishnan2016-03-18
| | | | | | | copying it. The copying showed up as a source of memory spikiness in an OOM. -- MOS_MIGRATED_REVID=117477133
* Fix style issue; convert comment to Javadoc format.Gravatar Ulf Adams2016-03-11
| | | | | | | ...so the docs show up on hover in an IDE. -- MOS_MIGRATED_REVID=116986129
* Use static creation method for SkyKey. This allows interning SkyKeys as they ↵Gravatar Janak Ramakrishnan2016-03-03
| | | | | | | are created, as opposed to when they are requested from the ParallelEvaluator. That delay can lead to large memory spikes and churn. -- MOS_MIGRATED_REVID=116224565
* Improve the error message when the ParallelEvaluator is unable to create a ↵Gravatar Luis Fernando Pino Duque2016-02-28
| | | | | | | SkyFunction -- MOS_MIGRATED_REVID=115671161
* Prevent new evaluations from starting if a done child's error is discovered. ↵Gravatar Janak Ramakrishnan2016-02-23
| | | | | | | Also delete some code that's been dead for a while, now that we eagerly shut down evaluation when we come across a child in error during a fail-fast evaluation. -- MOS_MIGRATED_REVID=115272603
* Make IntVersion package-private, since it isn't used outside of Skyframe core.Gravatar Janak Ramakrishnan2016-02-12
| | | | | -- MOS_MIGRATED_REVID=114488935
generated by cgit v1.2.3 (git 2.39.1) at 2024-07-04 14:58:09 +0000