aboutsummaryrefslogtreecommitdiffhomepage
path: root/src/main/java/com/google/devtools/build/skyframe
Commit message (Collapse)AuthorAge
* Introduce AbstractParallelEvaluatorGravatar mschaller2017-08-07
| | | | | | | | | | This change enables alternate evaluation strategies. Drive-by fix to GraphTester, making TestFunction static because it has no dependencies on its outer class, and adding #unsetConstantValue. RELNOTES: None. PiperOrigin-RevId: 164327055
* Adding a ConstantVersion to Skyframe, to be used when only a single ↵Gravatar kush2017-08-03
| | | | | | | evaluation of the graph with a single constant version is expected RELNOTES: None PiperOrigin-RevId: 164059941
* Add --toolchain_resolution_debug option to give more information aboutGravatar John Cater2017-07-26
| | | | | | | | | toolchain selection. Fixes #3431. Change-Id: Ia38415575b6a121cbb6a028bfc0276691cd11b6d PiperOrigin-RevId: 163196646
* Make SkyKey.argument a default methodGravatar ulfjack2017-07-25
| | | | | | | Almost all implementations simply return this, all of which can be removed now. PiperOrigin-RevId: 163046912
* Make InMemoryGraph public.Gravatar janakr2017-07-24
| | | | PiperOrigin-RevId: 162788157
* Tolerate injected nodes having deps.Gravatar janakr2017-07-21
| | | | | | | | | | We don't check explicitly that these are the only two ways, but this can happen if the error transience node is a dep of a node that's being injected, or if an injected node is an "external" file that needs to depend on an external package. The first possibility can happen if there was an IOException reading the node on the previous build. We handle the situation by just dirtying the node, not injecting it. Actual evaluation can handle the re-stat. PiperOrigin-RevId: 162622092
* Automated rollback of commit 6e72f78e1c2e573787ec862e671f3b3b4c33dc96.Gravatar kush2017-07-20
| | | | | | | | | | | | | | *** Reason for rollback *** Causing TGP issues with tool failures: b/63839245 Was finally able to repro the issue at HEAD, and didn't occur without this change. *** Original change description *** Small changes to skyframe package. RELNOTES: None PiperOrigin-RevId: 162565994
* Now that SkyKey is an interface, accept Iterable<? extends SkyKey>Gravatar ulfjack2017-07-19
| | | | | | | A subsequent CL makes TargetPatternKey implement SkyKey, and it's much nicer if I can pass lists of TargetPatternKey to the various Skyframe APIs. PiperOrigin-RevId: 162359843
* Fix Postable forwarding and replayGravatar ulfjack2017-07-18
| | | | | | | We were previously duplicate-posting Postable events posted to the Skyframe environment. PiperOrigin-RevId: 162323598
* Small changes to skyframe package.Gravatar kush2017-07-18
| | | | PiperOrigin-RevId: 162288376
* A bunch of unrelated cleanups:Gravatar nharmata2017-06-16
| | | | | | | | | | -Have SkylarkImportLookupFunction include causes in the SkyFunctionExceptions it throws. -Better transitive skyframe error declarations in ASTFileLookupFunction. -Have ErrorInfo differentiate between direct and transitive transience. -Introduce ErrorInfoManager and have ParallelEvaluator/ParallelEvaluatorContext use it. RELNOTES: None PiperOrigin-RevId: 159163186
* Propagate postable events further upGravatar Klaus Aehlig2017-06-14
| | | | | | | | | | | | With the introduction of the ExtendedEventHandler, SkyFunctions were given the possibility to post additional Postable events in addition to the standard events. As SkyFunctions have to be restartable, events are collected first and only posted after the function is finished. Make sure that this also applies to postable events and they are not dropped. Change-Id: Ie1c3a0134935c75ea984fa2cc924e7327a9da81f PiperOrigin-RevId: 158964337
* Also report stored posts further up in SkyframeGravatar Klaus Aehlig2017-05-31
| | | | | | | | | | As SkyFunctions are supposed to be restartable, e.g., if prerequisites are missing, events generated by them are temporarily stored. So are BuildEvents posted by them. However, once the evaluation of a SkyFunction is finalized, the stored posts need to be reposted. Do this. Change-Id: I4ce20266fbfcbb298e93eb53086fa9916874f5d8 PiperOrigin-RevId: 157575119
* Avoid waste in registerNewlyDiscovered... when there aren't anyGravatar mschaller2017-05-22
| | | | | RELNOTES: None. PiperOrigin-RevId: 156553687
* Add a EvaluationProgressReceiver#computing method.Gravatar nharmata2017-05-11
| | | | | RELNOTES: None PiperOrigin-RevId: 155665128
* Introduce CompoundEvaluationProgressReceiver for combining multiple ↵Gravatar nharmata2017-05-10
| | | | | | | EvaluationProgressReceivers. RELNOTES: None PiperOrigin-RevId: 155542146
* Introduce an EvaluationProgressReceiver implementation that does nothing.Gravatar nharmata2017-05-10
| | | | | | | | This is useful for dealing with all the existing implementations in the face of interface changes that are irrelevant. RELNOTES: None PiperOrigin-RevId: 155525021
* Clean up AbstractQueueVisitor's constructors.Gravatar janakr2017-05-09
| | | | | | | | The "concurrent" bit was supposedly around for testing purposes, but who knows if it even works anymore. Making other callsites explicitly state their ErrorClassifier gets us down to two constructors, one of which can delegate to the other. I think having both these constructors is useful because there's a linkage between creating a new executor service and specifying that the AQV should shut down the service at the end of the visitation. And using a static create() method doesn't work because of AQV's inheritance model. PiperOrigin-RevId: 155406771
* Make SkyKey an interface, and start the migration of not creating SkyKey ↵Gravatar janakr2017-05-04
| | | | | | wrapper objects: for OwnedArtifacts, which are the most numerous during builds, and for Labels for TransitiveTraversalValues, which are the most numerous during queries. PiperOrigin-RevId: 154989520
* Allow graph implementations to filter out deps that are known to be done ↵Gravatar janakr2017-04-10
| | | | | | when change pruning. This can speed up change pruning. PiperOrigin-RevId: 152538144
* Drop loading-phase values if --discard_analysis_cache is true and we're not ↵Gravatar janakr2017-03-31
| | | | | | keeping incremental state. PiperOrigin-RevId: 151639711
* Remove BuildingState, since it only has one field. Instead, keep the ↵Gravatar Janak Ramakrishnan2017-03-27
| | | | | | | | | | signaledDeps field directly in InMemoryNodeEntry. Should save ~24 bytes per freshly evaluating node entry (I haven't calculated object alignment for InMemoryNodeEntry now, so could be more or less). Also might save some memory for re-evaluating node entries, since the BuildingState class had to be padded out to a multiple of 8 bytes before the DirtyBuildingState fields could start. Don't actually know if that was happening. -- PiperOrigin-RevId: 151138224 MOS_MIGRATED_REVID=151138224
* Stop storing reverse deps to signal in BuildingState. Instead, re-use the ↵Gravatar Janak Ramakrishnan2017-03-24
| | | | | | | | | | | | | | reverseDepsToConsolidate field in InMemoryNodeEntry. As part of that, revamp our logic of how we store pending operations: store adds bare on initial evaluations, and checks bare on incremental evaluations and operations on done nodes. This should improve performance in two ways: BuildingState loses two fields, saving working memory intra-build. Storing pending reverse dep operations bare also saves memory intra-build. Note that neither of these changes helps resting memory state, only while a node is still evaluating. Because of this, we can simplify ReverseDepsUtil a bit, making ReverseDepsUtilImpl a static class, which it always wanted to be (what it really wants to be is a superclass of InMemoryNodeEntry, but I don't want to spend the object alignment bits). Finally, this makes it pretty tempting to get rid of BuildingState altogether on initial evaluations. We'd still keep DirtyBuildingState, but we could save another ~24 bytes by storing BuildingState's one remaining field, signaledDeps, directly inside InMemoryNodeEntry. -- PiperOrigin-RevId: 151048879 MOS_MIGRATED_REVID=151048879
* Tighten invariants around dirtiness checking. We should never need to create ↵Gravatar Janak Ramakrishnan2017-03-06
| | | | | | | | a child that's being checked by a parent (that was a legacy of when we delegated to enqueueChild), and such a child that is being checked should always be dirty or done, never fresh. -- PiperOrigin-RevId: 149136909 MOS_MIGRATED_REVID=149136909
* Add some more debugging info in case of rdep inconsistencies in Skyframe -- ↵Gravatar Janak Ramakrishnan2017-03-03
| | | | | | | | the full list of rdep mods that are being performed on this entry, not just the current one that failed. -- PiperOrigin-RevId: 149055655 MOS_MIGRATED_REVID=149055655
* In SkyFunctionEnvironment post progress-like events immediately Gravatar Klaus Aehlig2017-02-28
| | | | | | | | | | | | | In Skyframe Evaluation, events reporting about progress are meant to be seen immediately by the environment. More over, they are not to be replayed when taking this SkyFunction out of cache. So change the custom implementation of the StoredEventHandler in the SkyFunctionEnvironment to forward appropriately. -- Change-Id: I30baf0088595ef684f16270f11668e980e65aa41 Reviewed-on: https://cr.bazel.build/9112 PiperOrigin-RevId: 148766400 MOS_MIGRATED_REVID=148766400
* Provide more reporting options to SkyFunctions Gravatar Klaus Aehlig2017-02-27
| | | | | | | | | | | | | With more specific information to be reported by Skyfunctions, e.g., to inform the build-event protocol on missing files, the EventHandler interface is no longer enough. Therefore, provide an enriched context for reporting events. -- Change-Id: I2d06166fe4d5b9054e24ad8c752fafc039e3f9f8 Reviewed-on: https://cr.bazel.build/8794 PiperOrigin-RevId: 148463437 MOS_MIGRATED_REVID=148463437
* Inform progress receiver that node has been computed before any exception ↵Gravatar Janak Ramakrishnan2017-02-13
| | | | | | | | handling. -- PiperOrigin-RevId: 147185674 MOS_MIGRATED_REVID=147185674
* Add "CLEAN_UP" as a reason for retrieving a node.Gravatar Shreya Bhattarai2017-02-09
| | | | | | -- PiperOrigin-RevId: 146982656 MOS_MIGRATED_REVID=146982656
* Refactor DirtyBuildingState to allow custom implementation on storing theGravatar Googler2017-02-01
| | | | | | | | dependencies used in last build -- PiperOrigin-RevId: 146169454 MOS_MIGRATED_REVID=146169454
* Fix an issue with cached EventHandlers in SkyQueryEnvironment's resolverGravatar Mark Schaller2017-01-23
| | | | | | | | | | | | Because the event handler's inner handlers are removed after each query command, caching the handler caused a subset of subsequent commands' errors (those reported through the resolver's handler) to go unlogged. Also fix a bug with cycle detection in DelegatingWalkableGraph. -- PiperOrigin-RevId: 145124271 MOS_MIGRATED_REVID=145124271
* Add some logging in case we fail an evaluation. This should be generally ↵Gravatar Janak Ramakrishnan2017-01-13
| | | | | | | | useful, but may also help identify Bazel as the culprit in the linked bug if we're lucky and the timestamps are clear. -- PiperOrigin-RevId: 144359628 MOS_MIGRATED_REVID=144359628
* Unsynchronize InMemoryNodeEntry#isDone. The buildingState variable's state ↵Gravatar Shreya Bhattarai2017-01-09
| | | | | | | | changes are visibile to all other threads by volatility and we don't read the variable at any intermediate or inconsistent state (simply check against null or a constant). reverseDepsToSignal is also made volatile for subclasses that need volatile reads to it. -- PiperOrigin-RevId: 143787032 MOS_MIGRATED_REVID=143787032
* Remove WalkableGraph#exists and allow WalkableGraph#getValue and ↵Gravatar Janak Ramakrishnan2017-01-03
| | | | | | | | WalkableGraph#getException to be given non-existent keys without crashing. Add WalkableGraph#isCycle to fill the gap in testing for the difference between non-existence and depending on a cycle. -- PiperOrigin-RevId: 143205289 MOS_MIGRATED_REVID=143205289
* Global cleanup change.Gravatar Googler2016-12-07
| | | | | | -- PiperOrigin-RevId: 141307853 MOS_MIGRATED_REVID=141307853
* Introduce BlazeInterners, a Blaze-specific wrapper around Guava's Interners ↵Gravatar Nathan Harmata2016-11-24
| | | | | | | | | | | that makes an appropriate call to Interners.InternerBuilder#concurrencyLevel. For current readers of this CL, I used this class everywhere in the Blaze codebase. For future readers of this CL, this class should be used to create an Interner in the Blaze codebase. -- MOS_MIGRATED_REVID=140063271
* Cache all previously computed values in ↵Gravatar Janak Ramakrishnan2016-11-17
| | | | | | | | | SkyQueryEnvironment#beforeEvaluateQuery if possible to save on latency for small queries. This assumes that if the graph is up to date, then the data in SkyQueryEnvironment is also up to date. It also assumes that a ForkJoinPool remains usable until #shutdownNow is called. -- MOS_MIGRATED_REVID=139386363
* Re-use the precomputed universe key in WalkableGraphFactory#prepareAndGet.Gravatar Janak Ramakrishnan2016-11-17
| | | | | -- MOS_MIGRATED_REVID=139357569
* Add method getCurrentlyAvailableNodes to QueryableGraph and Walkable GraphGravatar Googler2016-10-31
| | | | | | | | | | It allows all graph implementations to return the list of nodes which are immediately available to be fetched. NOTE: Not-currently-available here does not mean the nodes do not exist in the graph. It simply means they are not ready to be fetched immediately yet. -- MOS_MIGRATED_REVID=137701432
* Rollback of commit e6191f7209ebf3f67480bc7aa9db5ac58beb2156.Gravatar Damien Martin-Guillerez2016-10-19
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | *** Reason for rollback *** Roll-forward with fix. Tested with `bazel build src:srcs //src/test/...` using a bootstrapped bazel. Fixes #1923. RELNOTES[INC]: Non test-only targets can no longer depends on test-only targets. *** Original change description *** Automated [] rollback of commit a9f20b0d6459d395444c45cf5e94a899f3443633. *** Reason for rollback *** Broke Bazel CI: ERROR: /home/ci/workspace/Bazel/JAVA_VERSION/1.8/PLATFORM_NAME/linux-x86_64/src/BUILD:284:1: in filegroup rule //src:srcs: non-test target '//src:srcs' depends on testonly target '//src/java_tools/junitrunner/java/com/google/testing/junit/junit4:srcs' and doesn't have testonly attribute set. ERROR: Analysis of target '//scripts/packages:packages' failed; build aborted. http://ci.bazel.io/view/Bazel%20b... *** -- MOS_MIGRATED_REVID=136589937
* Fix unnecessary static imports of typesGravatar Liam Miller-Cushon2016-10-18
| | | | | -- MOS_MIGRATED_REVID=136431274
* Stop storing a set in GroupedListHelper to deduplicate SkyKey dep requests. ↵Gravatar Janak Ramakrishnan2016-10-14
| | | | | | | Instead, deduplicate when the helper is actually added to a GroupedList. -- MOS_MIGRATED_REVID=136145321
* EvaluationProgressReceiver absorbs other progress trackers.Gravatar Chloe Calvarin2016-10-06
| | | | | | | | | | Remove DirtyKeyTracker and inflight node tracking as independent progress tracking bits of code, and instead use an internal-facing DirtyTrackingProgressReceiver wrapper of the user's EvaluationProgressReceiver. -- MOS_MIGRATED_REVID=135291924
* Fixed EvaluationProgressReceiver object namesGravatar Googler2016-09-27
| | | | | | | | | | EvaluationProgressReceiver objects have two common naming schemes currently, and calling them invalidationReceiver is misleading, so to make the naming convention standard, all object names are based on "progressReceiver." -- MOS_MIGRATED_REVID=134411011
* Introduce a Builder for ForkJoinQuiescingExecutor.Gravatar Nathan Harmata2016-09-21
| | | | | -- MOS_MIGRATED_REVID=133714902
* Allow reverse dependency lookups on Skyframe graph to throw InteruptedExceptionGravatar Googler2016-09-14
| | | | | -- MOS_MIGRATED_REVID=132999234
* Allow cycle detector to be injected into ParallelEvaluator, so that callers ↵Gravatar Janak Ramakrishnan2016-08-26
| | | | | | | can use different cycle detection algorithms if they wish. -- MOS_MIGRATED_REVID=131347160
* Factor out cycle detection.Gravatar Janak Ramakrishnan2016-08-26
| | | | | -- MOS_MIGRATED_REVID=131344134
* Refactor ParallelEvaluator in preparation for making it more modular with ↵Gravatar Janak Ramakrishnan2016-08-26
| | | | | | | | | | | respect to cycle checking. Reducing the size of ParallelEvaluator.java is also probably long overdue. I believe this change stands on its own, but if you don't think the third change is worth it, and this isn't worth it on its own, feel free to push back. -- MOS_MIGRATED_REVID=131340165
* Avoid copying SkyKeys into a set and then into a map when retrieving them. ↵Gravatar Janak Ramakrishnan2016-08-23
| | | | | | | | | | | | | Instead, just put them directly into a map. This avoids the memory churn and CPU cost of the set. As a result, we have to use HashMaps instead of ImmutableMap.Builders, which I hope is ok (especially since we're not keeping them around), and due to that, we have some nice nondeterminism in the returned order, which matters for some cycle-checking tests. Also, don't use a map at all when we don't need to (when building events). Note that, since we have to deduplicate at some point, this means that changing the return type of SkyFunction.Environment#getValues to not be a random-access map is probably not worth it. Changing the return type of ProcessableGraph#getBatch to not be a random access map might still be worthwhile, although it might require some funny operations. -- MOS_MIGRATED_REVID=131070418
generated by cgit v1.2.3 (git 2.39.1) at 2024-07-02 15:50:33 +0000