| Commit message (Collapse) | Author | Age |
|
|
|
|
|
|
|
|
|
| |
As SkyFunctions are supposed to be restartable, e.g., if prerequisites are
missing, events generated by them are temporarily stored. So are BuildEvents
posted by them. However, once the evaluation of a SkyFunction is finalized,
the stored posts need to be reposted. Do this.
Change-Id: I4ce20266fbfcbb298e93eb53086fa9916874f5d8
PiperOrigin-RevId: 157575119
|
|
|
|
|
| |
RELNOTES: None.
PiperOrigin-RevId: 156553687
|
|
|
|
|
| |
RELNOTES: None
PiperOrigin-RevId: 155665128
|
|
|
|
|
|
|
| |
EvaluationProgressReceivers.
RELNOTES: None
PiperOrigin-RevId: 155542146
|
|
|
|
|
|
|
|
| |
This is useful for dealing with all the existing implementations in the face of
interface changes that are irrelevant.
RELNOTES: None
PiperOrigin-RevId: 155525021
|
|
|
|
|
|
|
|
| |
The "concurrent" bit was supposedly around for testing purposes, but who knows if it even works anymore. Making other callsites explicitly state their ErrorClassifier gets us down to two constructors, one of which can delegate to the other.
I think having both these constructors is useful because there's a linkage between creating a new executor service and specifying that the AQV should shut down the service at the end of the visitation. And using a static create() method doesn't work because of AQV's inheritance model.
PiperOrigin-RevId: 155406771
|
|
|
|
|
|
| |
wrapper objects: for OwnedArtifacts, which are the most numerous during builds, and for Labels for TransitiveTraversalValues, which are the most numerous during queries.
PiperOrigin-RevId: 154989520
|
|
|
|
|
|
| |
when change pruning. This can speed up change pruning.
PiperOrigin-RevId: 152538144
|
|
|
|
|
|
| |
keeping incremental state.
PiperOrigin-RevId: 151639711
|
|
|
|
|
|
|
|
|
|
| |
signaledDeps field directly in InMemoryNodeEntry.
Should save ~24 bytes per freshly evaluating node entry (I haven't calculated object alignment for InMemoryNodeEntry now, so could be more or less). Also might save some memory for re-evaluating node entries, since the BuildingState class had to be padded out to a multiple of 8 bytes before the DirtyBuildingState fields could start. Don't actually know if that was happening.
--
PiperOrigin-RevId: 151138224
MOS_MIGRATED_REVID=151138224
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
reverseDepsToConsolidate field in InMemoryNodeEntry. As part of that, revamp our logic of how we store pending operations: store adds bare on initial evaluations, and checks bare on incremental evaluations and operations on done nodes.
This should improve performance in two ways: BuildingState loses two fields, saving working memory intra-build. Storing pending reverse dep operations bare also saves memory intra-build. Note that neither of these changes helps resting memory state, only while a node is still evaluating.
Because of this, we can simplify ReverseDepsUtil a bit, making ReverseDepsUtilImpl a static class, which it always wanted to be (what it really wants to be is a superclass of InMemoryNodeEntry, but I don't want to spend the object alignment bits).
Finally, this makes it pretty tempting to get rid of BuildingState altogether on initial evaluations. We'd still keep DirtyBuildingState, but we could save another ~24 bytes by storing BuildingState's one remaining field, signaledDeps, directly inside InMemoryNodeEntry.
--
PiperOrigin-RevId: 151048879
MOS_MIGRATED_REVID=151048879
|
|
|
|
|
|
|
|
| |
a child that's being checked by a parent (that was a legacy of when we delegated to enqueueChild), and such a child that is being checked should always be dirty or done, never fresh.
--
PiperOrigin-RevId: 149136909
MOS_MIGRATED_REVID=149136909
|
|
|
|
|
|
|
|
| |
the full list of rdep mods that are being performed on this entry, not just the current one that failed.
--
PiperOrigin-RevId: 149055655
MOS_MIGRATED_REVID=149055655
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
In Skyframe Evaluation, events reporting about progress are meant to be seen
immediately by the environment. More over, they are not to be replayed when
taking this SkyFunction out of cache. So change the custom implementation of the
StoredEventHandler in the SkyFunctionEnvironment to forward appropriately.
--
Change-Id: I30baf0088595ef684f16270f11668e980e65aa41
Reviewed-on: https://cr.bazel.build/9112
PiperOrigin-RevId: 148766400
MOS_MIGRATED_REVID=148766400
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
With more specific information to be reported by Skyfunctions, e.g.,
to inform the build-event protocol on missing files, the EventHandler
interface is no longer enough. Therefore, provide an enriched context
for reporting events.
--
Change-Id: I2d06166fe4d5b9054e24ad8c752fafc039e3f9f8
Reviewed-on: https://cr.bazel.build/8794
PiperOrigin-RevId: 148463437
MOS_MIGRATED_REVID=148463437
|
|
|
|
|
|
|
|
| |
handling.
--
PiperOrigin-RevId: 147185674
MOS_MIGRATED_REVID=147185674
|
|
|
|
|
|
| |
--
PiperOrigin-RevId: 146982656
MOS_MIGRATED_REVID=146982656
|
|
|
|
|
|
|
|
| |
dependencies used in last build
--
PiperOrigin-RevId: 146169454
MOS_MIGRATED_REVID=146169454
|
|
|
|
|
|
|
|
|
|
|
|
| |
Because the event handler's inner handlers are removed after each query
command, caching the handler caused a subset of subsequent commands'
errors (those reported through the resolver's handler) to go unlogged.
Also fix a bug with cycle detection in DelegatingWalkableGraph.
--
PiperOrigin-RevId: 145124271
MOS_MIGRATED_REVID=145124271
|
|
|
|
|
|
|
|
| |
useful, but may also help identify Bazel as the culprit in the linked bug if we're lucky and the timestamps are clear.
--
PiperOrigin-RevId: 144359628
MOS_MIGRATED_REVID=144359628
|
|
|
|
|
|
|
|
| |
changes are visibile to all other threads by volatility and we don't read the variable at any intermediate or inconsistent state (simply check against null or a constant). reverseDepsToSignal is also made volatile for subclasses that need volatile reads to it.
--
PiperOrigin-RevId: 143787032
MOS_MIGRATED_REVID=143787032
|
|
|
|
|
|
|
|
| |
WalkableGraph#getException to be given non-existent keys without crashing. Add WalkableGraph#isCycle to fill the gap in testing for the difference between non-existence and depending on a cycle.
--
PiperOrigin-RevId: 143205289
MOS_MIGRATED_REVID=143205289
|
|
|
|
|
|
| |
--
PiperOrigin-RevId: 141307853
MOS_MIGRATED_REVID=141307853
|
|
|
|
|
|
|
|
|
|
|
| |
that makes an appropriate call to Interners.InternerBuilder#concurrencyLevel.
For current readers of this CL, I used this class everywhere in the Blaze codebase.
For future readers of this CL, this class should be used to create an Interner in the Blaze codebase.
--
MOS_MIGRATED_REVID=140063271
|
|
|
|
|
|
|
|
|
| |
SkyQueryEnvironment#beforeEvaluateQuery if possible to save on latency for small queries.
This assumes that if the graph is up to date, then the data in SkyQueryEnvironment is also up to date. It also assumes that a ForkJoinPool remains usable until #shutdownNow is called.
--
MOS_MIGRATED_REVID=139386363
|
|
|
|
|
| |
--
MOS_MIGRATED_REVID=139357569
|
|
|
|
|
|
|
|
|
|
| |
It allows all graph implementations to return the list of nodes which are
immediately available to be fetched. NOTE: Not-currently-available here does
not mean the nodes do not exist in the graph. It simply means they are not
ready to be fetched immediately yet.
--
MOS_MIGRATED_REVID=137701432
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
*** Reason for rollback ***
Roll-forward with fix.
Tested with `bazel build src:srcs //src/test/...` using a bootstrapped bazel.
Fixes #1923.
RELNOTES[INC]: Non test-only targets can no longer depends on test-only targets.
*** Original change description ***
Automated [] rollback of commit a9f20b0d6459d395444c45cf5e94a899f3443633.
*** Reason for rollback ***
Broke Bazel CI:
ERROR: /home/ci/workspace/Bazel/JAVA_VERSION/1.8/PLATFORM_NAME/linux-x86_64/src/BUILD:284:1: in filegroup rule //src:srcs: non-test target '//src:srcs' depends on testonly target '//src/java_tools/junitrunner/java/com/google/testing/junit/junit4:srcs' and doesn't have testonly attribute set.
ERROR: Analysis of target '//scripts/packages:packages' failed; build aborted.
http://ci.bazel.io/view/Bazel%20b...
***
--
MOS_MIGRATED_REVID=136589937
|
|
|
|
|
| |
--
MOS_MIGRATED_REVID=136431274
|
|
|
|
|
|
|
| |
Instead, deduplicate when the helper is actually added to a GroupedList.
--
MOS_MIGRATED_REVID=136145321
|
|
|
|
|
|
|
|
|
|
| |
Remove DirtyKeyTracker and inflight node tracking as independent
progress tracking bits of code, and instead use an internal-facing
DirtyTrackingProgressReceiver wrapper of the user's
EvaluationProgressReceiver.
--
MOS_MIGRATED_REVID=135291924
|
|
|
|
|
|
|
|
|
|
| |
EvaluationProgressReceiver objects have two common naming schemes
currently, and calling them invalidationReceiver is misleading, so to
make the naming convention standard, all object names are based on
"progressReceiver."
--
MOS_MIGRATED_REVID=134411011
|
|
|
|
|
| |
--
MOS_MIGRATED_REVID=133714902
|
|
|
|
|
| |
--
MOS_MIGRATED_REVID=132999234
|
|
|
|
|
|
|
| |
can use different cycle detection algorithms if they wish.
--
MOS_MIGRATED_REVID=131347160
|
|
|
|
|
| |
--
MOS_MIGRATED_REVID=131344134
|
|
|
|
|
|
|
|
|
|
|
| |
respect to cycle checking.
Reducing the size of ParallelEvaluator.java is also probably long overdue.
I believe this change stands on its own, but if you don't think the third change is worth it, and this isn't worth it on its own, feel free to push back.
--
MOS_MIGRATED_REVID=131340165
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Instead, just put them directly into a map. This avoids the memory churn and CPU cost of the set.
As a result, we have to use HashMaps instead of ImmutableMap.Builders, which I hope is ok (especially since we're not keeping them around), and due to that, we have some nice nondeterminism in the returned order, which matters for some cycle-checking tests.
Also, don't use a map at all when we don't need to (when building events).
Note that, since we have to deduplicate at some point, this means that changing the return type of SkyFunction.Environment#getValues to not be a random-access map is probably not worth it. Changing the return type of ProcessableGraph#getBatch to not be a random access map might still be worthwhile, although it might require some funny operations.
--
MOS_MIGRATED_REVID=131070418
|
|
|
|
|
|
|
| |
The only place we now don't handle InterruptedException is in the action graph created after analysis, since I'm not sure that will be around for that much longer.
--
MOS_MIGRATED_REVID=130327770
|
|
|
|
|
|
|
|
|
| |
Make sure that ParallelEvaluator treats SchedulerExceptions as less severe than other RuntimeExceptions (it already did, but add a comment emphasizing this).
The combination of the above means that we don't ignore hypothetical crashes in ParallelEvaluator worker threads _after_ a SchedulerException (e.g. due to an error in nokeep_going evaluation).
--
MOS_MIGRATED_REVID=130044230
|
|
|
|
|
|
|
|
|
| |
nokeep_going build because it finished building in between the time it was first requested and when we checked it for done-ness after the SkyFunction evaluation.
This results in an IllegalStateException that gets ignored (and, importantly, not propagated) by AbstractQueueVisitor because AbstractQueueVisitor only records and propagates the first Throwable encountered. For nokeep_going evaluations, this will be the SchedulerException that we use for control flow. The IllegalStateException in question is benign in this case because it's merely from a Preconditions failure and doesn't leave anything in a bad state. It's possible, though, that we have other bugs that are being masked in this way.
--
MOS_MIGRATED_REVID=129919336
|
|
|
|
|
| |
--
MOS_MIGRATED_REVID=129895423
|
|
|
|
|
|
|
| |
QueryableGraph.Reason which conveys more information. Add a few more Reason enum values to make this refactor benign.
--
MOS_MIGRATED_REVID=129118462
|
|
|
|
|
|
|
| |
parameters conveying the requesting node (if any), the requested node(s), as well as a reason for the skyframe graph lookup. Alternate graph implementations may be interested in this information.
--
MOS_MIGRATED_REVID=128496089
|
|
|
|
|
|
|
| |
SkyFunction if it throws an exception but still has missing deps. Instead, pretend it didn't throw, and restart it when its known deps are all done, presumably to throw the same exception. This removes a basic source of non-determinism.
--
MOS_MIGRATED_REVID=127129202
|
|
|
|
|
|
|
| |
alternate graph implementations to optimize how they construct node entries.
--
MOS_MIGRATED_REVID=126932020
|
|
|
|
|
|
|
| |
interfaces, in preparation for further changes.
--
MOS_MIGRATED_REVID=126924789
|
|
|
|
|
|
|
|
|
| |
Collapse the "evaluating" boolean into the "signaledDeps" int field, since signaledDeps is always 0 if evaluating is false, so we can use the sentinel value -1 to indicate that evaluation has not yet started. This leads to a slightly less tolerant node entry: it must "start evaluating" before you can do things like set its value. Places that wasn't being done have been fixed, at least as far as we have test coverage for.
Also, factor the "dirty" parts of BuildingState out into a subclass. It would probably be cleaner to use composition here, but I don't want to pay the price of another object.
--
MOS_MIGRATED_REVID=126729331
|
|
|
|
|
|
|
| |
The biggest savings here is that we were not eagerly discarding the InMemoryNodeEntry#directDeps field after an entry was marked dirty, even though we would never read its value again. But rather than just fix that, by getting rid of the field in BuildingState, we can potentially save memory with smaller BuildingState objects as well.
--
MOS_MIGRATED_REVID=126709632
|
|
|
|
|
|
|
| |
last build's deps. This is only an issue with cycles, since there we try to maintain the invariant that a parent is not done before its children by removing its deps on its children before we construct its cycle value.
--
MOS_MIGRATED_REVID=126494009
|