aboutsummaryrefslogtreecommitdiffhomepage
path: root/src/main/java/com/google/devtools/build/skyframe
Commit message (Collapse)AuthorAge
* Add AutoProfiler-like API to ProfilerGravatar ulfjack2018-06-11
| | | | | | - migrate all startTask/completeTask pairs to the new API PiperOrigin-RevId: 200038703
* Make Artifact#equals take the owner into account for derived artifacts.Gravatar janakr2018-06-08
| | | | | | | | Derived artifacts' owners are important because they are used to determine the artifact's generating action. Source artifacts' owners are not used in this way, so I left them alone. This allows us to get rid of most uses of ArtifactSkyKey. We may be able to delete it entirely in a follow-up. PiperOrigin-RevId: 199836436
* Add hacky method to InMemoryNodeEntry for fast but unsafe iteration over ↵Gravatar janakr2018-06-04
| | | | | | in-progress reverse deps. PiperOrigin-RevId: 199209256
* Remove ConfiguredTarget from TargetCompletionValue, since it is no longer ↵Gravatar janakr2018-06-04
| | | | | | needed: we can get the ConfiguredTargetKey directly from the TargetCompletionKey. Since that was the only use of the actual value in EvaluationProgressReceiver#evaluated, remove it, instead just provide a boolean enum that gives whether or not evaluation succeeded. PiperOrigin-RevId: 199178047
* Add events and get rid of ErrorInfoEncoder. Clean up some signatures and ↵Gravatar janakr2018-05-22
| | | | | | visibility in Skyframe classes. PiperOrigin-RevId: 197665817
* Potentially allow children of a dirty node to be missing from the graph. ↵Gravatar janakr2018-05-15
| | | | | | Also pass the GraphInconsistencyReciever into SkyframeExecutor as a parameter. PiperOrigin-RevId: 196716642
* Use ConcurrentHashMap directly instead of MapMaker.Gravatar Googler2018-05-03
| | | | | RELNOTES: None. PiperOrigin-RevId: 195280015
* Clean up code that directly imports nested classes like Builder, Entry, etc.Gravatar jcater2018-05-02
| | | | PiperOrigin-RevId: 195100670
* Allow SkyFunctions to return a sentinel value indicating that all of a ↵Gravatar janakr2018-04-30
| | | | | | | | | | node's in-progress data should be forgotten, and its evaluation should be restarted from scratch, as if it were freshly created/dirtied. To guard against this happening unexpectedly, any such events are passed to a GraphInconsistencyReceiver, which can verify that the SkyFunction is behaving properly. This is the first change in a series to permit action rewinding when it is discovered that a previously generated input file is no longer available. When an action detects that one of its inputs is unusable, it can return this sentinel value, causing it to be re-evaluated from scratch. Follow-up changes will make the node corresponding to the input, and the node corresponding to the action that generated the input, dirty when this happens, causing the upstream action to be re-run, regenerating the desired input. Currently works for builds that do not keep edges, although follow-ups may make this possible for all builds. PiperOrigin-RevId: 194863097
* Add hook to inform SkyframeExecutor that build is finished.Gravatar janakr2018-04-24
| | | | PiperOrigin-RevId: 194099006
* Remove use of bare Immutable{List,Map,Set} Builder classes.Gravatar jcater2018-04-20
| | | | | | Always use the more-qualified class name for clarity at the site of use. There are too many classes named Builder. PiperOrigin-RevId: 193649193
* Add warning to registerDependencies javadocGravatar shreyax2018-04-19
| | | | PiperOrigin-RevId: 193559600
* Notify Skyframe listeners about additional node states: CHECK_DIRTY, ↵Gravatar felly2018-04-17
| | | | | | INIT_ENV, and COMMIT. PiperOrigin-RevId: 193261989
* Simplify AbstractSkyFunctionEnvironment to more directly call ↵Gravatar shreyax2018-04-10
| | | | | | getValueOrUntypedExceptions. PiperOrigin-RevId: 192329649
* In inlined skylark import lookup function caching, don't actually fetch deps ↵Gravatar shreyax2018-04-06
| | | | | | when all we want to do is register an edge and don't require the value. PiperOrigin-RevId: 191966203
* Change profiling to only accept strings for its "description" argument. ↵Gravatar janakr2018-04-01
| | | | | | Profiling can hold onto objects for the duration of the build, and some of those objects may be temporary that should not be persisted. In particular, UnixGlob and its inner classes should not outlive loading and analysis. For the most part, care was taken in this CL to only use strings that required no additional construction, mainly to minimize garbage (retaining references to newly created strings is not as great a concern since only the strings corresponding to the slowest K tasks are retained, for some relatively small values of K). Action descriptions for actually executing actions are eagerly expanded because that work is minimal compared to the work of actually executing an action. PiperOrigin-RevId: 191251488
* Fix accidental eager expansion of grouped list in in-memory nodes.Gravatar felly2018-03-27
| | | | PiperOrigin-RevId: 190656902
* Add DONE_CHECKING as a reason for requesting nodes.Gravatar shreyax2018-03-26
| | | | | | | | Split registering the unique new deps of a node between those where we're enqueueing a known dependency from a prior build and one where we're adding a new dependency. Replace prefetchBatch with getBatchAsync and add createIfAbsentBatchAsync. PiperOrigin-RevId: 190471980
* Don't check for deps being done in a keep-going build as we expect this to ↵Gravatar shreyax2018-03-20
| | | | | | always be the case. PiperOrigin-RevId: 189792299
* Automated rollback of commit 72d28f3efc2842510a34cacd930c0204143f7412.Gravatar shreyax2018-03-19
| | | | | | | | | | | | | | | | | | | | | | | | Fix skylark caching to properly include transitive dependencies when there is a diamond-like dependency in the loaded bzl files. Also add guards to make sure we're not attempting to cache skylark files that transitively request a dependency that is in error. *** Reason for rollback *** Looking for source of non-determinism *** Original change description *** Automated rollback of commit 7ba939dfd5df48903929e9c14ebd0449656403e4. *** Reason for rollback *** Likely cause for non-determinism in skyframe *** Original change description *** Cache SkylarkLookupImportValues in memory so that we don't recompute them multiple times. PiperOrigin-RevId: 189686604
* Automated rollback of commit 7ba939dfd5df48903929e9c14ebd0449656403e4.Gravatar shreyax2018-03-12
| | | | | | | | | | | | *** Reason for rollback *** Likely cause for non-determinism in skyframe *** Original change description *** Cache SkylarkLookupImportValues in memory so that we don't recompute them multiple times. PiperOrigin-RevId: 188729929
* Cache SkylarkLookupImportValues in memory so that we don't recompute them ↵Gravatar shreyax2018-03-05
| | | | | | multiple times. PiperOrigin-RevId: 187941859
* Replace LegacySkyKey by AbstractSkyKey or custom SkyKeys. AbstractSkyKey ↵Gravatar janakr2018-03-02
| | | | | | | | doesn't save memory in the 32-bit case, but makes it easier for people to see how many SkyKeys we have. There's some unnecessary interning in tests, but it was easier to copypasta and doesn't harm anything, I think. PiperOrigin-RevId: 187694309
* Optimize GC usage of iterating over all elements of GroupedLists when we ↵Gravatar shreyax2018-03-02
| | | | | | don't care about the group structure, and simplify the logic for prefetching old deps. PiperOrigin-RevId: 187681887
* Deletes AutoCodec.Strategy.SINGLETON now that we have @AutoCodec field tags.Gravatar shahan2018-02-28
| | | | | | This also gets rid of some static initialization cycles which we should try very hard to avoid in the future. PiperOrigin-RevId: 187334087
* Tags mapped singletons in SkyValueEncoderGravatar shahan2018-02-27
| | | | PiperOrigin-RevId: 187209783
* Expose SimpleCycleDetector. Also, allow MemoizingEvaluatorTest to support ↵Gravatar cpeyser2018-02-26
| | | | | | evaluators that do not store errors alongside values, but which still support error transience. PiperOrigin-RevId: 187058808
* Fixes skyframe evaluation to correctly handle interrupts during ↵Gravatar Googler2018-02-23
| | | | | | | | | | | | | #doMutatingEvaluation When multiple keys need to be evaluated, we may schedule keys before informing progress on done nodes, which can throw InterruptExceptions. If the main thread is interrupted during #informProgressReceiverThatValueIsDone, then we may not properly track evaluations which are already scheduled. It could cause threads continue to run after the main loop is closed. RELNOTES: None PiperOrigin-RevId: 186801930
* Automated rollback of commit 7fe59b98eefc96a6310f0b0221d4e0f18e2a9000.Gravatar shreyax2018-02-23
| | | | | | | | | | | | *** Reason for rollback *** Fixed bug due to TransitiveTargetFunction requesting multiple Package dependencies when computing its aspect deps by only applying the optimization to TransitiveTraversalFunction. *** Original change description *** Automated rollback of commit cce164aed44aba1de244f0d764cd33a5cc6980b2. PiperOrigin-RevId: 186766812
* Accept Durations in the CriticalPathComponent and Profiler.Gravatar ccalvarin2018-02-22
| | | | | RELNOTES: None. PiperOrigin-RevId: 186658512
* Store cycle detectors directly in AbstractParallelEvaluator.Gravatar shreyax2018-02-22
| | | | PiperOrigin-RevId: 186617412
* Add basic equality checking for ErrorInfo and tighten the interface for ↵Gravatar shreyax2018-02-21
| | | | | | ErrorInfoManager. PiperOrigin-RevId: 186524034
* Automated rollback of commit cce164aed44aba1de244f0d764cd33a5cc6980b2.Gravatar fwe2018-02-19
| | | | PiperOrigin-RevId: 186211672
* Re-use previously computed deps for TransitiveBaseTraversalFunction#compute ↵Gravatar shreyax2018-02-16
| | | | | | if we have them instead of re-computing them each time on a skyframe restart. PiperOrigin-RevId: 186017079
* Refactor WalkableGraph and BuildDriver interfacesGravatar Googler2018-02-13
| | | | | | | | Remove WalkableGraph#isUpToDate and BuildDriver#alreadyEvaluated and delegate the work to implementation. RELNOTES: None PiperOrigin-RevId: 185562370
* Refactors ParallelEvaluator to support eval-wide exceptions.Gravatar mschaller2018-02-01
| | | | | | | Also clarifies a comment on preventNewEvaluations. RELNOTES: None. PiperOrigin-RevId: 184198568
* Don't use null EvaluationProgressReceiver in MemoizingEvaluator (it was only ↵Gravatar janakr2018-01-30
| | | | | | null in AbstractPackageLoader and tests). PiperOrigin-RevId: 183896826
* Re-add WalkableGraph#isUpToDate and BuildDriver#alreadyEvaluatedGravatar Googler2018-01-18
| | | | | RELNOTES: None PiperOrigin-RevId: 182389876
* Remove synchronization from InMemoryNodeEntry#getValue{,MaybeWithMetadata} ↵Gravatar janakr2018-01-10
| | | | | | and #toValue. I saw significant contention from this method in some experiments, and the synchronization isn't needed, since we only call these methods on done nodes (as determined by #isDone), and a node that is observably done cannot racily change its value. PiperOrigin-RevId: 181507133
* Refactor WalkableGraph and BuildDriver interfacesGravatar Googler2017-12-21
| | | | | | | | Remove WalkableGraph#isUpToDate and BuildDriver#alreadyEvaluated and delegate the work to implementation. RELNOTES: None PiperOrigin-RevId: 179815374
* ParallelEvaluator: report events early for cache hitsGravatar ulfjack2017-12-21
| | | | | | | | | | | | | | | | | | In the case that a node is already done when evaluation starts, we now report events and postables early, rather than waiting until the end of evaluation. This makes reporting more timely, and ensures reporting even if the evaluation is interrupted. This caused a problem with moving the TargetCompleteEvent into Skyframe (unknown commit). I added two unit tests at the Skyframe level to cover the guarantees that we need for that. Note that the replay call in constructResult can duplicate the events from a cache hit - this is not a problem since the replaying visitor automatically removes duplicates (and it wasn't obvious which keys correspond to cache hits). PiperOrigin-RevId: 179788157
* Simplify tagged event handling.Gravatar ulfjack2017-11-30
| | | | | | | | | | | | Don't make copies of Events on replay. The same events may be replayed a lot, so it's better to copy before storing the events. Also avoid a copy if the tag doesn't actually change. The intent of this change is to reduce gc churn on incremental builds. When I wrote this change (~a year ago), this was a noticable source of gc churn in some benchmarks I ran at the time. PiperOrigin-RevId: 177450696
* Add a static method GroupedList#numElements to allow to count the number of ↵Gravatar shreyax2017-11-29
| | | | | | deps from a compressed GroupedList without uncompressing it. Also some minor GC improvements. PiperOrigin-RevId: 177338852
* Allow InMemoryNodeEntry subclasses to change which reverse dep operation to ↵Gravatar janakr2017-11-21
| | | | | | store bare. PiperOrigin-RevId: 176505963
* Clear interrupted bit in thread when throwing an interrupted exception that ↵Gravatar janakr2017-11-21
| | | | | | | | came from an AbstractParallelEvaluator evaluation. It's against the standard Java contract to throw but still have the thread's interrupted bit set. Also get rid of some unnecessary initializeTester() calls in MemoizingEvaluatorTest: we already call it via a @Before annotation. PiperOrigin-RevId: 176496034
* RELNOTES: --keep_incrementality_data flag allows Bazel servers to be run in ↵Gravatar janakr2017-11-12
| | | | | | | | | | | | | | memory-saving non-incremental mode independent of --batch and --discard_analysis_cache. A command run with --nokeep_incrementality_data will discard data that would be needed for incremental builds. Subsequent commands can be sent to the same server, but they will not get the benefit of incrementality from this command. However, if --keep_incrementality_data is specified on a subsequent command, the commands after that will get the benefits of incrementality. There are two benefits to not being dependent on --batch. First, this allows Bazel servers to be run in extreme memory-saving mode without the startup penalties (JVM startup, JITting) that --batch execution imposes. Second, this allows Bazel developers to inspect the state of a Bazel server after an extreme memory-saving build. In order to avoid discarding data unnecessarily (for instance, on a "bazel info used-heap-size-after-gc" or "bazel dump --skyframe=summary") the actual resetting of the graph is done lazily, right before its use in SequencedSkyframeExecutor#sync. This is morally a partial rollback of https://github.com/bazelbuild/bazel/commit/98cd82cbdcac7c48164a611c5a9aa8fc2f1720ef. For now, our tests specify all of the flags. After this change sticks, I plan to get rid of the --batch flag from these tests, which should allow for some clean-ups. Eventually --batch and --discard_analysis_cache may not imply that we don't keep incremental state: we can require that it be specified explicitly. PiperOrigin-RevId: 175335075
* Replace all usages of Blaze's Preconditions class with guava.Gravatar tomlu2017-11-09
| | | | | | | | Blaze had its own class to avoid GC from varargs array creation for the precondition happy path. Guava now (mostly) implements these, making it unnecessary to maintain our own. This change was almost entirely automated by search-and-replace. A few BUILD files needed fixing up since I removed an export of preconditions from lib:util, which was all done by add_deps. There was one incorrect usage of Preconditions that was caught by error prone (which checks Guava's version of Preconditions) that I had to change manually. PiperOrigin-RevId: 175033526
* Remove least important argument to 5+ arg Preconditions#checkState/checkNotNull.Gravatar tomlu2017-11-08
| | | | | | This change prepares a move to Guava's preconditions. Guava only has vararg-avoiding overloads up to 4 args. PiperOrigin-RevId: 175015502
* Make ErrorInfo#toString more informative.Gravatar janakr2017-11-06
| | | | PiperOrigin-RevId: 174508154
* Don't require --keep_going to discard graph edges. It's unnecessary.Gravatar janakr2017-11-02
| | | | PiperOrigin-RevId: 174202685
generated by cgit v1.2.3 (git 2.39.1) at 2024-07-05 12:02:01 +0000