aboutsummaryrefslogtreecommitdiffhomepage
path: root/src/main/java/com/google/devtools/build/skyframe/InMemoryMemoizingEvaluator.java
Commit message (Collapse)AuthorAge
* When no new configured targets have been analyzed, only check for artifact ↵Gravatar janakr2018-08-03
| | | | | | | | | | conflicts if the current set of configured targets is not a subset of the largest set of configured targets that have been checked for conflicts. Also rework the flow between SkyframeBuildView and SkyframeActionExecutor to remove the SkyframeExecutor middleman. Also reword the error message in case of an ArtifactPrefixConflictException, since a clean should no longer be necessary. PiperOrigin-RevId: 207322139
* Add the ParallelEvaluator.eval call to the ProfilerGravatar ulfjack2018-07-30
| | | | | | | We want to make sure that the profile captures all Skyframe invocations, so that we can catch cases where we accidentally introduce new ones. PiperOrigin-RevId: 206543307
* Skip invalidation entirely if the diff is emptyGravatar ulfjack2018-07-25
| | | | PiperOrigin-RevId: 205980620
* Add events and get rid of ErrorInfoEncoder. Clean up some signatures and ↵Gravatar janakr2018-05-22
| | | | | | visibility in Skyframe classes. PiperOrigin-RevId: 197665817
* Clean up code that directly imports nested classes like Builder, Entry, etc.Gravatar jcater2018-05-02
| | | | PiperOrigin-RevId: 195100670
* Allow SkyFunctions to return a sentinel value indicating that all of a ↵Gravatar janakr2018-04-30
| | | | | | | | | | node's in-progress data should be forgotten, and its evaluation should be restarted from scratch, as if it were freshly created/dirtied. To guard against this happening unexpectedly, any such events are passed to a GraphInconsistencyReceiver, which can verify that the SkyFunction is behaving properly. This is the first change in a series to permit action rewinding when it is discovered that a previously generated input file is no longer available. When an action detects that one of its inputs is unusable, it can return this sentinel value, causing it to be re-evaluated from scratch. Follow-up changes will make the node corresponding to the input, and the node corresponding to the action that generated the input, dirty when this happens, causing the upstream action to be re-run, regenerating the desired input. Currently works for builds that do not keep edges, although follow-ups may make this possible for all builds. PiperOrigin-RevId: 194863097
* Don't use null EvaluationProgressReceiver in MemoizingEvaluator (it was only ↵Gravatar janakr2018-01-30
| | | | | | null in AbstractPackageLoader and tests). PiperOrigin-RevId: 183896826
* RELNOTES: --keep_incrementality_data flag allows Bazel servers to be run in ↵Gravatar janakr2017-11-12
| | | | | | | | | | | | | | memory-saving non-incremental mode independent of --batch and --discard_analysis_cache. A command run with --nokeep_incrementality_data will discard data that would be needed for incremental builds. Subsequent commands can be sent to the same server, but they will not get the benefit of incrementality from this command. However, if --keep_incrementality_data is specified on a subsequent command, the commands after that will get the benefits of incrementality. There are two benefits to not being dependent on --batch. First, this allows Bazel servers to be run in extreme memory-saving mode without the startup penalties (JVM startup, JITting) that --batch execution imposes. Second, this allows Bazel developers to inspect the state of a Bazel server after an extreme memory-saving build. In order to avoid discarding data unnecessarily (for instance, on a "bazel info used-heap-size-after-gc" or "bazel dump --skyframe=summary") the actual resetting of the graph is done lazily, right before its use in SequencedSkyframeExecutor#sync. This is morally a partial rollback of https://github.com/bazelbuild/bazel/commit/98cd82cbdcac7c48164a611c5a9aa8fc2f1720ef. For now, our tests specify all of the flags. After this change sticks, I plan to get rid of the --batch flag from these tests, which should allow for some clean-ups. Eventually --batch and --discard_analysis_cache may not imply that we don't keep incremental state: we can require that it be specified explicitly. PiperOrigin-RevId: 175335075
* Replace all usages of Blaze's Preconditions class with guava.Gravatar tomlu2017-11-09
| | | | | | | | Blaze had its own class to avoid GC from varargs array creation for the precondition happy path. Guava now (mostly) implements these, making it unnecessary to maintain our own. This change was almost entirely automated by search-and-replace. A few BUILD files needed fixing up since I removed an export of preconditions from lib:util, which was all done by add_deps. There was one incorrect usage of Preconditions that was caught by error prone (which checks Guava's version of Preconditions) that I had to change manually. PiperOrigin-RevId: 175033526
* Stop injecting WorkspaceStatusAction into the Skyframe graph as a ↵Gravatar janakr2017-09-26
| | | | | | | | | | | | precomputed value. Instead, manually check if the value has changed, and if it has, invalidate its consuming WorkspaceStatusValue node, forcing its re-evaluation, where it will pick up the new value. This seems more awkward than the original code, but it is more correct in spirit: injecting a precomputed value which can change even while the source state does not is a smell. Long-term, the key for the WorkspaceStatusValue should incorporate a hash of the action, and that hash should be in the configuration, just as other configuration flags are. That isn't possible right now just because we don't have configuration trimming, and we drop all nodes on configuration changes, so putting workspace status options into the configuration would lose change pruning whenever we changed workspace status options. If/when those problems are fixed, we can extend this change to have WorkspaceStatusFunction continue to request the action out-of-band, but keyed by the hash. Then we can stop invalidating stale nodes. See also https://github.com/bazelbuild/bazel/issues/3785. PiperOrigin-RevId: 169947071
* Add --toolchain_resolution_debug option to give more information aboutGravatar John Cater2017-07-26
| | | | | | | | | toolchain selection. Fixes #3431. Change-Id: Ia38415575b6a121cbb6a028bfc0276691cd11b6d PiperOrigin-RevId: 163196646
* Tolerate injected nodes having deps.Gravatar janakr2017-07-21
| | | | | | | | | | We don't check explicitly that these are the only two ways, but this can happen if the error transience node is a dep of a node that's being injected, or if an injected node is an "external" file that needs to depend on an external package. The first possibility can happen if there was an IOException reading the node on the previous build. We handle the situation by just dirtying the node, not injecting it. Actual evaluation can handle the re-stat. PiperOrigin-RevId: 162622092
* Now that SkyKey is an interface, accept Iterable<? extends SkyKey>Gravatar ulfjack2017-07-19
| | | | | | | A subsequent CL makes TargetPatternKey implement SkyKey, and it's much nicer if I can pass lists of TargetPatternKey to the various Skyframe APIs. PiperOrigin-RevId: 162359843
* A bunch of unrelated cleanups:Gravatar nharmata2017-06-16
| | | | | | | | | | -Have SkylarkImportLookupFunction include causes in the SkyFunctionExceptions it throws. -Better transitive skyframe error declarations in ASTFileLookupFunction. -Have ErrorInfo differentiate between direct and transitive transience. -Introduce ErrorInfoManager and have ParallelEvaluator/ParallelEvaluatorContext use it. RELNOTES: None PiperOrigin-RevId: 159163186
* Drop loading-phase values if --discard_analysis_cache is true and we're not ↵Gravatar janakr2017-03-31
| | | | | | keeping incremental state. PiperOrigin-RevId: 151639711
* Provide more reporting options to SkyFunctions Gravatar Klaus Aehlig2017-02-27
| | | | | | | | | | | | | With more specific information to be reported by Skyfunctions, e.g., to inform the build-event protocol on missing files, the EventHandler interface is no longer enough. Therefore, provide an enriched context for reporting events. -- Change-Id: I2d06166fe4d5b9054e24ad8c752fafc039e3f9f8 Reviewed-on: https://cr.bazel.build/8794 PiperOrigin-RevId: 148463437 MOS_MIGRATED_REVID=148463437
* EvaluationProgressReceiver absorbs other progress trackers.Gravatar Chloe Calvarin2016-10-06
| | | | | | | | | | Remove DirtyKeyTracker and inflight node tracking as independent progress tracking bits of code, and instead use an internal-facing DirtyTrackingProgressReceiver wrapper of the user's EvaluationProgressReceiver. -- MOS_MIGRATED_REVID=135291924
* Fixed EvaluationProgressReceiver object namesGravatar Googler2016-09-27
| | | | | | | | | | EvaluationProgressReceiver objects have two common naming schemes currently, and calling them invalidationReceiver is misleading, so to make the naming convention standard, all object names are based on "progressReceiver." -- MOS_MIGRATED_REVID=134411011
* Refactor ParallelEvaluator in preparation for making it more modular with ↵Gravatar Janak Ramakrishnan2016-08-26
| | | | | | | | | | | respect to cycle checking. Reducing the size of ParallelEvaluator.java is also probably long overdue. I believe this change stands on its own, but if you don't think the third change is worth it, and this isn't worth it on its own, feel free to push back. -- MOS_MIGRATED_REVID=131340165
* Allow Skyframe graph lookups and value retrievals to throw InterruptedException.Gravatar Janak Ramakrishnan2016-08-16
| | | | | | | The only place we now don't handle InterruptedException is in the action graph created after analysis, since I'm not sure that will be around for that much longer. -- MOS_MIGRATED_REVID=130327770
* Augment the QueryableGraph#get[BatchWithFieldHints] method to take in ↵Gravatar Nathan Harmata2016-07-27
| | | | | | | parameters conveying the requesting node (if any), the requested node(s), as well as a reason for the skyframe graph lookup. Alternate graph implementations may be interested in this information. -- MOS_MIGRATED_REVID=128496089
* Refactor QueryableGraph and ThinNodeQueryableGraph to be independent ↵Gravatar Janak Ramakrishnan2016-07-11
| | | | | | | interfaces, in preparation for further changes. -- MOS_MIGRATED_REVID=126924789
* Minor change to add QueryableGraphBackedSkyFunctionEnvironment.Gravatar Janak Ramakrishnan2016-06-21
| | | | | -- MOS_MIGRATED_REVID=125362963
* Allow Skyframe tests to use the NotifyingGraph and DeterministicGraph ↵Gravatar Janak Ramakrishnan2016-05-11
| | | | | | | helpers to enforce concurrency synchronization points and determinism even if they are not using an InMemoryGraph-backed evaluator. -- MOS_MIGRATED_REVID=121977783
* Remove TransitivePackageLoader#getVisitedTargets, which was only used in ↵Gravatar Janak Ramakrishnan2016-02-11
| | | | | | | tests. Replace it in tests with direct Skyframe graph lookups. -- MOS_MIGRATED_REVID=114335937
* Intern IntVersions.Gravatar Janak Ramakrishnan2016-01-28
| | | | | -- MOS_MIGRATED_REVID=113197641
* Use Bazel Preconditions variant which avoids varargs array creationGravatar Mark Schaller2015-12-10
| | | | | | | Reduces garbage. -- MOS_MIGRATED_REVID=109914243
* Allow FilesystemValueChecker to operate on a WalkableGraph and add TODOs for ↵Gravatar Nathan Harmata2015-11-10
| | | | | | | | | replacing MemoizingEvaluator#getValues et al with WalkableGraph usage. I initially attempted to do this but punted once I realized how much work it would be. Also make DirectoryListingStateValue and FileStateValue public for use in outside callers of FilesystemValueChecker. -- MOS_MIGRATED_REVID=107447425
* Remove wildcard of SkyFunctionName, as it is finalGravatar Mark Schaller2015-10-22
| | | | | | | Nothing can extend SkyFunctionName. It's final. -- MOS_MIGRATED_REVID=105989976
* Don't iterate over children when building events if events won't be stored ↵Gravatar Janak Ramakrishnan2015-10-12
| | | | | | | anyway. Also batch retrievals of children when building events. -- MOS_MIGRATED_REVID=105174435
* Replace EvaluableGraph#createIfAbsent with the potentially more efficient ↵Gravatar Nathan Harmata2015-10-05
| | | | | | | EvaluableGraph#createIfAbsentBatch. -- MOS_MIGRATED_REVID=104534858
* Rationalize copyright headersGravatar Damien Martin-Guillerez2015-09-25
| | | | | | | | | | | The headers were modified with `find . -type f -exec 'sed' '-Ei' 's|Copyright 201([45]) Google|Copyright 201\1 The Bazel Authors|' '{}' ';'` And manual edit for not Google owned copyright. Because of the nature of ijar, I did not modified the header of file owned by Alan Donovan. The list of authors were extracted from the git log. It is missing older Google contributors that can be added on-demand. -- MOS_MIGRATED_REVID=103938715
* Allow Skylark import lookup values to be computed inline in order to avoid ↵Gravatar Janak Ramakrishnan2015-09-24
| | | | | | | reifying them in the Skyframe graph. -- MOS_MIGRATED_REVID=103758591
* Don't remove reverse deps until node is known to be changed. This helps ↵Gravatar Janak Ramakrishnan2015-09-23
| | | | | | | avoid mutating the deps of nodes that are still going to be deps after evaluation is finished. -- MOS_MIGRATED_REVID=103659429
* Fix Bazel breakageGravatar Damien Martin-Guillerez2015-09-11
| | | | | -- MOS_MIGRATED_REVID=102835622
* Delay cleaning of in-flight nodes until the following build. This allows us ↵Gravatar Janak Ramakrishnan2015-09-11
| | | | | | | | | to interrupt evaluation in constant time. Some ParallelEvaluator tests that implicitly relied on cleaning happening before the next evaluation were moved into MemoizingEvaluatorTest as a result. -- MOS_MIGRATED_REVID=102696653
* Pass filter for what events should be stored in the Skyframe graph into the ↵Gravatar Janak Ramakrishnan2015-07-16
| | | | | | | ParallelEvaluator, to allow users to customize which events they want to store. -- MOS_MIGRATED_REVID=98299604
* Add debug preconditions guard for null skykeys to ↵Gravatar Michajlo Matijkiw2015-07-10
| | | | | | | InMemoryMemoizingEvaluator#delete -- MOS_MIGRATED_REVID=97908428
* Add a SkyQueryEnvironment which runs queries by examining the Skyframe graph.Gravatar Janak Ramakrishnan2015-02-26
| | | | | | | | | This environment eagerly preloads the transitive closure of a specified query "universe", and so may not be as efficient as the standard query for limited-scope queries. It is activated when the universe is specified and ordered results are not requested (since it is currently unable to order results). Tests were modified/added to exercise this environment where deemed interesting. Some ugly hacks were done to add coverage in AbstractQueryTest and friends, because currently even if the full depot is loaded (using //...), individual target patterns most likely won't be present in the graph. A better way to deal with this situation, suggested by felly, is probably to extract target pattern resolution logic to an auxiliary function so that query is able to resolve target patterns without mutating the graph, and then call into the read-only graph with the resolved patterns. That may be done in a follow-up, in which case the "scope" of every query could be //... . -- MOS_MIGRATED_REVID=87257028
* Update from Google.Gravatar Han-Wen Nienhuys2015-02-25
-- MOE_MIGRATED_REVID=85702957
generated by cgit v1.2.3 (git 2.39.1) at 2024-06-30 22:27:37 +0000