aboutsummaryrefslogtreecommitdiffhomepage
path: root/src/main/java/com/google/devtools/build/lib/skyframe
Commit message (Collapse)AuthorAge
* Fix broken tests.Gravatar Marian Lobur2015-09-02
| | | | | -- MOS_MIGRATED_REVID=102134151
* Aspects can get information from their base rule.Gravatar Marian Lobur2015-09-02
| | | | | -- MOS_MIGRATED_REVID=102126786
* Inject the prelude path through the rule class provider.Gravatar Ulf Adams2015-09-01
| | | | | -- MOS_MIGRATED_REVID=102020499
* Allow for customization of the per-build system call caches used in Skyframe.Gravatar Eric Fellheimer2015-09-01
| | | | | -- MOS_MIGRATED_REVID=101984361
* Move global objects to RuntimeGravatar Francois-Rene Rideau2015-08-31
| | | | | | | | Move away global constants and global namespaces out of Environment and into a new file Runtime. -- MOS_MIGRATED_REVID=101940218
* Only print the label (and not the configuration hash key) for error messages ↵Gravatar Lukacs Berki2015-08-31
| | | | | | | with --keep_going. -- MOS_MIGRATED_REVID=101921704
* Check argument isn't null when creating TransitiveTraversalValue error valuesGravatar Michajlo Matijkiw2015-08-31
| | | | | -- MOS_MIGRATED_REVID=101812326
* Move PackageIdentifier to cmdlineGravatar Kristina Chodorow2015-08-31
| | | | | | | | This is necessary to have TargetResolver depend on it without making it depend on the packages target. First step of #389. -- MOS_MIGRATED_REVID=101790345
* Remove the WATCHFS_BLACKLIST from Constants.java.Gravatar Ulf Adams2015-08-28
| | | | | | | Instead, inject the list from the corresponding module. -- MOS_MIGRATED_REVID=101769355
* Make dangling symlinks in external repositories work to some degree.Gravatar Lukacs Berki2015-08-28
| | | | | | | | | | | | | | | | | This seems to be the least insane approach within the following boundaries: - Skyframe apparently doesn't allow GlobFunction to recover if FileFunction had already raised an exception that Skyframe knows about (this is somewhat surprising) - I didn't want to change FileFunction not to throw an exception for dangling symlinks because this part of the code is scary - I didn't want to revert to Skyframe-based symlink resolution for symlinks in immutable directories because that would be a performance hit - I didn't want to write yet another symlink resolver and the two existing ones (FileSystem#resolveSymlinks() and and FileFunction#getSymlinkTargetRootedPath()) don't work: the former cannot resolve just one level of symlinks and the latter cannot do its job without adding Skyframe dependencies I had to put in a placeholder value for realRootedPath when the FileValue represents a dangling symlink, because FileStateValue.create() relies on the symlink target being different than the symlink itself. RELNOTES: -- MOS_MIGRATED_REVID=101756189
* Be resilient to a non-decreasing clock in ↵Gravatar Nathan Harmata2015-08-28
| | | | | | | | | BlazeRuntime#recordLastExecutionTime. Also add @Nullable annotations as appropriate. Fixes #394. -- MOS_MIGRATED_REVID=101685096
* Make load() work in remote repositories too.Gravatar Lukacs Berki2015-08-27
| | | | | | | | | Previously, load() always looked up .bzl files in the main repository. Ideally, it would just take a label and then it would work by default, but for the time being, this quick fix will do. I had to put in an evil hack to make load() statements work in the prelude, because we currently have no way to distinguish load() statements from the prelude and from the BUILD file. Again, a proper label-based load() would solve this. -- MOS_MIGRATED_REVID=101677502
* Description redacted.Gravatar Michajlo Matijkiw2015-08-27
| | | | | -- MOS_MIGRATED_REVID=101635740
* Description redacted.Gravatar Michajlo Matijkiw2015-08-27
| | | | | -- MOS_MIGRATED_REVID=101598188
* Add toString() for PrepareDepsOfTargetsUnderDirectoryValue key.Gravatar Eric Fellheimer2015-08-26
| | | | | -- MOS_MIGRATED_REVID=101510850
* Implement the core structure for dynamic configurations.Gravatar Greg Estren2015-08-26
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is a big change, so let me walk you through the key pieces: 1) This cl provides an alternative mechanism for creating configurations and doing configuration transitions that's "dynamic" in that the configurations can be created on the fly and the transitions are arbitrary mappings from BuildOptions --> BuildOptions that can also be created on the fly. It also integrates this with ConfiguredTargetFunction, so the configured target graph automatically uses this framework. 2) It does *not* replace old-style (which we'll call "static") configurations. For a number of important reasons: It's not yet at feature parity (particularly: no LIPO). It's not remotely tested across real projects enough to have confidence that it's battle-ready. It doesn't yet handle certain "special" functions like BuildConfiguration.prepareToBuild() and BuildConfiguration.getRoots(). It still relies on the old static transition logic to determine what transitions to apply (eventually we'll distribute that logic out, but that's too much for a single cl). We need the option to toggle it on and off until we have enough confidence in it. So with this cl, builds can be done in either mode. 3) The new flag --experimental_dynamic_configs toggles use of dynamic configurations. 4) Dynamic configurations are created with the Skyframe function BuildConfigurationFunction (this was already created in an earlier change). This consumes a BuildOptions and a set of configuration fragments to produce a BuildConfiguration. 5) Dynamic transitions are instances of the new class PatchTransition, which simply maps an input BuildOptions to an output BuildOptions. 6) Since static and dynamic configurations have to co-exist (for now), this cl tries hard to keep today's transition logic defined in a single place (vs. forking a dedicated copy for each configuration style). This is done via the new interface BuildConfiguration.TransitionApplier. BuildConfiguration.evaluateTransition is modified to feed its transition logic into TransitionApplier's common API. Both dynamic and static configurations have their own implementations that "do the right thing" with the results. 7) The transition applier for dynamic configurations basically stores the Transition, then allows DependencyResolver (which calls BuildConfiguration.evaluateTransition) to return Dependency instances containing that Transition (vs. a BuildConfiguration, which they traditionally contain). 7.5) An earlier variation of the dynamic transition applier retained BuildOptions (e.g. when it got a Transition it immediately applied it to get its output BuildOptions, then stored that). This had the advantage of making composing of transitions easier, especially within BuildConfiguration.evaluateTransition (which can theoretically apply multiple transitions to the input configuration). But it turns out that applying transitions has a cost, and it's simply more performant to pass them through until they're really needed. 8) In dynamic configuration mode, after ConfiguredTargetFunction gets its deps (e.g. an <Attribute, Dependency> multimap), it "trims" the configurations for its dependencies by a) only including config fragments required by the deps' subtrees and b) applying the transitions that came from 7). This all happens in the new method ConfiguredTargetFunction.trimConfigurations. 9) trimConfigurations is heavily performance-optimized based on a lot of experience running this through a large project within Google. As it turns out, the cost of host transitions can be atrocious (because there are a lot of them). Also, BuildOptions.clone() is expensive. And just creating BuildConfiguration SkyKeys also has a cost (largely because of BuildOptions cloning), so that shouldn't be done except when really necessary. My experience with this convinced me it's worth making this method complicated for the sake of making it fast. Since it basically visits every edge in the configured target graph (at least), it really needs to be slick. 10) Since host transitions in particular are problematic w.r.t. speed, I compute the host *once* in ConfigurationCollectionFunction.getHostConfiguration() and expose that reference to ConfiguredTargetFunction and other Skyframe functions. This limits recomputation to just when the fragments are trimmed. 11) Since options cloning is expensive, I'm doing something scary: exposing a BuildConfiguration.getOptions() method that returns a direct reference. Since BuildOptions is mutable, this is dangerous in the wrong hands. I can experiment with going back to cloning (with the caching of host transitions it may not matter as much), but we may ultimately have to put up with this risk for the sake of performant analysis time. What would be *really* awesome would be to make BuildOptions immutable. But that's not going to happen in this cl. So in short, the key abstractions in this cl are: - PatchTransition - BuildConfiguration.TransitionApplier - ConfiguredTargetFunction.trimConfigurations The current implementation imposes no analysis time penalty -- MOS_MIGRATED_REVID=101474620
* Rename some variables missed by a codebase-wide renaming a long time ago.Gravatar Nathan Harmata2015-08-25
| | | | | -- MOS_MIGRATED_REVID=101388809
* Preserve specified order of keys when analyzing configured targets.Gravatar Janak Ramakrishnan2015-08-24
| | | | | -- MOS_MIGRATED_REVID=101235139
* Implement hashCode, equals for TransitiveTraversalValue and related SkyValuesGravatar Mark Schaller2015-08-24
| | | | | -- MOS_MIGRATED_REVID=101233095
* Add rbuildfiles function to queries, only for use when using ↵Gravatar Janak Ramakrishnan2015-08-21
| | | | | | | | | SkyQueryEnvironment. rbuildfiles returns the packages (in the form of BUILD files) that depend on the given source files as BUILD files or subincludes. -- MOS_MIGRATED_REVID=101157700
* Convert evaluated tracking to take a lazy SkyValueGravatar Mark Schaller2015-08-21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The EvaluationProgressReceiver#evaluated method took a SkyValue, but that parameter was only used by some of its implementations, and only under some conditions. Outside of tests, the main users are SkyframeBuildView's ConfiguredTargetValueInvalidationReceiver and SkyframeBuilder's ExecutionProgressReceiver. The former builds up a set of built ConfiguredTarget keys when the SkyValue is non-null and the EvaluationState is BUILT, and so its nullity check can live behind those two conditions. The latter cares about builting up a set of ConfiguredTargets, and raising events on the eventBus when a TARGET_COMPLETION or ASPECT_COMPLETION value is evaluated and is non-null. The extraction of these values can live behind the conditions that check the type of the SkyKey. By making the SkyValue parameter lazy, this change enforces that it's only accessed under these conditions. This CL introduces a semantic change that should be small in effect. The SkyframeBuildView keeps track of a set, dirtiedConfiguredTargetKeys, and ConfiguredTarget keys evaluated as CLEAN were removed from it if they had a non-null value. With this change, ConfiguredTarget keys evaluated as CLEAN get removed regardless of whether their values are null or non-null. The set is used to determine whether artifact conflict search has to be rerun, and the extra removals that result from this CL could cause fewer artifact conflict searches to run, but the only affected searches would be those that were caused by dirtied configured target values in error all of which were subsequently marked as clean, which is probably rare. -- MOS_MIGRATED_REVID=101144655
* Replace ref equality with .equals for SkyFunctionNames (2)Gravatar Mark Schaller2015-08-20
| | | | | | | | | | Ref equality for same-name SkyFunctionNames is not guaranteed. These were some instances I missed in my previous sweep. I searched for references to the SkyFunctions and SkyFunctionNames classes throughout the project to catch these remaining instances. -- MOS_MIGRATED_REVID=101061352
* A prototype implementation of top-level aspects.Gravatar Dmitry Lomov2015-08-20
| | | | | -- MOS_MIGRATED_REVID=101033236
* Log information about the initial baseline DiffAwareness.View. This is ↵Gravatar Nathan Harmata2015-08-20
| | | | | | | especially useful reproducing a clean build merely from the INFO log. -- MOS_MIGRATED_REVID=100984749
* Avoid Skyframe restarts in transitive targets due related to conservative ↵Gravatar Eric Fellheimer2015-08-20
| | | | | | | aspect deps. -- MOS_MIGRATED_REVID=100960261
* Allow for metadata lookup alongside the existing #prepareAndGet() graph ↵Gravatar Eric Fellheimer2015-08-17
| | | | | | | construction. -- MOS_MIGRATED_REVID=100709648
* TransitiveTraversalFunction now implements "conservative" (a la ↵Gravatar Eric Fellheimer2015-08-13
| | | | | | | ConservativeAspectResolver) aspect resolution. -- MOS_MIGRATED_REVID=100526575
* When we have enough information to determine that a file didn't change type, ↵Gravatar Nathan Harmata2015-08-13
| | | | | | | don't unnecessarily invalidate DirectoryListingStateValues. -- MOS_MIGRATED_REVID=100511935
* Use full package identifier in containing package checkGravatar Kristina Chodorow2015-08-12
| | | | | | | Otherwise a @x//a/b will be seen as crossing @y//a's package boundary. -- MOS_MIGRATED_REVID=100465538
* Replace ref equality with .equals for SkyFunctionNamesGravatar Mark Schaller2015-08-11
| | | | | | | Ref equality for same-name SkyFunctionNames is not guaranteed. -- MOS_MIGRATED_REVID=100322275
* Convert invalidated tracking from per-value to per-keyGravatar Mark Schaller2015-08-11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The primary user of invalidation tracking is the SkyframeBuildView, which monitored which ConfiguredTargetValues were invalidated. It did that so the SkyframeExecutor could limit its search for artifact conflicts to when the set of configured targets in the build changed. For the newly introduced set of dirtied keys, "dirtiedConfiguredTargetKeys" in SkyframeBuildView, to be as useful as the "dirtyConfiguredTargets" set it replaced, the ConfiguredTargetValueInvalidationReceiver must only remove a key from the set if it was found to be clean when it was re-evaluated. If it was rebuilt, then the key must stay in the set, to represent that the set of configured target values has truly changed. This CL introduces a semantic change that hopefully has a small effect, if any. Previously, the informInvalidationReceiver method in InvalidatingNodeVisitor only informed the invalidationReceiver when a non-null value was invalidated. (This is usually, but not always, equivalent to a non-error value. The uncommon exception is that in keep-going builds, some nodes with errors may also have values, and the invalidator would inform the receiver when such a node was invalidated.) Now, the receiver is informed that a node is invalidated regardless of whether its value is null. Because the receiver uses this information to determine whether artifact conflict search has to be rerun, and that search is expensive, it's possible this change will negatively impact performance. However, the only way an extra search could be invoked is if the invalidated configured target nodes are all in error. That seems like it would happen rarely, if at all. Further cleanup of unused SkyValues returned by markDirty to come in a subsequent CL. -- MOS_MIGRATED_REVID=100304744
* Move skylark import dependency registration to after the preprocessor.Gravatar Han-Wen Nienhuys2015-08-10
| | | | | | | RELNOTES: allow load() in subincluded files. -- MOS_MIGRATED_REVID=100125415
* Rephrase the ActionGraph in terms of the WalkableGraph. This should be more ↵Gravatar Eric Fellheimer2015-08-06
| | | | | | | efficient if we need to do many ActionGraph looups. -- MOS_MIGRATED_REVID=100060090
* Factor out import finding code.Gravatar Han-Wen Nienhuys2015-08-06
| | | | | -- MOS_MIGRATED_REVID=100038493
* Create a parameter to GlobFunction to determine whether it should deal with ↵Gravatar Eric Fellheimer2015-08-04
| | | | | | | | | non-pattern sub-expressions via file stat or directory listing. There is an inherent incrementality tradeoff here: A directory listing should be independent of any file edits in the directory, but may be overly conservative if an unrelated file is added or removed. The file stat is overly conservative when the file is modified, since glob() cares only about existence vs. non-existence. -- MOS_MIGRATED_REVID=99838654
* Remove stale comment.Gravatar Nathan Harmata2015-08-04
| | | | | | | RELNOTES: -- MOS_MIGRATED_REVID=99589366
* Fix ClassCastException.Gravatar Nathan Harmata2015-08-04
| | | | | -- MOS_MIGRATED_REVID=99487015
* FilesetEntryValue now overrides .equals and .hashCode.Gravatar Laszlo Csomor2015-07-30
| | | | | | | | | This means Skyframe's change pruning can work more efficiently. Without the overridden implementations we'd compare equal FEVs as inequal and unnecessarily re-evaluate SkyKeys that we could have verified clean. -- MOS_MIGRATED_REVID=99397188
* ActionExecutionValue now overrides .equals and .hashCode.Gravatar Laszlo Csomor2015-07-30
| | | | | | | | | This means Skyframe's change pruning can work more efficiently. Without the overridden implementations we'd compare equal AEVs as inequal and unnecessarily re-evaluate SkyKeys that we could have verified clean. -- MOS_MIGRATED_REVID=99396001
* Adds TransitiveTraversalFunction that loads transitive deps of a targetGravatar Mark Schaller2015-07-30
| | | | | | | | | Unlike TransitiveTargetFunction, it does not return nested sets of the traversed targets. Used primarily for its side effects of loading the transitive targets into the graph. -- MOS_MIGRATED_REVID=99388411
* Extract base class from TransitiveTargetFunctionGravatar Mark Schaller2015-07-30
| | | | | -- MOS_MIGRATED_REVID=99386094
* Cleanup of TransitiveTarget and TargetMarker functionsGravatar Mark Schaller2015-07-30
| | | | | | | | | | Added more information to TargetMarkerFunction documentation. Cleaned up and rearranged some code in TransitiveTargetFunction to help with future refactoring. -- MOS_MIGRATED_REVID=99384291
* Elegantly handle unbounded file symlink resolutions, e.g. 'a' -> 'b' -> ↵Gravatar Nathan Harmata2015-07-29
| | | | | | | 'a/nope'. -- MOS_MIGRATED_REVID=99337668
* Allow modules to specify additional nodes in the graph to be invalidated and ↵Gravatar Janak Ramakrishnan2015-07-29
| | | | | | | | | | | use this functionality to properly invalidate http download nodes if the downloaded zip file doesn't match the specified hash. This still means that the actual files downloaded may not match, but checking all such files may be too expensive. This helps with #336 but before that issue can be closed all remote repositories (git, etc.), should have similar functionality added. -- MOS_MIGRATED_REVID=99317085
* Some additional timing logging for discarding analysis cache, multi group ↵Gravatar Eric Fellheimer2015-07-28
| | | | | | | accounting, and artifact conflicts. -- MOS_MIGRATED_REVID=99217433
* Make some skyframe and lib/skyframe classes public.Gravatar Nathan Harmata2015-07-28
| | | | | -- MOS_MIGRATED_REVID=99197069
* Remove unused constructor and inline create method.Gravatar Janak Ramakrishnan2015-07-27
| | | | | | | -- Change-Id: Ica96bd719fe9d7e152fbe78e71669fddefe12e92 Reviewed-on: https://bazel-review.googlesource.com/#/c/1640/ MOS_MIGRATED_REVID=99182210
* Make the creation of the external package much simpler.Gravatar Lukacs Berki2015-07-23
| | | | | | | A previous change made the loading-time external label resolution unused, thus, now we can do away with a lot of machinery. The only unfortunate side effect is that instead of a nice and clear "No Android SDK found" error message, you'll get a more cryptic "external label //external:android/sdk is unbound" one. I think it's a fair tradeoff. -- MOS_MIGRATED_REVID=98813719
* --Gravatar Michajlo Matijkiw2015-07-21
| | | | MOS_MIGRATED_REVID=98737473
* Invalidate external repository files when the WORKSPACE changesGravatar Kristina Chodorow2015-07-17
| | | | | | | Fixes #236. -- MOS_MIGRATED_REVID=98402890
generated by cgit v1.2.3 (git 2.39.1) at 2024-07-08 12:31:42 +0000