aboutsummaryrefslogtreecommitdiffhomepage
path: root/src/main/java/com/google/devtools/build/lib/skyframe/AspectFunction.java
Commit message (Collapse)AuthorAge
* Native aspects can opt-in to apply to files.Gravatar dslomov2017-04-25
| | | | | | | Only works for top-level targets. RELNOTES: None. PiperOrigin-RevId: 154176914
* Aspect propagation should not lose transitively-visible aspects.Gravatar dslomov2017-04-11
| | | | | | | | If aspect a3 sees aspect a2, and aspect a2 sees aspect a1, propagation of the aspect list [a1,a2,a3] should not lose any aspects. RELNOTES: None. PiperOrigin-RevId: 152786900
* If --batch, --keep_going, --discard_analysis_cache, and the new ↵Gravatar janakr2017-03-31
| | | | | | | | | | | | | | | --noexperimental_enable_critical_path_profiling flags are all specified, then Bazel will delete Actions from ActionLookupValues as they are executed in order to save memory. Because an already-run action may output an artifact that is only requested later in the build, we need to maintain a way for the artifact to look up the action. But in most cases we don't need to keep the action itself, just its output metadata. Some actions unfortunately are needed post-execution, and so we special-case them. Also includes dependency change with description: Move action out of key. This keeps action references from polluting the graph -- actions are just stored in one SkyValue, instead of being present in SkyKeys. This does mean additional memory used: we have a separate ActionLookupData object per Action executed. That may reach ~24M for million-action builds. PiperOrigin-RevId: 151756383
* Do not execute aspect implementation if target advertizes but fails to ↵Gravatar Dmitry Lomov2017-03-01
| | | | | | | | | | | provide a provider. Previously we always executed the function, but didn't add the aspect to the deps. -- PiperOrigin-RevId: 148887089 MOS_MIGRATED_REVID=148887089
* Report inconsistent aspect order error to the user.Gravatar Dmitry Lomov2017-02-24
| | | | | | -- PiperOrigin-RevId: 148342788 MOS_MIGRATED_REVID=148342788
* Restrict aspects visible to other aspects according to their advertised ↵Gravatar Dmitry Lomov2017-02-15
| | | | | | | | providers. -- PiperOrigin-RevId: 147526961 MOS_MIGRATED_REVID=147526961
* Names of extra-action protos now take into account all aspect names.Gravatar Dmitry Lomov2016-12-16
| | | | | | | | | | | | | If an Aspect registered an action that an extra-action is shadowing, its name is used when creating the extra-action's ID and name. Since recently, an aspect can see other aspects applied to the same target. This CL record the names of other aspects applied to the target as well, disambiguating the action owners. -- PiperOrigin-RevId: 142264153 MOS_MIGRATED_REVID=142264153
* Aspects-on-aspects implementation.Gravatar Dmitry Lomov2016-11-15
| | | | | -- MOS_MIGRATED_REVID=139189444
* Do not crash when aspects provide duplicate things.Gravatar Dmitry Lomov2016-11-11
| | | | | -- MOS_MIGRATED_REVID=138860974
* Enable aspect invocations to be matched by output filters.Gravatar Michael Staib2016-09-19
| | | | | | | | | | | | | Currently aspects have no tag, i.e., they don't get checked against output filters at all. This changes aspects to have a tag of the same form as the rule they are attached to (i.e., the label of the target). Since the default output filters match on ^//package[:/], this should cover most uses of the output filter, while still allowing for finer control for those who crave it. -- MOS_MIGRATED_REVID=133425215
* Make aspects attached to aliases work when their actual= attribute is a select.Gravatar Lukacs Berki2016-08-31
| | | | | | | Fixes https://github.com/bazelbuild/e4b/issues/6 . -- MOS_MIGRATED_REVID=131698950
* Allow Skyframe graph lookups and value retrievals to throw InterruptedException.Gravatar Janak Ramakrishnan2016-08-16
| | | | | | | The only place we now don't handle InterruptedException is in the action graph created after analysis, since I'm not sure that will be around for that much longer. -- MOS_MIGRATED_REVID=130327770
* Changes DependencyResolver <Attribute, Dep> map from a ListMultimap to new classGravatar Greg Estren2016-08-12
| | | | | | | | | | | | | | | | | | | | | | OrderedSetMultimap. This maintains insertion order while eliminating duplicates. Certain rules, in particular, otherwise break this invariant: https://github.com/bazelbuild/bazel/blo[]e3e28274cca5b87f48abe33884edb84016dd3/src/main/java/com/google/devtools/build/lib/skyframe/ConfiguredTargetFunction.java#L403 There's no reason (to my knowledge) to need multiple instances of the same <Attribute, Dependency> pair. More context from Google code review: (Michael Staib): > There are many things which pass around a dependentNodeMap or help construct one or modify one. We want an interface which has the right guarantees. > ListMultimap is not the right interface because it has no guarantee of unique elements, which we want - we don't want the problem that this CL ran into, and there's no reason (that we know of, to be confirmed) that anyone would want multiple identical Dependencies. > SetMultimap is not the right interface because it has no guarantee of deterministic iteration order or efficient iteration, which we want - dependency order sometimes matters (e.g., Java classpath or C++ link order). > We agreed that the best way to get what we want is to define our own interface with its own simultaneous uniqueness and iterability guarantees. Unspoken in the discussion was why we wouldn't just use LinkedHashMultimap as the thing we pass around. IMO the reason for that is that we don't care that it be a LinkedHashMultimap specifically; if tomorrow Guava comes out with a faster cooler map that has deterministic and efficient iteration and guarantees element uniqueness, we want it. > In this case we're going to make the "interface" be a (final?) class: OrderedSetMultimap, an extension of ForwardingSetMultimap which delegates to LinkedHashMultimap, an implementation which does support both of those guarantees. > I had mentioned in the conversation that none of the Multimap implementations make guarantees about key iteration order, but this is not true - LinkedHashMultimap preserves key insertion order. We should perhaps declare this as part of the OrderedSetMultimap contract as well. -- MOS_MIGRATED_REVID=130037643
* Add an overview comment about how the analysis phase works and some pointers ↵Gravatar Lukacs Berki2016-05-19
| | | | | | | to it. -- MOS_MIGRATED_REVID=122718503
* Print an error message when the wrong type of input is given as an aspectGravatar Kristina Chodorow2016-05-11
| | | | | | | | | And add a test. Another issue I found while attempting to reproduce #1228, but not actually a fix for that bug. -- MOS_MIGRATED_REVID=121998625
* Add an "alias" rule.Gravatar Lukacs Berki2016-05-10
| | | | | | | This will be used to replace RedirectChaser so that we don't need to load packages during configuration creation anymore. -- MOS_MIGRATED_REVID=121935989
* Expose parameterized aspects to Skylark.Gravatar Googler2016-05-09
| | | | | | | | | | | | | | | There are no syntactic changes within Skylark; the only difference is that aspects may have non-implicit attributes, so long as they have type 'string' and use the 'values' restriction. Such aspects may only be requested by rules which have attributes with types and names matching the non-implicit, non-defaulted attributes of the aspect. This is not yet a complete match for native AspectParameters functionality since implicit attributes cannot yet be affected by attribute values, but that will be added later. Implicit aspects are still required to have default values. Non-implicit aspect attributes are considered "required" unless they have a default value. An error will occur if they are applied to a rule that does not "cover" all required attributes by having attributes of matching name and type. While non-implicit aspect attributes with a default are not required, they will still take on the value of a rule attribute with the same name and type if it is present. Aspects with non-implicit, non-defaulted ("required") attributes cannot be requested on the command line, only by a rule. RELNOTES: Expose parameterized aspects to Skylark. -- MOS_MIGRATED_REVID=121711715
* Delete the interface NativeAspectFactory and make native aspects extend from ↵Gravatar Luis Fernando Pino Duque2016-04-27
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | NativeAspectClass. This a large refactoring of the aspects, currently we have the following: - AspectClasses: The interface AspectClass is a implemented by either SkylarkAspectClass or NativeAspectClass<NativeAspectFactory>. They are wrappers for the AspectFactories and they hold the information about the Class<> of the factory. - AspectFactories (FooAspect.java): Represented by the interfaces ConfiguredAspectFactory and NativeAspectFactory, also by the interface ConfiguredNativeAspectFactory which is the union of the two aforementioned interfaces. All aspects implement ConfiguredNativeAspectFactory except Skylark aspects which implement only ConfiguredAspectFactory. After this CL the distinction between NativeAspectFactories and NativeAspectClasses dissappear, namely aspect that extends NativeAspectClass is considered native and if it implements ConfiguredAspectFactory it is configured. Therefore the interfaces NativeAspectFactory and ConfiguredNativeAspectFactory both disappear. With this refactoring the aspectFactoryMap in the ConfiguredRuleClassProvider changes its type from (String -> Class<? extends NativeAspectClass>) to (String -> NativeAspectClass) which means it is now able to have an instance of the aspect instead of its Class only. By doing this, it is now possible to pass parameters when creating an aspect in the ConfiguredRuleClassProvider. -- MOS_MIGRATED_REVID=120819647
* Split ActionMetadata into ActionAnalysisMetadata and ActionExecutionMetadata.Gravatar Rumou Duan2016-04-26
| | | | | | | Except in action execution logic (ActionExecutionFunction, SkyframeActionExecutor, etc.), switch Action interface references to either ActionAnalysisMetadata if possible or ActionExecutionMetadata. -- MOS_MIGRATED_REVID=120723431
* Make aspects work through bind(). Gravatar Lukacs Berki2016-04-22
| | | | | | | | | | | bind() is assumed to be able to provide any provider. This is suboptimal, but beats the alternative of traversing the dependency graph to an arbitrary depth. The reason for the removal of the iteration ability in TransitiveInfoCollection is that now aspects can be attached to BindConfiguredTarget, too, which is not a RuleConfiguredTarget. Whereas I could have implemented the iterator, it was used only in BindConfiguredTarget anyway, so there didn't seem to be much reason to. Some work towards #952. -- MOS_MIGRATED_REVID=120549877
* Remove SkylarkAspectClass.getDefintion.Gravatar Dmitry Lomov2016-04-07
| | | | | -- MOS_MIGRATED_REVID=119162307
* Remove AspectClass.getDefinitionGravatar Dmitry Lomov2016-04-07
| | | | | | | | | | Aspect becomes a triple (AspectClass, AspectDefinition, AspectParameters) and loses its equals() method. After this CL, SkylarkAspectClass.getDefintion still exists and is deprecated. -- MOS_MIGRATED_REVID=119159653
* Use the correct Aspect in AspectFunction for Skylark aspects.Gravatar Dmitry Lomov2016-04-07
| | | | | | | | | | | | | Previously AspectFunction was using an Aspect from the SkyKey, which might have been stale. This CL fixes the bug as uncovered in the test (see SkylarkAspectsTest), but further refactoring is needed since SkylarkAspectClass equals() is incorrect, and in fact obtaining the Skylark aspect definition should always introduce Skyframe dependency. -- MOS_MIGRATED_REVID=119137636
* Use two configurations for AspectKeys.Gravatar Michael Staib2016-03-01
| | | | | | | | | | | | In order for Aspects to support dynamic configuration, they need to have two configurations: one to instantiate the Aspect with, containing all the fragment dependencies of the Aspect itself, and one to create the ConfiguredTargetValue.key with, containing only the dependencies of the Rule. This expands AspectKey to have a second configuration, although it currently does not populate that key with anything different. -- MOS_MIGRATED_REVID=115997454
* Refactor DependencyResolver to collect and return loading errors.Gravatar Ulf Adams2016-01-28
| | | | | | | | | This should never be triggered in production, where we always run a loading phase first and only analyze targets that load successfully. I.e., this is just plumbing which will be hooked up in a subsequent change. -- MOS_MIGRATED_REVID=113258593
* Share some error handling code between aspects and top-level aspects.Gravatar Ulf Adams2016-01-25
| | | | | -- MOS_MIGRATED_REVID=112948493
* Make AspectFunction error handling match TopLevelAspectFunction.Gravatar Ulf Adams2016-01-25
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Also match ConfiguredTargetFunction for target loading. It isn't currently possible to trigger either of these code paths - the loading phase ensures that we never attempt to analyze targets that fail to load - the Skylark import or conversion cannot fail, because Skylark checks during .bzl execution that all referenced symbols are Skylark aspects Therefore, the only way to trigger this would be if there was a native rule requesting a non-existent or broken Skylark aspect for its dependencies, but that is currently not possible - native rules can only request native aspects. However, for interleaved loading and analysis, we need to limit the set of exception classes that can be thrown from AspectFunction - we do that here by changing the constructor of AspectFunctionException to only accept either NoSuchThingException or AspectCreationException. That in turn requires that we re-throw the Skylark import and conversion exceptions as AspectCreationException, which is exactly what TopLevelAspectFunction is already doing, and necessary for correct error handling if we do ever support Skylark aspects in native rules. Alternatively, I could change the code path to crash Bazel, but that seems strictly worse. Even if we can't test this code, it's conceptually the right way to handle these errors. I'll move part of the error handling into loadSkylarkAspect in a subsequent change. -- MOS_MIGRATED_REVID=112938284
* Move analysis root cause tracking to the ConfiguredTargetFunction.Gravatar Ulf Adams2016-01-25
| | | | | | | | | | | | | | | | | | | | | | | | | The main remaining problem with interleaved loading and analysis is error handling. When interleaving, we don't run a real loading phase anymore, and loading errors can occur during the analysis phase, and need to be handled there. The plan is to have ConfiguredTargetFunction throw a ConfiguredValueCreationException with a list of all loading root causes, which requires that we also catch ConfiguredValueCreationException here, which in turn breaks analysis root cause handling, as that is currently relying on Skyframe root cause tracking. Moving analysis root cause handling into CTFunction makes it possible to subsequently also implement loading root cause handling here. This is also necessary if we want to have complete root cause handling in the general case: a target may have any number (and combination) of loading and analysis root causes at the same time. For now, we only pass a single analysis root cause, which mirrors the current Skyframe-based implementation. -- MOS_MIGRATED_REVID=112930871
* Fix a couple of bugs related to error handling for top-level aspects.Gravatar Ulf Adams2016-01-20
| | | | | -- MOS_MIGRATED_REVID=112561390
* General cleanup for the configured target / aspect creation code.Gravatar Ulf Adams2016-01-19
| | | | | | | | | | | | | | - update some comments - add some comments to make it easier to follow - delete some dead code, in particular the SkyframeDependencyResolver can never be null; remove an non-applicable @Nullable annotation I'm trying to figure out how the error handling code works, in order to add support for interleaved loading+analysis, which requires handling loading errors in this code path. -- MOS_MIGRATED_REVID=112456674
* Use Bazel Preconditions variant which avoids varargs array creationGravatar Mark Schaller2015-12-10
| | | | | | | Reduces garbage. -- MOS_MIGRATED_REVID=109914243
* Support skylark aspects in serialization/deserialization.Gravatar Dmitry Lomov2015-11-30
| | | | | -- MOS_MIGRATED_REVID=108964575
* Enable load() statement in the WORKSPACE file.Gravatar Damien Martin-Guillerez2015-11-30
| | | | | | | | | | | RELNOTES[NEW]: Skylark macros are now enabled in WORKSPACE file. Design document at https://docs.google.com/document/d/1jKbNXOVp2T1zJD_iRnVr8k5D0xZKgO8blMVDlXOksJg/preview Fixes #337 -- MOS_MIGRATED_REVID=108860301
* Use Labels, rather than PathFragments, to represent Skylark loads ↵Gravatar John Field2015-11-13
| | | | | | | | | | | | | | | | | | | | | internally. The load location for a Skylark Aspect is specified via a PathFragment, for consistency with current non-Aspect Skylark loads. This should be a semantics-preserving change for users. In a subsequent CL, I'll change the Skylark syntax to allow load statements to use labels as well as paths, with the goal of eventually deprecating the latter. Also: - Removed the hack for handling relative loads in the prelude file. - Refactored some redundant functionality in PackageFunction and SkylarkImportLookupFunction for handling loads. - Removed the ability to put the BUILD file for the package containing a Skylark file under a different package root than the Skylark file itself. This functionality isn't currently used and is inconsistent with Blaze's handling of the package path elsewhere. - Added BUILD files to a number of tests that load Skylark files; this is consistent with the requirement that all Skylark files need to be part of some package. - Changed the constants used to set the location of the prelude file from paths to labels. -- MOS_MIGRATED_REVID=107741568
* Inline SkyframeBuildView.createAspect into its only call-site, to reduce ↵Gravatar Carmi Grushko2015-11-13
| | | | | | | stack depth. -- MOS_MIGRATED_REVID=107688035
* Aspect terminology update.Gravatar Dmitry Lomov2015-11-10
| | | | | | | | Aspect => ConfiguredAspect AspectWithParameters => Aspect -- MOS_MIGRATED_REVID=107375211
* Pass AspectWithParameters instead of separately passing AspectParameters and ↵Gravatar Carmi Grushko2015-11-04
| | | | | | | | | aspect-attributes. The former contains both. -- MOS_MIGRATED_REVID=106961145
* When creating RuleContext, explicitly pass the set of attributes an attached ↵Gravatar Carmi Grushko2015-11-02
| | | | | | | Aspect provides. -- MOS_MIGRATED_REVID=106882046
* Parametrize aspect definition with AspectParameters.Gravatar Dmitry Lomov2015-11-02
| | | | | -- MOS_MIGRATED_REVID=106848269
* Implement propagation along dependencies for Skylark aspects.Gravatar Dmitry Lomov2015-11-02
| | | | | -- MOS_MIGRATED_REVID=106694515
* Implement aspect(...) Skylark function.Gravatar Dmitry Lomov2015-10-20
| | | | | -- MOS_MIGRATED_REVID=105844221
* Rollback of commit ac7195025b073948785db8c6975a53fc305c5087.Gravatar Peter Schmitt2015-10-20
| | | | | | | | | | | | | *** Reason for rollback *** Uses tests that don't run on Bazel *** Original change description *** Implement aspect(...) Skylark function. -- MOS_MIGRATED_REVID=105808850
* Implement aspect(...) Skylark function.Gravatar Dmitry Lomov2015-10-16
| | | | | -- MOS_MIGRATED_REVID=105596479
* Rationalize copyright headersGravatar Damien Martin-Guillerez2015-09-25
| | | | | | | | | | | The headers were modified with `find . -type f -exec 'sed' '-Ei' 's|Copyright 201([45]) Google|Copyright 201\1 The Bazel Authors|' '{}' ';'` And manual edit for not Google owned copyright. Because of the nature of ijar, I did not modified the header of file owned by Alan Donovan. The list of authors were extracted from the git log. It is missing older Google contributors that can be added on-demand. -- MOS_MIGRATED_REVID=103938715
* Stop throwing an exception if a Package was successfully created but ↵Gravatar Janak Ramakrishnan2015-09-17
| | | | | | | | | | | contains errors. Instead, require callers to process the package and throw if they need to. This allows us to avoid embedding a Package in an exception, which is icky. This also allows us to remove Package#containsTemporaryErrors. Most callers' changes are fairly straightforward. The exception is EnvironmentBackedRecursivePackageProvider, which cannot throw an exception of its own in case of a package with errors (because it doesn't do that in keep_going mode), but whose request for a package with errors *should* shut down the build in case of nokeep_going mode. To do this in Skyframe, we have a new PackageErrorFunction which is to be called only in this situation, and will unconditionally throw. EnvironmentBackedRecursivePackageProvider can then catch this exception and continue on as usual, except that the exception will shut down the thread pool in a nokeep_going build. -- MOS_MIGRATED_REVID=103247761
* Prevented catching/wrapping of InterruptedExceptions, especially in ↵Gravatar Florian Weikert2015-09-14
| | | | | | | BaseFunction. -- MOS_MIGRATED_REVID=102988766
* Fix a bug, when we have dependency from one aspect to another ↵Gravatar Marian Lobur2015-09-14
| | | | | | | aspect1->aspect2 and aspect2 fails then aspect1 throws IllegalStateException. -- MOS_MIGRATED_REVID=102976139
* Save information about transitive packages in ConfiguredTargetValue and ↵Gravatar Marian Lobur2015-09-09
| | | | | | | AspectValue. -- MOS_MIGRATED_REVID=102643564
* Aspects can get information from their base rule.Gravatar Marian Lobur2015-09-02
| | | | | -- MOS_MIGRATED_REVID=102126786
* Implement the core structure for dynamic configurations.Gravatar Greg Estren2015-08-26
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is a big change, so let me walk you through the key pieces: 1) This cl provides an alternative mechanism for creating configurations and doing configuration transitions that's "dynamic" in that the configurations can be created on the fly and the transitions are arbitrary mappings from BuildOptions --> BuildOptions that can also be created on the fly. It also integrates this with ConfiguredTargetFunction, so the configured target graph automatically uses this framework. 2) It does *not* replace old-style (which we'll call "static") configurations. For a number of important reasons: It's not yet at feature parity (particularly: no LIPO). It's not remotely tested across real projects enough to have confidence that it's battle-ready. It doesn't yet handle certain "special" functions like BuildConfiguration.prepareToBuild() and BuildConfiguration.getRoots(). It still relies on the old static transition logic to determine what transitions to apply (eventually we'll distribute that logic out, but that's too much for a single cl). We need the option to toggle it on and off until we have enough confidence in it. So with this cl, builds can be done in either mode. 3) The new flag --experimental_dynamic_configs toggles use of dynamic configurations. 4) Dynamic configurations are created with the Skyframe function BuildConfigurationFunction (this was already created in an earlier change). This consumes a BuildOptions and a set of configuration fragments to produce a BuildConfiguration. 5) Dynamic transitions are instances of the new class PatchTransition, which simply maps an input BuildOptions to an output BuildOptions. 6) Since static and dynamic configurations have to co-exist (for now), this cl tries hard to keep today's transition logic defined in a single place (vs. forking a dedicated copy for each configuration style). This is done via the new interface BuildConfiguration.TransitionApplier. BuildConfiguration.evaluateTransition is modified to feed its transition logic into TransitionApplier's common API. Both dynamic and static configurations have their own implementations that "do the right thing" with the results. 7) The transition applier for dynamic configurations basically stores the Transition, then allows DependencyResolver (which calls BuildConfiguration.evaluateTransition) to return Dependency instances containing that Transition (vs. a BuildConfiguration, which they traditionally contain). 7.5) An earlier variation of the dynamic transition applier retained BuildOptions (e.g. when it got a Transition it immediately applied it to get its output BuildOptions, then stored that). This had the advantage of making composing of transitions easier, especially within BuildConfiguration.evaluateTransition (which can theoretically apply multiple transitions to the input configuration). But it turns out that applying transitions has a cost, and it's simply more performant to pass them through until they're really needed. 8) In dynamic configuration mode, after ConfiguredTargetFunction gets its deps (e.g. an <Attribute, Dependency> multimap), it "trims" the configurations for its dependencies by a) only including config fragments required by the deps' subtrees and b) applying the transitions that came from 7). This all happens in the new method ConfiguredTargetFunction.trimConfigurations. 9) trimConfigurations is heavily performance-optimized based on a lot of experience running this through a large project within Google. As it turns out, the cost of host transitions can be atrocious (because there are a lot of them). Also, BuildOptions.clone() is expensive. And just creating BuildConfiguration SkyKeys also has a cost (largely because of BuildOptions cloning), so that shouldn't be done except when really necessary. My experience with this convinced me it's worth making this method complicated for the sake of making it fast. Since it basically visits every edge in the configured target graph (at least), it really needs to be slick. 10) Since host transitions in particular are problematic w.r.t. speed, I compute the host *once* in ConfigurationCollectionFunction.getHostConfiguration() and expose that reference to ConfiguredTargetFunction and other Skyframe functions. This limits recomputation to just when the fragments are trimmed. 11) Since options cloning is expensive, I'm doing something scary: exposing a BuildConfiguration.getOptions() method that returns a direct reference. Since BuildOptions is mutable, this is dangerous in the wrong hands. I can experiment with going back to cloning (with the caching of host transitions it may not matter as much), but we may ultimately have to put up with this risk for the sake of performant analysis time. What would be *really* awesome would be to make BuildOptions immutable. But that's not going to happen in this cl. So in short, the key abstractions in this cl are: - PatchTransition - BuildConfiguration.TransitionApplier - ConfiguredTargetFunction.trimConfigurations The current implementation imposes no analysis time penalty -- MOS_MIGRATED_REVID=101474620
generated by cgit v1.2.3 (git 2.39.1) at 2024-07-03 06:10:12 +0000