| Commit message (Collapse) | Author | Age |
|---|
| |
|
|
|
|
|
|
| |
Add test coverage by re-running BuildViewTest with the new Skyframe loading
phase runner.
--
MOS_MIGRATED_REVID=113517509
|
| |
|
|
|
|
|
|
|
| |
This should never be triggered in production, where we always run a loading
phase first and only analyze targets that load successfully. I.e., this is
just plumbing which will be hooked up in a subsequent change.
--
MOS_MIGRATED_REVID=113258593
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Dependencies are the data structure which needs to propagate the configuration for each
aspect as created by trimConfigurations down to the point where it's actually used. We
need this to store different configurations for different aspects in a world where aspects
have their own configurations, which may have more fragments than the target they're
attached to.
That world is on its way.
Also in this CL:
* Refactor Dependency to be an abstract parent class with separate implementations for
Attribute.Transitions and BuildConfigurations, as well as null configurations, to avoid
having to check nullness in various places. Users of the API will not see this, but get
factory methods instead of constructors. As a consequence of this, refactor Dependency
to be its own top-level class instead of a nested class in DependencyResolver.
--
MOS_MIGRATED_REVID=113109615
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Also match ConfiguredTargetFunction for target loading.
It isn't currently possible to trigger either of these code paths
- the loading phase ensures that we never attempt to analyze targets that fail
to load
- the Skylark import or conversion cannot fail, because Skylark checks during
.bzl execution that all referenced symbols are Skylark aspects
Therefore, the only way to trigger this would be if there was a native rule
requesting a non-existent or broken Skylark aspect for its dependencies, but
that is currently not possible - native rules can only request native aspects.
However, for interleaved loading and analysis, we need to limit the set of
exception classes that can be thrown from AspectFunction - we do that here
by changing the constructor of AspectFunctionException to only accept either
NoSuchThingException or AspectCreationException.
That in turn requires that we re-throw the Skylark import and conversion
exceptions as AspectCreationException, which is exactly what
TopLevelAspectFunction is already doing, and necessary for correct error
handling if we do ever support Skylark aspects in native rules.
Alternatively, I could change the code path to crash Bazel, but that seems
strictly worse. Even if we can't test this code, it's conceptually the right
way to handle these errors.
I'll move part of the error handling into loadSkylarkAspect in a subsequent
change.
--
MOS_MIGRATED_REVID=112938284
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The main remaining problem with interleaved loading and analysis is error
handling. When interleaving, we don't run a real loading phase anymore, and
loading errors can occur during the analysis phase, and need to be handled
there.
The plan is to have ConfiguredTargetFunction throw a
ConfiguredValueCreationException with a list of all loading root causes, which
requires that we also catch ConfiguredValueCreationException here, which in
turn breaks analysis root cause handling, as that is currently relying on
Skyframe root cause tracking.
Moving analysis root cause handling into CTFunction makes it possible to
subsequently also implement loading root cause handling here. This is also
necessary if we want to have complete root cause handling in the general case:
a target may have any number (and combination) of loading and analysis root
causes at the same time.
For now, we only pass a single analysis root cause, which mirrors the current
Skyframe-based implementation.
--
MOS_MIGRATED_REVID=112930871
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
Note that this can never happen in production - the loading phase ensures that
we only attempt to analyze targets that load successfully. That's also why
there are no tests for this.
This code path will become live in a later change, with corresponding test
coverage; the existing tests cover this in principle, we just need to make
them run with the reduced loading phase.
--
MOS_MIGRATED_REVID=112926211
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
- update some comments
- add some comments to make it easier to follow
- delete some dead code, in particular the SkyframeDependencyResolver can
never be null; remove an non-applicable @Nullable annotation
I'm trying to figure out how the error handling code works, in order to add
support for interleaved loading+analysis, which requires handling loading
errors in this code path.
--
MOS_MIGRATED_REVID=112456674
|
| |
|
|
|
|
|
|
|
|
|
| |
statements work in remote repositories.
This work is somewhat sisyphean; the principled thing to do would be to just do this resolution when labels are parsed.
Fixes #783.
--
MOS_MIGRATED_REVID=112137996
|
| |
|
|
|
|
|
|
|
|
|
|
| |
The ConfiguredTargetFunction can only throw a NoSuch{Target,Package}Exception
if it's run for a non-existent target. However, it will never request a CT for
a non-existent target: all labels are run through DependencyResolver in order
to determine the configuration transition, which requires loading the target.
If it doesn't exist, the label is never returned from the DependencyResolver,
and errors are swallowed (we rely on the loading phase to report the error).
--
MOS_MIGRATED_REVID=112127012
|
| |
|
|
|
|
|
| |
Reduces garbage.
--
MOS_MIGRATED_REVID=109914243
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
as its input.
e.g. given input {"attr": [':a', ':b']}
the trimmed version also keeps ':a' in front of ':b'.
This preserves the expected invariant that prerequisites are
navigatable in BUILD declaration order.
--
MOS_MIGRATED_REVID=108170921
|
| |
|
|
|
|
|
|
|
|
|
| |
in ConfiguredTargetFunction.trimConfigurations to preserve
key insertion order.
This ensures that RuleContext.getPrerequisites returns an
attribute's deps in the same order they were declared.
--
MOS_MIGRATED_REVID=108063142
|
| |
|
|
|
|
|
|
|
| |
configuration transition *before* trimming the options. This
is because the transition may read fragments that aren't
in the dep itself.
--
MOS_MIGRATED_REVID=107624186
|
| |
|
|
|
|
|
|
| |
Aspect => ConfiguredAspect
AspectWithParameters => Aspect
--
MOS_MIGRATED_REVID=107375211
|
| |
|
|
|
| |
--
MOS_MIGRATED_REVID=106848269
|
| |
|
|
|
| |
--
MOS_MIGRATED_REVID=106694515
|
| |
|
|
|
|
|
| |
For native aspects, AspectClass is a facade for Class<AspectFactory<...>>.
--
MOS_MIGRATED_REVID=105986390
|
| |
|
|
|
| |
--
MOS_MIGRATED_REVID=105844221
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
*** Reason for rollback ***
Uses tests that don't run on Bazel
*** Original change description ***
Implement aspect(...) Skylark function.
--
MOS_MIGRATED_REVID=105808850
|
| |
|
|
|
| |
--
MOS_MIGRATED_REVID=105596479
|
| |
|
|
|
|
|
|
|
|
| |
(following PackageGroup's precedent).
Also cleanup: generalize the pattern by which we determine
non-configurable target types.
--
MOS_MIGRATED_REVID=104125803
|
| |
|
|
|
|
|
|
|
|
|
| |
The headers were modified with
`find . -type f -exec 'sed' '-Ei' 's|Copyright 201([45]) Google|Copyright 201\1 The Bazel Authors|' '{}' ';'`
And manual edit for not Google owned copyright. Because of the nature of ijar, I did not modified the header of file owned by Alan Donovan.
The list of authors were extracted from the git log. It is missing older Google contributors that can be added on-demand.
--
MOS_MIGRATED_REVID=103938715
|
| |
|
|
|
|
|
|
|
|
| |
- Label parsing can be simplified
- lib.syntax is only contains the code for Skylark and is reasonably independent from the problem domain of building things
This change is mostly only changes to imports declarations. The rest is reversing the dependency between :cmdline and :syntax and moving a tiny amount of code between Printer and FilesetEntry and the addition of SkylarkPrintableValue that I couldn't be bothered to separate out into its own change.
--
MOS_MIGRATED_REVID=103527877
|
| |
|
|
|
| |
--
MOS_MIGRATED_REVID=103374106
|
| |
|
|
|
|
|
|
|
|
|
| |
contains errors. Instead, require callers to process the package and throw if they need to.
This allows us to avoid embedding a Package in an exception, which is icky. This also allows us to remove Package#containsTemporaryErrors.
Most callers' changes are fairly straightforward. The exception is EnvironmentBackedRecursivePackageProvider, which cannot throw an exception of its own in case of a package with errors (because it doesn't do that in keep_going mode), but whose request for a package with errors *should* shut down the build in case of nokeep_going mode. To do this in Skyframe, we have a new PackageErrorFunction which is to be called only in this situation, and will unconditionally throw. EnvironmentBackedRecursivePackageProvider can then catch this exception and continue on as usual, except that the exception will shut down the thread pool in a nokeep_going build.
--
MOS_MIGRATED_REVID=103247761
|
| |
|
|
|
|
|
| |
BaseFunction.
--
MOS_MIGRATED_REVID=102988766
|
| |
|
|
|
|
|
| |
aspect1->aspect2 and aspect2 fails then aspect1 throws IllegalStateException.
--
MOS_MIGRATED_REVID=102976139
|
| |
|
|
|
|
|
| |
AspectValue.
--
MOS_MIGRATED_REVID=102643564
|
| |
|
|
|
| |
--
MOS_MIGRATED_REVID=102332437
|
| |
|
|
|
| |
--
MOS_MIGRATED_REVID=102126786
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This is a big change, so let me walk you through the key pieces:
1) This cl provides an alternative mechanism for creating configurations and doing configuration transitions that's "dynamic" in that the configurations can be created on the fly and the transitions are arbitrary mappings from BuildOptions --> BuildOptions that can also be created on the fly. It also integrates this with ConfiguredTargetFunction, so the configured target graph automatically uses this framework.
2) It does *not* replace old-style (which we'll call "static") configurations. For a number of important reasons: It's not yet at feature parity (particularly: no LIPO). It's not remotely tested across real projects enough to have confidence that it's battle-ready. It doesn't yet handle certain "special" functions like BuildConfiguration.prepareToBuild() and BuildConfiguration.getRoots(). It still relies on the old static transition logic to determine what transitions to apply (eventually we'll distribute that logic out, but that's too much for a single cl). We need the option to toggle it on and off until we have enough confidence in it. So with this cl, builds can be done in either mode.
3) The new flag --experimental_dynamic_configs toggles use of dynamic configurations.
4) Dynamic configurations are created with the Skyframe function BuildConfigurationFunction (this was already created in an earlier change). This consumes a BuildOptions and a set of configuration fragments to produce a BuildConfiguration.
5) Dynamic transitions are instances of the new class PatchTransition, which simply maps an input BuildOptions to an output BuildOptions.
6) Since static and dynamic configurations have to co-exist (for now), this cl tries hard to keep today's transition logic defined in a single place (vs. forking a dedicated copy for each configuration style). This is done via the new interface BuildConfiguration.TransitionApplier. BuildConfiguration.evaluateTransition is modified to feed its transition logic into TransitionApplier's common API. Both dynamic and static configurations have their own implementations that "do the right thing" with the results.
7) The transition applier for dynamic configurations basically stores the Transition, then allows DependencyResolver (which calls BuildConfiguration.evaluateTransition) to return Dependency instances containing that Transition (vs. a BuildConfiguration, which they traditionally contain).
7.5) An earlier variation of the dynamic transition applier retained BuildOptions (e.g. when it got a Transition it immediately applied it to get its output BuildOptions, then stored that). This had the advantage of making composing of transitions easier, especially within BuildConfiguration.evaluateTransition (which can theoretically apply multiple transitions to the input configuration). But it turns out that applying transitions has a cost, and it's simply more performant to pass them through until they're really needed.
8) In dynamic configuration mode, after ConfiguredTargetFunction gets its deps (e.g. an <Attribute, Dependency> multimap), it "trims" the configurations for its dependencies by a) only including config fragments required by the deps' subtrees and b) applying the transitions that came from 7). This all happens in the new method ConfiguredTargetFunction.trimConfigurations.
9) trimConfigurations is heavily performance-optimized based on a lot of experience running this through a large project within Google. As it turns out, the cost of host transitions can be atrocious (because there are a lot of them). Also, BuildOptions.clone() is expensive. And just creating BuildConfiguration SkyKeys also has a cost (largely because of BuildOptions cloning), so that shouldn't be done except when really necessary. My experience with this convinced me it's worth making this method complicated for the sake of making it fast. Since it basically visits every edge in the configured target graph (at least), it really needs to be slick.
10) Since host transitions in particular are problematic w.r.t. speed, I compute the host *once* in ConfigurationCollectionFunction.getHostConfiguration() and expose that reference to ConfiguredTargetFunction and other Skyframe functions. This limits recomputation to just when the fragments are trimmed.
11) Since options cloning is expensive, I'm doing something scary: exposing a BuildConfiguration.getOptions() method that returns a direct reference. Since BuildOptions is mutable, this is dangerous in the wrong hands. I can experiment with going back to cloning (with the caching of host transitions it may not matter as much), but we may ultimately have to put up with this risk for the sake of performant analysis time. What would be *really* awesome would be to make BuildOptions immutable. But that's not going to happen in this cl.
So in short, the key abstractions in this cl are:
- PatchTransition
- BuildConfiguration.TransitionApplier
- ConfiguredTargetFunction.trimConfigurations
The current implementation imposes no analysis time penalty
--
MOS_MIGRATED_REVID=101474620
|
| |
|
|
|
| |
--
MOS_MIGRATED_REVID=101033236
|
| |
|
|
|
|
|
| |
A previous change made the loading-time external label resolution unused, thus, now we can do away with a lot of machinery. The only unfortunate side effect is that instead of a nice and clear "No Android SDK found" error message, you'll get a more cryptic "external label //external:android/sdk is unbound" one. I think it's a fair tradeoff.
--
MOS_MIGRATED_REVID=98813719
|
| |
|
|
|
|
|
|
|
|
|
| |
//external:android/sdk anywhere so that no Android-specific package is loaded when the user doesn't use an android_sdk_repository rule.
To this end, SkyframePackageLoaderWithValueEnvironment.getLoadedTarget() doesn't resolve //external: labels anymore. This was only needed for JVM resolution, which was dealt with by adding and extra RedirectChaser.followRedirect() call to JvmConfigurationLoader. One hack less.
On the flip side, BazelConfigurationCollection.collectTransitiveClosure() grew a hack to handle bind(), but that method is awful enough as it is anyway.
--
MOS_MIGRATED_REVID=97307779
|
| |
|
|
|
|
|
|
|
| |
Dynamic configuration transitions require access to Skyframe (since they instantiate BuildConfigurations as Skyframe nodes). There are various places in Bazel where static transitions are done with no convenient Skyframe access. This cl shuffles host transitions, in particular, to places that are more amenable.
This change also assumes one host configuration per invocation. While this isn't strictly true (each target configuration can have its own host, and multiple target configurations are possible per build), we don't leverage that functionality in any meaningful way today. So until we have a proper interface for multiple host configurations, let's not block dynamic config progress on it.
--
MOS_MIGRATED_REVID=97008479
|
| |
|
|
|
|
|
|
|
| |
TransitiveTargetFunction only prints an error message if the package names
match, so it was just exiting with "loading failed" when there was an error
with external dependencies.
--
MOS_MIGRATED_REVID=96204337
|
| |
|
|
|
|
|
| |
different attributes, to which we have attached aspect.
--
MOS_MIGRATED_REVID=93412457
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
Instead of computing flags based on the configuration in two places, we pass
in the configuration and let the common code figure it out.
Also change it so each CT only depends on the build info collection for its
specific configuration. This should be safe, as the API doesn't allow passing
a configuration in - it instead takes ruleContext.getConfiguration, which
should be consistent with the dependency declarations in SkyframeBuildView in
all cases.
--
MOS_MIGRATED_REVID=91819844
|
| |
|
|
|
| |
--
MOS_MIGRATED_REVID=90574562
|
| |
|
|
|
|
|
| |
and fail hard in other cases.
--
MOS_MIGRATED_REVID=89720528
|
| |
|
|
|
| |
--
MOS_MIGRATED_REVID=89704303
|
| |
|
|
|
| |
--
MOS_MIGRATED_REVID=85722639
|
|
|
--
MOE_MIGRATED_REVID=85702957
|
Ulf Adams
Michael Staib
Lukacs Berki
Mark Schaller
Greg Estren
Dmitry Lomov
Peter Schmitt
Damien Martin-Guillerez
Janak Ramakrishnan
Florian Weikert
Marian Lobur
Kristina Chodorow
Nathan Harmata
Michajlo Matijkiw
Laszlo Csomor
Han-Wen Nienhuys