| Commit message (Collapse) | Author | Age |
|---|
| |
|
|
|
|
|
| |
This is accomplished by moving it to ConfiguredRuleClassProvider. This also suggests a neat way to get rid of logic in ShellConfiguration.Loader() by moving the determination of the shell executable, there, too, but not in this change.
RELNOTES: None.
PiperOrigin-RevId: 192411609
|
| |
|
|
|
|
|
|
|
| |
encoding the same in ConfiguredRuleClassProvider.
This is a step towards dumbing down BuildConfiguration.Fragment and the ConfigurationFactoryLoader, which is in needed so that we can rewrite C++/Java/Python rules in Skylark without having to introduce the concept of "configuration loader" in Skylark, too.
RELNOTES: None.
PiperOrigin-RevId: 192104912
|
| |
|
|
|
|
| |
from the provided defaults in Options classes. These are used to create BuildOptionsDiffForReconstruction, which lets us store only the diffs in our BuildConfigurationValue.Keys.
PiperOrigin-RevId: 190117455
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This accomplishes a few goals:
1. Removes the outdated BuildConfiguration.StaticConfigurationApplier code.
2. Removes the TransitionApplier abstraction completely. This was an awkward
bridge meant to support both static and dynamic implementations.
3. Moves transition logic to its own dedicated class: ConfigurationResolver.
This no longer belongs in BuildConfiguration, which we ultimately want to
become a simple <key, value> map.
Part of the static config cleanup effort.
PiperOrigin-RevId: 165736955
|
| |
|
|
| |
PiperOrigin-RevId: 165578449
|
| |
|
|
| |
PiperOrigin-RevId: 161432622
|
| |
|
|
|
|
|
|
|
|
|
|
| |
This is a legacy dependency on the configuration transition table, which is
only needed for static configurations. Dynamic configurations didn't actually
use anything in that table: this was just a convenience interface that could
have equally been defined somewhere else. So this cl defines it somewhere else.
There's still one last dependency: Transitions.configurationHook. We'll tackle
that in a followup cl.
PiperOrigin-RevId: 161141650
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
*** Reason for rollback ***
Roll forward of directory name change
*** Original change description ***
Automated g4 rollback of commit 1d9e1ac90197b1d3d7b137ba3c1ada67bb9ba31b.
*** Reason for rollback ***
Breaks //src/test/shell/integration:force_delete_output_test
*** Original change description ***
Symlink output directories to the correct directory name
If the workspace directory is /path/to/my/proj and the name in the WORKSPACE
file is "floop", this will symlink the output directories to
output_base/execroot/floop instead of output_base/execroot/proj.
More prep for #1262, fixes #1681.
PiperOrigin-RevId: 156892980
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
1) Remove obsolete originalOptions field. This was originally added to
support "parent" transitions, which supported config1 -> config2 -> config1
transitions by having config2 store config1's options.
The purpose of this feature was to support LIPO (which has a DATA -> TARGET
transition). But https://github.com/bazelbuild/bazel/commit/ff29c0b39cf936a2699b05edd54f483f1a037d93
makes this unnecessary.
2) Support the "disable actions" feature of the LIPO context collector configuration.
Putting this in BuildOptions make this dynamic config-compatible.
This change intentionally doesn't add disableOptions to BuildOptions.equals() or
BuildOptions.hashCode(). It'd be great to do that. But that has semantic consequences.
And we've run into really tricky bugs in the past with dynamic configurations and
BuildOptions.equals / BuildConfiguration.equals. So it's best to experiment with
that in its own change.
PiperOrigin-RevId: 154999718
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
*** Reason for rollback ***
Breaks //src/test/shell/integration:force_delete_output_test
*** Original change description ***
Symlink output directories to the correct directory name
If the workspace directory is /path/to/my/proj and the name in the WORKSPACE
file is "floop", this will symlink the output directories to
output_base/execroot/floop instead of output_base/execroot/proj.
More prep for #1262, fixes #1681.
PiperOrigin-RevId: 152126545
|
| |
|
|
|
|
|
|
|
|
| |
If the workspace directory is /path/to/my/proj and the name in the WORKSPACE
file is "floop", this will symlink the output directories to
output_base/execroot/floop instead of output_base/execroot/proj.
More prep for #1262, fixes #1681.
PiperOrigin-RevId: 151712384
|
| |
|
|
|
|
|
| |
The only place we now don't handle InterruptedException is in the action graph created after analysis, since I'm not sure that will be around for that much longer.
--
MOS_MIGRATED_REVID=130327770
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
options actually needed by the fragment. This protects against, e.g.,
unnecessarily duplicating CppConfiguration instances when only Java flags
change.
This is a recommit of ca1b21ac6d8a58041db822725b42de151b163dee which was
rolled back because it broke LIPO.
This change is particularly important for dynamic configurations, which may
mix and match fragments arbitrarily throughout a build. This not only has
performance implications, but also correctness implications: code that
expects two configured targets to have the same fragment (value) shouldn't
break just because the second CT's configuration is a trimmed version of the
first's.
The original change breaks FDO/LIPO because CppConfiguration can't be
shared across configurations. That's because it mutates state when
prepareHook() is called, and each configuration calls prepareHook. We
should ultimately solve this by refactoring the FDO/LIPO implementation
but don't want to block dynamic configuration progress on that. So this
change only enables trimming for dynamic configurations.
--
MOS_MIGRATED_REVID=113570250
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The main remaining problem with interleaved loading and analysis is error
handling. When interleaving, we don't run a real loading phase anymore, and
loading errors can occur during the analysis phase, and need to be handled
there.
The plan is to have ConfiguredTargetFunction throw a
ConfiguredValueCreationException with a list of all loading root causes, which
requires that we also catch ConfiguredValueCreationException here, which in
turn breaks analysis root cause handling, as that is currently relying on
Skyframe root cause tracking.
Moving analysis root cause handling into CTFunction makes it possible to
subsequently also implement loading root cause handling here. This is also
necessary if we want to have complete root cause handling in the general case:
a target may have any number (and combination) of loading and analysis root
causes at the same time.
For now, we only pass a single analysis root cause, which mirrors the current
Skyframe-based implementation.
--
MOS_MIGRATED_REVID=112930871
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
*** Reason for rollback ***
Broke some alipo builds. I had to leave in the change to TransitiveTargetFunctionTest, which is relied upon by follow-up CLs.
*** Original change description ***
Pre-trim build options for all remaining calls to ConfigurationFragment.key. Move
the trimming logic into key() itself to:
a) eliminate code redundancy
b) guarantee all future calls to key() also do this.
--
MOS_MIGRATED_REVID=109683849
|
| |
|
|
|
|
|
|
|
|
|
|
| |
Move
the trimming logic into key() itself to:
a) eliminate code redundancy
b) guarantee all future calls to key() also do this.
--
MOS_MIGRATED_REVID=107713353
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
trim the BuildOptions to only those needed by the fragment.
This guarantees that the same fragment only gets instantiated
once (for example: we don't want to have two copies of a
CppConfiguration because one copy had PythonOptions and the
other didn't).
This also prevents crashes, e.g. in the Preconditions check in
CcLibraryHelper.addDeps.
--
MOS_MIGRATED_REVID=107609270
|
| |
|
|
|
|
|
|
|
|
|
| |
The headers were modified with
`find . -type f -exec 'sed' '-Ei' 's|Copyright 201([45]) Google|Copyright 201\1 The Bazel Authors|' '{}' ';'`
And manual edit for not Google owned copyright. Because of the nature of ijar, I did not modified the header of file owned by Alan Donovan.
The list of authors were extracted from the git log. It is missing older Google contributors that can be added on-demand.
--
MOS_MIGRATED_REVID=103938715
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This is a big change, so let me walk you through the key pieces:
1) This cl provides an alternative mechanism for creating configurations and doing configuration transitions that's "dynamic" in that the configurations can be created on the fly and the transitions are arbitrary mappings from BuildOptions --> BuildOptions that can also be created on the fly. It also integrates this with ConfiguredTargetFunction, so the configured target graph automatically uses this framework.
2) It does *not* replace old-style (which we'll call "static") configurations. For a number of important reasons: It's not yet at feature parity (particularly: no LIPO). It's not remotely tested across real projects enough to have confidence that it's battle-ready. It doesn't yet handle certain "special" functions like BuildConfiguration.prepareToBuild() and BuildConfiguration.getRoots(). It still relies on the old static transition logic to determine what transitions to apply (eventually we'll distribute that logic out, but that's too much for a single cl). We need the option to toggle it on and off until we have enough confidence in it. So with this cl, builds can be done in either mode.
3) The new flag --experimental_dynamic_configs toggles use of dynamic configurations.
4) Dynamic configurations are created with the Skyframe function BuildConfigurationFunction (this was already created in an earlier change). This consumes a BuildOptions and a set of configuration fragments to produce a BuildConfiguration.
5) Dynamic transitions are instances of the new class PatchTransition, which simply maps an input BuildOptions to an output BuildOptions.
6) Since static and dynamic configurations have to co-exist (for now), this cl tries hard to keep today's transition logic defined in a single place (vs. forking a dedicated copy for each configuration style). This is done via the new interface BuildConfiguration.TransitionApplier. BuildConfiguration.evaluateTransition is modified to feed its transition logic into TransitionApplier's common API. Both dynamic and static configurations have their own implementations that "do the right thing" with the results.
7) The transition applier for dynamic configurations basically stores the Transition, then allows DependencyResolver (which calls BuildConfiguration.evaluateTransition) to return Dependency instances containing that Transition (vs. a BuildConfiguration, which they traditionally contain).
7.5) An earlier variation of the dynamic transition applier retained BuildOptions (e.g. when it got a Transition it immediately applied it to get its output BuildOptions, then stored that). This had the advantage of making composing of transitions easier, especially within BuildConfiguration.evaluateTransition (which can theoretically apply multiple transitions to the input configuration). But it turns out that applying transitions has a cost, and it's simply more performant to pass them through until they're really needed.
8) In dynamic configuration mode, after ConfiguredTargetFunction gets its deps (e.g. an <Attribute, Dependency> multimap), it "trims" the configurations for its dependencies by a) only including config fragments required by the deps' subtrees and b) applying the transitions that came from 7). This all happens in the new method ConfiguredTargetFunction.trimConfigurations.
9) trimConfigurations is heavily performance-optimized based on a lot of experience running this through a large project within Google. As it turns out, the cost of host transitions can be atrocious (because there are a lot of them). Also, BuildOptions.clone() is expensive. And just creating BuildConfiguration SkyKeys also has a cost (largely because of BuildOptions cloning), so that shouldn't be done except when really necessary. My experience with this convinced me it's worth making this method complicated for the sake of making it fast. Since it basically visits every edge in the configured target graph (at least), it really needs to be slick.
10) Since host transitions in particular are problematic w.r.t. speed, I compute the host *once* in ConfigurationCollectionFunction.getHostConfiguration() and expose that reference to ConfiguredTargetFunction and other Skyframe functions. This limits recomputation to just when the fragments are trimmed.
11) Since options cloning is expensive, I'm doing something scary: exposing a BuildConfiguration.getOptions() method that returns a direct reference. Since BuildOptions is mutable, this is dangerous in the wrong hands. I can experiment with going back to cloning (with the caching of host transitions it may not matter as much), but we may ultimately have to put up with this risk for the sake of performant analysis time. What would be *really* awesome would be to make BuildOptions immutable. But that's not going to happen in this cl.
So in short, the key abstractions in this cl are:
- PatchTransition
- BuildConfiguration.TransitionApplier
- ConfiguredTargetFunction.trimConfigurations
The current implementation imposes no analysis time penalty
--
MOS_MIGRATED_REVID=101474620
|
|
|
configuration fragments needed by a rule's transitive
closure.
Also add a Skyframe BuildConfiguration node.
Memory and performance profiling shows no noticeable
performance hit in loading or analysis and a 0.35%
memory increase for moderately sized (by Google
standards) build graphs when these are depended
upon in ConfiguredTargetFunction.
--
MOS_MIGRATED_REVID=94517099
|
lberki
mjhalupka
gregce
kchodorow
hlopko
Janak Ramakrishnan
Ulf Adams
Damien Martin-Guillerez