| Commit message (Collapse) | Author | Age |
|---|
| |
|
|
|
| |
--
MOS_MIGRATED_REVID=104009600
|
| |
|
|
|
|
|
|
|
|
|
| |
The headers were modified with
`find . -type f -exec 'sed' '-Ei' 's|Copyright 201([45]) Google|Copyright 201\1 The Bazel Authors|' '{}' ';'`
And manual edit for not Google owned copyright. Because of the nature of ijar, I did not modified the header of file owned by Alan Donovan.
The list of authors were extracted from the git log. It is missing older Google contributors that can be added on-demand.
--
MOS_MIGRATED_REVID=103938715
|
| |
|
|
|
|
|
| |
Removes mutable global state.
--
MOS_MIGRATED_REVID=103837106
|
| |
|
|
|
|
|
|
|
| |
Instead, pass an appropriate EventHandler instance in. This is in preparation
for creating a per-command EventHandler, in preparation for allowing multiple
commands to run in parallel. This is removal of shared global state.
--
MOS_MIGRATED_REVID=103828963
|
| |
|
|
|
|
|
|
| |
Also move ownership of ArtifactFactory to SkyframeBuildView; simplify the
code.
--
MOS_MIGRATED_REVID=103722228
|
| |
|
|
|
|
|
|
|
|
| |
- Label parsing can be simplified
- lib.syntax is only contains the code for Skylark and is reasonably independent from the problem domain of building things
This change is mostly only changes to imports declarations. The rest is reversing the dependency between :cmdline and :syntax and moving a tiny amount of code between Printer and FilesetEntry and the addition of SkylarkPrintableValue that I couldn't be bothered to separate out into its own change.
--
MOS_MIGRATED_REVID=103527877
|
| |
|
|
|
|
|
| |
evaluation, test_suite expansion and configuration creation is still there). Also remove some unused code.
--
MOS_MIGRATED_REVID=103177839
|
| |
|
|
|
|
|
| |
SkyFunction#compute calls.
--
MOS_MIGRATED_REVID=102268773
|
| |
|
|
|
| |
--
MOS_MIGRATED_REVID=102126786
|
| |
|
|
|
|
|
| |
with --keep_going.
--
MOS_MIGRATED_REVID=101921704
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This is a big change, so let me walk you through the key pieces:
1) This cl provides an alternative mechanism for creating configurations and doing configuration transitions that's "dynamic" in that the configurations can be created on the fly and the transitions are arbitrary mappings from BuildOptions --> BuildOptions that can also be created on the fly. It also integrates this with ConfiguredTargetFunction, so the configured target graph automatically uses this framework.
2) It does *not* replace old-style (which we'll call "static") configurations. For a number of important reasons: It's not yet at feature parity (particularly: no LIPO). It's not remotely tested across real projects enough to have confidence that it's battle-ready. It doesn't yet handle certain "special" functions like BuildConfiguration.prepareToBuild() and BuildConfiguration.getRoots(). It still relies on the old static transition logic to determine what transitions to apply (eventually we'll distribute that logic out, but that's too much for a single cl). We need the option to toggle it on and off until we have enough confidence in it. So with this cl, builds can be done in either mode.
3) The new flag --experimental_dynamic_configs toggles use of dynamic configurations.
4) Dynamic configurations are created with the Skyframe function BuildConfigurationFunction (this was already created in an earlier change). This consumes a BuildOptions and a set of configuration fragments to produce a BuildConfiguration.
5) Dynamic transitions are instances of the new class PatchTransition, which simply maps an input BuildOptions to an output BuildOptions.
6) Since static and dynamic configurations have to co-exist (for now), this cl tries hard to keep today's transition logic defined in a single place (vs. forking a dedicated copy for each configuration style). This is done via the new interface BuildConfiguration.TransitionApplier. BuildConfiguration.evaluateTransition is modified to feed its transition logic into TransitionApplier's common API. Both dynamic and static configurations have their own implementations that "do the right thing" with the results.
7) The transition applier for dynamic configurations basically stores the Transition, then allows DependencyResolver (which calls BuildConfiguration.evaluateTransition) to return Dependency instances containing that Transition (vs. a BuildConfiguration, which they traditionally contain).
7.5) An earlier variation of the dynamic transition applier retained BuildOptions (e.g. when it got a Transition it immediately applied it to get its output BuildOptions, then stored that). This had the advantage of making composing of transitions easier, especially within BuildConfiguration.evaluateTransition (which can theoretically apply multiple transitions to the input configuration). But it turns out that applying transitions has a cost, and it's simply more performant to pass them through until they're really needed.
8) In dynamic configuration mode, after ConfiguredTargetFunction gets its deps (e.g. an <Attribute, Dependency> multimap), it "trims" the configurations for its dependencies by a) only including config fragments required by the deps' subtrees and b) applying the transitions that came from 7). This all happens in the new method ConfiguredTargetFunction.trimConfigurations.
9) trimConfigurations is heavily performance-optimized based on a lot of experience running this through a large project within Google. As it turns out, the cost of host transitions can be atrocious (because there are a lot of them). Also, BuildOptions.clone() is expensive. And just creating BuildConfiguration SkyKeys also has a cost (largely because of BuildOptions cloning), so that shouldn't be done except when really necessary. My experience with this convinced me it's worth making this method complicated for the sake of making it fast. Since it basically visits every edge in the configured target graph (at least), it really needs to be slick.
10) Since host transitions in particular are problematic w.r.t. speed, I compute the host *once* in ConfigurationCollectionFunction.getHostConfiguration() and expose that reference to ConfiguredTargetFunction and other Skyframe functions. This limits recomputation to just when the fragments are trimmed.
11) Since options cloning is expensive, I'm doing something scary: exposing a BuildConfiguration.getOptions() method that returns a direct reference. Since BuildOptions is mutable, this is dangerous in the wrong hands. I can experiment with going back to cloning (with the caching of host transitions it may not matter as much), but we may ultimately have to put up with this risk for the sake of performant analysis time. What would be *really* awesome would be to make BuildOptions immutable. But that's not going to happen in this cl.
So in short, the key abstractions in this cl are:
- PatchTransition
- BuildConfiguration.TransitionApplier
- ConfiguredTargetFunction.trimConfigurations
The current implementation imposes no analysis time penalty
--
MOS_MIGRATED_REVID=101474620
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The EvaluationProgressReceiver#evaluated method took a SkyValue,
but that parameter was only used by some of its implementations,
and only under some conditions.
Outside of tests, the main users are SkyframeBuildView's
ConfiguredTargetValueInvalidationReceiver and SkyframeBuilder's
ExecutionProgressReceiver.
The former builds up a set of built ConfiguredTarget keys when the
SkyValue is non-null and the EvaluationState is BUILT, and so its
nullity check can live behind those two conditions.
The latter cares about builting up a set of ConfiguredTargets, and
raising events on the eventBus when a TARGET_COMPLETION or
ASPECT_COMPLETION value is evaluated and is non-null. The extraction
of these values can live behind the conditions that check the type of
the SkyKey.
By making the SkyValue parameter lazy, this change enforces that it's
only accessed under these conditions.
This CL introduces a semantic change that should be small in effect.
The SkyframeBuildView keeps track of a set, dirtiedConfiguredTargetKeys,
and ConfiguredTarget keys evaluated as CLEAN were removed from it if
they had a non-null value. With this change, ConfiguredTarget keys
evaluated as CLEAN get removed regardless of whether their values are
null or non-null. The set is used to determine whether artifact conflict
search has to be rerun, and the extra removals that result from this CL
could cause fewer artifact conflict searches to run, but the only
affected searches would be those that were caused by dirtied configured
target values in error all of which were subsequently marked as clean,
which is probably rare.
--
MOS_MIGRATED_REVID=101144655
|
| |
|
|
|
| |
--
MOS_MIGRATED_REVID=101033236
|
| |
|
|
|
|
|
| |
Ref equality for same-name SkyFunctionNames is not guaranteed.
--
MOS_MIGRATED_REVID=100322275
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The primary user of invalidation tracking is the SkyframeBuildView,
which monitored which ConfiguredTargetValues were invalidated. It
did that so the SkyframeExecutor could limit its search for artifact
conflicts to when the set of configured targets in the build changed.
For the newly introduced set of dirtied keys,
"dirtiedConfiguredTargetKeys" in SkyframeBuildView, to be as useful as
the "dirtyConfiguredTargets" set it replaced, the
ConfiguredTargetValueInvalidationReceiver must only remove a key from
the set if it was found to be clean when it was re-evaluated. If it
was rebuilt, then the key must stay in the set, to represent that the
set of configured target values has truly changed.
This CL introduces a semantic change that hopefully has a small effect,
if any. Previously, the informInvalidationReceiver method in
InvalidatingNodeVisitor only informed the invalidationReceiver when a
non-null value was invalidated. (This is usually, but not always,
equivalent to a non-error value. The uncommon exception is that in
keep-going builds, some nodes with errors may also have values, and
the invalidator would inform the receiver when such a node was
invalidated.) Now, the receiver is informed that a node is invalidated
regardless of whether its value is null. Because the receiver uses this
information to determine whether artifact conflict search has to be
rerun, and that search is expensive, it's possible this change will
negatively impact performance. However, the only way an extra search
could be invoked is if the invalidated configured target nodes are
all in error. That seems like it would happen rarely, if at all.
Further cleanup of unused SkyValues returned by markDirty to come in
a subsequent CL.
--
MOS_MIGRATED_REVID=100304744
|
| |
|
|
|
|
|
| |
efficient if we need to do many ActionGraph looups.
--
MOS_MIGRATED_REVID=100060090
|
| |
|
|
|
|
|
|
|
| |
Dynamic configuration transitions require access to Skyframe (since they instantiate BuildConfigurations as Skyframe nodes). There are various places in Bazel where static transitions are done with no convenient Skyframe access. This cl shuffles host transitions, in particular, to places that are more amenable.
This change also assumes one host configuration per invocation. While this isn't strictly true (each target configuration can have its own host, and multiple target configurations are possible per build), we don't leverage that functionality in any meaningful way today. So until we have a proper interface for multiple host configurations, let's not block dynamic config progress on it.
--
MOS_MIGRATED_REVID=97008479
|
| |
|
|
|
|
|
|
|
| |
1. Make --analysis_warnings_as_errors a no-op.
2. Stop doing a sanity check that we didn't succeed in analysis even with errors emitted.
3. As a result, emit an error about shared actions to the proper listener.
--
MOS_MIGRATED_REVID=92262247
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
As far as I can tell, the dependency on the BuildConfigurationCollection node
is not required anymore. The BuildConfiguration is part of the SkyKey, and it
uses object identity.
This shouldn't affect garbage collection due to the explicit
BuildView.dropConfiguredTargets. If we change that in the future, we will have
to be (even more) careful.
--
MOS_MIGRATED_REVID=92134490
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
Instead of computing flags based on the configuration in two places, we pass
in the configuration and let the common code figure it out.
Also change it so each CT only depends on the build info collection for its
specific configuration. This should be safe, as the API doesn't allow passing
a configuration in - it instead takes ruleContext.getConfiguration, which
should be consistent with the dependency declarations in SkyframeBuildView in
all cases.
--
MOS_MIGRATED_REVID=91819844
|
| |
|
|
|
|
|
| |
and fail hard in other cases.
--
MOS_MIGRATED_REVID=89720528
|
| |
|
|
|
|
|
|
|
| |
This allows the pending action counter to be decremented correctly when in --keep_going mode.
As part of this, there's a small refactor in ParallelEvaluator that also fixes a potential bug, that nodes in error were signaling their parents with the graph version as opposed to their actual version. Because errors never compare equal (ErrorInfo doesn't override equality), this isn't an issue in practice. But it could be in the future.
--
MOS_MIGRATED_REVID=88395500
|
|
|
--
MOE_MIGRATED_REVID=85702957
|
Ulf Adams
Damien Martin-Guillerez
Lukacs Berki
Marian Lobur
Nathan Harmata
Greg Estren
Mark Schaller
Dmitry Lomov
Eric Fellheimer
Janak Ramakrishnan
Han-Wen Nienhuys