| Commit message (Collapse) | Author | Age |
|---|
| |
|
|
|
| |
--
MOS_MIGRATED_REVID=102134151
|
| |
|
|
|
| |
--
MOS_MIGRATED_REVID=102126786
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
We no longer generate baseline coverage for all transitive source files in
every target; instead, we generate baseline coverage for the files in the
current target and collect all of them transitively. That means much smaller
but more baseline coverage files; the total content is smaller if you were
providing more than one target with overlapping transitive closures on the
command line.
In addition, we now collect baseline coverage for all targets in the transitive
closure of the top-level targets. Previously, if you only passed test targets,
you would not get any baseline coverage.
--
MOS_MIGRATED_REVID=101929897
|
| |
|
|
|
|
|
|
|
|
| |
The baseline artifacts are part of the instrumented files provider now, and
are strongly tied to the collect_code_coverage flag. It seems to be simpler
to collect them explicitly in the BuildView (which already collects them for
post-processing), than to rely on the output group selection.
--
MOS_MIGRATED_REVID=101926341
|
| |
|
|
|
|
|
|
| |
This is necessary to have TargetResolver depend on it without making it depend
on the packages target. First step of #389.
--
MOS_MIGRATED_REVID=101790345
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This is a big change, so let me walk you through the key pieces:
1) This cl provides an alternative mechanism for creating configurations and doing configuration transitions that's "dynamic" in that the configurations can be created on the fly and the transitions are arbitrary mappings from BuildOptions --> BuildOptions that can also be created on the fly. It also integrates this with ConfiguredTargetFunction, so the configured target graph automatically uses this framework.
2) It does *not* replace old-style (which we'll call "static") configurations. For a number of important reasons: It's not yet at feature parity (particularly: no LIPO). It's not remotely tested across real projects enough to have confidence that it's battle-ready. It doesn't yet handle certain "special" functions like BuildConfiguration.prepareToBuild() and BuildConfiguration.getRoots(). It still relies on the old static transition logic to determine what transitions to apply (eventually we'll distribute that logic out, but that's too much for a single cl). We need the option to toggle it on and off until we have enough confidence in it. So with this cl, builds can be done in either mode.
3) The new flag --experimental_dynamic_configs toggles use of dynamic configurations.
4) Dynamic configurations are created with the Skyframe function BuildConfigurationFunction (this was already created in an earlier change). This consumes a BuildOptions and a set of configuration fragments to produce a BuildConfiguration.
5) Dynamic transitions are instances of the new class PatchTransition, which simply maps an input BuildOptions to an output BuildOptions.
6) Since static and dynamic configurations have to co-exist (for now), this cl tries hard to keep today's transition logic defined in a single place (vs. forking a dedicated copy for each configuration style). This is done via the new interface BuildConfiguration.TransitionApplier. BuildConfiguration.evaluateTransition is modified to feed its transition logic into TransitionApplier's common API. Both dynamic and static configurations have their own implementations that "do the right thing" with the results.
7) The transition applier for dynamic configurations basically stores the Transition, then allows DependencyResolver (which calls BuildConfiguration.evaluateTransition) to return Dependency instances containing that Transition (vs. a BuildConfiguration, which they traditionally contain).
7.5) An earlier variation of the dynamic transition applier retained BuildOptions (e.g. when it got a Transition it immediately applied it to get its output BuildOptions, then stored that). This had the advantage of making composing of transitions easier, especially within BuildConfiguration.evaluateTransition (which can theoretically apply multiple transitions to the input configuration). But it turns out that applying transitions has a cost, and it's simply more performant to pass them through until they're really needed.
8) In dynamic configuration mode, after ConfiguredTargetFunction gets its deps (e.g. an <Attribute, Dependency> multimap), it "trims" the configurations for its dependencies by a) only including config fragments required by the deps' subtrees and b) applying the transitions that came from 7). This all happens in the new method ConfiguredTargetFunction.trimConfigurations.
9) trimConfigurations is heavily performance-optimized based on a lot of experience running this through a large project within Google. As it turns out, the cost of host transitions can be atrocious (because there are a lot of them). Also, BuildOptions.clone() is expensive. And just creating BuildConfiguration SkyKeys also has a cost (largely because of BuildOptions cloning), so that shouldn't be done except when really necessary. My experience with this convinced me it's worth making this method complicated for the sake of making it fast. Since it basically visits every edge in the configured target graph (at least), it really needs to be slick.
10) Since host transitions in particular are problematic w.r.t. speed, I compute the host *once* in ConfigurationCollectionFunction.getHostConfiguration() and expose that reference to ConfiguredTargetFunction and other Skyframe functions. This limits recomputation to just when the fragments are trimmed.
11) Since options cloning is expensive, I'm doing something scary: exposing a BuildConfiguration.getOptions() method that returns a direct reference. Since BuildOptions is mutable, this is dangerous in the wrong hands. I can experiment with going back to cloning (with the caching of host transitions it may not matter as much), but we may ultimately have to put up with this risk for the sake of performant analysis time. What would be *really* awesome would be to make BuildOptions immutable. But that's not going to happen in this cl.
So in short, the key abstractions in this cl are:
- PatchTransition
- BuildConfiguration.TransitionApplier
- ConfiguredTargetFunction.trimConfigurations
The current implementation imposes no analysis time penalty
--
MOS_MIGRATED_REVID=101474620
|
| |
|
|
|
| |
--
MOS_MIGRATED_REVID=101033236
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
checking that are compatible with upcoming dynamic configurations (https://docs.google.com/document/d/1uoU8t7loTOu6uyzez-ilhcYgEXg4xjViJu0aqrf69TY/edit#heading=h.xsc8wmorka3u).
With today's configuration machinery, configuration equality means reference equality. With dynamic configurations, that won't necessarily be the case. This is primarily for two reasons:
1) Fragment-limited configurations means that a target's configuration only includes fragments needed by the target and its transitive closure. So if a java_library depends on a cc_library with the same configuration, the cc_library's instance won't contain the JavaConfiguration fragment.
2) The (upcoming) first pass of dynamic configurations uses old-style configurations for top-level targets, and dynamic configs for everything underneath. This can impact test calls that request a configured target both directly and as a dependency of another target. This distinction will eventually go away, but not yet.
Configurations that share the same options and fragments will continue to be reference-equal because of Skyframe caching.
--
MOS_MIGRATED_REVID=99610132
|
| |
|
|
|
|
|
| |
[].
--
MOS_MIGRATED_REVID=99592972
|
| |
|
|
|
|
|
|
|
| |
from previous builds with a different CPU do not end up in the APK.
Fixes #344.
--
MOS_MIGRATED_REVID=99550457
|
| |
|
|
|
|
|
|
|
|
|
| |
use this functionality to properly invalidate http download nodes if the downloaded zip file doesn't match the specified hash.
This still means that the actual files downloaded may not match, but checking all such files may be too expensive.
This helps with #336 but before that issue can be closed all remote repositories (git, etc.), should have similar functionality added.
--
MOS_MIGRATED_REVID=99317085
|
| |
|
|
|
| |
--
MOS_MIGRATED_REVID=99181437
|
| |
|
|
|
|
|
| |
Currently, the best (and only?) way to access an empty action inside a test is via an extra action and an action listener.
--
MOS_MIGRATED_REVID=98628097
|
| |
|
|
|
|
|
|
| |
One of them moves up to BuildViewTestCase, the other is done as a static
import where needed.
--
MOS_MIGRATED_REVID=98484132
|
| |
|
|
|
|
|
|
|
|
|
| |
This involved quite a few changes, mainly changing a bunch of places where we refer to packages by a PathFragment to PackageIdentifier.
The only wart is the code in PathPackageLocator: ideally, it would just call into PackageLookupFunction. Unfortunately, it is (through globbing and Parser.include) called from within a Skyframe function, and we don't want to have two eval() calls going on at the same time, so we cannot use that.
There is a potential correctness issue there: PathPackageLocator now assumes where external repositories are put and assumes that they are there when it gets control, but my understanding is that the associated RepositoryValue is always evaluated before, so it works out okay.
--
MOS_MIGRATED_REVID=97751539
|
| |
|
|
|
| |
--
MOS_MIGRATED_REVID=97648982
|
| |
|
|
|
|
|
|
|
|
|
| |
This involved quite a few changes, mainly changing a bunch of places where we refer to packages by a PathFragment to PackageIdentifier.
The only wart is the code in PathPackageLocator: ideally, it would just call into PackageLookupFunction. Unfortunately, it is (through globbing and Parser.include) called from within a Skyframe function, and we don't want to have two eval() calls going on at the same time, so we cannot use that.
There is a potential correctness issue there: PathPackageLocator now assumes where external repositories are put and assumes that they are there when it gets control, but my understanding is that the associated RepositoryValue is always evaluated before, so it works out okay.
--
MOS_MIGRATED_REVID=97647787
|
| |
|
|
|
|
|
| |
This was only used in error reporting, and the interface of BuildConfiguration could use some more simplicity. The comment on BuildConfiguration.getShortName() was confusing: that field is *not* actually used to compute the output directory name, which took me a few minutes to realize.
--
MOS_MIGRATED_REVID=97128287
|
| |
|
|
|
|
|
|
|
|
|
| |
ParameterFileWriteAction.
Performance changes:
- output files of actions require an extra system call
+ incremental builds no longer require re-writing the .param file (typically)
--
MOS_MIGRATED_REVID=95842983
|
| |
|
|
|
| |
--
MOS_MIGRATED_REVID=95135804
|
| |
|
|
|
|
|
| |
thread pool.
--
MOS_MIGRATED_REVID=94236393
|
| |
|
|
|
|
|
| |
support for Aspects.
--
MOS_MIGRATED_REVID=93547419
|
| |
|
|
|
|
|
| |
different attributes, to which we have attached aspect.
--
MOS_MIGRATED_REVID=93412457
|
| |
|
|
|
| |
--
MOS_MIGRATED_REVID=92727637
|
| |
|
|
|
|
|
|
|
| |
1. Make --analysis_warnings_as_errors a no-op.
2. Stop doing a sanity check that we didn't succeed in analysis even with errors emitted.
3. As a result, emit an error about shared actions to the proper listener.
--
MOS_MIGRATED_REVID=92262247
|
| |
|
|
|
| |
--
MOS_MIGRATED_REVID=92128998
|
| |
|
|
|
| |
--
MOS_MIGRATED_REVID=91907246
|
| |
|
|
|
|
|
|
|
| |
Instead of passing BuildConfigurationKey instances around, just pass in the
little data we actually need. This allows removing the BuildConfigurationKey
class.
--
MOS_MIGRATED_REVID=91865340
|
| |
|
|
|
|
|
| |
ActionContextConsumers.
--
MOS_MIGRATED_REVID=91827715
|
| |
|
|
|
|
|
|
|
|
|
| |
BuildConfiguration.Options.testEnvironment instead of special-casing it in a large number of classes.
The variables in the client environment are read in BlazeRuntime#beforeCommand() now.
Note that this entails a slight loss of caching: before, "--test_env=a=A,b=B" and "--test_env=b=B,a=A" were equivalent, now they are not, since instead of comparing Map<String, String>, List<Map.Entry<String,String>> instances are compared.
--
MOS_MIGRATED_REVID=91570828
|
| |
|
|
|
| |
--
MOS_MIGRATED_REVID=91405317
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- update the MOCK_CROSSTOOL to provide more stuff needed by tests
- add a THIS_IS_BAZEL constant to allow disabling individual test cases
- disable some tests in Bazel
The disabled tests are mainly due to differences in the test setup - making
the test setups more similar will largely fix that. I think we'll make some
changes to our internal setup, too, not just the external one. For example,
the use of 'k8' and 'piii' to refer to 'x86_64' and 'x86' seems archaic.
I decided to leave the dependency on the C++ and Java configurations rather
than rewriting the tests to use mock configurations. That would be nicer, but
also requires significantly more work.
--
MOS_MIGRATED_REVID=91399406
|
| |
|
|
|
|
|
| |
computing the test environment as early as possible, and passing that along.
--
MOS_MIGRATED_REVID=91388451
|
| |
|
|
|
|
|
| |
This requires writing a workspace file for the JDK.
--
MOS_MIGRATED_REVID=91287178
|
| |
|
|
|
| |
--
MOS_MIGRATED_REVID=91286405
|
| |
|
|
|
| |
--
MOS_MIGRATED_REVID=91284411
|
| |
|
|
|
| |
--
MOS_MIGRATED_REVID=90984191
|
| |
|
|
|
| |
--
MOS_MIGRATED_REVID=89680998
|
| |
|
|
|
|
|
| |
of BuildConfigurationTest into a new test class.
--
MOS_MIGRATED_REVID=89113080
|
| |
|
|
|
| |
--
MOS_MIGRATED_REVID=88944458
|
| |
|
|
|
|
|
|
|
| |
unit tests and main code have fallen out of sync a few times, so
consolidating the defaults here to make mistakes harder later on.
one could argue this also gives a little better mental locality.
--
MOS_MIGRATED_REVID=88921641
|
| |
|
|
|
|
|
| |
Specifically, move AnalysisTestCase and SkyframeExecutorTestUtil.
--
MOS_MIGRATED_REVID=88715957
|
| |
|
|
|
| |
--
MOS_MIGRATED_REVID=87942730
|
| |
|
|
|
|
|
|
| |
methods available doesn't seem to carry its weight, and this makes it easier
for us to migrate to JUnit 4.
--
MOS_MIGRATED_REVID=87597162
|
| |
|
|
|
| |
--
MOS_MIGRATED_REVID=87596401
|
| |
|
|
|
| |
--
MOS_MIGRATED_REVID=87594852
|
| |
|
|
|
| |
--
MOS_MIGRATED_REVID=87345558
|
| |
|
|
|
|
|
|
|
|
|
| |
A recent change resulted in TopLevelArtifactHelper.getTopLevelArtifacts()
returning an empty result when passed TopLevelArtifactContext.DEFAULT.
This only happens in tests and wasn't caught because tests were never
fully checking the results. Update MultiGroupForgeResourceAccountantTest
to better consider/expose this failure and fixed.
--
MOS_MIGRATED_REVID=87254923
|
|
|
the tests under analysis/actions. They don't run yet, because the mock client
setup is still missing.
--
MOS_MIGRATED_REVID=87149625
|
Marian Lobur
Ulf Adams
Kristina Chodorow
Greg Estren
Dmitry Lomov
Lukacs Berki
Janak Ramakrishnan
Laszlo Csomor
Florian Weikert
Eric Fellheimer
Rumou Duan
Googler
Michajlo Matijkiw
Manuel Klimek