| Commit message (Collapse) | Author | Age |
|
|
|
|
| |
--
MOS_MIGRATED_REVID=114249806
|
|
|
|
|
|
|
|
|
|
|
|
| |
This is the main step toward supporting load statement of remote
repository in the WORKSPACE file (bug #824). Load statement that
refers to a remote repository will depend on the previous fragment
of the workspace file.
Issue #824 Step 3.
--
MOS_MIGRATED_REVID=114237027
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This is a bit odd - the legacy loading phase runner reports a loading error,
but no target pattern error in keep_going mode, even though it's clearly an
error in the referenced target itself, rather than in its transitive closure.
This happens because the target pattern eval swallows such errors in keep_going
mode, and doesn't even report the error. I tried changing that, but it's a
fairly large refactoring, and that code path is dead if we switch to the new
one.
In the Skyframe-based implementation, both keep_going and nokeep_going paths
now report a target pattern error. (Note that the new code can never report a
loading error, because it doesn't perform transitive loading.)
The corresponding test is moved from SkyframeLoadingAndAnalysisTest to
LoadingPhaseRunnerTest - we don't need any integration test setup for that.
--
MOS_MIGRATED_REVID=114236897
|
|
|
|
|
|
|
|
| |
Note that this only affects the new Skyframe-based implementation of the
LoadingPhaseRunner, which isn't live yet.
--
MOS_MIGRATED_REVID=114230017
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This only applies to interleaved loading and analysis - the production code
is fine.
Add tests for the RedirectChaser, the fdoOptimize code path, the XcodeConfig,
and the Jvm loader. Unfortunately, the configuration factory we internally
create by default contains a mock Jvm loader implementation. Since that is one
Yak too many right now, I'm adding a temporary method to the AnalysisMock.
I added the tests to BuildViewTest for now; technically, they ought to go
into the language-specific test cases, but that would require more refactoring
as those don't currently run with interleaved loading and analysis.
--
MOS_MIGRATED_REVID=114221476
|
|
|
|
|
| |
--
MOS_MIGRATED_REVID=114204404
|
|
|
|
|
| |
--
MOS_MIGRATED_REVID=114202845
|
|
|
|
|
|
|
| |
ArtifactFunction.
--
MOS_MIGRATED_REVID=114174899
|
|
|
|
|
|
|
| |
ArtifactFiles.
--
MOS_MIGRATED_REVID=114166208
|
|
|
|
|
| |
--
MOS_MIGRATED_REVID=114086842
|
|
|
|
|
|
|
|
|
| |
load statements
Issue #824 Step 2.
--
MOS_MIGRATED_REVID=113986176
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The WORKSPACE file parsing needs to be separated into several parts
to enable load of labels in the WORKSPACE file.
This change adds an intermediate SkyFunction, ExternalPackageFunction,
that requires all the WORKSPACE file part to be parsed to resolve
//external: labels.
Issue #824 Step 1.
--
MOS_MIGRATED_REVID=113984026
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
first try to get a skyframe cache-hit; otherwise, fall back to legacy globbing. This gives us the best of both worlds: no extra skyframe restarts on glob-dep-misses, and much better incremental performance in the common case that a package's globs haven't changed. See the class-comment for PackageFunction.SkyframeHybridGlobber for a detailed description and explanation.
This CL has no impact on semantics and is a strict performance win.
Bazel users: Here's an example benchmark (does an incremental loading phase of a target T but forces all packages to be reloaded):
nharmata@nharmata:~/bazel$ N=10; B="output/bazel"; T=//src/main/java/com/google/devtools/build/lib:bazel/BazelServer_deploy.jar; CMD="build --noanalyze $T"; $B clean &> /dev/null; P=base_workspace/tools/build_rules/prelude_bazel; rm $P; touch base_workspace/tools/build_rules/BUILD; $B $CMD &> /dev/null; time for i in $(seq 1 $N); do echo "#hi" >> $P; $B $CMD &> /dev/null; done
For a very large internal Google target, this CL improves the benchmark performance by ~6%. A more targeted benchmark would be for loading a single package that has lots of expensive globs. For example, the time to incrementally load a single pathological Google-internal package was reduced by ~36%.
Alternatives considered: Introduce skyframe native globbing, gated by flags for both globbing during preprocessing and globbing during regular BUILD file evaluation. The approach in this CL is superior performance-wise.
--
MOS_MIGRATED_REVID=113899687
|
|
|
|
|
| |
--
MOS_MIGRATED_REVID=113893917
|
|
|
|
|
|
|
|
| |
In the interleaved case, loading errors can occur during configuration creation
and we need to correctly report such errors in that case.
--
MOS_MIGRATED_REVID=113826273
|
|
|
|
|
|
|
|
|
| |
by having it not declare a dep on the PackageLookupValue for the package. This optimization means that a BUILD file edit doesn't (necessarily) invalidate all the globs in the package; the PackageLookupValue node would get change-pruned but we still pay the very small cost of invalidating unnecessarily.
Also slightly improve variable naming in GlobFunctionTest.
--
MOS_MIGRATED_REVID=113799936
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
In the interleaved case, loading errors can now also be discovered during the
analysis phase. Add a boolean flag to the SkyframeAnalysisResult to indicate
that such an error happened, and pass it through in BuildView.
Also refactor BuildView to simplify the code a bit - simply pass the
SkyframeAnalysisResult to the createResult method.
There is already an integration test for this - I'm adding a faster unit test
in BuildViewTest, as this is part of the BuildView contract.
--
MOS_MIGRATED_REVID=113716870
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
options actually needed by the fragment. This protects against, e.g.,
unnecessarily duplicating CppConfiguration instances when only Java flags
change.
This is a recommit of ca1b21ac6d8a58041db822725b42de151b163dee which was
rolled back because it broke LIPO.
This change is particularly important for dynamic configurations, which may
mix and match fragments arbitrarily throughout a build. This not only has
performance implications, but also correctness implications: code that
expects two configured targets to have the same fragment (value) shouldn't
break just because the second CT's configuration is a trimmed version of the
first's.
The original change breaks FDO/LIPO because CppConfiguration can't be
shared across configurations. That's because it mutates state when
prepareHook() is called, and each configuration calls prepareHook. We
should ultimately solve this by refactoring the FDO/LIPO implementation
but don't want to block dynamic configuration progress on that. So this
change only enables trimming for dynamic configurations.
--
MOS_MIGRATED_REVID=113570250
|
|
|
|
|
|
|
|
| |
This exposes an issue with the new Skyframe-based implementation, which is
fixed here as well.
--
MOS_MIGRATED_REVID=113556169
|
|
|
|
|
|
|
| |
--
Change-Id: I86b90ff6eda6a16c2efeb8fffca301e2594c254e
Reviewed-on: https://bazel-review.googlesource.com/#/c/2784/
MOS_MIGRATED_REVID=113529297
|
|
|
|
|
|
|
|
| |
Add test coverage by re-running BuildViewTest with the new Skyframe loading
phase runner.
--
MOS_MIGRATED_REVID=113517509
|
|
|
|
|
|
|
|
|
| |
Moved RepositoryValue to RepositoryDirectoryValue so that it could be cached
(and not re-downloaded) even if the WorkspaceAST caused a Skyframe restart
(as mentioned in https://github.com/bazelbuild/bazel/blob/master/src/main/java/com/google/devtools/build/lib/rules/repository/RepositoryDelegatorFunction.java#L130-L133).
--
MOS_MIGRATED_REVID=113358489
|
|
|
|
|
| |
--
MOS_MIGRATED_REVID=113338481
|
|
|
|
|
|
|
| |
type-safety, we now must pass in the exception type of the callback.
--
MOS_MIGRATED_REVID=113313312
|
|
|
|
|
|
|
|
|
| |
Also add some TODOs for potentially improving package loading performance.
This CL has no semantic impact.
--
MOS_MIGRATED_REVID=113301656
|
|
|
|
|
|
|
|
|
| |
This should never be triggered in production, where we always run a loading
phase first and only analyze targets that load successfully. I.e., this is
just plumbing which will be hooked up in a subsequent change.
--
MOS_MIGRATED_REVID=113258593
|
|
|
|
|
|
|
|
|
|
|
| |
- if there are failed top-level aspects
- if there are conflicting actions
I ended up rewriting how errors are signaled from the SkyframeBuildView. I
think this is safe, but please review carefully.
--
MOS_MIGRATED_REVID=113150100
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Dependencies are the data structure which needs to propagate the configuration for each
aspect as created by trimConfigurations down to the point where it's actually used. We
need this to store different configurations for different aspects in a world where aspects
have their own configurations, which may have more fragments than the target they're
attached to.
That world is on its way.
Also in this CL:
* Refactor Dependency to be an abstract parent class with separate implementations for
Attribute.Transitions and BuildConfigurations, as well as null configurations, to avoid
having to check nullness in various places. Users of the API will not see this, but get
factory methods instead of constructors. As a consequence of this, refactor Dependency
to be its own top-level class instead of a nested class in DependencyResolver.
--
MOS_MIGRATED_REVID=113109615
|
|
|
|
|
|
|
|
|
|
| |
As it's about to get more important in its own right.
The only change that isn't just moving code around is making
RepositoryName.validate package-private.
--
MOS_MIGRATED_REVID=112956571
|
|
|
|
|
| |
--
MOS_MIGRATED_REVID=112948493
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Also match ConfiguredTargetFunction for target loading.
It isn't currently possible to trigger either of these code paths
- the loading phase ensures that we never attempt to analyze targets that fail
to load
- the Skylark import or conversion cannot fail, because Skylark checks during
.bzl execution that all referenced symbols are Skylark aspects
Therefore, the only way to trigger this would be if there was a native rule
requesting a non-existent or broken Skylark aspect for its dependencies, but
that is currently not possible - native rules can only request native aspects.
However, for interleaved loading and analysis, we need to limit the set of
exception classes that can be thrown from AspectFunction - we do that here
by changing the constructor of AspectFunctionException to only accept either
NoSuchThingException or AspectCreationException.
That in turn requires that we re-throw the Skylark import and conversion
exceptions as AspectCreationException, which is exactly what
TopLevelAspectFunction is already doing, and necessary for correct error
handling if we do ever support Skylark aspects in native rules.
Alternatively, I could change the code path to crash Bazel, but that seems
strictly worse. Even if we can't test this code, it's conceptually the right
way to handle these errors.
I'll move part of the error handling into loadSkylarkAspect in a subsequent
change.
--
MOS_MIGRATED_REVID=112938284
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The main remaining problem with interleaved loading and analysis is error
handling. When interleaving, we don't run a real loading phase anymore, and
loading errors can occur during the analysis phase, and need to be handled
there.
The plan is to have ConfiguredTargetFunction throw a
ConfiguredValueCreationException with a list of all loading root causes, which
requires that we also catch ConfiguredValueCreationException here, which in
turn breaks analysis root cause handling, as that is currently relying on
Skyframe root cause tracking.
Moving analysis root cause handling into CTFunction makes it possible to
subsequently also implement loading root cause handling here. This is also
necessary if we want to have complete root cause handling in the general case:
a target may have any number (and combination) of loading and analysis root
causes at the same time.
For now, we only pass a single analysis root cause, which mirrors the current
Skyframe-based implementation.
--
MOS_MIGRATED_REVID=112930871
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Note that this can never happen in production - the loading phase ensures that
we only attempt to analyze targets that load successfully. That's also why
there are no tests for this.
This code path will become live in a later change, with corresponding test
coverage; the existing tests cover this in principle, we just need to make
them run with the reduced loading phase.
--
MOS_MIGRATED_REVID=112926211
|
|
|
|
|
|
|
|
|
| |
action fails, and propagate the exit code to BuildFailedException.
And update all call sites that copy or wrap ActionExecutionExceptions.
--
MOS_MIGRATED_REVID=112811857
|
|
|
|
|
|
|
|
|
|
|
| |
The WORKSPACE file AST is now parsed as a separate SkyFunction
and this will be used to have multiple SkyValue for the same
WORKSPACE file, splitting the execution of the AST after load
statements to enable load statement of external dependencies
in the WORKSPACE file.
--
MOS_MIGRATED_REVID=112768897
|
|
|
|
|
|
|
|
|
| |
--noimplicit_deps and/or --nohost_deps.
RELNOTES: --noimplicit_deps and --nohost_deps work correctly for Aspect attributes.
--
MOS_MIGRATED_REVID=112724917
|
|
|
|
|
| |
--
MOS_MIGRATED_REVID=112717648
|
|
|
|
|
|
|
| |
is a partial rollback of commit f1e257d because it turns out that loading sequentially can be a bottleneck.
--
MOS_MIGRATED_REVID=112628616
|
|
|
|
|
| |
--
MOS_MIGRATED_REVID=112561390
|
|
|
|
|
| |
--
MOS_MIGRATED_REVID=112507181
|
|
|
|
|
|
|
| |
all so that custom implementations can have custom options passed around.
--
MOS_MIGRATED_REVID=112502778
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- update some comments
- add some comments to make it easier to follow
- delete some dead code, in particular the SkyframeDependencyResolver can
never be null; remove an non-applicable @Nullable annotation
I'm trying to figure out how the error handling code works, in order to add
support for interleaved loading+analysis, which requires handling loading
errors in this code path.
--
MOS_MIGRATED_REVID=112456674
|
|
|
|
|
|
|
|
|
| |
to do such a load will result in a skyframe circular dependency exception.
As a side effect of this change, SkylarkImportFailedExceptions thrown by SkylarkImportLookupFunction are now caught by PackageFunction and wrapped and rethrown as a PackageFunctionException. Previously, the first exception wasn't caught, generating an uncaught exception error at top level.
--
MOS_MIGRATED_REVID=112328755
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This is in preparation for moving the output dumping to the Reporter for
two reasons:
- we're currently relying on there being a Reporter which locks on itself;
i.e., this code is tightly coupled with the Reporter code
- the current Blaze output is confusing; we first print an info event, then
the actual output, and then an error event - putting the Reporter in control
allows us to fix that
--
MOS_MIGRATED_REVID=112240684
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Previously, for some types of errors, you'd just see "WORKSPACE file
could not be parsed" if there was a syntax error (instead of what the
syntax error was).
Change-Id: Ia278cf23b79f8daba02917cdfb4cc515a87080b1
--
Change-Id: Ia278cf23b79f8daba02917cdfb4cc515a87080b1
Reviewed-on: https://github.com/bazelbuild/bazel/pull/774
MOS_MIGRATED_REVID=112190322
|
|
|
|
|
|
|
|
|
|
|
| |
statements work in remote repositories.
This work is somewhat sisyphean; the principled thing to do would be to just do this resolution when labels are parsed.
Fixes #783.
--
MOS_MIGRATED_REVID=112137996
|
|
|
|
|
|
|
|
|
|
|
|
| |
The ConfiguredTargetFunction can only throw a NoSuch{Target,Package}Exception
if it's run for a non-existent target. However, it will never request a CT for
a non-existent target: all labels are run through DependencyResolver in order
to determine the configuration transition, which requires loading the target.
If it doesn't exist, the label is never returned from the DependencyResolver,
and errors are swallowed (we rely on the loading phase to report the error).
--
MOS_MIGRATED_REVID=112127012
|
|
|
|
|
|
|
| |
WalkableGraphFactory interface with a hook to inform the factory when the WalkableGraph is done being used.
--
MOS_MIGRATED_REVID=112074284
|
|
|
|
|
|
|
|
|
| |
TargetPatternKey, since that is what is needed by callers.
Also, since the PathFragments come from packages, they are guaranteed to be well-formed, so the checks we were doing were unnecessary.
--
MOS_MIGRATED_REVID=112059930
|
|
|
|
|
|
|
| |
from the current package batch.
--
MOS_MIGRATED_REVID=111858978
|