| Commit message (Collapse) | Author | Age |
|
|
|
|
|
| |
This makes it clearer that the path fragments in question are relative *to the root*. Confusingly, when the root is absolute, the root relative fragment is also absolute. This makes it a tiny bit clearer that the path fragment may be absolute.
PiperOrigin-RevId: 182544893
|
|
|
|
|
|
| |
path class.
PiperOrigin-RevId: 182526427
|
|
|
|
|
|
|
|
|
|
|
| |
This class represents a root (such as a package path or an output root) used for file lookups and artifacts. It is meant to be as opaque as possible in order to hide the user's environment from sky keys and sky functions.
Roots are used by RootedPaths and ArtifactRoots.
This CL attempts to make the minimum number of modifications necessary to change RootedPath and ArtifactRoot to use these fields. Deprecated methods and invasive accessors are permitted to minimise the risk of any observable changes.
RELNOTES: None
PiperOrigin-RevId: 182271759
|
|
|
|
|
|
|
| |
Fixes #4056.
Change-Id: Ia7425c2146f15e9293605ee3da53007805e82275
PiperOrigin-RevId: 177813070
|
|
|
|
|
|
|
|
| |
Blaze had its own class to avoid GC from varargs array creation for the precondition happy path. Guava now (mostly) implements these, making it unnecessary to maintain our own.
This change was almost entirely automated by search-and-replace. A few BUILD files needed fixing up since I removed an export of preconditions from lib:util, which was all done by add_deps. There was one incorrect usage of Preconditions that was caught by error prone (which checks Guava's version of Preconditions) that I had to change manually.
PiperOrigin-RevId: 175033526
|
|
|
|
|
|
|
|
|
| |
RecursivePackageProvider#getPackagesUnderDirectory assumed that the given directory needed to be strictly underneath all directories in the given blacklistedSubdirectories set, yet the callsites were calling this method such that the given directory was not-necessarily-strictly underneath the blacklistedSubdirectories.
N.B. There is no end-to-end bug in EnvironmentBackedRecursivePackageProvider because TargetPatternFunction's usage of TargetPattern#eval is always with blacklistedSubdirectories=ImmutableSet.of(). Also note that the existing fast-path in GraphBackedRecursivePackageProvider is dead code because the Preconditions check would fail first :(
RELNOTES: None
PiperOrigin-RevId: 173924216
|
|
|
|
|
|
|
| |
Turns out, my previous search expression didn't find the ones that were not "final LOG" or "final log" and a surprising number of places were missing the "final" tag.
RELNOTES: None.
PiperOrigin-RevId: 167547507
|
|
|
|
|
|
|
| |
RecursivePackageProvider dealing with the concept of "excluded directories".
RELNOTES: None
PiperOrigin-RevId: 163074794
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
We need to lookup repositories as part of converting exec paths to artifacts,
which in turn is needed for action cache lookups. These lookups should not
cause a Skyframe exit, so we must not throw an exception here, unless the
error makes it impossible to continue. Instead, we need to leave the decision
whether to error out or not to the caller.
Note that we may unnecessarily fetch a remote repository in order to do the
action cache lookup, even if the action no longer depends on the input file,
although this should only be possible for C++ compile actions. It's possible
that there's another bug in the C++ compile action key computation that also
contributes.
This change also makes it so that the post-resolution action cache code
ignores any errors wrt. repository lookup rather than throwing. If any of the
paths could not be found, then the action cache lookup fails and we re-execute
the corresponding action, which is exactly what should happen.
Fixes #2759.
PiperOrigin-RevId: 153696243
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The error message logged during query (and build) when a package has a
broken Skylark load statement was not specific. Previously, it said
"package contains errors:" and then the package name.
Also, this error message was not emitted when using SkyQueryEnvironment
and evaluating a query containing a "TargetsBelowDirectory" pattern
(such as //foo/...) when a package below the specified directory had
such an error.
The approach taken by this CL is to include any package loading error
message in the SkyValue produced by
CollectPackagesUnderDirectoryFunction, and report them during
evaluation of a TargetsBelowDirectory pattern.
RELNOTES: Evaluation of commands on TargetsBelowDirectory patterns
(e.g. //foo/...) matching packages that fail to load now report more
detailed error messages in keep_going mode.
--
PiperOrigin-RevId: 149802362
MOS_MIGRATED_REVID=149802362
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
With more specific information to be reported by Skyfunctions, e.g.,
to inform the build-event protocol on missing files, the EventHandler
interface is no longer enough. Therefore, provide an enriched context
for reporting events.
--
Change-Id: I2d06166fe4d5b9054e24ad8c752fafc039e3f9f8
Reviewed-on: https://cr.bazel.build/8794
PiperOrigin-RevId: 148463437
MOS_MIGRATED_REVID=148463437
|
|
|
|
|
|
| |
--
PiperOrigin-RevId: 144494739
MOS_MIGRATED_REVID=144494739
|
|
|
|
|
|
|
|
| |
WalkableGraph#getException to be given non-existent keys without crashing. Add WalkableGraph#isCycle to fill the gap in testing for the difference between non-existence and depending on a cycle.
--
PiperOrigin-RevId: 143205289
MOS_MIGRATED_REVID=143205289
|
|
|
|
|
|
|
| |
to load must have failed because of an exception, which is false in the case of cycles;
--
MOS_MIGRATED_REVID=135742248
|
|
|
|
|
|
|
| |
The only place we now don't handle InterruptedException is in the action graph created after analysis, since I'm not sure that will be around for that much longer.
--
MOS_MIGRATED_REVID=130327770
|
|
|
|
|
| |
--
MOS_MIGRATED_REVID=125592223
|
|
|
|
|
| |
--
MOS_MIGRATED_REVID=125582928
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Previously, this would get thrown when referring to the same package
from both the main and default repositories:
java.lang.IllegalArgumentException: Multiple entries with same key: tools/cpp=/home/brian/971-Robot-Code and tools/cpp=/home/brian/971-Robot-Code
at com.google.common.collect.ImmutableMap.checkNoConflict(ImmutableMap.java:136)
at com.google.common.collect.RegularImmutableMap.checkNoConflictInKeyBucket(RegularImmutableMap.java:98)
at com.google.common.collect.RegularImmutableMap.fromEntryArray(RegularImmutableMap.java:84)
at com.google.common.collect.ImmutableMap$Builder.build(ImmutableMap.java:295)
at com.google.devtools.build.lib.buildtool.BuildTool.transformPackageRoots(BuildTool.java:301)
at com.google.devtools.build.lib.buildtool.BuildTool.buildTargets(BuildTool.java:209)
at com.google.devtools.build.lib.buildtool.BuildTool.processRequest(BuildTool.java:334)
at com.google.devtools.build.lib.runtime.commands.TestCommand.doTest(TestCommand.java:119)
at com.google.devtools.build.lib.runtime.commands.TestCommand.exec(TestCommand.java:104)
at com.google.devtools.build.lib.runtime.BlazeCommandDispatcher.exec(BlazeCommandDispatcher.java:371)
at com.google.devtools.build.lib.runtime.BlazeRuntime$3.exec(BlazeRuntime.java:1016)
at com.google.devtools.build.lib.server.RPCService.executeRequest(RPCService.java:65)
at com.google.devtools.build.lib.server.RPCServer.executeRequest(RPCServer.java:434)
at com.google.devtools.build.lib.server.RPCServer.serve(RPCServer.java:229)
at com.google.devtools.build.lib.runtime.BlazeRuntime.serverMain(BlazeRuntime.java:975)
at com.google.devtools.build.lib.runtime.BlazeRuntime.main(BlazeRuntime.java:772)
at com.google.devtools.build.lib.bazel.BazelMain.main(BazelMain.java:55)
And this would get thrown for any packages in the main repository loaded
from other repositories:
java.lang.RuntimeException: Unrecoverable error while evaluating node 'PACKAGE:@//tools/build_rules/go/toolchain' (requested by nodes )
at com.google.devtools.build.skyframe.ParallelEvaluator$Evaluate.run(ParallelEvaluator.java:982)
at com.google.devtools.build.lib.concurrent.AbstractQueueVisitor$WrappedRunnable.run(AbstractQueueVisitor.java:499)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: Invalid BUILD file name for package '@//tools/build_rules/go/toolchain': /home/brian/bazel/tools/build_rules/go/toolchain/BUILD
at com.google.devtools.build.lib.packages.Package.finishInit(Package.java:299)
at com.google.devtools.build.lib.packages.Package$Builder.finishBuild(Package.java:1308)
at com.google.devtools.build.lib.skyframe.PackageFunction.compute(PackageFunction.java:501)
at com.google.devtools.build.skyframe.ParallelEvaluator$Evaluate.run(ParallelEvaluator.java:933)
... 4 more
Sponsor's comment: note the abundance of new Label.resolveRepositoryRelative() calls. They are ugly, but it's only making existing ugliness explicit. Yes, we should fix it, especially in the implementation of configurable attributes.
Refs #940
--
Change-Id: I8bd7f7b00bec58a7157507595421bc50c81b404c
Reviewed-on: https://bazel-review.googlesource.com/#/c/2591
MOS_MIGRATED_REVID=117429733
|
|
|
|
|
|
|
|
|
| |
all packages were successful.
Also add Preconditions check to ensure we get the expected Package response when retrieving packages in bulk when we know the set of Package identifiers correspond to valid packages.
--
MOS_MIGRATED_REVID=116580093
|
|
|
|
|
|
|
|
|
| |
Moved RepositoryValue to RepositoryDirectoryValue so that it could be cached
(and not re-downloaded) even if the WorkspaceAST caused a Skyframe restart
(as mentioned in https://github.com/bazelbuild/bazel/blob/master/src/main/java/com/google/devtools/build/lib/rules/repository/RepositoryDelegatorFunction.java#L130-L133).
--
MOS_MIGRATED_REVID=113358489
|
|
|
|
|
|
|
| |
type-safety, we now must pass in the exception type of the callback.
--
MOS_MIGRATED_REVID=113313312
|
|
|
|
|
|
|
|
|
|
| |
As it's about to get more important in its own right.
The only change that isn't just moving code around is making
RepositoryName.validate package-private.
--
MOS_MIGRATED_REVID=112956571
|
|
|
|
|
|
|
| |
directory traversal and package loading, and the other which requests deps on all the transitive targets. We need values from the first half, but the second half can fail to evaluate because of a target cycle. By splitting them, we ensure that there will be values in the graph, so we can get the targets below a directory even if there are cycles present.
--
MOS_MIGRATED_REVID=111609889
|
|
|
|
|
| |
--
MOS_MIGRATED_REVID=111130363
|
|
|
|
|
| |
--
MOS_MIGRATED_REVID=110797095
|
|
|
|
|
|
|
| |
Reduces garbage.
--
MOS_MIGRATED_REVID=109914243
|
|
|
|
|
|
|
| |
Ideally, PrepareDepsOfPatternFunction and maybe even RecursivePkgFunction would also be changed to take a PackageIdentifier instead of RootedPath because the less places we store the set of roots, the better, but I've done enough refactoring in the past weeks to not be thrilled by the idea of doing more.
--
MOS_MIGRATED_REVID=105486561
|
|
|
|
|
|
|
| |
preparation for eventually implementing recursive target patterns for remote repositories.
--
MOS_MIGRATED_REVID=105412566
|
|
|
|
|
|
|
|
|
|
|
| |
The headers were modified with
`find . -type f -exec 'sed' '-Ei' 's|Copyright 201([45]) Google|Copyright 201\1 The Bazel Authors|' '{}' ';'`
And manual edit for not Google owned copyright. Because of the nature of ijar, I did not modified the header of file owned by Alan Donovan.
The list of authors were extracted from the git log. It is missing older Google contributors that can be added on-demand.
--
MOS_MIGRATED_REVID=103938715
|
|
|
|
|
|
|
|
|
| |
RecursivePkgFunction.
This introduces some redundancy with RootedPath, but only in the case of remote repositories. There doesn't seem to be a good way of removing this redundancy.
--
MOS_MIGRATED_REVID=103621610
|
|
|
|
|
|
|
|
|
|
| |
- Label parsing can be simplified
- lib.syntax is only contains the code for Skylark and is reasonably independent from the problem domain of building things
This change is mostly only changes to imports declarations. The rest is reversing the dependency between :cmdline and :syntax and moving a tiny amount of code between Printer and FilesetEntry and the addition of SkylarkPrintableValue that I couldn't be bothered to separate out into its own change.
--
MOS_MIGRATED_REVID=103527877
|
|
|
|
|
|
|
|
|
|
|
| |
contains errors. Instead, require callers to process the package and throw if they need to.
This allows us to avoid embedding a Package in an exception, which is icky. This also allows us to remove Package#containsTemporaryErrors.
Most callers' changes are fairly straightforward. The exception is EnvironmentBackedRecursivePackageProvider, which cannot throw an exception of its own in case of a package with errors (because it doesn't do that in keep_going mode), but whose request for a package with errors *should* shut down the build in case of nokeep_going mode. To do this in Skyframe, we have a new PackageErrorFunction which is to be called only in this situation, and will unconditionally throw. EnvironmentBackedRecursivePackageProvider can then catch this exception and continue on as usual, except that the exception will shut down the thread pool in a nokeep_going build.
--
MOS_MIGRATED_REVID=103247761
|
|
|
|
|
|
|
|
| |
This is necessary to have TargetResolver depend on it without making it depend
on the packages target. First step of #389.
--
MOS_MIGRATED_REVID=101790345
|
|
|
|
|
|
|
|
|
|
|
| |
This involved quite a few changes, mainly changing a bunch of places where we refer to packages by a PathFragment to PackageIdentifier.
The only wart is the code in PathPackageLocator: ideally, it would just call into PackageLookupFunction. Unfortunately, it is (through globbing and Parser.include) called from within a Skyframe function, and we don't want to have two eval() calls going on at the same time, so we cannot use that.
There is a potential correctness issue there: PathPackageLocator now assumes where external repositories are put and assumes that they are there when it gets control, but my understanding is that the associated RepositoryValue is always evaluated before, so it works out okay.
--
MOS_MIGRATED_REVID=97751539
|
|
|
|
|
|
|
|
| |
Hooks up the recently introduced interleaved loading functions to
normal graph loading.
--
MOS_MIGRATED_REVID=97679451
|
|
|
|
|
| |
--
MOS_MIGRATED_REVID=97648982
|
|
|
|
|
|
|
|
|
|
|
| |
This involved quite a few changes, mainly changing a bunch of places where we refer to packages by a PathFragment to PackageIdentifier.
The only wart is the code in PathPackageLocator: ideally, it would just call into PackageLookupFunction. Unfortunately, it is (through globbing and Parser.include) called from within a Skyframe function, and we don't want to have two eval() calls going on at the same time, so we cannot use that.
There is a potential correctness issue there: PathPackageLocator now assumes where external repositories are put and assumes that they are there when it gets control, but my understanding is that the associated RepositoryValue is always evaluated before, so it works out okay.
--
MOS_MIGRATED_REVID=97647787
|
|
|
|
|
|
|
|
|
| |
TransitiveTargetFunction only prints an error message if the package names
match, so it was just exiting with "loading failed" when there was an error
with external dependencies.
--
MOS_MIGRATED_REVID=96204337
|
|
|
|
|
|
|
|
|
|
|
|
| |
RecursivePkgFunction now expects both a rooted path to load packages
beneath and a set of paths to exclude. This also augments existing
machinery to deliver this set of paths to exclude.
This leads toward more efficient processing of target patterns in
target pattern sequence evaluation.
--
MOS_MIGRATED_REVID=94020331
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This change means that tests no longer need to specify the patterns
they use themselves, unless the test involves excluding some pattern.
Also fixes a bug in GraphBackedRecursivePackageProvider revealed by
the test changes where it behaved differently from
EnvironmentBackedRecursivePackageProvider when looking up packages
with errors.
--
MOS_MIGRATED_REVID=89297008
|
|
Moves pattern resolving logic from TargetPatternFunction.Resolver to
a top level class. Adds a layer of abstraction to the Resolver
implementation enabling it to be backed by either an Environment or
a Graph, for use in SkyFunction evaluation or on-the-fly evaluation,
respectively. Finally, SkyQueryEnvironment#preloadOrThrow now checks
to see if each target pattern exists in the graph, and any that
don't will be resolved on-the-fly.
--
MOS_MIGRATED_REVID=88861201
|