aboutsummaryrefslogtreecommitdiffhomepage
path: root/src/main/java/com/google/devtools/build/lib/skyframe/PrepareDepsOfPatternFunction.java
Commit message (Collapse)AuthorAge
* Remove CachingPackageLocator interface from the PackageProvider hierarchyGravatar ulfjack2018-06-06
| | | | | | This simplifies the code quite a bit. PiperOrigin-RevId: 199427943
* Consolidate fields and methods of recursive package providers into an ↵Gravatar juliexxia2018-05-03
| | | | | | abstract class. This is also helpful for the new recursive package provider I'll be adding to support recursive patterns in cquery. PiperOrigin-RevId: 195279920
* Introduce Root class.Gravatar tomlu2018-01-17
| | | | | | | | | | | This class represents a root (such as a package path or an output root) used for file lookups and artifacts. It is meant to be as opaque as possible in order to hide the user's environment from sky keys and sky functions. Roots are used by RootedPaths and ArtifactRoots. This CL attempts to make the minimum number of modifications necessary to change RootedPath and ArtifactRoot to use these fields. Deprecated methods and invasive accessors are permitted to minimise the risk of any observable changes. RELNOTES: None PiperOrigin-RevId: 182271759
* Fix getBuildFiles to not assume BUILD is the name of the build file.Gravatar John Cater2017-12-04
| | | | | | | Fixes #4056. Change-Id: Ia7425c2146f15e9293605ee3da53007805e82275 PiperOrigin-RevId: 177813070
* Replace all usages of Blaze's Preconditions class with guava.Gravatar tomlu2017-11-09
| | | | | | | | Blaze had its own class to avoid GC from varargs array creation for the precondition happy path. Guava now (mostly) implements these, making it unnecessary to maintain our own. This change was almost entirely automated by search-and-replace. A few BUILD files needed fixing up since I removed an export of preconditions from lib:util, which was all done by add_deps. There was one incorrect usage of Preconditions that was caught by error prone (which checks Guava's version of Preconditions) that I had to change manually. PiperOrigin-RevId: 175033526
* Fix bug where all three implementations of ↵Gravatar nharmata2017-10-31
| | | | | | | | | RecursivePackageProvider#getPackagesUnderDirectory assumed that the given directory needed to be strictly underneath all directories in the given blacklistedSubdirectories set, yet the callsites were calling this method such that the given directory was not-necessarily-strictly underneath the blacklistedSubdirectories. N.B. There is no end-to-end bug in EnvironmentBackedRecursivePackageProvider because TargetPatternFunction's usage of TargetPattern#eval is always with blacklistedSubdirectories=ImmutableSet.of(). Also note that the existing fast-path in GraphBackedRecursivePackageProvider is dead code because the Preconditions check would fail first :( RELNOTES: None PiperOrigin-RevId: 173924216
* Generalize some of methods in TargetPattern, PrepareDepsOfPatternValue, and ↵Gravatar nharmata2017-07-26
| | | | | | | RecursivePackageProvider dealing with the concept of "excluded directories". RELNOTES: None PiperOrigin-RevId: 163074794
* Rewrite repository lookup to return a failed value rather than throwGravatar ulfjack2017-04-20
| | | | | | | | | | | | | | | | | | | | | | | We need to lookup repositories as part of converting exec paths to artifacts, which in turn is needed for action cache lookups. These lookups should not cause a Skyframe exit, so we must not throw an exception here, unless the error makes it impossible to continue. Instead, we need to leave the decision whether to error out or not to the caller. Note that we may unnecessarily fetch a remote repository in order to do the action cache lookup, even if the action no longer depends on the input file, although this should only be possible for C++ compile actions. It's possible that there's another bug in the C++ compile action key computation that also contributes. This change also makes it so that the post-resolution action cache code ignores any errors wrt. repository lookup rather than throwing. If any of the paths could not be found, then the action cache lookup fails and we re-execute the corresponding action, which is exactly what should happen. Fixes #2759. PiperOrigin-RevId: 153696243
* Description redacted.Gravatar Nathan Harmata2017-03-09
| | | | | | -- PiperOrigin-RevId: 149585165 MOS_MIGRATED_REVID=149585165
* Rollback of commit 822c37816ac669e51bec3853b41849a19ec5e230.Gravatar Nathan Harmata2017-03-01
| | | | | | -- PiperOrigin-RevId: 148844518 MOS_MIGRATED_REVID=148844518
* Reimplement blaze query using an async evaluation model. Use a concurrent ↵Gravatar Nathan Harmata2017-02-28
| | | | | | | | | | | | | | | | | | | backend for SkyQueryEnvironment's implementation in order to achieve parallelism. Advantages: -New design has no flaws that the old design had. -Code is structured so that deadlocks due to thread starvation are impossible (yup!). Disadvantages: -The meat of this change needs to all be in a single CL because every single QueryFunction and QueryExpression needs to be rewritten in the async style. Still TODO: -Fully embrace the async model in all QueryFunctions (e.g. 'rdeps', 'allpaths'). -Use concurrency in BlazeQueryEnvironment to achieve parallel evaluation for (non SkyQuery) 'blaze query' and genquery. -- PiperOrigin-RevId: 148690279 MOS_MIGRATED_REVID=148690279
* Make TargetPattern evaluation during query evaluation more parallel-friendly ↵Gravatar Nathan Harmata2016-11-15
| | | | | | | by introducing TargetPattern#parEval, which allows TargetPatterns' evaluations to explicitly have parallel implementations (no need to secretly use a FJP). -- MOS_MIGRATED_REVID=139101922
* Allow Skyframe graph lookups and value retrievals to throw InterruptedException.Gravatar Janak Ramakrishnan2016-08-16
| | | | | | | The only place we now don't handle InterruptedException is in the action graph created after analysis, since I'm not sure that will be around for that much longer. -- MOS_MIGRATED_REVID=130327770
* Fix SkyQuery bug where we weren't respecting the package blacklist. We do ↵Gravatar Nathan Harmata2016-04-18
| | | | | | | | | this by changing both the relevant Skyframe and the SkyQuery code to propagate (minimal!) blacklist information in the SkyKeys themselves. There are other approaches to solving this problem, but I like how this solution doesn't involve duplication of logic. Also, it has the following nice benefit: previously, RecursiveDirectoryTraversalFunction would declare a dep on the blacklist for every directory traversed which adds an edge for each directory traversed. -- MOS_MIGRATED_REVID=120049635
* Don't keep packages in the default repository around after loading.Gravatar Brian Silverman2016-03-17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Previously, this would get thrown when referring to the same package from both the main and default repositories: java.lang.IllegalArgumentException: Multiple entries with same key: tools/cpp=/home/brian/971-Robot-Code and tools/cpp=/home/brian/971-Robot-Code at com.google.common.collect.ImmutableMap.checkNoConflict(ImmutableMap.java:136) at com.google.common.collect.RegularImmutableMap.checkNoConflictInKeyBucket(RegularImmutableMap.java:98) at com.google.common.collect.RegularImmutableMap.fromEntryArray(RegularImmutableMap.java:84) at com.google.common.collect.ImmutableMap$Builder.build(ImmutableMap.java:295) at com.google.devtools.build.lib.buildtool.BuildTool.transformPackageRoots(BuildTool.java:301) at com.google.devtools.build.lib.buildtool.BuildTool.buildTargets(BuildTool.java:209) at com.google.devtools.build.lib.buildtool.BuildTool.processRequest(BuildTool.java:334) at com.google.devtools.build.lib.runtime.commands.TestCommand.doTest(TestCommand.java:119) at com.google.devtools.build.lib.runtime.commands.TestCommand.exec(TestCommand.java:104) at com.google.devtools.build.lib.runtime.BlazeCommandDispatcher.exec(BlazeCommandDispatcher.java:371) at com.google.devtools.build.lib.runtime.BlazeRuntime$3.exec(BlazeRuntime.java:1016) at com.google.devtools.build.lib.server.RPCService.executeRequest(RPCService.java:65) at com.google.devtools.build.lib.server.RPCServer.executeRequest(RPCServer.java:434) at com.google.devtools.build.lib.server.RPCServer.serve(RPCServer.java:229) at com.google.devtools.build.lib.runtime.BlazeRuntime.serverMain(BlazeRuntime.java:975) at com.google.devtools.build.lib.runtime.BlazeRuntime.main(BlazeRuntime.java:772) at com.google.devtools.build.lib.bazel.BazelMain.main(BazelMain.java:55) And this would get thrown for any packages in the main repository loaded from other repositories: java.lang.RuntimeException: Unrecoverable error while evaluating node 'PACKAGE:@//tools/build_rules/go/toolchain' (requested by nodes ) at com.google.devtools.build.skyframe.ParallelEvaluator$Evaluate.run(ParallelEvaluator.java:982) at com.google.devtools.build.lib.concurrent.AbstractQueueVisitor$WrappedRunnable.run(AbstractQueueVisitor.java:499) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.IllegalArgumentException: Invalid BUILD file name for package '@//tools/build_rules/go/toolchain': /home/brian/bazel/tools/build_rules/go/toolchain/BUILD at com.google.devtools.build.lib.packages.Package.finishInit(Package.java:299) at com.google.devtools.build.lib.packages.Package$Builder.finishBuild(Package.java:1308) at com.google.devtools.build.lib.skyframe.PackageFunction.compute(PackageFunction.java:501) at com.google.devtools.build.skyframe.ParallelEvaluator$Evaluate.run(ParallelEvaluator.java:933) ... 4 more Sponsor's comment: note the abundance of new Label.resolveRepositoryRelative() calls. They are ugly, but it's only making existing ugliness explicit. Yes, we should fix it, especially in the implementation of configurable attributes. Refs #940 -- Change-Id: I8bd7f7b00bec58a7157507595421bc50c81b404c Reviewed-on: https://bazel-review.googlesource.com/#/c/2591 MOS_MIGRATED_REVID=117429733
* Parse the workspace name when a repository is loadedGravatar Kristina Chodorow2016-01-29
| | | | | | | | | Moved RepositoryValue to RepositoryDirectoryValue so that it could be cached (and not re-downloaded) even if the WorkspaceAST caused a Skyframe restart (as mentioned in https://github.com/bazelbuild/bazel/blob/master/src/main/java/com/google/devtools/build/lib/rules/repository/RepositoryDelegatorFunction.java#L130-L133). -- MOS_MIGRATED_REVID=113358489
* Parallelize Package retrieval during Sky-query operation. To maintain ↵Gravatar Eric Fellheimer2016-01-29
| | | | | | | type-safety, we now must pass in the exception type of the callback. -- MOS_MIGRATED_REVID=113313312
* Move RepositoryName to its own top-level classGravatar Kristina Chodorow2016-01-26
| | | | | | | | | | As it's about to get more important in its own right. The only change that isn't just moving code around is making RepositoryName.validate package-private. -- MOS_MIGRATED_REVID=112956571
* Perform package loading in parallel with transitive target visitation. This ↵Gravatar Janak Ramakrishnan2016-01-21
| | | | | | | is a partial rollback of commit f1e257d because it turns out that loading sequentially can be a bottleneck. -- MOS_MIGRATED_REVID=112628616
* Store excluded subdirectories as PathFragment instead of String inside ↵Gravatar Janak Ramakrishnan2016-01-14
| | | | | | | | | TargetPatternKey, since that is what is needed by callers. Also, since the PathFragments come from packages, they are guaranteed to be well-formed, so the checks we were doing were unnecessary. -- MOS_MIGRATED_REVID=112059930
* Stream TargetPattern#eval implementations' results to a callback rather than ↵Gravatar Janak Ramakrishnan2016-01-07
| | | | | | | | | returning a ResolvedTargets set. This is the second step in a series to allow processing large sets of targets in query target patterns via streaming batches rather than all at once. This should also be a functional no-op. -- MOS_MIGRATED_REVID=111611858
* Stream results of targets below directory to a callback rather than ↵Gravatar Janak Ramakrishnan2016-01-07
| | | | | | | | | returning a ResolvedTargets set. This is the first step in a series to allow processing large sets of targets in query target patterns via streaming batches rather than all at once. This should be a functional no-op. -- MOS_MIGRATED_REVID=111609309
* Use Bazel Preconditions variant which avoids varargs array creationGravatar Mark Schaller2015-12-10
| | | | | | | Reduces garbage. -- MOS_MIGRATED_REVID=109914243
* Make recursive package wildcards work in remote repositories.Gravatar Lukacs Berki2015-10-15
| | | | | | | Ideally, PrepareDepsOfPatternFunction and maybe even RecursivePkgFunction would also be changed to take a PackageIdentifier instead of RootedPath because the less places we store the set of roots, the better, but I've done enough refactoring in the past weeks to not be thrilled by the idea of doing more. -- MOS_MIGRATED_REVID=105486561
* Rationalize copyright headersGravatar Damien Martin-Guillerez2015-09-25
| | | | | | | | | | | The headers were modified with `find . -type f -exec 'sed' '-Ei' 's|Copyright 201([45]) Google|Copyright 201\1 The Bazel Authors|' '{}' ';'` And manual edit for not Google owned copyright. Because of the nature of ijar, I did not modified the header of file owned by Alan Donovan. The list of authors were extracted from the git log. It is missing older Google contributors that can be added on-demand. -- MOS_MIGRATED_REVID=103938715
* Make intra-package wildcards work for remote repositories and clean up ↵Gravatar Lukacs Berki2015-09-24
| | | | | | | | | target pattern parsing just a tiny little bit. This wounds #389 dealing 4d6 fire damage (recursive wildcards, e.g. /... and friends still don't work) -- MOS_MIGRATED_REVID=103822319
* Make TargetPatternResolver#{isPackage, getTargetsInPackage} take a ↵Gravatar Lukacs Berki2015-09-22
| | | | | | | | | PackageIdentifier instead of a String. This remarkably fiddly CL is a step towards making wildcards pattern work with remote repositories. I originally wanted to refactor findTargetsBeneathDirectory(), too, but it turns out that it's a much more complicated affair. -- MOS_MIGRATED_REVID=103622420
* Thread the repository name all the way from TargetPatternResolver to ↵Gravatar Lukacs Berki2015-09-22
| | | | | | | | | RecursivePkgFunction. This introduces some redundancy with RootedPath, but only in the case of remote repositories. There doesn't seem to be a good way of removing this redundancy. -- MOS_MIGRATED_REVID=103621610
* Move Label from the lib.syntax to the lib.cmdline package so that:Gravatar Lukacs Berki2015-09-21
| | | | | | | | | | - Label parsing can be simplified - lib.syntax is only contains the code for Skylark and is reasonably independent from the problem domain of building things This change is mostly only changes to imports declarations. The rest is reversing the dependency between :cmdline and :syntax and moving a tiny amount of code between Printer and FilesetEntry and the addition of SkylarkPrintableValue that I couldn't be bothered to separate out into its own change. -- MOS_MIGRATED_REVID=103527877
* Fix some warnings.Gravatar Ulf Adams2015-09-04
| | | | | -- MOS_MIGRATED_REVID=102332437
* Move PackageIdentifier to cmdlineGravatar Kristina Chodorow2015-08-31
| | | | | | | | This is necessary to have TargetResolver depend on it without making it depend on the packages target. First step of #389. -- MOS_MIGRATED_REVID=101790345
* Adds TransitiveTraversalFunction that loads transitive deps of a targetGravatar Mark Schaller2015-07-30
| | | | | | | | | Unlike TransitiveTargetFunction, it does not return nested sets of the traversed targets. Used primarily for its side effects of loading the transitive targets into the graph. -- MOS_MIGRATED_REVID=99388411
* Make globs work in remote repositories.Gravatar Lukacs Berki2015-07-08
| | | | | | | | | | | This involved quite a few changes, mainly changing a bunch of places where we refer to packages by a PathFragment to PackageIdentifier. The only wart is the code in PathPackageLocator: ideally, it would just call into PackageLookupFunction. Unfortunately, it is (through globbing and Parser.include) called from within a Skyframe function, and we don't want to have two eval() calls going on at the same time, so we cannot use that. There is a potential correctness issue there: PathPackageLocator now assumes where external repositories are put and assumes that they are there when it gets control, but my understanding is that the associated RepositoryValue is always evaluated before, so it works out okay. -- MOS_MIGRATED_REVID=97751539
* Activate interleaved package and transitive target loadingGravatar Mark Schaller2015-07-08
| | | | | | | | Hooks up the recently introduced interleaved loading functions to normal graph loading. -- MOS_MIGRATED_REVID=97679451
* Rollback of accidentally submitted change.Gravatar Lukacs Berki2015-07-07
| | | | | -- MOS_MIGRATED_REVID=97648982
* Make globs work in remote repositories.Gravatar Lukacs Berki2015-07-07
| | | | | | | | | | | This involved quite a few changes, mainly changing a bunch of places where we refer to packages by a PathFragment to PackageIdentifier. The only wart is the code in PathPackageLocator: ideally, it would just call into PackageLookupFunction. Unfortunately, it is (through globbing and Parser.include) called from within a Skyframe function, and we don't want to have two eval() calls going on at the same time, so we cannot use that. There is a potential correctness issue there: PathPackageLocator now assumes where external repositories are put and assumes that they are there when it gets control, but my understanding is that the associated RepositoryValue is always evaluated before, so it works out okay. -- MOS_MIGRATED_REVID=97647787
* Introduce interleaved package and transitive target loadingGravatar Mark Schaller2015-07-01
Adds SkyFunctions and assorted values that implement interleaved loading of packages and their targets' transitive dependencies. They are not hooked up to any graph loading components, yet. -- MOS_MIGRATED_REVID=97278368
generated by cgit v1.2.3 (git 2.39.1) at 2024-06-12 20:50:06 +0000