aboutsummaryrefslogtreecommitdiffhomepage
path: root/src/main/java/com/google/devtools/build/lib/skyframe/RecursivePackageProviderBackedTargetPatternResolver.java
Commit message (Collapse)AuthorAge
* Rename VariableContext to the more general purpose QueryExpressionContext ↵Gravatar shreyax2018-06-12
| | | | | | and thread it through to graph traversal functions. Some other light refactorings as well. PiperOrigin-RevId: 200292556
* Replace all usages of Blaze's Preconditions class with guava.Gravatar tomlu2017-11-09
| | | | | | | | Blaze had its own class to avoid GC from varargs array creation for the precondition happy path. Guava now (mostly) implements these, making it unnecessary to maintain our own. This change was almost entirely automated by search-and-replace. A few BUILD files needed fixing up since I removed an export of preconditions from lib:util, which was all done by add_deps. There was one incorrect usage of Preconditions that was caught by error prone (which checks Guava's version of Preconditions) that I had to change manually. PiperOrigin-RevId: 175033526
* Generalize some of methods in TargetPattern, PrepareDepsOfPatternValue, and ↵Gravatar nharmata2017-07-26
| | | | | | | RecursivePackageProvider dealing with the concept of "excluded directories". RELNOTES: None PiperOrigin-RevId: 163074794
* Refactor SkyQueryEnvironment to allow preprocessing or postprocessing targetsGravatar Googler2017-06-27
| | | | | | | | Refactor SkyQueryEnvironment and a few other query helpers to make it easier to work with targets. RELNOTES: None PiperOrigin-RevId: 160165398
* Migrate Executor-less Futures callback methods to the ones with an Executor, ↵Gravatar Googler2017-04-25
| | | | | | while applying MoreExecutors.directExecutor() for identical behavior. PiperOrigin-RevId: 154129536
* Remove usage of SynchronizedBatchCallback findTargetsBeneathDirectoryAsync; ↵Gravatar Nathan Harmata2017-03-12
| | | | | | | | | | by design, the callback passed in there is thread safe. A previous change accidentally introduced this unnecessary synchronization. Also move SynchronizedBatchCallback to the file with its only current usage. -- PiperOrigin-RevId: 149809083 MOS_MIGRATED_REVID=149809083
* Improve query error msg when a package has a broken Skylark loadGravatar Mark Schaller2017-03-12
| | | | | | | | | | | | | | | | | | | | | | | | The error message logged during query (and build) when a package has a broken Skylark load statement was not specific. Previously, it said "package contains errors:" and then the package name. Also, this error message was not emitted when using SkyQueryEnvironment and evaluating a query containing a "TargetsBelowDirectory" pattern (such as //foo/...) when a package below the specified directory had such an error. The approach taken by this CL is to include any package loading error message in the SkyValue produced by CollectPackagesUnderDirectoryFunction, and report them during evaluation of a TargetsBelowDirectory pattern. RELNOTES: Evaluation of commands on TargetsBelowDirectory patterns (e.g. //foo/...) matching packages that fail to load now report more detailed error messages in keep_going mode. -- PiperOrigin-RevId: 149802362 MOS_MIGRATED_REVID=149802362
* Description redacted.Gravatar Nathan Harmata2017-03-09
| | | | | | -- PiperOrigin-RevId: 149585165 MOS_MIGRATED_REVID=149585165
* Rollback of commit 822c37816ac669e51bec3853b41849a19ec5e230.Gravatar Nathan Harmata2017-03-01
| | | | | | -- PiperOrigin-RevId: 148844518 MOS_MIGRATED_REVID=148844518
* Reimplement blaze query using an async evaluation model. Use a concurrent ↵Gravatar Nathan Harmata2017-02-28
| | | | | | | | | | | | | | | | | | | backend for SkyQueryEnvironment's implementation in order to achieve parallelism. Advantages: -New design has no flaws that the old design had. -Code is structured so that deadlocks due to thread starvation are impossible (yup!). Disadvantages: -The meat of this change needs to all be in a single CL because every single QueryFunction and QueryExpression needs to be rewritten in the async style. Still TODO: -Fully embrace the async model in all QueryFunctions (e.g. 'rdeps', 'allpaths'). -Use concurrency in BlazeQueryEnvironment to achieve parallel evaluation for (non SkyQuery) 'blaze query' and genquery. -- PiperOrigin-RevId: 148690279 MOS_MIGRATED_REVID=148690279
* Provide more reporting options to SkyFunctions Gravatar Klaus Aehlig2017-02-27
| | | | | | | | | | | | | With more specific information to be reported by Skyfunctions, e.g., to inform the build-event protocol on missing files, the EventHandler interface is no longer enough. Therefore, provide an enriched context for reporting events. -- Change-Id: I2d06166fe4d5b9054e24ad8c752fafc039e3f9f8 Reviewed-on: https://cr.bazel.build/8794 PiperOrigin-RevId: 148463437 MOS_MIGRATED_REVID=148463437
* Make TargetPattern evaluation during query evaluation more parallel-friendly ↵Gravatar Nathan Harmata2016-11-15
| | | | | | | by introducing TargetPattern#parEval, which allows TargetPatterns' evaluations to explicitly have parallel implementations (no need to secretly use a FJP). -- MOS_MIGRATED_REVID=139101922
* Add a mechanism for bounding the number of Packages SkyQueryEnvironment's ↵Gravatar Nathan Harmata2016-11-11
| | | | | | | expensive parallel operations can operate on at once. -- MOS_MIGRATED_REVID=138779172
* Introduce MoreFutures#waitForAllInterruptiblyFailFast and use this in the ↵Gravatar Nathan Harmata2016-09-26
| | | | | | | | | places we wait for tasks (plural!) submitted to a ForkJoinPool to finish since we actually want to do so interruptibly. As was to be expected, testing this was tricky :) -- MOS_MIGRATED_REVID=134128019
* Use ForkJoinPool, rather than ListeningExecutorService, for parallel query ↵Gravatar Nathan Harmata2016-09-22
| | | | | | | evaluation in SkyQueryEnvironment. FJP is nicer to program against, imo. -- MOS_MIGRATED_REVID=133844508
* Allow Skyframe graph lookups and value retrievals to throw InterruptedException.Gravatar Janak Ramakrishnan2016-08-16
| | | | | | | The only place we now don't handle InterruptedException is in the action graph created after analysis, since I'm not sure that will be around for that much longer. -- MOS_MIGRATED_REVID=130327770
* Create one TargetPatternResolver per SkyQueryEnvironment initializationGravatar Mark Schaller2016-06-22
| | | | | | | | | | | | | | | | | | Instead of creating one GraphBackedRecursivePackageProvider, thread pool, and RecursivePackageProviderBackedTargetPatternResolver per call to getTargetsMatchingPattern, create them once during field initialization and in the init method. Previously, Recursive[..]TargetPatternResolver expected to be given ownership of the executor, so it could shut it down. With this change, the resolver now employs a different technique for blocking on the completion of its asynchronous calls. The change also fixes an issue where the use of the resolver along with the EnvironmentBackedRecursivePackageProvider worked only because the provided executor was a direct executor. -- MOS_MIGRATED_REVID=125499330
* Fix hypothetical bug when querying for packages in bulk. We were assuming ↵Gravatar Eric Fellheimer2016-03-08
| | | | | | | | | all packages were successful. Also add Preconditions check to ensure we get the expected Package response when retrieving packages in bulk when we know the set of Package identifiers correspond to valid packages. -- MOS_MIGRATED_REVID=116580093
* Parallelize Package retrieval during Sky-query operation. To maintain ↵Gravatar Eric Fellheimer2016-01-29
| | | | | | | type-safety, we now must pass in the exception type of the callback. -- MOS_MIGRATED_REVID=113313312
* Move RepositoryName to its own top-level classGravatar Kristina Chodorow2016-01-26
| | | | | | | | | | As it's about to get more important in its own right. The only change that isn't just moving code around is making RepositoryName.validate package-private. -- MOS_MIGRATED_REVID=112956571
* Store excluded subdirectories as PathFragment instead of String inside ↵Gravatar Janak Ramakrishnan2016-01-14
| | | | | | | | | TargetPatternKey, since that is what is needed by callers. Also, since the PathFragments come from packages, they are guaranteed to be well-formed, so the checks we were doing were unnecessary. -- MOS_MIGRATED_REVID=112059930
* Inside findTargetsBeneathDirectory(), invoke the callback with all targets ↵Gravatar Eric Fellheimer2016-01-11
| | | | | | | from the current package batch. -- MOS_MIGRATED_REVID=111858978
* Stream results of targets below directory to a callback rather than ↵Gravatar Janak Ramakrishnan2016-01-07
| | | | | | | | | returning a ResolvedTargets set. This is the first step in a series to allow processing large sets of targets in query target patterns via streaming batches rather than all at once. This should be a functional no-op. -- MOS_MIGRATED_REVID=111609309
* Add bulk package lookup for use during target pattern resolution.Gravatar Eric Fellheimer2016-01-04
| | | | | -- MOS_MIGRATED_REVID=111130363
* Make recursive package wildcards work in remote repositories.Gravatar Lukacs Berki2015-10-15
| | | | | | | Ideally, PrepareDepsOfPatternFunction and maybe even RecursivePkgFunction would also be changed to take a PackageIdentifier instead of RootedPath because the less places we store the set of roots, the better, but I've done enough refactoring in the past weeks to not be thrilled by the idea of doing more. -- MOS_MIGRATED_REVID=105486561
* Rationalize copyright headersGravatar Damien Martin-Guillerez2015-09-25
| | | | | | | | | | | The headers were modified with `find . -type f -exec 'sed' '-Ei' 's|Copyright 201([45]) Google|Copyright 201\1 The Bazel Authors|' '{}' ';'` And manual edit for not Google owned copyright. Because of the nature of ijar, I did not modified the header of file owned by Alan Donovan. The list of authors were extracted from the git log. It is missing older Google contributors that can be added on-demand. -- MOS_MIGRATED_REVID=103938715
* Make intra-package wildcards work for remote repositories and clean up ↵Gravatar Lukacs Berki2015-09-24
| | | | | | | | | target pattern parsing just a tiny little bit. This wounds #389 dealing 4d6 fire damage (recursive wildcards, e.g. /... and friends still don't work) -- MOS_MIGRATED_REVID=103822319
* Make TargetPatternResolver#{isPackage, getTargetsInPackage} take a ↵Gravatar Lukacs Berki2015-09-22
| | | | | | | | | PackageIdentifier instead of a String. This remarkably fiddly CL is a step towards making wildcards pattern work with remote repositories. I originally wanted to refactor findTargetsBeneathDirectory(), too, but it turns out that it's a much more complicated affair. -- MOS_MIGRATED_REVID=103622420
* Thread the repository name all the way from TargetPatternResolver to ↵Gravatar Lukacs Berki2015-09-22
| | | | | | | | | RecursivePkgFunction. This introduces some redundancy with RootedPath, but only in the case of remote repositories. There doesn't seem to be a good way of removing this redundancy. -- MOS_MIGRATED_REVID=103621610
* Move Label from the lib.syntax to the lib.cmdline package so that:Gravatar Lukacs Berki2015-09-21
| | | | | | | | | | - Label parsing can be simplified - lib.syntax is only contains the code for Skylark and is reasonably independent from the problem domain of building things This change is mostly only changes to imports declarations. The rest is reversing the dependency between :cmdline and :syntax and moving a tiny amount of code between Printer and FilesetEntry and the addition of SkylarkPrintableValue that I couldn't be bothered to separate out into its own change. -- MOS_MIGRATED_REVID=103527877
* Stop throwing an exception if a Package was successfully created but ↵Gravatar Janak Ramakrishnan2015-09-17
| | | | | | | | | | | contains errors. Instead, require callers to process the package and throw if they need to. This allows us to avoid embedding a Package in an exception, which is icky. This also allows us to remove Package#containsTemporaryErrors. Most callers' changes are fairly straightforward. The exception is EnvironmentBackedRecursivePackageProvider, which cannot throw an exception of its own in case of a package with errors (because it doesn't do that in keep_going mode), but whose request for a package with errors *should* shut down the build in case of nokeep_going mode. To do this in Skyframe, we have a new PackageErrorFunction which is to be called only in this situation, and will unconditionally throw. EnvironmentBackedRecursivePackageProvider can then catch this exception and continue on as usual, except that the exception will shut down the thread pool in a nokeep_going build. -- MOS_MIGRATED_REVID=103247761
* Parse the label syntax "@//a:b" so that eventually we can make that the ↵Gravatar Lukacs Berki2015-09-15
| | | | | | | | | syntax that means "refer to the main repository". There isn't an overarching plan for what we are going to do with the cmdline package, which seems to be separated from the .syntax one in all sorts of awkward ways. -- MOS_MIGRATED_REVID=103088960
* Move PackageIdentifier to cmdlineGravatar Kristina Chodorow2015-08-31
| | | | | | | | This is necessary to have TargetResolver depend on it without making it depend on the packages target. First step of #389. -- MOS_MIGRATED_REVID=101790345
* Make globs work in remote repositories.Gravatar Lukacs Berki2015-07-08
| | | | | | | | | | | This involved quite a few changes, mainly changing a bunch of places where we refer to packages by a PathFragment to PackageIdentifier. The only wart is the code in PathPackageLocator: ideally, it would just call into PackageLookupFunction. Unfortunately, it is (through globbing and Parser.include) called from within a Skyframe function, and we don't want to have two eval() calls going on at the same time, so we cannot use that. There is a potential correctness issue there: PathPackageLocator now assumes where external repositories are put and assumes that they are there when it gets control, but my understanding is that the associated RepositoryValue is always evaluated before, so it works out okay. -- MOS_MIGRATED_REVID=97751539
* Rollback of accidentally submitted change.Gravatar Lukacs Berki2015-07-07
| | | | | -- MOS_MIGRATED_REVID=97648982
* Make globs work in remote repositories.Gravatar Lukacs Berki2015-07-07
| | | | | | | | | | | This involved quite a few changes, mainly changing a bunch of places where we refer to packages by a PathFragment to PackageIdentifier. The only wart is the code in PathPackageLocator: ideally, it would just call into PackageLookupFunction. Unfortunately, it is (through globbing and Parser.include) called from within a Skyframe function, and we don't want to have two eval() calls going on at the same time, so we cannot use that. There is a potential correctness issue there: PathPackageLocator now assumes where external repositories are put and assumes that they are there when it gets control, but my understanding is that the associated RepositoryValue is always evaluated before, so it works out okay. -- MOS_MIGRATED_REVID=97647787
* Refactor reusable target parsing functions from RPPBTPRGravatar Mark Schaller2015-06-23
| | | | | | | | Adds them to TargetPatternResolverUtil alongside other useful target parsing utility functions. -- MOS_MIGRATED_REVID=96611975
* Cleanup findTargetsBeneathDirectoryGravatar Mark Schaller2015-06-11
| | | | | | | | | | Improves local variable names, moves building of excluded path fragments outside of the package path entries loop, and moves the explanatory comment for the no-targets-found check up to where the check is performed. -- MOS_MIGRATED_REVID=95660427
* Teach skyframe about excluded directories, pathsGravatar Mark Schaller2015-05-21
| | | | | | | | | | | | RecursivePkgFunction now expects both a rooted path to load packages beneath and a set of paths to exclude. This also augments existing machinery to deliver this set of paths to exclude. This leads toward more efficient processing of target patterns in target pattern sequence evaluation. -- MOS_MIGRATED_REVID=94020331
* On-the-fly target pattern resolution in SkyQueryEnvironmentGravatar Mark Schaller2015-03-18
Moves pattern resolving logic from TargetPatternFunction.Resolver to a top level class. Adds a layer of abstraction to the Resolver implementation enabling it to be backed by either an Environment or a Graph, for use in SkyFunction evaluation or on-the-fly evaluation, respectively. Finally, SkyQueryEnvironment#preloadOrThrow now checks to see if each target pattern exists in the graph, and any that don't will be resolved on-the-fly. -- MOS_MIGRATED_REVID=88861201
generated by cgit v1.2.3 (git 2.39.1) at 2024-06-20 19:54:28 +0000