aboutsummaryrefslogtreecommitdiffhomepage
path: root/src/main/java/com/google/devtools/build/lib/query2/BlazeQueryEnvironment.java
Commit message (Collapse)AuthorAge
* Replace PackageProvider with TargetProvider where possibleGravatar Ulf Adams2017-02-09
| | | | | | | | | | I was trying to simplify GenQuery, and found this during an audit of the code; it's not sufficient to make any larger changes, but it may help eventuall. -- PiperOrigin-RevId: 147023626 MOS_MIGRATED_REVID=147023626
* Remove dead code in SkyframeLabelVisitorGravatar Ulf Adams2017-02-07
| | | | | | | | | | Two of the functions are never used, several of the parameter are not used or can be simplified (require callers to convert from Target -> Label). -- PiperOrigin-RevId: 146787380 MOS_MIGRATED_REVID=146787380
* Rollback of commit e0a330577d9fe98169645cb68d9fc22cc787eeb6.Gravatar Janak Ramakrishnan2016-12-21
| | | | | | | | | | *** Reason for rollback *** Investigating if causes deadlock/thread starvation. -- PiperOrigin-RevId: 142575769 MOS_MIGRATED_REVID=142575769
* For all function expressions of the form f(..., e1, ..., e2, ..., eK, ...), ↵Gravatar Nathan Harmata2016-12-02
| | | | | | | | ensure that all of e1, e2, ..., and eK are elligble for parallel evaluation. This is _not_ the same as providing a parallel implementation of f, which we can do separately in followup CLs. -- PiperOrigin-RevId: 140861694 MOS_MIGRATED_REVID=140861694
* Make TargetPattern evaluation during query evaluation more parallel-friendly ↵Gravatar Nathan Harmata2016-11-15
| | | | | | | by introducing TargetPattern#parEval, which allows TargetPatterns' evaluations to explicitly have parallel implementations (no need to secretly use a FJP). -- MOS_MIGRATED_REVID=139101922
* Unify AbstractBlazeQueryEnvironment#evaluateQuery with its subclass ↵Gravatar Nathan Harmata2016-10-04
| | | | | | | overrides. Also, have AbstractBlazeQueryEnvironment#evaluateQuery take an OutputFormatterCallback instance rather than a Callback instance. This is more sensible since the latter is only intended to be used intra-query, while the former is intended for usage in end-to-end query evaluation. This lets us slightly simplify QueryCommand, by shifting the responsibility for managing the OutputFormatterCallback to AbstractBlazeQueryEnvironment#evaluateQuery. -- MOS_MIGRATED_REVID=134827588
* Make AbstractBlazeQueryEnvironment no longer implement AutoCloseable. ↵Gravatar Nathan Harmata2016-09-29
| | | | | | | | | | | Instead have SkyQueryEnvironment#evaluateQuery be responsible for handling cleanup of its internal ForkJoinPool. In addition to being a more sensible way of organizing the code (imo, it makes sense for SkyQueryEnvironment to clean up after itself), this fixes several issues: (i) If query evaluation is interrupted, the AbstractBlazeQueryEnvironment#close call at the end of the try-with-resources statement in QueryCommand would be blocking and non-urgent. N.B. This was not an issue with the current ForkJoinPool usage, but was an issue with the old ThreadPoolExecutor usage. (ii) Because of how the code in QueryCommand was structured, OutputFormatterCallback#close would happen _before_ the AbstractBlazeQueryEnvironment#close call. If query evaluation is interrupted, threads executing query tasks may be invoking the callback after the callback had been shut down! -- MOS_MIGRATED_REVID=134573395
* A bunch of small changes to prepare SkyQueryEnvironment for full-parallel ↵Gravatar Nathan Harmata2016-09-08
| | | | | | | | | | | | | | evaluation: -Rename QueryExpression#evalConcurrently to QueryExpression#parEval. (parallelism is not concurrency! See https://existentialtype.wordpress.com/2011/03/17/parallelism-is-not-concurrency/) -Have SkyQueryEnvironment#eval (used recursively in #evaluateQuery) dynamically call QueryExpression#parEval when appropriate. -Delete QueryExpression#canEvalConcurrently. -Add ThreadSafety annotations in a bunch of relevant places in the query codebase. -A bunch of testing infrastructure to test parallel query evaluation. -TODOs for implementing parallel evaluation of all QueryExpression nodes. -- MOS_MIGRATED_REVID=132436340
* In BlazeQueryEnvironment, stop wrapping InterruptedExceptions in ↵Gravatar Janak Ramakrishnan2016-08-17
| | | | | | | | | QueryExceptions now that all relevant methods declare that they throw InterruptedException. Small side benefit of commit 3c0adb26bac6d756fb97e4bcc6d4e5b2cefa5eeb. -- MOS_MIGRATED_REVID=130402917
* Allow Skyframe graph lookups and value retrievals to throw InterruptedException.Gravatar Janak Ramakrishnan2016-08-16
| | | | | | | The only place we now don't handle InterruptedException is in the action graph created after analysis, since I'm not sure that will be around for that much longer. -- MOS_MIGRATED_REVID=130327770
* Re-implement variables in the blaze query language. Instead of using a ↵Gravatar Nathan Harmata2016-07-14
| | | | | | | | | mutable global context of variable bindings, pass around immutable local contexts. The motivation is so we can safely evaluate all blaze query expressions concurrently under the hood. A global context is hostile to this goal. -- MOS_MIGRATED_REVID=127324600
* Fix threadpool leak in SkyQueryEnvironmentGravatar Mark Schaller2016-06-27
| | | | | | | | Shutdown the SkyQueryEnvironment's threadpool after query evaluation is complete and the environment is ready for disposal. -- MOS_MIGRATED_REVID=125975317
* Resolve target patterns on the fly in SkyQueryEnvironment. Cache only the ↵Gravatar Janak Ramakrishnan2016-01-07
| | | | | | | | | label sets that are precomputed in the graph. This is the fourth step in a series to allow processing large sets of targets in query target patterns via streaming batches rather than all at once. This may make SkyQueryEnvironment slower when evaluating queries with repeated target patterns, or many target patterns that would benefit from graph lookups that were batched across all patterns. But that is not currently a bottleneck we're concerned about. -- MOS_MIGRATED_REVID=111626483
* Make QueryEnvironment#getTargetsMatchingPattern pass its result to a callback.Gravatar Janak Ramakrishnan2016-01-07
| | | | | | | This is the third step in a series to allow processing large sets of targets in query target patterns via streaming batches rather than all at once. This should also be a functional no-op. -- MOS_MIGRATED_REVID=111620668
* RELNOTES: add loadfiles() query operator, to find skylark files loaded by ↵Gravatar Han-Wen Nienhuys2015-12-16
| | | | | | | targets. -- MOS_MIGRATED_REVID=110275427
* Use Bazel Preconditions variant which avoids varargs array creationGravatar Mark Schaller2015-12-10
| | | | | | | Reduces garbage. -- MOS_MIGRATED_REVID=109914243
* Allow output formatters to work in stream mode.Gravatar Miguel Alcon Pinto2015-12-10
| | | | | -- MOS_MIGRATED_REVID=109908202
* Rollback of commit 8e7da8704dba78dba47fbaa0eafce3a91c2deb13.Gravatar Peter Schmitt2015-12-01
| | | | | | | *** Reason for rollback *** -- MOS_MIGRATED_REVID=109005097
* Allow output formatters to work in stream mode.Gravatar Miguel Alcon Pinto2015-11-19
| | | | | -- MOS_MIGRATED_REVID=108142169
* Transform Blaze query to be able to work in streamed mode.Gravatar Miguel Alcon Pinto2015-11-06
| | | | | -- MOS_MIGRATED_REVID=107249788
* Rationalize copyright headersGravatar Damien Martin-Guillerez2015-09-25
| | | | | | | | | | | The headers were modified with `find . -type f -exec 'sed' '-Ei' 's|Copyright 201([45]) Google|Copyright 201\1 The Bazel Authors|' '{}' ';'` And manual edit for not Google owned copyright. Because of the nature of ijar, I did not modified the header of file owned by Alan Donovan. The list of authors were extracted from the git log. It is missing older Google contributors that can be added on-demand. -- MOS_MIGRATED_REVID=103938715
* Move Label from the lib.syntax to the lib.cmdline package so that:Gravatar Lukacs Berki2015-09-21
| | | | | | | | | | - Label parsing can be simplified - lib.syntax is only contains the code for Skylark and is reasonably independent from the problem domain of building things This change is mostly only changes to imports declarations. The rest is reversing the dependency between :cmdline and :syntax and moving a tiny amount of code between Printer and FilesetEntry and the addition of SkylarkPrintableValue that I couldn't be bothered to separate out into its own change. -- MOS_MIGRATED_REVID=103527877
* Parse the label syntax "@//a:b" so that eventually we can make that the ↵Gravatar Lukacs Berki2015-09-15
| | | | | | | | | syntax that means "refer to the main repository". There isn't an overarching plan for what we are going to do with the cmdline package, which seems to be separated from the .syntax one in all sorts of awkward ways. -- MOS_MIGRATED_REVID=103088960
* Fix some warnings.Gravatar Ulf Adams2015-09-04
| | | | | -- MOS_MIGRATED_REVID=102332437
* Use Set<Target> instead of ResolvedTargets<Target> when saving resolved ↵Gravatar Janak Ramakrishnan2015-08-18
| | | | | | | | | target patterns in query. We never use anything but the getTargets() field, so it's useless to store other things. -- MOS_MIGRATED_REVID=100846573
* Throw InterruptedException directly if a query is interrupted, instead of ↵Gravatar Janak Ramakrishnan2015-07-10
| | | | | | | | | wrapping it in a QueryException. QueryException should usually indicate a persistent failure, while an InterruptedException is transient. Wrapping the InterruptedException in a QueryException just obfuscates state. -- MOS_MIGRATED_REVID=97815388
* Clean up AllPathsFunction and get rid of getFwdDeps(Target) and ↵Gravatar Janak Ramakrishnan2015-06-19
| | | | | | | getReverseDeps(Target) from the QueryEnvironment interface, since they're no longer needed by any query functions, and also WalkableGraph#get{Direct,Reverse}Deps(SkyKey). -- MOS_MIGRATED_REVID=96361760
* Create batch versions of query environment methods getFwdDeps and ↵Gravatar Janak Ramakrishnan2015-06-17
| | | | | | | | | getReverseDeps, and migrate DepsFunction and RdepsFunction to use them. This makes our (unordered) results potentially dependent on the iteration order of hash sets, but what do we care? -- MOS_MIGRATED_REVID=96117626
* Teach FakeSubincludeTarget its package, because rdeps wants itGravatar Mark Schaller2015-06-05
| | | | | | | | | | | | While evaluating the transitive closure of the universe, if the universe contains the buildfiles of a package with a subinclude, the RdepsFunction would ask a FakeSubincludeTarget for its package. Prior to this change, this would result in a crash. Now, FakeSubincludeTargets remember the package they belong to. -- MOS_MIGRATED_REVID=95212925
* Preload the universe for query testsGravatar Mark Schaller2015-03-24
| | | | | | | | | | | | | This change means that tests no longer need to specify the patterns they use themselves, unless the test involves excluding some pattern. Also fixes a bug in GraphBackedRecursivePackageProvider revealed by the test changes where it behaved differently from EnvironmentBackedRecursivePackageProvider when looking up packages with errors. -- MOS_MIGRATED_REVID=89297008
* Add a SkyQueryEnvironment which runs queries by examining the Skyframe graph.Gravatar Janak Ramakrishnan2015-02-26
| | | | | | | | | This environment eagerly preloads the transitive closure of a specified query "universe", and so may not be as efficient as the standard query for limited-scope queries. It is activated when the universe is specified and ordered results are not requested (since it is currently unable to order results). Tests were modified/added to exercise this environment where deemed interesting. Some ugly hacks were done to add coverage in AbstractQueryTest and friends, because currently even if the full depot is loaded (using //...), individual target patterns most likely won't be present in the graph. A better way to deal with this situation, suggested by felly, is probably to extract target pattern resolution logic to an auxiliary function so that query is able to resolve target patterns without mutating the graph, and then call into the read-only graph with the resolved patterns. That may be done in a follow-up, in which case the "scope" of every query could be //... . -- MOS_MIGRATED_REVID=87257028
* Refactor Query code a bit. Get rid of separate SkyframeQueryEnvironment, ↵Gravatar Janak Ramakrishnan2015-02-11
| | | | | | | | | move some utility methods to utility class. The real goal of this CL is to make sure all users of BlazeQueryEnvironment are constructing one through EvaluableBlazeQueryEnvironment#newQueryEnvironment, and not directly. This ensures that alternate implementations of EvaluableBlazeQueryEnvironment will be automatically injected via #newQueryEnvironment. -- MOS_MIGRATED_REVID=86112439
* Extract an EvaluableBlazeQueryEnvironment abstract class from ↵Gravatar Janak Ramakrishnan2015-02-11
| | | | | | | | | | | BlazeQueryEnvironment that exposes an evaluateQuery method, and also implements the non-LabelVisitor-specific parts of BlazeQueryEnvironment, for other implementations' uses. Most of the code is just a straight refactoring of BlazeQueryEnvironment into EvaluableBlazeQueryEnvironment (and BlazeTargetAccessor). Ignoring whitespace changes in [] may be your friend for seeing that it's a straight move. This also allows us to change tests to use QueryCommand.newQueryEnvironment, in preparation for newQueryEnvironment potentially returning a different EvaluableBlazeQueryEnvironment subclass depending on the circumstances. -- MOS_MIGRATED_REVID=86088953
* Remove unnecessary wrapper in return value of ↵Gravatar Janak Ramakrishnan2015-02-06
| | | | | | | BlazeQueryEnvironment#getTarget(Label). -- MOS_MIGRATED_REVID=85744195
* Update from Google.Gravatar Han-Wen Nienhuys2015-02-25
-- MOE_MIGRATED_REVID=85702957
generated by cgit v1.2.3 (git 2.39.1) at 2024-07-30 12:18:16 +0000