aboutsummaryrefslogtreecommitdiffhomepage
path: root/src/main/java/com/google/devtools/build/lib/query2/engine
Commit message (Collapse)AuthorAge
...
* Introduce a failFast mode to OutputFormatterCallback#close.Gravatar Nathan Harmata2016-11-18
| | | | | -- MOS_MIGRATED_REVID=139508838
* Make TargetPattern evaluation during query evaluation more parallel-friendly ↵Gravatar Nathan Harmata2016-11-15
| | | | | | | by introducing TargetPattern#parEval, which allows TargetPatterns' evaluations to explicitly have parallel implementations (no need to secretly use a FJP). -- MOS_MIGRATED_REVID=139101922
* Have query Lexer operated directly on StringGravatar Michajlo Matijkiw2016-11-02
| | | | | | | | No need for the char[] in the middle, prevents us from accidentally modifying input, or sucking up ram on huge queries. -- MOS_MIGRATED_REVID=137872573
* Unify AbstractBlazeQueryEnvironment#evaluateQuery with its subclass ↵Gravatar Nathan Harmata2016-10-04
| | | | | | | overrides. Also, have AbstractBlazeQueryEnvironment#evaluateQuery take an OutputFormatterCallback instance rather than a Callback instance. This is more sensible since the latter is only intended to be used intra-query, while the former is intended for usage in end-to-end query evaluation. This lets us slightly simplify QueryCommand, by shifting the responsibility for managing the OutputFormatterCallback to AbstractBlazeQueryEnvironment#evaluateQuery. -- MOS_MIGRATED_REVID=134827588
* Provide parallel implementations of 'allrdeps' and 'rbuildfiles', using a ↵Gravatar Nathan Harmata2016-09-29
| | | | | | | shared naive BFS implementation. Also implement RegexFilterExpression#parEval. -- MOS_MIGRATED_REVID=134598046
* Use ForkJoinPool, rather than ListeningExecutorService, for parallel query ↵Gravatar Nathan Harmata2016-09-22
| | | | | | | evaluation in SkyQueryEnvironment. FJP is nicer to program against, imo. -- MOS_MIGRATED_REVID=133844508
* Have SkyQueryEnvironment#getRBuildFiles not visit skyframe nodes more than ↵Gravatar Nathan Harmata2016-09-12
| | | | | | | once. This is helpful when there are multiple skylark reverse 'load' paths from the same popular .bzl file to a single BUILD file. As an implementation detail, introduce a few ThreadSafeUniquifier things (these will also be used in followup CLs). -- MOS_MIGRATED_REVID=132680777
* Boilerplate for parallel evaluation of FunctionExpression query expression ↵Gravatar Nathan Harmata2016-09-09
| | | | | | | nodes. -- MOS_MIGRATED_REVID=132558031
* A bunch of small changes to prepare SkyQueryEnvironment for full-parallel ↵Gravatar Nathan Harmata2016-09-08
| | | | | | | | | | | | | | evaluation: -Rename QueryExpression#evalConcurrently to QueryExpression#parEval. (parallelism is not concurrency! See https://existentialtype.wordpress.com/2011/03/17/parallelism-is-not-concurrency/) -Have SkyQueryEnvironment#eval (used recursively in #evaluateQuery) dynamically call QueryExpression#parEval when appropriate. -Delete QueryExpression#canEvalConcurrently. -Add ThreadSafety annotations in a bunch of relevant places in the query codebase. -A bunch of testing infrastructure to test parallel query evaluation. -TODOs for implementing parallel evaluation of all QueryExpression nodes. -- MOS_MIGRATED_REVID=132436340
* Allow Skyframe graph lookups and value retrievals to throw InterruptedException.Gravatar Janak Ramakrishnan2016-08-16
| | | | | | | The only place we now don't handle InterruptedException is in the action graph created after analysis, since I'm not sure that will be around for that much longer. -- MOS_MIGRATED_REVID=130327770
* Stream the right-hand side of subtractive query expressions. This means only ↵Gravatar Eric Fellheimer2016-08-12
| | | | | | | | | the left-hand side needs to be pinned fully into memory. Intersection is not associative, so we can't do the same thing there. For example, "abc" ^ "abc" = "abc", but "abc" ^ "a" ^ "b" ^ "c" = <null>. -- MOS_MIGRATED_REVID=130015098
* Fix embarassing typo in VariableContext#with.Gravatar Nathan Harmata2016-07-15
| | | | | | | | | Turns out that our existing test coverage was insufficient for catching this (I misread the tests during development of unknown commit and thus incorrectly thought we had test coverage for this). -- MOS_MIGRATED_REVID=127461716
* Re-implement variables in the blaze query language. Instead of using a ↵Gravatar Nathan Harmata2016-07-14
| | | | | | | | | mutable global context of variable bindings, pass around immutable local contexts. The motivation is so we can safely evaluate all blaze query expressions concurrently under the hood. A global context is hostile to this goal. -- MOS_MIGRATED_REVID=127324600
* No longer allow QueryExpressionMapper#map to throw a QueryException. This ↵Gravatar Nathan Harmata2016-07-04
| | | | | | | was unnecessary. -- MOS_MIGRATED_REVID=126457804
* (1) Allow QueryExpressionMapper#map to throw a QueryException.Gravatar Nathan Harmata2016-07-01
| | | | | | | | (2) Refactor the simple query concurrency support by introducing QueryExpression#canEvalConcurrently and removing the primitive BinaryOperatorExpression#evalConcurrently and the hardcoded SkyQueryEnvironment logic around it. This way each QueryExpression can safely manage its own concurrency. A followup CL will ensure that concurrent evaluation occurs for as much of the query expression as possible, rather than just for the top-level query expression node. (3) Make a few query internals public. -- MOS_MIGRATED_REVID=126324637
* Add SkyQuery-specific allrdeps implementation to allow batch getReverseDepsGravatar Googler2016-06-27
| | | | | | | with maximum limit -- MOS_MIGRATED_REVID=125959807
* Parallelize (some of) BinaryOperatorExpressionGravatar Mark Schaller2016-06-22
| | | | | | | | | | | | | | | | | | | | | | | | Adds evalConcurrently to QueryExpression so that expression implementations that support concurrent evaluation can do so using the supplied executor service. Implements concurrent evaluation for the PLUS/UNION cases of BinaryOperatorExpression. Because evalConcurrently requires its callback to be threadsafe, but the callback passed to evaluateQuery may only be called by one thread at a time, this change makes the BatchStreamedCallback constructed by SkyQueryEnvironment threadsafe, including its uniquifier. However, there is a thread-safety problem when the operands of BinaryOperatorExpression are LetExpressions, because their evaluation involves mutating state in the query environment. A future change will fix that. For now, concurrent evaluation is only attempted when the query expression is a BinaryOperatorExpression and all its operands are target literals. -- MOS_MIGRATED_REVID=125505247
* Make RegexFilterExpression publicGravatar Chris Parsons2016-05-11
| | | | | -- MOS_MIGRATED_REVID=121982887
* In SkyQueryEnvironment, rewrite queries using the semantics-preserving ↵Gravatar Nathan Harmata2016-03-02
| | | | | | | | | transformation 'rdeps(<sky_query_environment_universe_scope>, T, depth)' -> 'allrdeps(T, depth)'. SkyQueryEnvironment can evaluate such allrdeps queries much more efficiently since it doesn't need to bother filtering out targets outside of universe, meaning it doesn't need to have all targets in the universe in memory at the same time. -- MOS_MIGRATED_REVID=116075008
* Typo fixGravatar Shreya Bhattarai2016-02-03
| | | | | -- MOS_MIGRATED_REVID=113681840
* Stream result of TargetPattern#eval to a callback instead of returning it ↵Gravatar Janak Ramakrishnan2016-01-11
| | | | | | | | | directly, and pass a Query callback in when resolving target patterns. This means that the targets a pattern resolves to can be processed incrementally. This is the fifth step in a series to allow processing large sets of targets in query target patterns via streaming batches rather than all at once. This should improve performance for SkyQueryEnvironment for certain classes of large queries. -- MOS_MIGRATED_REVID=111696713
* Make QueryEnvironment#getTargetsMatchingPattern pass its result to a callback.Gravatar Janak Ramakrishnan2016-01-07
| | | | | | | This is the third step in a series to allow processing large sets of targets in query target patterns via streaming batches rather than all at once. This should also be a functional no-op. -- MOS_MIGRATED_REVID=111620668
* Stream results of targets below directory to a callback rather than ↵Gravatar Janak Ramakrishnan2016-01-07
| | | | | | | | | returning a ResolvedTargets set. This is the first step in a series to allow processing large sets of targets in query target patterns via streaming batches rather than all at once. This should be a functional no-op. -- MOS_MIGRATED_REVID=111609309
* RELNOTES: add loadfiles() query operator, to find skylark files loaded by ↵Gravatar Han-Wen Nienhuys2015-12-16
| | | | | | | targets. -- MOS_MIGRATED_REVID=110275427
* Use Bazel Preconditions variant which avoids varargs array creationGravatar Mark Schaller2015-12-10
| | | | | | | Reduces garbage. -- MOS_MIGRATED_REVID=109914243
* Allow output formatters to work in stream mode.Gravatar Miguel Alcon Pinto2015-12-10
| | | | | -- MOS_MIGRATED_REVID=109908202
* Rollback of commit 8e7da8704dba78dba47fbaa0eafce3a91c2deb13.Gravatar Peter Schmitt2015-12-01
| | | | | | | *** Reason for rollback *** -- MOS_MIGRATED_REVID=109005097
* Allow output formatters to work in stream mode.Gravatar Miguel Alcon Pinto2015-11-19
| | | | | -- MOS_MIGRATED_REVID=108142169
* Transform Blaze query to be able to work in streamed mode.Gravatar Miguel Alcon Pinto2015-11-06
| | | | | -- MOS_MIGRATED_REVID=107249788
* Rationalize copyright headersGravatar Damien Martin-Guillerez2015-09-25
| | | | | | | | | | | The headers were modified with `find . -type f -exec 'sed' '-Ei' 's|Copyright 201([45]) Google|Copyright 201\1 The Bazel Authors|' '{}' ';'` And manual edit for not Google owned copyright. Because of the nature of ijar, I did not modified the header of file owned by Alan Donovan. The list of authors were extracted from the git log. It is missing older Google contributors that can be added on-demand. -- MOS_MIGRATED_REVID=103938715
* Fix some warnings.Gravatar Ulf Adams2015-09-04
| | | | | -- MOS_MIGRATED_REVID=102332437
* Code cleanupGravatar Laurent Le Brun2015-09-03
| | | | | -- MOS_MIGRATED_REVID=102239051
* Add rbuildfiles function to queries, only for use when using ↵Gravatar Janak Ramakrishnan2015-08-21
| | | | | | | | | SkyQueryEnvironment. rbuildfiles returns the packages (in the form of BUILD files) that depend on the given source files as BUILD files or subincludes. -- MOS_MIGRATED_REVID=101157700
* Throw InterruptedException directly if a query is interrupted, instead of ↵Gravatar Janak Ramakrishnan2015-07-10
| | | | | | | | | wrapping it in a QueryException. QueryException should usually indicate a persistent failure, while an InterruptedException is transient. Wrapping the InterruptedException in a QueryException just obfuscates state. -- MOS_MIGRATED_REVID=97815388
* Add "allrdeps" function that computes rdeps in whatever universe happens to ↵Gravatar Janak Ramakrishnan2015-06-26
| | | | | | | | | | | be loaded. This avoids having to materialize the full universe scope in memory when doing an rdeps query over "//...". Another option considered was special-casing the string "//..." if it was the first argument to rdeps, but this seemed cleaner. -- MOS_MIGRATED_REVID=96878999
* Clean up AllPathsFunction and get rid of getFwdDeps(Target) and ↵Gravatar Janak Ramakrishnan2015-06-19
| | | | | | | getReverseDeps(Target) from the QueryEnvironment interface, since they're no longer needed by any query functions, and also WalkableGraph#get{Direct,Reverse}Deps(SkyKey). -- MOS_MIGRATED_REVID=96361760
* Create batch versions of query environment methods getFwdDeps and ↵Gravatar Janak Ramakrishnan2015-06-17
| | | | | | | | | getReverseDeps, and migrate DepsFunction and RdepsFunction to use them. This makes our (unordered) results potentially dependent on the iteration order of hash sets, but what do we care? -- MOS_MIGRATED_REVID=96117626
* Add a visibility predicate to blaze query.Gravatar Liam Miller-Cushon2015-04-10
| | | | | | | | | | The visible(x, y) query expression computes the subset of nodes in y visible from all nodes in x. RELNOTES: Add a visibility predicate to blaze query. -- MOS_MIGRATED_REVID=90765071
* Add toString method to QueryEvalResult.Gravatar Janak Ramakrishnan2015-02-20
| | | | | -- MOS_MIGRATED_REVID=86826447
* Update from Google.Gravatar Han-Wen Nienhuys2015-02-25
-- MOE_MIGRATED_REVID=85702957
generated by cgit v1.2.3 (git 2.39.1) at 2024-06-30 23:50:18 +0000