| Commit message (Collapse) | Author | Age |
|
|
|
|
|
|
|
|
|
| |
*** Reason for rollback ***
Investigating if causes deadlock/thread starvation.
--
PiperOrigin-RevId: 142575769
MOS_MIGRATED_REVID=142575769
|
|
|
|
|
|
|
|
| |
ensure that all of e1, e2, ..., and eK are elligble for parallel evaluation. This is _not_ the same as providing a parallel implementation of f, which we can do separately in followup CLs.
--
PiperOrigin-RevId: 140861694
MOS_MIGRATED_REVID=140861694
|
|
|
|
|
|
|
| |
by introducing TargetPattern#parEval, which allows TargetPatterns' evaluations to explicitly have parallel implementations (no need to secretly use a FJP).
--
MOS_MIGRATED_REVID=139101922
|
|
|
|
|
|
|
| |
overrides. Also, have AbstractBlazeQueryEnvironment#evaluateQuery take an OutputFormatterCallback instance rather than a Callback instance. This is more sensible since the latter is only intended to be used intra-query, while the former is intended for usage in end-to-end query evaluation. This lets us slightly simplify QueryCommand, by shifting the responsibility for managing the OutputFormatterCallback to AbstractBlazeQueryEnvironment#evaluateQuery.
--
MOS_MIGRATED_REVID=134827588
|
|
|
|
|
|
|
|
|
|
|
| |
Instead have SkyQueryEnvironment#evaluateQuery be responsible for handling cleanup of its internal ForkJoinPool.
In addition to being a more sensible way of organizing the code (imo, it makes sense for SkyQueryEnvironment to clean up after itself), this fixes several issues:
(i) If query evaluation is interrupted, the AbstractBlazeQueryEnvironment#close call at the end of the try-with-resources statement in QueryCommand would be blocking and non-urgent. N.B. This was not an issue with the current ForkJoinPool usage, but was an issue with the old ThreadPoolExecutor usage.
(ii) Because of how the code in QueryCommand was structured, OutputFormatterCallback#close would happen _before_ the AbstractBlazeQueryEnvironment#close call. If query evaluation is interrupted, threads executing query tasks may be invoking the callback after the callback had been shut down!
--
MOS_MIGRATED_REVID=134573395
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
evaluation:
-Rename QueryExpression#evalConcurrently to QueryExpression#parEval. (parallelism is not concurrency! See https://existentialtype.wordpress.com/2011/03/17/parallelism-is-not-concurrency/)
-Have SkyQueryEnvironment#eval (used recursively in #evaluateQuery) dynamically call QueryExpression#parEval when appropriate.
-Delete QueryExpression#canEvalConcurrently.
-Add ThreadSafety annotations in a bunch of relevant places in the query codebase.
-A bunch of testing infrastructure to test parallel query evaluation.
-TODOs for implementing parallel evaluation of all QueryExpression nodes.
--
MOS_MIGRATED_REVID=132436340
|
|
|
|
|
|
|
|
|
| |
QueryExceptions now that all relevant methods declare that they throw InterruptedException.
Small side benefit of commit 3c0adb26bac6d756fb97e4bcc6d4e5b2cefa5eeb.
--
MOS_MIGRATED_REVID=130402917
|
|
|
|
|
|
|
| |
The only place we now don't handle InterruptedException is in the action graph created after analysis, since I'm not sure that will be around for that much longer.
--
MOS_MIGRATED_REVID=130327770
|
|
|
|
|
|
|
|
|
| |
mutable global context of variable bindings, pass around immutable local contexts.
The motivation is so we can safely evaluate all blaze query expressions concurrently under the hood. A global context is hostile to this goal.
--
MOS_MIGRATED_REVID=127324600
|
|
|
|
|
|
|
|
| |
Shutdown the SkyQueryEnvironment's threadpool after query evaluation
is complete and the environment is ready for disposal.
--
MOS_MIGRATED_REVID=125975317
|
|
|
|
|
|
|
|
|
| |
label sets that are precomputed in the graph.
This is the fourth step in a series to allow processing large sets of targets in query target patterns via streaming batches rather than all at once. This may make SkyQueryEnvironment slower when evaluating queries with repeated target patterns, or many target patterns that would benefit from graph lookups that were batched across all patterns. But that is not currently a bottleneck we're concerned about.
--
MOS_MIGRATED_REVID=111626483
|
|
|
|
|
|
|
| |
This is the third step in a series to allow processing large sets of targets in query target patterns via streaming batches rather than all at once. This should also be a functional no-op.
--
MOS_MIGRATED_REVID=111620668
|
|
|
|
|
|
|
| |
targets.
--
MOS_MIGRATED_REVID=110275427
|
|
|
|
|
|
|
| |
Reduces garbage.
--
MOS_MIGRATED_REVID=109914243
|
|
|
|
|
| |
--
MOS_MIGRATED_REVID=109908202
|
|
|
|
|
|
|
| |
*** Reason for rollback ***
--
MOS_MIGRATED_REVID=109005097
|
|
|
|
|
| |
--
MOS_MIGRATED_REVID=108142169
|
|
|
|
|
| |
--
MOS_MIGRATED_REVID=107249788
|
|
|
|
|
|
|
|
|
|
|
| |
The headers were modified with
`find . -type f -exec 'sed' '-Ei' 's|Copyright 201([45]) Google|Copyright 201\1 The Bazel Authors|' '{}' ';'`
And manual edit for not Google owned copyright. Because of the nature of ijar, I did not modified the header of file owned by Alan Donovan.
The list of authors were extracted from the git log. It is missing older Google contributors that can be added on-demand.
--
MOS_MIGRATED_REVID=103938715
|
|
|
|
|
|
|
|
|
|
| |
- Label parsing can be simplified
- lib.syntax is only contains the code for Skylark and is reasonably independent from the problem domain of building things
This change is mostly only changes to imports declarations. The rest is reversing the dependency between :cmdline and :syntax and moving a tiny amount of code between Printer and FilesetEntry and the addition of SkylarkPrintableValue that I couldn't be bothered to separate out into its own change.
--
MOS_MIGRATED_REVID=103527877
|
|
|
|
|
|
|
|
|
| |
syntax that means "refer to the main repository".
There isn't an overarching plan for what we are going to do with the cmdline package, which seems to be separated from the .syntax one in all sorts of awkward ways.
--
MOS_MIGRATED_REVID=103088960
|
|
|
|
|
| |
--
MOS_MIGRATED_REVID=102332437
|
|
|
|
|
|
|
|
|
| |
target patterns in query.
We never use anything but the getTargets() field, so it's useless to store other things.
--
MOS_MIGRATED_REVID=100846573
|
|
|
|
|
|
|
|
|
| |
wrapping it in a QueryException.
QueryException should usually indicate a persistent failure, while an InterruptedException is transient. Wrapping the InterruptedException in a QueryException just obfuscates state.
--
MOS_MIGRATED_REVID=97815388
|
|
|
|
|
|
|
| |
getReverseDeps(Target) from the QueryEnvironment interface, since they're no longer needed by any query functions, and also WalkableGraph#get{Direct,Reverse}Deps(SkyKey).
--
MOS_MIGRATED_REVID=96361760
|
|
|
|
|
|
|
|
|
| |
getReverseDeps, and migrate DepsFunction and RdepsFunction to use them.
This makes our (unordered) results potentially dependent on the iteration order of hash sets, but what do we care?
--
MOS_MIGRATED_REVID=96117626
|
|
|
|
|
|
|
|
|
|
|
|
| |
While evaluating the transitive closure of the universe, if the
universe contains the buildfiles of a package with a subinclude, the
RdepsFunction would ask a FakeSubincludeTarget for its package.
Prior to this change, this would result in a crash. Now,
FakeSubincludeTargets remember the package they belong to.
--
MOS_MIGRATED_REVID=95212925
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This change means that tests no longer need to specify the patterns
they use themselves, unless the test involves excluding some pattern.
Also fixes a bug in GraphBackedRecursivePackageProvider revealed by
the test changes where it behaved differently from
EnvironmentBackedRecursivePackageProvider when looking up packages
with errors.
--
MOS_MIGRATED_REVID=89297008
|
|
|
|
|
|
|
|
|
| |
This environment eagerly preloads the transitive closure of a specified query "universe", and so may not be as efficient as the standard query for limited-scope queries. It is activated when the universe is specified and ordered results are not requested (since it is currently unable to order results).
Tests were modified/added to exercise this environment where deemed interesting. Some ugly hacks were done to add coverage in AbstractQueryTest and friends, because currently even if the full depot is loaded (using //...), individual target patterns most likely won't be present in the graph. A better way to deal with this situation, suggested by felly, is probably to extract target pattern resolution logic to an auxiliary function so that query is able to resolve target patterns without mutating the graph, and then call into the read-only graph with the resolved patterns. That may be done in a follow-up, in which case the "scope" of every query could be //... .
--
MOS_MIGRATED_REVID=87257028
|
|
|
|
|
|
|
|
|
| |
move some utility methods to utility class.
The real goal of this CL is to make sure all users of BlazeQueryEnvironment are constructing one through EvaluableBlazeQueryEnvironment#newQueryEnvironment, and not directly. This ensures that alternate implementations of EvaluableBlazeQueryEnvironment will be automatically injected via #newQueryEnvironment.
--
MOS_MIGRATED_REVID=86112439
|
|
|
|
|
|
|
|
|
|
|
| |
BlazeQueryEnvironment that exposes an evaluateQuery method, and also implements the non-LabelVisitor-specific parts of BlazeQueryEnvironment, for other implementations' uses.
Most of the code is just a straight refactoring of BlazeQueryEnvironment into EvaluableBlazeQueryEnvironment (and BlazeTargetAccessor). Ignoring whitespace changes in [] may be your friend for seeing that it's a straight move.
This also allows us to change tests to use QueryCommand.newQueryEnvironment, in preparation for newQueryEnvironment potentially returning a different EvaluableBlazeQueryEnvironment subclass depending on the circumstances.
--
MOS_MIGRATED_REVID=86088953
|
|
|
|
|
|
|
| |
BlazeQueryEnvironment#getTarget(Label).
--
MOS_MIGRATED_REVID=85744195
|
|
--
MOE_MIGRATED_REVID=85702957
|