aboutsummaryrefslogtreecommitdiffhomepage
path: root/src/main/java/com/google/devtools/build/lib/query2
Commit message (Collapse)AuthorAge
* Refactor SkyQueryEnvironment to allow preprocessing or postprocessing targetsGravatar Googler2017-06-27
| | | | | | | | Refactor SkyQueryEnvironment and a few other query helpers to make it easier to work with targets. RELNOTES: None PiperOrigin-RevId: 160165398
* Use targets' labels' package identifier for query output=packageGravatar mschaller2017-06-22
| | | | | | | | | | Subinclude targets' packages are the package containing the load statement. Subinclude targets' labels are the bzl files they refer to. The latter has the appropriate package to display for --output=package. Fixes #3122. PiperOrigin-RevId: 159750840
* In the Blaze Query implementation, use Set and Map implementations backed by ↵Gravatar nharmata2017-06-22
| | | | | | | the same KeyExtractor used that the Uniquifier implementation uses. This fixes a hypothetical issue where we were previously relying on Target#equals/hashCode. RELNOTES: None PiperOrigin-RevId: 159741545
* Automated rollback of commit 7dec00574aa91327693f6ba7e90bff5bc834253e.Gravatar ajmichael2017-06-22
| | | | | | | | | *** Reason for rollback *** Tickled a hidden bug in Blaze query. RELNOTES: None PiperOrigin-RevId: 159718294
* Ban the combination of buildfiles/loadfiles and --output=location.Gravatar nharmata2017-06-19
| | | | | | RELNOTES[INC]: The --output=location flag to 'bazel query' cannot be used with query expressions that involve the 'buildfiles' or 'loadfiles' operators. This also applies to 'genquery' rules. PiperOrigin-RevId: 159259061
* Don't call function names keyword in the query reference because they are not.Gravatar lberki2017-06-13
| | | | | RELNOTES: None. PiperOrigin-RevId: 158721043
* Remove ErrorSensingEventHandler#resetErrors().Gravatar lberki2017-06-12
| | | | | | | The call sites in QueryEnvironment implementations were not needed because QueryEnvironments are always made afresh. RELNOTES: None. PiperOrigin-RevId: 158698881
* Make PackageOutputFormatter use PackageIdentifier instead of package name.Gravatar ajmichael2017-06-08
| | | | | | | Fixes #3122. RELNOTES: bazel query --output package now displays packages from external repository with the format "@reponame//package". Packages in the main repository continue to have the format "package". PiperOrigin-RevId: 158327492
* Fix a bug in ParallelVisitor which prevents visitation task from being ↵Gravatar Googler2017-06-02
| | | | | | | | | | | | | | interrupted eagerly The original code swallows the InterruptedException, sets the interrupt bit and stops new tasks from being submitted. However it does not actively send an interrupt signal to all running and pending tasks already in the pool. It is partly due to the misleading syntax of awaitTermination, which actually quitely waits for all tasks to exit even if interruptWorkers is set to true. Both errors are fixed in this change. RELNOTES: None PiperOrigin-RevId: 157762029
* Don't throw runtime exception on interruption - since we know the future is ↵Gravatar Googler2017-05-26
| | | | | | already done, we should leave the thread in an interrupted state and proceed. This fixes a blaze crash when a 'genquery' execution is interrupted at the right time. PiperOrigin-RevId: 157000269
* Expose the target map from Package.Gravatar carmi2017-05-19
| | | | | | | | This doesn't really expose new information, because one can iterate over the existing getTargets() and recreate the map. Moreover, the map is immutable, so there's no risk in returning it to the user. RELNOTES: None PiperOrigin-RevId: 156349797
* Have TransitiveTraversalValues store kind of targets which have errors when ↵Gravatar Googler2017-05-16
| | | | | | | computing TransitiveTraversalValues. RELNOTES: None PiperOrigin-RevId: 156138657
* Clean up AbstractQueueVisitor's constructors.Gravatar janakr2017-05-09
| | | | | | | | The "concurrent" bit was supposedly around for testing purposes, but who knows if it even works anymore. Making other callsites explicitly state their ErrorClassifier gets us down to two constructors, one of which can delegate to the other. I think having both these constructors is useful because there's a linkage between creating a new executor service and specifying that the AQV should shut down the service at the end of the visitation. And using a static create() method doesn't work because of AQV's inheritance model. PiperOrigin-RevId: 155406771
* Have QueryExpressionMapper#compose take arbitrary number of mappersGravatar Googler2017-05-04
| | | | | | | AttrFunction and LabelsFunction are also made public. RELNOTES: None PiperOrigin-RevId: 155108260
* Make SkyKey an interface, and start the migration of not creating SkyKey ↵Gravatar janakr2017-05-04
| | | | | | wrapper objects: for OwnedArtifacts, which are the most numerous during builds, and for Labels for TransitiveTraversalValues, which are the most numerous during queries. PiperOrigin-RevId: 154989520
* Add static methods to return target kinds for built-in targetsGravatar Googler2017-05-03
| | | | | | | | Add static methods to return target kinds of InputFile and OutFile, as well as suffix of target kind of Rule. RELNOTES: None PiperOrigin-RevId: 154866926
* Reword commentGravatar laurentlb2017-04-18
| | | | | RELNOTES: None. PiperOrigin-RevId: 153449924
* Split out option usage restriction from option category.Gravatar ccalvarin2017-04-18
| | | | | | | | | These are two different concepts. Do not remove category overload compatibility in this CL, to keep this change limited to converting the current uses of category. With some flyby formatting fixes on affected OptionsBases. RELNOTES: None. PiperOrigin-RevId: 153390002
* Update comments and remove obsolete field.Gravatar laurentlb2017-04-13
| | | | | RELNOTES: None. PiperOrigin-RevId: 153058374
* Add select() support for "query --output=build".Gravatar gregce2017-04-13
| | | | | | | | | | | | | | | | | | | | | | | | | Before this change, attributes with select() were completely skipped. This doesn't attempt to merge "trivial" selects, e.g. even though: attr = [":foo"] + select({"//conditions:default": [":bar"]}) always resolves to: attr = [":foo", ":bar"] this change still produces: attr = [":foo"] + [":bar"] We could merge these in a future change, if desired. But it's not even clear that's desired. There's conceptual value in keeping the lists separate since that's how they were originally written. That gives users a cue to maybe refactor their rules. RELNOTES[NEW]: "query --output=build" now includes select()s PiperOrigin-RevId: 152956939
* Refactor all ctor callsites of PathFragment to instead call a static ↵Gravatar nharmata2017-04-05
| | | | | | | | | | | | 'create' method. This paves the way for changing PathFragment to e.g. an abstract class with multiple subclasses. This way we can split out the windows-specific stuff into one of these concrete classes, making the code more readable and also saving memory (since the shallow heap size of the NonWindowsPathFragment subclass will hopefully be smaller than that of the current PathFragment). This also lets us pursue gc churn optimizations. We can now do interning in PathFragment#create and can also get rid of unnecessary intermediate PathFragment allocations. RELNOTES: None PiperOrigin-RevId: 152145768
* Delete STRING_DICT_UNARYGravatar michajlo2017-03-31
| | | | | | | This isn't used anymore, it's the same as STRING_DICT, deleting so no one tries to use it. PiperOrigin-RevId: 151738915
* Global cleanup change.Gravatar Googler2017-03-23
| | | | | | -- PiperOrigin-RevId: 151019690 MOS_MIGRATED_REVID=151019690
* Fix inadvertent performance regression introduced by the recent rewrite of ↵Gravatar Nathan Harmata2017-03-14
| | | | | | | | | | | | | | | | | | 'blaze query'. The "streaming" callbacks used by some query functions, e.g. 'deps', make calls to QueryEnvironment#buildTransitiveClosure. For a cold blaze server, these calls do package loading via LabelVisitor (which calls into Skyframe via a top-level #evaluate call). So we'd prefer a single massive call which can make full use of blaze's loading-phase parallelism via Skyframe over a bunch of sequential small calls. For a hot blaze server, there are two problems: (1) LabelVisitor's meager up-to-date check isn't useful (as in we cannot reuse old visitations) when we do a whole bunch of small visitations instead of one massive one. (2) The actual work of the LabelVisitor (building up a portion of a temporary graph) isn't being effectively parallelized when we do it sequentially in small chunks. This issue is yet another subtle reason why the old BlazeQueryEnvironment#eval made sense (and why it was unfortunately not compatible with the streaming query evaluation model from the beginning). -- PiperOrigin-RevId: 150081619 MOS_MIGRATED_REVID=150081619
* Configurably block errors from universe eval in query environmentsGravatar Mark Schaller2017-03-12
| | | | | | | | | | | | Universe evaluation only occurs in the SkyQueryEnvironment implementation. This setting is a no-op for other QueryEnvironment implementations. This support is needed to correctly test an upcoming bugfix. -- PiperOrigin-RevId: 149786616 MOS_MIGRATED_REVID=149786616
* Description redacted.Gravatar Nathan Harmata2017-03-09
| | | | | | -- PiperOrigin-RevId: 149585165 MOS_MIGRATED_REVID=149585165
* Fix bug with streaming bounded deps/allrdeps/rdeps.Gravatar Nathan Harmata2017-03-08
| | | | | | -- PiperOrigin-RevId: 149431500 MOS_MIGRATED_REVID=149431500
* Global cleanup change.Gravatar Googler2017-03-03
| | | | | | -- PiperOrigin-RevId: 149089903 MOS_MIGRATED_REVID=149089903
* Do not use additional scheduling threads during parallel evaluation to ↵Gravatar Googler2017-03-02
| | | | | | | | | | | | prevent thread starvation This change gets rid of the additional thread needed for task scheduling during BFS visitation, which eliminates the possibility of thread starvation while a single thread pool is used for multiple concurrent evaluations. -- PiperOrigin-RevId: 148911346 MOS_MIGRATED_REVID=148911346
* Rollback of commit 822c37816ac669e51bec3853b41849a19ec5e230.Gravatar Nathan Harmata2017-03-01
| | | | | | -- PiperOrigin-RevId: 148844518 MOS_MIGRATED_REVID=148844518
* Reimplement blaze query using an async evaluation model. Use a concurrent ↵Gravatar Nathan Harmata2017-02-28
| | | | | | | | | | | | | | | | | | | backend for SkyQueryEnvironment's implementation in order to achieve parallelism. Advantages: -New design has no flaws that the old design had. -Code is structured so that deadlocks due to thread starvation are impossible (yup!). Disadvantages: -The meat of this change needs to all be in a single CL because every single QueryFunction and QueryExpression needs to be rewritten in the async style. Still TODO: -Fully embrace the async model in all QueryFunctions (e.g. 'rdeps', 'allpaths'). -Use concurrency in BlazeQueryEnvironment to achieve parallel evaluation for (non SkyQuery) 'blaze query' and genquery. -- PiperOrigin-RevId: 148690279 MOS_MIGRATED_REVID=148690279
* Handle TriState type in `bazel query --output=build`Gravatar Laurent Le Brun2017-02-27
| | | | | | -- PiperOrigin-RevId: 148643993 MOS_MIGRATED_REVID=148643993
* Reword comment about the current implementation of ^.Gravatar Nathan Harmata2017-02-27
| | | | | | -- PiperOrigin-RevId: 148488719 MOS_MIGRATED_REVID=148488719
* Provide more reporting options to SkyFunctions Gravatar Klaus Aehlig2017-02-27
| | | | | | | | | | | | | With more specific information to be reported by Skyfunctions, e.g., to inform the build-event protocol on missing files, the EventHandler interface is no longer enough. Therefore, provide an enriched context for reporting events. -- Change-Id: I2d06166fe4d5b9054e24ad8c752fafc039e3f9f8 Reviewed-on: https://cr.bazel.build/8794 PiperOrigin-RevId: 148463437 MOS_MIGRATED_REVID=148463437
* Add the LABEL_KEYED_STRING_DICT type for attributes.Gravatar Michael Staib2017-02-24
| | | | | | | | | This enables both native and Skylark rules to declare attributes which have labels/Targets as keys, and have string values. -- PiperOrigin-RevId: 148365033 MOS_MIGRATED_REVID=148365033
* Fix a hypothetical issue with 'buildfiles' and 'loadfiles' with duplicate ↵Gravatar Nathan Harmata2017-02-16
| | | | | | | | | | | | | | | | | | targets. These two functions don't use a Uniquifier internally so if the same bzl file foo.bzl is loaded multiple ways in 'e' in 'loadfiles(e)' then 'f' in 'f(loadfiles(e))' will observe foo.bzl multiple times. This isn't observable in practice (i.e. I cannot write a black-box query test that would fail without this CL) because: (1) BlazeQueryEnvironment#eval(QueryExpression, VariableContext<Target>, Callback<Target>) uses an intermediate aggregating callback and thus doesn't use the streaming query evaluation model. This means that the internal deduping in BlazeQueryEnvironment#getBuildFiles is sufficient. (2) SkyQueryEnvironment uses an outer BatchStreamedCallback which internally uses a Uniquifier. Still, this CL is useful for SkyQuery because without we're doing wasted work (e.g. in my example above, 'f' will wastefully do duplicate work with the .bzl files from the inner loadfiles(e)) because the deduping happens on the final query result, not on the intermediate result from 'buildfiles'/'loadfiles'. For 'buildfiles', there's an additional wart with (1) above because FakeSubincludeTarget doesn't override Object#equals/hashCode. -- PiperOrigin-RevId: 147656183 MOS_MIGRATED_REVID=147656183
* Give a more helpful error message on malformed regexes.Gravatar Nathan Harmata2017-02-16
| | | | | | -- PiperOrigin-RevId: 147621296 MOS_MIGRATED_REVID=147621296
* Remove type checking requirement from AttributeMap.has.Gravatar Greg Estren2017-02-15
| | | | | | | | | | | | | | This overrides the traditional has(String name, Type<>T> type) with has(String name) and removes the type check outright from isConfigurable. Ideally we'd remove the old version in this same change. But there are enough uses of it that that's not a risk-free change and is safer as followup changes. -- PiperOrigin-RevId: 147513593 MOS_MIGRATED_REVID=147513593
* Remove special handling of name attribute. Fixes #278Gravatar Googler2017-02-14
| | | | | | | | | | | | | | | | | | | | | | | | | The name attribute gets special treatment in the codebase, in that it's not simply yet another attribute but stored in it's own field. Thus, every callside dealing with attributes needs to be aware of this special case and explicitly handle the name attribute. It's easy to see that this can lead to bugs. For example, querying for the name attribute is currently broken due the querying code not being aware of the special case [1]. Discussions with experienced bazel developers came to the conclusion that there is no need (anymore) to treat the name attribute specially and thus we decided it's best to remove the special treatment and handle the name attribute as any other attribute. This change removes the handling of name attributes and also adds a test case to verify that bug [1] is fixed. [1] https://github.com/bazelbuild/bazel/issues/278 -- PiperOrigin-RevId: 147446345 MOS_MIGRATED_REVID=147446345
* Replace PackageProvider with TargetProvider where possibleGravatar Ulf Adams2017-02-09
| | | | | | | | | | I was trying to simplify GenQuery, and found this during an audit of the code; it's not sufficient to make any larger changes, but it may help eventuall. -- PiperOrigin-RevId: 147023626 MOS_MIGRATED_REVID=147023626
* Remove dead code in SkyframeLabelVisitorGravatar Ulf Adams2017-02-07
| | | | | | | | | | Two of the functions are never used, several of the parameter are not used or can be simplified (require callers to convert from Target -> Label). -- PiperOrigin-RevId: 146787380 MOS_MIGRATED_REVID=146787380
* Global cleanup change.Gravatar Googler2017-01-25
| | | | | | -- PiperOrigin-RevId: 145473478 MOS_MIGRATED_REVID=145473478
* Fix an issue with cached EventHandlers in SkyQueryEnvironment's resolverGravatar Mark Schaller2017-01-23
| | | | | | | | | | | | Because the event handler's inner handlers are removed after each query command, caching the handler caused a subset of subsequent commands' errors (those reported through the resolver's handler) to go unlogged. Also fix a bug with cycle detection in DelegatingWalkableGraph. -- PiperOrigin-RevId: 145124271 MOS_MIGRATED_REVID=145124271
* Remove WalkableGraph#exists and allow WalkableGraph#getValue and ↵Gravatar Janak Ramakrishnan2017-01-03
| | | | | | | | WalkableGraph#getException to be given non-existent keys without crashing. Add WalkableGraph#isCycle to fill the gap in testing for the difference between non-existence and depending on a cycle. -- PiperOrigin-RevId: 143205289 MOS_MIGRATED_REVID=143205289
* Remove dead code referencing deleted "suites" attribute.Gravatar Greg Estren2016-12-21
| | | | | | | | | This used to be how test_suites depended on other test_suites. Now they just go in "tests". -- PiperOrigin-RevId: 142607603 MOS_MIGRATED_REVID=142607603
* Rollback of commit e0a330577d9fe98169645cb68d9fc22cc787eeb6.Gravatar Janak Ramakrishnan2016-12-21
| | | | | | | | | | *** Reason for rollback *** Investigating if causes deadlock/thread starvation. -- PiperOrigin-RevId: 142575769 MOS_MIGRATED_REVID=142575769
* Add some logging when query evaluation halts abruptly.Gravatar Nathan Harmata2016-12-16
| | | | | | -- PiperOrigin-RevId: 142295816 MOS_MIGRATED_REVID=142295816
* Some improvements to ParallelQueryUtils.Gravatar Nathan Harmata2016-12-16
| | | | | | | | | | | | | | | | | | | | | | | | (i) Use a CountDownLatch in ParallelQueryUtils#executeQueryTasksAndWaitInterruptibly to avoid busy-looping while waiting for query subtask completion (this busy-looping unnecessarily ties up a thread). But we still retain the fail-fast semantics we want (I renamed the method to emphasize this). (ii) Also have a special-case in ParallelQueryUtils#executeQueryTasksAndWaitInterruptibly for evaluating one query subtask so we don't wastefully use another thread. (iii) Also add ThreadSafety annotations to ParallelQueryUtils. ---- (i) and (ii) combine to address the following theoretical issue. Suppose we're evaluating a query expression of the form "(e1 - e2) + (e3 - e4)". The old code would (with the worst-case FJP thread scheduling) have the following threads at the _same_ time: Main QueryCommand thread - executeQueryTasksAndWaitInterruptibly(queryTasks = [(e1 - e2), (e3 - e4)] FJP thread - executeQueryTasksAndWaitInterruptibly(queryTasks = [e2]) FJP thread - eval(e2) FJP thread - executeQueryTasksAndWaitInterruptibly(queryTasks = [e4]) FJP thread - eval(e4) So of those 5 concurrent threads, 3 would be doing busy-loop waiting. For more pathological query expressions, we could end up tying up lots of threads doing wasteful busy-loops. -- PiperOrigin-RevId: 142215680 MOS_MIGRATED_REVID=142215680
* Make Argument factory methods public.Gravatar Nathan Harmata2016-12-13
| | | | | | -- PiperOrigin-RevId: 141920696 MOS_MIGRATED_REVID=141920696
* Rephrase query transformation in terms of composition of QueryExpressionMappers.Gravatar Nathan Harmata2016-12-13
| | | | | | -- PiperOrigin-RevId: 141904124 MOS_MIGRATED_REVID=141904124
generated by cgit v1.2.3 (git 2.39.1) at 2024-07-03 22:45:03 +0000