aboutsummaryrefslogtreecommitdiffhomepage
path: root/src/main/java/com/google/devtools/build/lib/packages/AggregatingAttributeMapper.java
Commit message (Collapse)AuthorAge
* Improve efficiency warnings in AggregatingAttributeMapper javadoc.Gravatar gregce2018-02-13
| | | | PiperOrigin-RevId: 185595397
* Move BazelRuleClassProvider.J2OBJC_RULES to its own class.Gravatar lpino2018-02-08
| | | | PiperOrigin-RevId: 184983111
* Add getAttributeLocation to AttributeMap and related classes.Gravatar jcater2017-11-10
| | | | PiperOrigin-RevId: 175230893
* Replace all usages of Blaze's Preconditions class with guava.Gravatar tomlu2017-11-09
| | | | | | | | Blaze had its own class to avoid GC from varargs array creation for the precondition happy path. Guava now (mostly) implements these, making it unnecessary to maintain our own. This change was almost entirely automated by search-and-replace. A few BUILD files needed fixing up since I removed an export of preconditions from lib:util, which was all done by add_deps. There was one incorrect usage of Preconditions that was caught by error prone (which checks Guava's version of Preconditions) that I had to change manually. PiperOrigin-RevId: 175033526
* Refactor ConfiguredAttributeMapper to lib/packages from lib/analysis.Gravatar cparsons2017-10-11
| | | | | | | | | This requires moving the convenience constructor using RuleConfiguredTarget to be owned by RuleConfiguredTarget. This refactoring is required by later work to allow SplitTransitionProvider to use configurable attributes. This would require packages/Attribute.java -> analysis/ConfiguredAttributeMapper.java, where in general, the 'analysis' package depends on the 'packages' package. This is the easiest way to prevent a circular dependency. RELNOTES: None. PiperOrigin-RevId: 171741620
* Do not take all possible values of an attribute for selector lists of list ↵Gravatar shreyax2017-06-28
| | | | | | | | types when running query output. Instead, simply return a list of list types, each list type being the concatenation of each of the possible outputs in each select. RELNOTES: Make querying attributes formed by selector lists of list types more efficient by no longer listing every possible combination of attribute value but by more compactly storing the possible values of the list. PiperOrigin-RevId: 160326041
* Rephrase AbstractAttributeMapper#visitLabels such that we can avoid creating ↵Gravatar nharmata2017-04-05
| | | | | | | | a temporary Type.LabelVisitor instance per Attribute being visited. Instead, we can now create one temporary object per visitation. Getting rid of this dimension of scaling reduces the amount of garbage created. RELNOTES: None PiperOrigin-RevId: 152161836
* Remove type checking requirement from AttributeMap.has.Gravatar Greg Estren2017-02-15
| | | | | | | | | | | | | | This overrides the traditional has(String name, Type<>T> type) with has(String name) and removes the type check outright from isConfigurable. Ideally we'd remove the old version in this same change. But there are enough uses of it that that's not a risk-free change and is safer as followup changes. -- PiperOrigin-RevId: 147513593 MOS_MIGRATED_REVID=147513593
* bazel query: Gravatar Greg Estren2016-10-28
| | | | | | | | | - Extend https://github.com/bazelbuild/bazel/commit/83ffb5e7bc034ea2dd1e957249079b3884a9450d to --output=xml, --output=proto, too. This makes query much more resilient against freezing / OOMing from "select() + select() + select() + ..." patterns. - Make the above logic accessible to any output type. - Move AggregatingAttributeMapper.flattenAttributeValues to ProtoOutputFormatter, since this is really a utility method for ProtoOutputFormatter. -- MOS_MIGRATED_REVID=137429116
* bazel query --output=xml: avoid iterating over all possible values forGravatar Greg Estren2016-10-26
| | | | | | | | attr = select({"a": LABEL_LIST, "b": LABEL_LIST, ...}). This can be exponential (w.r.t. the # of select conditions) and produce OOMs. -- MOS_MIGRATED_REVID=137200979
* Improve memory overhead of calculating all computed default valuesGravatar Michajlo Matijkiw2016-10-07
| | | | | | | | Instead of copying the accumulator map each call, reuse the same map, overwriting previous values and taking snapshots in the base case. -- MOS_MIGRATED_REVID=135394631
* Be honest about type in Type#visitLabelsGravatar Michajlo Matijkiw2016-09-22
| | | | | | | | | | We avoided referencing Label directly, but as the code evolved it became inevitable. Also fix a typo I introduced earlier. -- MOS_MIGRATED_REVID=133831955
* Implement label visitation using visitorGravatar Michajlo Matijkiw2016-09-21
| | | | | | | | | As opposed to building up a collection. These collections, and all their iterators, add up creating a lot of garbage. This saves us at least an ImmutableList + Iterator per label. -- MOS_MIGRATED_REVID=133754998
* Roll forward of commit 19db71413329da3f5d22b5fc7681471f3d971d88 (Skylark: ↵Gravatar Florian Weikert2016-09-06
| | | | | | | | | | | | | Replaced late bound attributes with computed default attributes) with two bug fixes: 1. Unlike SkylarkComputedDefault, SkylarkComputedDefaultTemplate did not sort the names of its attribute dependencies. Consequently, lookup operations failed when callback functions in bzl files specified the names of their required attributes in a non-alphabetical order since the order of the key tuples was different (e.g. [1, 2] vs [2, 1]). It would be less error prone to always sort the dependencies in createDependencyAssignmentTuple(), but this would impact performance. 2. SkylarkCallbackFunction ignores the legacy "cfg" parameter in callback functions. This special case should be deleted once all cfg parameters have been removed from the depot. -- MOS_MIGRATED_REVID=132235927
* Rollback of commit 19db71413329da3f5d22b5fc7681471f3d971d88.Gravatar Tobias Werth2016-09-06
| | | | | -- MOS_MIGRATED_REVID=132058819
* Skylark: Replaced late bound attributes with computed default attributesGravatar Mark Schaller2016-09-02
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Motivation: Compared to computed default attributes, late bound attributes are evaluated in a later phase (analysis instead of loading phase). While both mechanisms provide access to other attributes of a rule, only late bound attributes additionally provide access to the build configuration. However, late bound attributes could be used to return new labels that have not been loaded before. Since this happens in the analysis phase, it can break one of Blaze's underlying principles, thus introducing a serious correctness bug. We decided to replace late bound attributes in Skylark with computed default attributes since this moves the evaluation of values into the loading phase, thus fixing this bug. Moreover, none of the existing users of this mechanism required access to the build configuration, which means that the user impact of this change is minimal. Implementation details: Unlike attributes of non-Skylark rules, however, Skylark computed defaults need to be able to depend on more than two configurable attributes since all attributes of Skylark rules are configurable by default. This has two implications: 1. Unlike "normal" Skylark attributes, Skylark computed defaults need to know about the existence of other attributes in order to declare a dependency on them. This CL takes advantage of work previously done to require parameter names to be the names of attributes used by the computed default function. 2. Since Bazel computes the combinations of all possible attribute values, a Skylark rule with a computed default that depends on a lot of configurable attributes could crash Bazel. Consequently, this CL also introduces an upper bound (64) for the number of valid combinations. Caveats: 1. Getting the depended-on attributes' names from function paramters is mildly surprising. Alternatives: The best solution would be to keep SkylarkLateBound, but restrict it in a way that it can only return already loaded labels. This is not possible right now. -- MOS_MIGRATED_REVID=131967238
* Allow Skyframe graph lookups and value retrievals to throw InterruptedException.Gravatar Janak Ramakrishnan2016-08-16
| | | | | | | The only place we now don't handle InterruptedException is in the action graph created after analysis, since I'm not sure that will be around for that much longer. -- MOS_MIGRATED_REVID=130327770
* Clarify comments and function names in AggregatingAttributeMapperGravatar Mark Schaller2016-08-16
| | | | | | | | A small cleanup and refactoring in advance of heavier work to be done for Skylark computed defaults. -- MOS_MIGRATED_REVID=130140706
* Don't include null values in AggregatingAttributeMapper#visitAttribute. Null ↵Gravatar Nathan Harmata2016-06-16
| | | | | | | values arise when a None is used as the value of a Selector for a type without a default value. -- MOS_MIGRATED_REVID=124961388
* Remove LABEL_LIST_DICT type. This was only used for abi_deps,Gravatar Greg Estren2016-05-11
| | | | | | | which was removed in c231574. -- MOS_MIGRATED_REVID=121974686
* Let select() "unset" values via: select({"//some:condition: None }).Gravatar Greg Estren2016-04-29
| | | | | | | | | | | | | | This not only uses the default value when applicable, but also causes ConfiguredAttributeMapper.isAttributeValueExplicitlySpecified to return false. Note the default value can come from two places: from the rule definition if specified, otherwise from the type default. RELNOTES[NEW]: select({"//some:condition: None }) is now possible (this "unsets" the attribute). -- MOS_MIGRATED_REVID=121029815
* Don't keep packages in the default repository around after loading.Gravatar Brian Silverman2016-03-17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Previously, this would get thrown when referring to the same package from both the main and default repositories: java.lang.IllegalArgumentException: Multiple entries with same key: tools/cpp=/home/brian/971-Robot-Code and tools/cpp=/home/brian/971-Robot-Code at com.google.common.collect.ImmutableMap.checkNoConflict(ImmutableMap.java:136) at com.google.common.collect.RegularImmutableMap.checkNoConflictInKeyBucket(RegularImmutableMap.java:98) at com.google.common.collect.RegularImmutableMap.fromEntryArray(RegularImmutableMap.java:84) at com.google.common.collect.ImmutableMap$Builder.build(ImmutableMap.java:295) at com.google.devtools.build.lib.buildtool.BuildTool.transformPackageRoots(BuildTool.java:301) at com.google.devtools.build.lib.buildtool.BuildTool.buildTargets(BuildTool.java:209) at com.google.devtools.build.lib.buildtool.BuildTool.processRequest(BuildTool.java:334) at com.google.devtools.build.lib.runtime.commands.TestCommand.doTest(TestCommand.java:119) at com.google.devtools.build.lib.runtime.commands.TestCommand.exec(TestCommand.java:104) at com.google.devtools.build.lib.runtime.BlazeCommandDispatcher.exec(BlazeCommandDispatcher.java:371) at com.google.devtools.build.lib.runtime.BlazeRuntime$3.exec(BlazeRuntime.java:1016) at com.google.devtools.build.lib.server.RPCService.executeRequest(RPCService.java:65) at com.google.devtools.build.lib.server.RPCServer.executeRequest(RPCServer.java:434) at com.google.devtools.build.lib.server.RPCServer.serve(RPCServer.java:229) at com.google.devtools.build.lib.runtime.BlazeRuntime.serverMain(BlazeRuntime.java:975) at com.google.devtools.build.lib.runtime.BlazeRuntime.main(BlazeRuntime.java:772) at com.google.devtools.build.lib.bazel.BazelMain.main(BazelMain.java:55) And this would get thrown for any packages in the main repository loaded from other repositories: java.lang.RuntimeException: Unrecoverable error while evaluating node 'PACKAGE:@//tools/build_rules/go/toolchain' (requested by nodes ) at com.google.devtools.build.skyframe.ParallelEvaluator$Evaluate.run(ParallelEvaluator.java:982) at com.google.devtools.build.lib.concurrent.AbstractQueueVisitor$WrappedRunnable.run(AbstractQueueVisitor.java:499) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.IllegalArgumentException: Invalid BUILD file name for package '@//tools/build_rules/go/toolchain': /home/brian/bazel/tools/build_rules/go/toolchain/BUILD at com.google.devtools.build.lib.packages.Package.finishInit(Package.java:299) at com.google.devtools.build.lib.packages.Package$Builder.finishBuild(Package.java:1308) at com.google.devtools.build.lib.skyframe.PackageFunction.compute(PackageFunction.java:501) at com.google.devtools.build.skyframe.ParallelEvaluator$Evaluate.run(ParallelEvaluator.java:933) ... 4 more Sponsor's comment: note the abundance of new Label.resolveRepositoryRelative() calls. They are ugly, but it's only making existing ugliness explicit. Yes, we should fix it, especially in the implementation of configurable attributes. Refs #940 -- Change-Id: I8bd7f7b00bec58a7157507595421bc50c81b404c Reviewed-on: https://bazel-review.googlesource.com/#/c/2591 MOS_MIGRATED_REVID=117429733
* Add a few other Label.resolveRepositoryRelative() calls to make select() ↵Gravatar Lukacs Berki2016-01-15
| | | | | | | | | | | statements work in remote repositories. This work is somewhat sisyphean; the principled thing to do would be to just do this resolution when labels are parsed. Fixes #783. -- MOS_MIGRATED_REVID=112137996
* Permit proto serialization of configured attribute valuesGravatar Mark Schaller2016-01-04
| | | | | | | | | This commit adds proto messages that represent configurable values, and modifies attribute value serialization code to handle those values, which are called SelectorLists. -- MOS_MIGRATED_REVID=111149272
* Use Bazel Preconditions variant which avoids varargs array creationGravatar Mark Schaller2015-12-10
| | | | | | | Reduces garbage. -- MOS_MIGRATED_REVID=109914243
* Precompute non-configurable attributes at RuleClass levelGravatar Michajlo Matijkiw2015-12-09
| | | | | | | | | Previously we created this collection for each AggregatingAttributeMapper, which we create at least every attribute encountered. Calculate the collection up front to avoid wasting time and memory. -- MOS_MIGRATED_REVID=109805907
* Rationalize copyright headersGravatar Damien Martin-Guillerez2015-09-25
| | | | | | | | | | | The headers were modified with `find . -type f -exec 'sed' '-Ei' 's|Copyright 201([45]) Google|Copyright 201\1 The Bazel Authors|' '{}' ';'` And manual edit for not Google owned copyright. Because of the nature of ijar, I did not modified the header of file owned by Alan Donovan. The list of authors were extracted from the git log. It is missing older Google contributors that can be added on-demand. -- MOS_MIGRATED_REVID=103938715
* Move Label from the lib.syntax to the lib.cmdline package so that:Gravatar Lukacs Berki2015-09-21
| | | | | | | | | | - Label parsing can be simplified - lib.syntax is only contains the code for Skylark and is reasonably independent from the problem domain of building things This change is mostly only changes to imports declarations. The rest is reversing the dependency between :cmdline and :syntax and moving a tiny amount of code between Printer and FilesetEntry and the addition of SkylarkPrintableValue that I couldn't be bothered to separate out into its own change. -- MOS_MIGRATED_REVID=103527877
* Separate build-specific types and types inherent to Skylark.Gravatar Lukacs Berki2015-09-21
| | | | | -- MOS_MIGRATED_REVID=103374106
* Refactor Type.getLabels() so that instead of collecting labels, it flattens ↵Gravatar Lukacs Berki2015-09-17
| | | | | | | | | composite types and remove the mention of the type "Label" from the context argument of Type.convert(). This is in preparation of separating build-specific types and types inherent to Skylark. -- MOS_MIGRATED_REVID=103266869
* Stop "no generating action" exceptions for configurableGravatar Greg Estren2015-06-19
| | | | | | | | | | | | | | | | | | | cc_library "srcs". The underlying problem is that Bazel determines whether or not to generate a .so by the contents of "srcs", but this happens in the loading phase before configurations are known. So, when "srcs" is configurable, it conservatively assumes a .so needs to be generated. In cc_library analysis, when the final values for "srcs" is properly bound, the .so may not actually be needed after all. This cl fixes the discrepancy by registering a FailAction for the .so under those circumstances - we don't actually need the .so anywhere but we still have to ensure it's registered with an owning action. -- MOS_MIGRATED_REVID=96405079
* Refine duplicate label checking for embedded selects. Before, we wereGravatar Greg Estren2015-05-15
| | | | | | | | | | | | | | | | | | | coarsely checking for duplicates *anywhere*, e.g.: select({':a': ['a.cc'], ':b': ['a.cc']}) + select({':a': ['b.cc'], ':b': ['b.cc']}) would fail. But this case is okay because these duplicates are in mutually exclusive select paths (so they could never appear together anyway). The new checking logic is: - Duplicates can appear in different paths of the same select. - Duplicates can *not* appear within the same path of a select. - Duplicates can *not* appear across multiple selects (no matter what path - this is still stricter than we need to be, but there's no strong case for refining this case now). -- MOS_MIGRATED_REVID=93447979
* Optimize AggregatingAttributeMapper.visitAttribute (which returnsGravatar Greg Estren2015-05-15
| | | | | | | | | | | | | | | | | | | | | | | | | every possible value an attribute can take) for attributes with multiple selects: Given attr = select({':a': 'w', ':b': 'x'}) + select({':a': 'y', ':b': 'z'} the naive approach is to combine every possible value of the first select with every possible value of the second select (producing 4 possible values from 2^2 visitations). But since these selects have the same exact conditions, only two values are actually possible ("wy", "xz") from 2 visitations. This change efficiently considers that case. More generally, given n concatenated selects with the same conditions, it brings evaluation time down from O(2^n) to O(n) (assuming two conditions per select). It also works for partial matches: given a concatenation of 6 selects where 1, 3, and 5 have the same conditions and 2, 4, and 6 have the same conditions, evaluation time goes from O(2^6) to O(2^2). -- MOS_MIGRATED_REVID=93325787
* Further reduction of calls to AggregatingAttributeMapper.visitAttribute.Gravatar Greg Estren2015-04-16
| | | | | | | This is now not called on the regular build path except for computed defaults. -- MOS_MIGRATED_REVID=91306062
* Minimize calls to AggregatingAttributeMapper.visitAttribute. Attributes withGravatar Greg Estren2015-04-15
| | | | | | | | | multiple selects run the risk of exponential value growth, so we shouldn't request a full iteration of possible values unless that's really what the caller needs. -- MOS_MIGRATED_REVID=91118257
* Configurable attributes: support embeddable selects. With thisGravatar Greg Estren2015-04-14
| | | | | | | | | | | | | | | | change, the following syntax: deps = [':always'] + select({':a': [':adep'], ':b': [':bdep']}) or deps = select({':a': [':adep'], ':b': [':bdep']}) + select({':c': [':cdep'], ':d': [':ddep']}) works. -- MOS_MIGRATED_REVID=91016337
* Remove "obsolete" and "default_obsolete" from the BUILD language.Gravatar Greg Estren2015-03-18
| | | | | | | | Anyone who needs this kind of functionality in the future can redefine it through user-defined constraints. -- MOS_MIGRATED_REVID=88871811
* AttributeMap#has(String, Type) is added to check whether an attribute with a ↵Gravatar Googler2015-03-13
| | | | | | | | | given name is present. Rule implementations are cleaned up. -- MOS_MIGRATED_REVID=88430329
* Update from Google.Gravatar Han-Wen Nienhuys2015-02-25
-- MOE_MIGRATED_REVID=85702957
generated by cgit v1.2.3 (git 2.39.1) at 2024-07-07 07:29:13 +0000