aboutsummaryrefslogtreecommitdiffhomepage
path: root/src/main/java/com/google/devtools/build/lib/packages
Commit message (Collapse)AuthorAge
* Fix workspace name setting when the WORKSPACE file is splittedGravatar Damien Martin-Guillerez2016-03-31
| | | | | | | | If a load statement is present in the WORKSPACE file, the workspace name attribute will be lost from the WORKSPACE file. -- MOS_MIGRATED_REVID=118577315
* --Gravatar Chris Parsons2016-03-28
| | | | MOS_MIGRATED_REVID=118246740
* Removed ending . in error messageGravatar Damien Martin-Guillerez2016-03-23
| | | | | | | They are automatically added when printing the error message. -- MOS_MIGRATED_REVID=117927312
* Make labels in .bzl files in remote repos resolve relative to their repoGravatar Kristina Chodorow2016-03-21
| | | | | | | | | | | | | | | | | | For example, if you have a BUILD file that does: load('@foo//bar:baz.bzl', 'my_rule') my_rule(...) If baz.bzl uses Label('//whatever'), this change makes //whatever resolve to @foo//whatever. Previous to this change, it would be resolved to the repository the BUILD file using my_rule was in. RELNOTES[INC]: Labels in .bzl files in remote repositories will be resolved relative to their repository (instead of the repository the Skylark rule is used in). -- MOS_MIGRATED_REVID=117720181
* Fix equality and comparison for SkylarkAspectClass.Gravatar Dmitry Lomov2016-03-17
| | | | | | | | AspectClass (also SkylarkApsectClass as its implementation) is a part of AspectKey. DependencyResolver gets the definition of aspect (including -- MOS_MIGRATED_REVID=117433907
* Don't keep packages in the default repository around after loading.Gravatar Brian Silverman2016-03-17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Previously, this would get thrown when referring to the same package from both the main and default repositories: java.lang.IllegalArgumentException: Multiple entries with same key: tools/cpp=/home/brian/971-Robot-Code and tools/cpp=/home/brian/971-Robot-Code at com.google.common.collect.ImmutableMap.checkNoConflict(ImmutableMap.java:136) at com.google.common.collect.RegularImmutableMap.checkNoConflictInKeyBucket(RegularImmutableMap.java:98) at com.google.common.collect.RegularImmutableMap.fromEntryArray(RegularImmutableMap.java:84) at com.google.common.collect.ImmutableMap$Builder.build(ImmutableMap.java:295) at com.google.devtools.build.lib.buildtool.BuildTool.transformPackageRoots(BuildTool.java:301) at com.google.devtools.build.lib.buildtool.BuildTool.buildTargets(BuildTool.java:209) at com.google.devtools.build.lib.buildtool.BuildTool.processRequest(BuildTool.java:334) at com.google.devtools.build.lib.runtime.commands.TestCommand.doTest(TestCommand.java:119) at com.google.devtools.build.lib.runtime.commands.TestCommand.exec(TestCommand.java:104) at com.google.devtools.build.lib.runtime.BlazeCommandDispatcher.exec(BlazeCommandDispatcher.java:371) at com.google.devtools.build.lib.runtime.BlazeRuntime$3.exec(BlazeRuntime.java:1016) at com.google.devtools.build.lib.server.RPCService.executeRequest(RPCService.java:65) at com.google.devtools.build.lib.server.RPCServer.executeRequest(RPCServer.java:434) at com.google.devtools.build.lib.server.RPCServer.serve(RPCServer.java:229) at com.google.devtools.build.lib.runtime.BlazeRuntime.serverMain(BlazeRuntime.java:975) at com.google.devtools.build.lib.runtime.BlazeRuntime.main(BlazeRuntime.java:772) at com.google.devtools.build.lib.bazel.BazelMain.main(BazelMain.java:55) And this would get thrown for any packages in the main repository loaded from other repositories: java.lang.RuntimeException: Unrecoverable error while evaluating node 'PACKAGE:@//tools/build_rules/go/toolchain' (requested by nodes ) at com.google.devtools.build.skyframe.ParallelEvaluator$Evaluate.run(ParallelEvaluator.java:982) at com.google.devtools.build.lib.concurrent.AbstractQueueVisitor$WrappedRunnable.run(AbstractQueueVisitor.java:499) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.IllegalArgumentException: Invalid BUILD file name for package '@//tools/build_rules/go/toolchain': /home/brian/bazel/tools/build_rules/go/toolchain/BUILD at com.google.devtools.build.lib.packages.Package.finishInit(Package.java:299) at com.google.devtools.build.lib.packages.Package$Builder.finishBuild(Package.java:1308) at com.google.devtools.build.lib.skyframe.PackageFunction.compute(PackageFunction.java:501) at com.google.devtools.build.skyframe.ParallelEvaluator$Evaluate.run(ParallelEvaluator.java:933) ... 4 more Sponsor's comment: note the abundance of new Label.resolveRepositoryRelative() calls. They are ugly, but it's only making existing ugliness explicit. Yes, we should fix it, especially in the implementation of configurable attributes. Refs #940 -- Change-Id: I8bd7f7b00bec58a7157507595421bc50c81b404c Reviewed-on: https://bazel-review.googlesource.com/#/c/2591 MOS_MIGRATED_REVID=117429733
* Fix glob performance regression introduced by commit ↵Gravatar Nathan Harmata2016-03-17
| | | | | | | | | | | | | 3a95f353704dc2f7061e2c0786c2459ac1db0fd1. AbstractSet#removeAll has unexpected, yet oddly intentional (and documented), performance characteristics. Suppose we are evaluating 'set.removeAll(collection)' and 'collection.contains(x)' is 'O(e)'. Then 'set.removeAll(collection)' is 'O(set.size())' when 'set.size() <= collection.size()' and 'O(set.size()) * e' otherwise. When 'collection' is e.g. an ArrayList, 'e' is 'collection.size()' and so 'set.removeAll(collection)' is 'O(set.size() * collection.size())', which is bad. This meant we had poor performance when the excludes patterns of a glob matched more files than the includes patterns. Note that, while GlobCache#glob() *did* also use removeAll (potentially inefficiently), it was doing so for each list of exclude glob matches individually. So legacy globbing would have suboptimal performance for 'glob(includes=[i_1, i_2, ...i_k], excludes = [e_1, e_2, ..., e_j])' whenever the result of any e_i was larger than the union of all the includes matches. (But skyframe hybrid globbing has the performance issue when the union of the excludes matches is larger than the union of the includes matches, which is more likely to happen in practice.) I fixed this hypothetical problem too. -- MOS_MIGRATED_REVID=117367755
* Adds bazel_version to the native module for workspace macros.Gravatar Damien Martin-Guillerez2016-03-15
| | | | | | | | | | | | | | | | | | | | | One can now invoke native.bazel_version to get the version of Bazel from the WORKSPACE file. This can be used to do version check, for instance: version.bzl: def check_version(x): if native.bazel_version < x: fail("Current Bazel version is {}, expected at least {}".format(native.bazel_version, x)) WORKSPACE: load("//:version.bzl", "check_bersion") check_version("0.2") Fixes #1014. -- MOS_MIGRATED_REVID=117231557
* fix typo: "is produce" -> "is produced"Gravatar Cal Peyser2016-03-15
| | | | | -- MOS_MIGRATED_REVID=117202268
* Improve error when allowedFileTypesForLabels is not setGravatar Liam Miller-Cushon2016-03-11
| | | | | -- MOS_MIGRATED_REVID=116980421
* Python provider is now available in SkylarkGravatar Yun Peng2016-03-03
| | | | | | | | | | | | | Using mandatoryProvidersList to validate python rules' dependency. Added a SkylarkProvider named 'py' which is a SkylarkClassObject in Java and a struct in Skylark. Native python rule and Skylark python rule should have this provider so that they can depend on each other. RELNOTES[NEW]: Native python rule can depend on skylark rule as long as skylark rule provides 'py' provider. -- MOS_MIGRATED_REVID=116241504
* Changed mandatoryProviders to mandatoryProvidersListGravatar Yun Peng2016-02-22
| | | | | | | | "mandatoryProvidersList" is a list of sets of providers. For any rule, if it provides all the providers from one of those sets, we consider the dependency valid. -- MOS_MIGRATED_REVID=115221394
* Introduce SkylarkRepositoryModuleGravatar Damien Martin-Guillerez2016-02-17
| | | | | | | | | | | | | | | | | The SkylarkRepositoryModule declare the `repository_rule` function to Skylark to define new remote repository types (http://goo.gl/OZV3o0). The work is delagated to the `SkylarkRepositoryFunction` by the `RepositoryDelegatorFunction`. `SkylarkRepositoryContext` defines the `ctx` object passed to the `repository_rule` implementation function. This change also introduce a `SkylarkPath` and the necessary methods in `SkylarkRepositoryContext` to showcase the creation of a `local_repository` like repository. Issue #893: step 3 of the roadmap http://goo.gl/OZV3o0. -- MOS_MIGRATED_REVID=114895003
* Reinstate mutable maps, again.Gravatar Francois-Rene Rideau2016-02-17
| | | | | -- MOS_MIGRATED_REVID=114860576
* Add a method to get a single value from AspectParameters.Gravatar Carmi Grushko2016-02-17
| | | | | | | I found this to be so common I think we should just provide it on the AspectParameters class. -- MOS_MIGRATED_REVID=114803710
* Include real type in type casting error message.Gravatar Tobias Werth2016-02-16
| | | | | -- MOS_MIGRATED_REVID=114751682
* Remove deprecated aggregation functions.Gravatar Han-Wen Nienhuys2016-02-16
| | | | | -- MOS_MIGRATED_REVID=114736521
* Prevent load statements in remote repository's WORKSPACE file to breakGravatar Damien Martin-Guillerez2016-02-12
| | | | | | | | | | | | | | | If a load statements was present in a remote repository's WORKSPACE file, the parsing of the WORKSPACE file to get the workspace name will break the build because it tries to load the Skylark extension from the main repository. This change replace the parsing of the whole remote repository's WORKSPACE file by a parsing of only it first chunk (before the first load statement). This change also enforce that the workpace() function to be called only from the top of the WORKSPACE file. -- MOS_MIGRATED_REVID=114528640
* Forbid overloading of a repository outside of the first part of the ↵Gravatar Damien Martin-Guillerez2016-02-11
| | | | | | | | | workspace file Fixes #824. -- MOS_MIGRATED_REVID=114326952
* Implement TODO from: ↵Gravatar Greg Estren2016-02-11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | https://github.com/bazelbuild/bazel/blo[]fbbd6a32b95ba746f09dae1eaeaccf675cd5b3/src/main/java/com/google/devtools/build/lib/packages/Attribute.java#L1045 This allows the default value computation for latebound attributes to consider the values of configurable attributes. This is most directly useful for user-definable Skylark defaults, which have full access to the values of all non-latebound attributes. Without this change, this kind of scenario crashes Bazel. For example: ------------------ select_rules.bzl: ------------------ def _impl(ctx): ctx.file_action( output=ctx.outputs.out_file, content=ctx.attr.string_value, ) return struct() # Bug does not manifest without using this as a default. def _derived_value(attrs, _): return Label("//some:dep") selector_rule = rule( implementation=_impl, attrs={ "string_value": attr.string(default=""), "out_file": attr.output(), "_derived": attr.label(default=_derived_value), }, output_to_genfiles=True, ) def selector_macro(name, out_file="", string_value=""): # This will fail with selectors. selector_rule( name="%s_skylark" % name, string_value=string_value, out_file=out_file + ".skylark", ) # This does not. native.genrule( name="%s_genrule" % name, cmd="echo '" + string_value + "' > $@", outs=[out_file + ".genrule"], ) native.filegroup( name=name, srcs=[":%s_genrule" % name, "%s_skylark" % name], ) ------------------ BUILD.bzl: ------------------ config_setting( name = "selector", values = {"compilation_mode": "opt"}, ) selector_macro( name = "this_rule", string_value = """soup? """ + select({ ":selector": "no, thank you.", "//conditions:default": "yes, please!!", }), out_file = "this_rule.txt", ) -- MOS_MIGRATED_REVID=114326474
* Split the execution of the WORKSPACE file after each load statementGravatar Damien Martin-Guillerez2016-02-10
| | | | | | | | | | | | This is the main step toward supporting load statement of remote repository in the WORKSPACE file (bug #824). Load statement that refers to a remote repository will depend on the previous fragment of the workspace file. Issue #824 Step 3. -- MOS_MIGRATED_REVID=114237027
* Use a clever hybrid approach for evaluating globs during package loading: ↵Gravatar Nathan Harmata2016-02-05
| | | | | | | | | | | | | | | | | first try to get a skyframe cache-hit; otherwise, fall back to legacy globbing. This gives us the best of both worlds: no extra skyframe restarts on glob-dep-misses, and much better incremental performance in the common case that a package's globs haven't changed. See the class-comment for PackageFunction.SkyframeHybridGlobber for a detailed description and explanation. This CL has no impact on semantics and is a strict performance win. Bazel users: Here's an example benchmark (does an incremental loading phase of a target T but forces all packages to be reloaded): nharmata@nharmata:~/bazel$ N=10; B="output/bazel"; T=//src/main/java/com/google/devtools/build/lib:bazel/BazelServer_deploy.jar; CMD="build --noanalyze $T"; $B clean &> /dev/null; P=base_workspace/tools/build_rules/prelude_bazel; rm $P; touch base_workspace/tools/build_rules/BUILD; $B $CMD &> /dev/null; time for i in $(seq 1 $N); do echo "#hi" >> $P; $B $CMD &> /dev/null; done For a very large internal Google target, this CL improves the benchmark performance by ~6%. A more targeted benchmark would be for loading a single package that has lots of expensive globs. For example, the time to incrementally load a single pathological Google-internal package was reduced by ~36%. Alternatives considered: Introduce skyframe native globbing, gated by flags for both globbing during preprocessing and globbing during regular BUILD file evaluation. The approach in this CL is superior performance-wise. -- MOS_MIGRATED_REVID=113899687
* Move the Globber interface into its own file.Gravatar Nathan Harmata2016-02-05
| | | | | -- MOS_MIGRATED_REVID=113893917
* Fileset: introduce support for workspace-relative strip_prefix values.Gravatar Laszlo Csomor2016-02-04
| | | | | -- MOS_MIGRATED_REVID=113830075
* Add back documentation and better error message for workspace namesGravatar Kristina Chodorow2016-02-03
| | | | | -- MOS_MIGRATED_REVID=113658912
* Rollback of commit f941d56acfad5f8c819c81b494f806ea74ea7fd8.Gravatar Carmi Grushko2016-02-02
| | | | | | | | | | | | | | | | | | *** Reason for rollback *** Breaks many targets, see [] *** Original change description *** Reinstate mutable SkylarkDict Add <String, Object> annotation to optionMap invocation in SkylarkAttr, to make JDK 1.7 happy. Give the visible name "aspect" to class SkylarkAspect. -- MOS_MIGRATED_REVID=113543873
* Improve the error message on a missing implicit dep. The old error message ↵Gravatar Nathan Harmata2016-02-01
| | | | | | | didn't include the reason why the target was missing, and it also spammed Blaze users with a "tip" that's only relevant to Blaze developers debugging newly written integration tests. -- MOS_MIGRATED_REVID=113401767
* Reinstate mutable SkylarkDictGravatar Francois-Rene Rideau2016-02-01
| | | | | | | | | | Add <String, Object> annotation to optionMap invocation in SkylarkAttr, to make JDK 1.7 happy. Give the visible name "aspect" to class SkylarkAspect. -- MOS_MIGRATED_REVID=113394826
* Rollback of commit c0a8c58b9230a1f5d76269eb7dc6b11e18f19686.Gravatar Damien Martin-Guillerez2016-01-29
| | | | | | | | | | | | | | | | | | | | | | | *** Reason for rollback *** Break Java 1.7 builds of Bazel. See http://ci.bazel.io/job/Bazel/JAVA_VERSION=1.7,PLATFORM_NAME=linux-x86_64/327/console Test: git clone ... && git revert c0a8c58 && export JAVA_VERSION=1.7 && export BAZEL_COMPILE_TARGET=compile && bash -c "source scripts/ci/build.sh; bazel_build" *** Original change description *** Make Skylark dicts mutable Represent Skylark dict using a new subclass SkylarkDict<K, V> of Map<K, V>. Back it with a TreeMap to provide a deterministic iteration order. Also make SkylarkList generic in its element type <E>. Have Artifact implement Comparable<Object> so it can be used as TreeMap key. -- MOS_MIGRATED_REVID=113359718
* Parse the workspace name when a repository is loadedGravatar Kristina Chodorow2016-01-29
| | | | | | | | | Moved RepositoryValue to RepositoryDirectoryValue so that it could be cached (and not re-downloaded) even if the WorkspaceAST caused a Skyframe restart (as mentioned in https://github.com/bazelbuild/bazel/blob/master/src/main/java/com/google/devtools/build/lib/rules/repository/RepositoryDelegatorFunction.java#L130-L133). -- MOS_MIGRATED_REVID=113358489
* Clear up some confusion about glob prefetching (the old comment was wrong). ↵Gravatar Nathan Harmata2016-01-29
| | | | | | | | | Also add some TODOs for potentially improving package loading performance. This CL has no semantic impact. -- MOS_MIGRATED_REVID=113301656
* Make Skylark dicts mutableGravatar Francois-Rene Rideau2016-01-29
| | | | | | | | | | Represent Skylark dict using a new subclass SkylarkDict<K, V> of Map<K, V>. Back it with a TreeMap to provide a deterministic iteration order. Also make SkylarkList generic in its element type <E>. Have Artifact implement Comparable<Object> so it can be used as TreeMap key. -- MOS_MIGRATED_REVID=113277489
* Resolve repository-relative labels within the current repositoryGravatar Kristina Chodorow2016-01-28
| | | | | | | | | | | Using $(location //foo) from an external repository was resolving to @//foo, not @repo//foo, which generally wouldn't be in the main repository. This may also fix other cases where getRelative was resolving incorrectly. Fixes #819. -- MOS_MIGRATED_REVID=113256854
* Rename native.rule and native.rules to {existing_rule,existing_rules}Gravatar Han-Wen Nienhuys2016-01-28
| | | | | | | | | | | | This is to avoid confusion between rule(), which declares a new build rules, and native.rule(), which can only be used in macros to inspect the BUILD file processed so far. native.{rule,rules} is maintained and marked deprecated to smooth the transition for early adopters. -- MOS_MIGRATED_REVID=113250194
* Inject the TOOLS_REPOSITORY constant in the RuleClassProvider instead of ↵Gravatar Luis Fernando Pino Duque2016-01-28
| | | | | | | | | | | using Constants.java It also includes one example on how to use the new mechanism in BazelCppRuleClasses. This is the first phase for the removal of the TOOLS_REPOSITORY constant. -- MOS_MIGRATED_REVID=113244399
* Ensure repository names are valid workspace namesGravatar Kristina Chodorow2016-01-28
| | | | | | | RELNOTES: Repository rules must use names that are valid workspace names. -- MOS_MIGRATED_REVID=113197588
* Describe some of the more likely license typesGravatar Kristina Chodorow2016-01-27
| | | | | | | Fixes #643. -- MOS_MIGRATED_REVID=113055336
* Return opaque value for native.rule() on a select() statement.Gravatar Han-Wen Nienhuys2016-01-27
| | | | | | | | | | | | | | | | | | | | | | | | | This is imperfect in many ways: 1) the value is opaque, and not a BUILD value, so it cannot be used in rule arguments 2) its representation has a pointer address, so it breaks hermeticity. Despite its problem, we return this value because otherwise native.rules() fails if there is any rule using a select() in the BUILD file. A future solution would be to convert BuildType.SelectorList back to syntax.SelectorList. To do so, we would have to 1) recurse into the Selector contents of SelectorList, so those values are run through skylarkifyValue too 2) get the right Class<?> value. We could probably get at that by looking at ((SelectorList)val).getSelectors().first().getEntries().first().getClass(). -- MOS_MIGRATED_REVID=113051612
* Add documentation for workspace functionGravatar Kristina Chodorow2016-01-26
| | | | | -- MOS_MIGRATED_REVID=112971086
* Fix some generic warnings.Gravatar Ulf Adams2016-01-25
| | | | | -- MOS_MIGRATED_REVID=112952552
* Make select() statements in remote repositories with selector labels ↵Gravatar Lukacs Berki2016-01-25
| | | | | | | | | | | pointing back to the main repository ("@//:...") work. In principle, we should just use main repository labels instead of default repository labels in all the SkyKeys that take labels. @bsilver8192 is working on it, but in the meantime, this will serve to unblock @chin33z. Fixes #811. -- MOS_MIGRATED_REVID=112787545
* Extract parsing of the WORKSPACE file in two partsGravatar Damien Martin-Guillerez2016-01-22
| | | | | | | | | | | The WORKSPACE file AST is now parsed as a separate SkyFunction and this will be used to have multiple SkyValue for the same WORKSPACE file, splitting the execution of the AST after load statements to enable load statement of external dependencies in the WORKSPACE file. -- MOS_MIGRATED_REVID=112768897
* Skylark, JS: nicer error message when a source file has an unknown type.Gravatar Laszlo Csomor2016-01-22
| | | | | -- MOS_MIGRATED_REVID=112766003
* Properly filter out Aspect attributes when `blaze query` requests ↵Gravatar Dmitry Lomov2016-01-22
| | | | | | | | | --noimplicit_deps and/or --nohost_deps. RELNOTES: --noimplicit_deps and --nohost_deps work correctly for Aspect attributes. -- MOS_MIGRATED_REVID=112724917
* Extract AttributeInfoProvider interface for DependencyFilters.Gravatar Dmitry Lomov2016-01-22
| | | | | | | | This is in preparation of implementing dependency filtering correctly for aspects. -- MOS_MIGRATED_REVID=112721440
* Refactor dependency filtering out of Rule class.Gravatar Dmitry Lomov2016-01-22
| | | | | -- MOS_MIGRATED_REVID=112717648
* Fix NullPointerException in DelegatingWalkableGraphGravatar Damien Martin-Guillerez2016-01-22
| | | | | | | | | | Package name comparison did not included the repository name which leads to a crash when trying to build :* targets. Fixes #792. -- MOS_MIGRATED_REVID=112708531
* Raise error if we find an unknown type in native.rule().Gravatar Han-Wen Nienhuys2016-01-21
| | | | | | | | | | | | Handle more types: * Boolean * TriState * SkylarkValue (eg. FileSetEntry) * skip Licenses, Distribs. -- MOS_MIGRATED_REVID=112690550
* Fileset (Google internal): code cleanup.Gravatar Laszlo Csomor2016-01-19
| | | | | -- MOS_MIGRATED_REVID=112466264
* Description redacted.Gravatar Googler2016-01-18
| | | | | -- MOS_MIGRATED_REVID=112394431
generated by cgit v1.2.3 (git 2.39.1) at 2024-07-07 16:55:12 +0000