| Commit message (Collapse) | Author | Age |
|
|
|
|
|
|
| |
*** Reason for rollback ***
--
MOS_MIGRATED_REVID=136448178
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
*** Reason for rollback ***
[]
*** Original change description ***
Slight refactor of ExternalFilesHelper:
-Make FileType and ExternalFileAction public.
-Have producers use ExternalFileAction, rather than a boolean, to specify the desired behavior.
And a big change in semantics (doesn't affect Bazel):
-Replace ExternalFileAction.ERROR_OUT with ExternalFileAction.ASSUME_NON_EXISTENT_AND_IMMUTABLE, which does what it sounds like. This new action, like the old ERROR_OUT, is _not_ used in Bazel.
--
MOS_MIGRATED_REVID=136206810
|
|
|
|
|
|
|
|
|
|
|
| |
-Make FileType and ExternalFileAction public.
-Have producers use ExternalFileAction, rather than a boolean, to specify the desired behavior.
And a big change in semantics (doesn't affect Bazel):
-Replace ExternalFileAction.ERROR_OUT with ExternalFileAction.ASSUME_NON_EXISTENT_AND_IMMUTABLE, which does what it sounds like. This new action, like the old ERROR_OUT, is _not_ used in Bazel.
--
MOS_MIGRATED_REVID=136063159
|
|
|
|
|
|
|
| |
loaded by a single package. This also fixes a bug in which Skylark code expecting reference equality for an object imported multiple ways were not getting it.
--
MOS_MIGRATED_REVID=135738789
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
manual rollback of []
*** Reason for rollback ***
Depot has been fixed / is in the process of being fixed. See the work tracked on []
*** Original change description ***
Automated [] rollback of commit bb5d5efb4b50710241b5b374eb67084f4bf08278.
--
MOS_MIGRATED_REVID=131095905
|
|
|
|
|
| |
--
MOS_MIGRATED_REVID=130941264
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
sorted. Previously, it would return a list formed by concatenating the sorted results of each pattern in the 'includes' list.
A bunch of cleanups and one bug fix:
-Remove the unused-except-for tests GlobCache#globsUpToDate. This code has been dead for a very very long time, ever since we switched to using Skyframe.
-Change the semantics of the 'glob' function as described above.
-Change UnixGlob to return unsorted results. Document this in UnixGlob and GlobCache.
-Change LegacyGlobber to conditionally return sorted results. Have users other than PackageFunction get sorted results (as described above). Have PackageFunction's use case get completely unsorted results, and have PackageFunction do the sorting itself.
-Have PackageFunction's HybridGlobber unconditionally sort the glob result list. This ensure deterministic glob results, fixing a bug where the order of the elements of the result depended on the contents of the Skyframe graph, which of course depends on the sequence of incremental Blaze commands.
--
MOS_MIGRATED_REVID=130540152
|
|
|
|
|
|
|
| |
The only place we now don't handle InterruptedException is in the action graph created after analysis, since I'm not sure that will be around for that much longer.
--
MOS_MIGRATED_REVID=130327770
|
|
|
|
|
|
|
| |
the PackageLookupValue to reduce the number of references to the filename "BUILD".
--
MOS_MIGRATED_REVID=129203257
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
execroot change
This is in prep for making the execution root path for external repositories
../repo_name (instead of external/repo_name). Right now, the getRunfilesPath() returns that path, so that is renamed getExecRoot() (since the runfiles are really just a reflection of the execRoot structure). getSourceRoot() replaces getPathFragment, which has always been a confusing name (it's not clear from the name
what the difference is between it and getPackageFragment()). It returns the relative path to source files for external repositories (external/repo_name).
Also renamed/moved to more sensible class a few static RepositoryName fields.
--
MOS_MIGRATED_REVID=128594419
|
|
|
|
|
|
|
|
|
| |
Skyframe globbing. This adds a log(n) factor to uses of globs, but getting globs to be returned in a reasonable order that can be emulated by legacy globbing is hard and bug-prone right now, and we must sort anyway if we are merging legacy and Skyframe globs.
Note that this log(n) factor is already present on clean builds with legacy globbing. If we end up seeing performance issues on incremental loading, we can investigate making GlobFunction efficiently return elements in sorted order. (We would still need to sort if merging legacy and Skyframe globs, but that should be a relatively rare occurrence, and can be dealt with by a more efficient merge sort if necessary.)
--
MOS_MIGRATED_REVID=127752554
|
|
|
|
|
|
|
| |
PackageFunction, leading to InconsistentFilesystemExceptions being the root cause of a NoSuchPackageException. They are persistent and should be treated as the real root cause of the NoSuchPackageException.
--
MOS_MIGRATED_REVID=125683253
|
|
|
|
|
|
|
| |
don't want to throw IOExceptions unless there is an actual IOException. Syntax errors in WORKSPACE files don't qualify, and neither do badly written Skylark rules. I may have misunderstood some code here, so please do push back if the root causes here really can be filesystem issues.
--
MOS_MIGRATED_REVID=125591177
|
|
|
|
|
|
|
| |
after the Package is fully loaded.
--
MOS_MIGRATED_REVID=124260910
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
To give a better understanding of which packages are on the critical
path during loading and analysis, provide information in the same way
as during execution: show the earliest started, but not yet completed
package. As not all packages looked at during the analysis phase are
reported to the progress receiver, use a custom class to aggregate those
data.
--
Change-Id: I03c25efdecb4124e1bc06fce8be9175dc56b5500
Reviewed-on: https://bazel-review.googlesource.com/#/c/3700
MOS_MIGRATED_REVID=123408689
|
|
|
|
|
|
|
| |
3a95f353704dc2f7061e2c0786c2459ac1db0fd1).
--
MOS_MIGRATED_REVID=123107954
|
|
|
|
|
|
|
| |
already have Label.EXTERNAL_PACKAGE_NAME.
--
MOS_MIGRATED_REVID=120828276
|
|
|
|
|
|
|
|
|
| |
BadWorkspaceFileException and BadPreludeFileException.
BadWorkspaceFileException and BadPreludeFileException are not doing anythings special rather than providing a custom message. Replace them by using NoSuchTargetException directly.
--
MOS_MIGRATED_REVID=119530878
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Previously, this would get thrown when referring to the same package
from both the main and default repositories:
java.lang.IllegalArgumentException: Multiple entries with same key: tools/cpp=/home/brian/971-Robot-Code and tools/cpp=/home/brian/971-Robot-Code
at com.google.common.collect.ImmutableMap.checkNoConflict(ImmutableMap.java:136)
at com.google.common.collect.RegularImmutableMap.checkNoConflictInKeyBucket(RegularImmutableMap.java:98)
at com.google.common.collect.RegularImmutableMap.fromEntryArray(RegularImmutableMap.java:84)
at com.google.common.collect.ImmutableMap$Builder.build(ImmutableMap.java:295)
at com.google.devtools.build.lib.buildtool.BuildTool.transformPackageRoots(BuildTool.java:301)
at com.google.devtools.build.lib.buildtool.BuildTool.buildTargets(BuildTool.java:209)
at com.google.devtools.build.lib.buildtool.BuildTool.processRequest(BuildTool.java:334)
at com.google.devtools.build.lib.runtime.commands.TestCommand.doTest(TestCommand.java:119)
at com.google.devtools.build.lib.runtime.commands.TestCommand.exec(TestCommand.java:104)
at com.google.devtools.build.lib.runtime.BlazeCommandDispatcher.exec(BlazeCommandDispatcher.java:371)
at com.google.devtools.build.lib.runtime.BlazeRuntime$3.exec(BlazeRuntime.java:1016)
at com.google.devtools.build.lib.server.RPCService.executeRequest(RPCService.java:65)
at com.google.devtools.build.lib.server.RPCServer.executeRequest(RPCServer.java:434)
at com.google.devtools.build.lib.server.RPCServer.serve(RPCServer.java:229)
at com.google.devtools.build.lib.runtime.BlazeRuntime.serverMain(BlazeRuntime.java:975)
at com.google.devtools.build.lib.runtime.BlazeRuntime.main(BlazeRuntime.java:772)
at com.google.devtools.build.lib.bazel.BazelMain.main(BazelMain.java:55)
And this would get thrown for any packages in the main repository loaded
from other repositories:
java.lang.RuntimeException: Unrecoverable error while evaluating node 'PACKAGE:@//tools/build_rules/go/toolchain' (requested by nodes )
at com.google.devtools.build.skyframe.ParallelEvaluator$Evaluate.run(ParallelEvaluator.java:982)
at com.google.devtools.build.lib.concurrent.AbstractQueueVisitor$WrappedRunnable.run(AbstractQueueVisitor.java:499)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: Invalid BUILD file name for package '@//tools/build_rules/go/toolchain': /home/brian/bazel/tools/build_rules/go/toolchain/BUILD
at com.google.devtools.build.lib.packages.Package.finishInit(Package.java:299)
at com.google.devtools.build.lib.packages.Package$Builder.finishBuild(Package.java:1308)
at com.google.devtools.build.lib.skyframe.PackageFunction.compute(PackageFunction.java:501)
at com.google.devtools.build.skyframe.ParallelEvaluator$Evaluate.run(ParallelEvaluator.java:933)
... 4 more
Sponsor's comment: note the abundance of new Label.resolveRepositoryRelative() calls. They are ugly, but it's only making existing ugliness explicit. Yes, we should fix it, especially in the implementation of configurable attributes.
Refs #940
--
Change-Id: I8bd7f7b00bec58a7157507595421bc50c81b404c
Reviewed-on: https://bazel-review.googlesource.com/#/c/2591
MOS_MIGRATED_REVID=117429733
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
3a95f353704dc2f7061e2c0786c2459ac1db0fd1.
AbstractSet#removeAll has unexpected, yet oddly intentional (and documented), performance characteristics. Suppose we are evaluating 'set.removeAll(collection)' and 'collection.contains(x)' is 'O(e)'. Then 'set.removeAll(collection)' is 'O(set.size())' when 'set.size() <= collection.size()' and 'O(set.size()) * e' otherwise. When 'collection' is e.g. an ArrayList, 'e' is 'collection.size()' and so 'set.removeAll(collection)' is 'O(set.size() * collection.size())', which is bad.
This meant we had poor performance when the excludes patterns of a glob matched more files than the includes patterns.
Note that, while GlobCache#glob() *did* also use removeAll (potentially inefficiently), it was doing so for each list of exclude glob matches individually. So legacy globbing would have suboptimal performance for 'glob(includes=[i_1, i_2, ...i_k], excludes = [e_1, e_2, ..., e_j])' whenever the result of any e_i was larger than the union of all the includes matches. (But skyframe hybrid globbing has the performance issue when the union of the excludes matches is larger than the union of the includes matches, which is more likely to happen in practice.) I fixed this hypothetical problem too.
--
MOS_MIGRATED_REVID=117367755
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The WORKSPACE file parsing needs to be separated into several parts
to enable load of labels in the WORKSPACE file.
This change adds an intermediate SkyFunction, ExternalPackageFunction,
that requires all the WORKSPACE file part to be parsed to resolve
//external: labels.
Issue #824 Step 1.
--
MOS_MIGRATED_REVID=113984026
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
first try to get a skyframe cache-hit; otherwise, fall back to legacy globbing. This gives us the best of both worlds: no extra skyframe restarts on glob-dep-misses, and much better incremental performance in the common case that a package's globs haven't changed. See the class-comment for PackageFunction.SkyframeHybridGlobber for a detailed description and explanation.
This CL has no impact on semantics and is a strict performance win.
Bazel users: Here's an example benchmark (does an incremental loading phase of a target T but forces all packages to be reloaded):
nharmata@nharmata:~/bazel$ N=10; B="output/bazel"; T=//src/main/java/com/google/devtools/build/lib:bazel/BazelServer_deploy.jar; CMD="build --noanalyze $T"; $B clean &> /dev/null; P=base_workspace/tools/build_rules/prelude_bazel; rm $P; touch base_workspace/tools/build_rules/BUILD; $B $CMD &> /dev/null; time for i in $(seq 1 $N); do echo "#hi" >> $P; $B $CMD &> /dev/null; done
For a very large internal Google target, this CL improves the benchmark performance by ~6%. A more targeted benchmark would be for loading a single package that has lots of expensive globs. For example, the time to incrementally load a single pathological Google-internal package was reduced by ~36%.
Alternatives considered: Introduce skyframe native globbing, gated by flags for both globbing during preprocessing and globbing during regular BUILD file evaluation. The approach in this CL is superior performance-wise.
--
MOS_MIGRATED_REVID=113899687
|
|
|
|
|
| |
--
MOS_MIGRATED_REVID=113893917
|
|
|
|
|
|
|
|
|
| |
by having it not declare a dep on the PackageLookupValue for the package. This optimization means that a BUILD file edit doesn't (necessarily) invalidate all the globs in the package; the PackageLookupValue node would get change-pruned but we still pay the very small cost of invalidating unnecessarily.
Also slightly improve variable naming in GlobFunctionTest.
--
MOS_MIGRATED_REVID=113799936
|
|
|
|
|
|
|
| |
--
Change-Id: I86b90ff6eda6a16c2efeb8fffca301e2594c254e
Reviewed-on: https://bazel-review.googlesource.com/#/c/2784/
MOS_MIGRATED_REVID=113529297
|
|
|
|
|
|
|
|
|
| |
Also add some TODOs for potentially improving package loading performance.
This CL has no semantic impact.
--
MOS_MIGRATED_REVID=113301656
|
|
|
|
|
|
|
|
|
| |
to do such a load will result in a skyframe circular dependency exception.
As a side effect of this change, SkylarkImportFailedExceptions thrown by SkylarkImportLookupFunction are now caught by PackageFunction and wrapped and rethrown as a PackageFunctionException. Previously, the first exception wasn't caught, generating an uncaught exception error at top level.
--
MOS_MIGRATED_REVID=112328755
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
files in external repositories.
In addition:
- Cleaned up and refactored some tests to reflect the new loading behavior.
Deferred to future CLs:
- Updating Bazel Skylark documentation to reflect the new load form.
- Enabling command-line loading of Aspects via labels.
RELNOTES: Skylark load statements may now reference .bzl files via build labels, in addition to paths. In particular, such labels can be used to reference Skylark files in external repositories; e.g., load("@my_external_repo//some_pkg:some_file.bzl", ...). Path-based loads are now deprecated and may be disabled in the future. Caveats: Skylark files currently do not respect package visibility; i.e., all Skylark files are effectively public. Also, loads may not reference the special //external package.
--
MOS_MIGRATED_REVID=110786452
|
|
|
|
|
|
|
| |
Reduces garbage.
--
MOS_MIGRATED_REVID=109914243
|
|
|
|
|
|
|
|
|
| |
package and the prefix of external packages in the execroot.
This is in preparation for renaming that path segment from "external" to "bazel-external" so that we can provide a symlink to it from the source tree so that the source tree looks like the execroot.
--
MOS_MIGRATED_REVID=109882753
|
|
|
|
|
|
|
|
|
|
|
| |
RELNOTES[NEW]: Skylark macros are now enabled in WORKSPACE file.
Design document at https://docs.google.com/document/d/1jKbNXOVp2T1zJD_iRnVr8k5D0xZKgO8blMVDlXOksJg/preview
Fixes #337
--
MOS_MIGRATED_REVID=108860301
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
internally. The load location for a Skylark Aspect is specified via a PathFragment, for consistency with current non-Aspect Skylark loads.
This should be a semantics-preserving change for users. In a subsequent CL, I'll change the Skylark syntax to allow load statements to use labels as well as paths, with the goal of eventually deprecating the latter.
Also:
- Removed the hack for handling relative loads in the prelude file.
- Refactored some redundant functionality in PackageFunction and SkylarkImportLookupFunction for handling loads.
- Removed the ability to put the BUILD file for the package containing a Skylark file under a different package root than the Skylark file itself. This functionality isn't currently used and is inconsistent with Blaze's handling of the package path elsewhere.
- Added BUILD files to a number of tests that load Skylark files; this is consistent with the requirement that all Skylark files need to be part of some package.
- Changed the constants used to set the location of the prelude file from paths to labels.
--
MOS_MIGRATED_REVID=107741568
|
|
|
|
|
|
|
| |
Otherwise we have potential correctness and performance problems on a missing Skylark import dep.
--
MOS_MIGRATED_REVID=107605200
|
|
|
|
|
|
|
|
|
| |
file rather than a ParserInputSource.
This is part of a series of changes with the net result being that we open, read, and parse each BUILD file exactly once.
--
MOS_MIGRATED_REVID=105911557
|
|
|
|
|
|
|
| |
FileStatus#isSpecialFile to help disambiguate between a regular file and a special file, since the file size of a special file cannot be trusted.
--
MOS_MIGRATED_REVID=105903622
|
|
|
|
|
| |
--
MOS_MIGRATED_REVID=105734972
|
|
|
|
|
| |
--
MOS_MIGRATED_REVID=105648425
|
|
|
|
|
|
|
| |
Minimize indirection wrt singletons, turns out we had one that was completely unused.
--
MOS_MIGRATED_REVID=105621494
|
|
|
|
|
|
|
|
|
| |
former uses the latter). This way we parse BUILD files exactly once.
This is part of a series of changes with the net result being that we open, read, and parse each BUILD file exactly once.
--
MOS_MIGRATED_REVID=105528075
|
|
|
|
|
|
|
| |
because it breaks some internal tests.
--
MOS_MIGRATED_REVID=105511798
|
|
|
|
|
|
|
|
|
| |
result.
This is a step in the right direction towards the goal of opening and reading each BUILD file exactly once.
--
MOS_MIGRATED_REVID=105338761
|
|
|
|
|
|
|
|
|
| |
error from SkylarkImportResult dep.
This is part of a series of changes with the net result being that we open, read, and parse each BUILD file exactly once.
--
MOS_MIGRATED_REVID=105253634
|
|
|
|
|
|
|
|
|
|
| |
We don't have any strong guarantees around the stability of SkyKey#toString(),
which we rely on to canonicalize cycles. We can reasonably expect
PackageIdentifier#toString() to be stable enough though. We also hide away the
cycle detection details from external code in this change.
--
MOS_MIGRATED_REVID=105093095
|
|
|
|
|
|
|
|
|
| |
present.
This is nice because this removes the last thing Bazel needs to be able to function without a tools/ directory that is not already embedded into the binary.
--
MOS_MIGRATED_REVID=105054134
|
|
|
|
|
|
|
|
|
| |
repository.
This is necessary because we copy labels from the command line into tools/defaults/BUILD and currently there is no syntax to make a label in a remote repository refer to one in the main repository.
--
MOS_MIGRATED_REVID=104755206
|
|
|
|
|
| |
--
MOS_MIGRATED_REVID=104403040
|
|
|
|
|
|
|
|
|
|
|
| |
The headers were modified with
`find . -type f -exec 'sed' '-Ei' 's|Copyright 201([45]) Google|Copyright 201\1 The Bazel Authors|' '{}' ';'`
And manual edit for not Google owned copyright. Because of the nature of ijar, I did not modified the header of file owned by Alan Donovan.
The list of authors were extracted from the git log. It is missing older Google contributors that can be added on-demand.
--
MOS_MIGRATED_REVID=103938715
|
|
|
|
|
|
|
|
|
| |
Instead, pass an appropriate EventHandler instance in. This is in preparation
for creating a per-command EventHandler, in preparation for allowing multiple
commands to run in parallel. This is removal of shared global state.
--
MOS_MIGRATED_REVID=103828963
|
|
|
|
|
|
|
| |
reifying them in the Skyframe graph.
--
MOS_MIGRATED_REVID=103758591
|
|
|
|
|
|
|
| |
throw errors more eagerly in SkylarkImportLookupFunction -- don't try to request deps if the ast is in error.
--
MOS_MIGRATED_REVID=103607939
|