aboutsummaryrefslogtreecommitdiffhomepage
path: root/src/main/java/com/google/devtools/build/lib/skyframe/RecursiveDirectoryTraversalFunction.java
Commit message (Collapse)AuthorAge
* When requesting all transitive traversal values recursively, request ↵Gravatar Janak Ramakrishnan2016-10-12
| | | | | | | | | | | packages/targets in parallel with subdirectories. The previous implementation was vulnerable to incomplete traversal in case of cycles: if a subdirectory had a cycle, the targets in the package would never be requested. Requesting the package first would create a different problem, where the subdirectories would never be requested if the package depended on a cycle. Also stop uniquifying Skylark import cycles when inlining. While uniquification is nice, this was leading to us emitting an error during queries that we weren't actually emitting when not inlining. Putting the cycle into the exception error message should be enough information, and since we emit an error per exception anyway, the number of events will still be lower (although each event might be more verbose). -- MOS_MIGRATED_REVID=135846847
* Allow Skyframe graph lookups and value retrievals to throw InterruptedException.Gravatar Janak Ramakrishnan2016-08-16
| | | | | | | The only place we now don't handle InterruptedException is in the action graph created after analysis, since I'm not sure that will be around for that much longer. -- MOS_MIGRATED_REVID=130327770
* Fix typo.Gravatar Nathan Harmata2016-05-27
| | | | | -- MOS_MIGRATED_REVID=123365501
* Tolerate NoSuchPackageException when processing subdirectories.Gravatar Janak Ramakrishnan2016-04-18
| | | | | | | | | | | In RecursiveDirectoryTraversalFunction, we must tolerate NoSuchPackageException being thrown by subdirectories' nodes, since that can happen in a nokeep_going build. -- Change-Id: Id9a48256aa209775f27130186c58e03c788d20a9 Reviewed-on: https://bazel-review.googlesource.com/#/c/3392/5 MOS_MIGRATED_REVID=120081575
* Fix SkyQuery bug where we weren't respecting the package blacklist. We do ↵Gravatar Nathan Harmata2016-04-18
| | | | | | | | | this by changing both the relevant Skyframe and the SkyQuery code to propagate (minimal!) blacklist information in the SkyKeys themselves. There are other approaches to solving this problem, but I like how this solution doesn't involve duplication of logic. Also, it has the following nice benefit: previously, RecursiveDirectoryTraversalFunction would declare a dep on the blacklist for every directory traversed which adds an edge for each directory traversed. -- MOS_MIGRATED_REVID=120049635
* Batch package and subdirectory requests -- might as well avoid serializing ↵Gravatar Janak Ramakrishnan2016-04-12
| | | | | | | requests wherever we can. -- MOS_MIGRATED_REVID=119531512
* Don't keep packages in the default repository around after loading.Gravatar Brian Silverman2016-03-17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Previously, this would get thrown when referring to the same package from both the main and default repositories: java.lang.IllegalArgumentException: Multiple entries with same key: tools/cpp=/home/brian/971-Robot-Code and tools/cpp=/home/brian/971-Robot-Code at com.google.common.collect.ImmutableMap.checkNoConflict(ImmutableMap.java:136) at com.google.common.collect.RegularImmutableMap.checkNoConflictInKeyBucket(RegularImmutableMap.java:98) at com.google.common.collect.RegularImmutableMap.fromEntryArray(RegularImmutableMap.java:84) at com.google.common.collect.ImmutableMap$Builder.build(ImmutableMap.java:295) at com.google.devtools.build.lib.buildtool.BuildTool.transformPackageRoots(BuildTool.java:301) at com.google.devtools.build.lib.buildtool.BuildTool.buildTargets(BuildTool.java:209) at com.google.devtools.build.lib.buildtool.BuildTool.processRequest(BuildTool.java:334) at com.google.devtools.build.lib.runtime.commands.TestCommand.doTest(TestCommand.java:119) at com.google.devtools.build.lib.runtime.commands.TestCommand.exec(TestCommand.java:104) at com.google.devtools.build.lib.runtime.BlazeCommandDispatcher.exec(BlazeCommandDispatcher.java:371) at com.google.devtools.build.lib.runtime.BlazeRuntime$3.exec(BlazeRuntime.java:1016) at com.google.devtools.build.lib.server.RPCService.executeRequest(RPCService.java:65) at com.google.devtools.build.lib.server.RPCServer.executeRequest(RPCServer.java:434) at com.google.devtools.build.lib.server.RPCServer.serve(RPCServer.java:229) at com.google.devtools.build.lib.runtime.BlazeRuntime.serverMain(BlazeRuntime.java:975) at com.google.devtools.build.lib.runtime.BlazeRuntime.main(BlazeRuntime.java:772) at com.google.devtools.build.lib.bazel.BazelMain.main(BazelMain.java:55) And this would get thrown for any packages in the main repository loaded from other repositories: java.lang.RuntimeException: Unrecoverable error while evaluating node 'PACKAGE:@//tools/build_rules/go/toolchain' (requested by nodes ) at com.google.devtools.build.skyframe.ParallelEvaluator$Evaluate.run(ParallelEvaluator.java:982) at com.google.devtools.build.lib.concurrent.AbstractQueueVisitor$WrappedRunnable.run(AbstractQueueVisitor.java:499) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.IllegalArgumentException: Invalid BUILD file name for package '@//tools/build_rules/go/toolchain': /home/brian/bazel/tools/build_rules/go/toolchain/BUILD at com.google.devtools.build.lib.packages.Package.finishInit(Package.java:299) at com.google.devtools.build.lib.packages.Package$Builder.finishBuild(Package.java:1308) at com.google.devtools.build.lib.skyframe.PackageFunction.compute(PackageFunction.java:501) at com.google.devtools.build.skyframe.ParallelEvaluator$Evaluate.run(ParallelEvaluator.java:933) ... 4 more Sponsor's comment: note the abundance of new Label.resolveRepositoryRelative() calls. They are ugly, but it's only making existing ugliness explicit. Yes, we should fix it, especially in the implementation of configurable attributes. Refs #940 -- Change-Id: I8bd7f7b00bec58a7157507595421bc50c81b404c Reviewed-on: https://bazel-review.googlesource.com/#/c/2591 MOS_MIGRATED_REVID=117429733
* Move RepositoryName to its own top-level classGravatar Kristina Chodorow2016-01-26
| | | | | | | | | | As it's about to get more important in its own right. The only change that isn't just moving code around is making RepositoryName.validate package-private. -- MOS_MIGRATED_REVID=112956571
* Use Bazel Preconditions variant which avoids varargs array creationGravatar Mark Schaller2015-12-10
| | | | | | | Reduces garbage. -- MOS_MIGRATED_REVID=109914243
* Do not recurse into the convenience symlinks when evaluating the "//..." ↵Gravatar Lukacs Berki2015-11-20
| | | | | | | | | target pattern. Apart from the tests, I also tested this manually running "bazel query //..." in a tree with convenience symlinks. -- MOS_MIGRATED_REVID=108325454
* RELNOTES: Symlink dirents of directories containing a file named ↵Gravatar Nathan Harmata2015-11-13
| | | | | | | | | | | | | | "DONT_FOLLOW_SYMLINKS_WHEN_TRAVERSING_THIS_DIRECTORY_VIA_A_RECURSIVE_TARGET_PATTERN" will *not* be traversed for transitive target patterns. The motivation here is to allow directories that intentionally contain wonky symlinks (e.g. foo/bar -> foo) to opt out of being consumed by Blaze. For example, given <workspace>/foo bar bad -> . DONT_FOLLOW_SYMLINKS_WHEN_TRAVERSING_THIS_DIRECTORY_VIA_A_RECURSIVE_TARGET_PATTERN the 'bad' symlink will *not* be traversed by the pattern '//foo/...'. -- MOS_MIGRATED_REVID=107738930
* Allow package blacklisting to be done via a file checked into the depot. By ↵Gravatar Eric Fellheimer2015-11-12
| | | | | | | default this is disabled. -- MOS_MIGRATED_REVID=107644420
* Add a Skyframe injected value representing a Blacklist of package patterns. ↵Gravatar Eric Fellheimer2015-11-10
| | | | | | | | | | | | | Any Package name starting with a blacklisted prefix is treated as a deleted package. Note the relation to prior art in Blaze: 1. Deleted packages indicates to Blaze to pretend certain packages do not exist. The deleted packages are passed in as fully resolved package names. This is important because the invalidation logic when these values changes references the exact package names, as opposed to prefix patterns. 2. Negative patterns in target parsing. (eg, "foo/..., -foo/bar/..."). Note that if //foo:biz depends on //foo/bar:bang, we still visit that dependency. In other words, the negative patterns just control the target pattern parsing, but not the later evaluation of those targets. Blacklisting acts like both (1) and (2), but in the form of a Precomputed injected value (so modifying the value would be more expensive than modifying deleted_packages). -- MOS_MIGRATED_REVID=107431993
* Avoid some Skyframe restarts.Gravatar Nathan Harmata2015-10-19
| | | | | -- MOS_MIGRATED_REVID=105648425
* Make the /... recursive package target wildcard traverse symlinks.Gravatar Lukacs Berki2015-10-13
| | | | | | | | | This works because we resolve the FileValue for every directory found during the traversal, therefore, symlink cycles and symlink structures giving rise to infinite directory trees are handled correctly (that is to say, the same way as they are handled during globbing) Fixes #274. -- MOS_MIGRATED_REVID=105290852
* Intern PackageIdentifiers as a memory optimization.Gravatar Miguel Alcon Pinto2015-10-01
| | | | | -- MOS_MIGRATED_REVID=104403040
* Remove unnecessary env.valuesMissing() call.Gravatar Nathan Harmata2015-09-28
| | | | | -- MOS_MIGRATED_REVID=103975426
* Rationalize copyright headersGravatar Damien Martin-Guillerez2015-09-25
| | | | | | | | | | | The headers were modified with `find . -type f -exec 'sed' '-Ei' 's|Copyright 201([45]) Google|Copyright 201\1 The Bazel Authors|' '{}' ';'` And manual edit for not Google owned copyright. Because of the nature of ijar, I did not modified the header of file owned by Alan Donovan. The list of authors were extracted from the git log. It is missing older Google contributors that can be added on-demand. -- MOS_MIGRATED_REVID=103938715
* Thread the repository name all the way from TargetPatternResolver to ↵Gravatar Lukacs Berki2015-09-22
| | | | | | | | | RecursivePkgFunction. This introduces some redundancy with RootedPath, but only in the case of remote repositories. There doesn't seem to be a good way of removing this redundancy. -- MOS_MIGRATED_REVID=103621610
* Stop throwing an exception if a Package was successfully created but ↵Gravatar Janak Ramakrishnan2015-09-17
| | | | | | | | | | | contains errors. Instead, require callers to process the package and throw if they need to. This allows us to avoid embedding a Package in an exception, which is icky. This also allows us to remove Package#containsTemporaryErrors. Most callers' changes are fairly straightforward. The exception is EnvironmentBackedRecursivePackageProvider, which cannot throw an exception of its own in case of a package with errors (because it doesn't do that in keep_going mode), but whose request for a package with errors *should* shut down the build in case of nokeep_going mode. To do this in Skyframe, we have a new PackageErrorFunction which is to be called only in this situation, and will unconditionally throw. EnvironmentBackedRecursivePackageProvider can then catch this exception and continue on as usual, except that the exception will shut down the thread pool in a nokeep_going build. -- MOS_MIGRATED_REVID=103247761
* Fix some warnings.Gravatar Ulf Adams2015-09-04
| | | | | -- MOS_MIGRATED_REVID=102332437
* Move PackageIdentifier to cmdlineGravatar Kristina Chodorow2015-08-31
| | | | | | | | This is necessary to have TargetResolver depend on it without making it depend on the packages target. First step of #389. -- MOS_MIGRATED_REVID=101790345
* Elegantly handle unbounded file symlink resolutions, e.g. 'a' -> 'b' -> ↵Gravatar Nathan Harmata2015-07-29
| | | | | | | 'a/nope'. -- MOS_MIGRATED_REVID=99337668
* Log error description, not implementation detailsGravatar Mark Schaller2015-07-10
| | | | | | | | RecursiveDirectoryTraversalFunction should provide a description of what went wrong to the user instead of logging class names. -- MOS_MIGRATED_REVID=97878795
* Handle errors from DirectoryListingFunctionGravatar Mark Schaller2015-07-10
| | | | | | | | | | | | | | | | | | | SkyFunctions that call DirectoryListingFunction directly, as in the case of RecursiveDirectoryTraversalFunction, or transitively, as in the case of IncludeParser's call to GlobFunction, had been failing to handle the exceptions that DirectoryListingFunction can throw. DirectoryListingFunction can throw InconsistentFilesystemException directly, but it can also throw IOException and FileOutsidePackageRootsException because of its call to DirectoryListingStateFunction without any of its own error handling. RecursiveFilesystemTraversalFunction also calls DirectoryListingFunction, but is not yet in use. A follow-up CL will take care of its error handling needs. -- MOS_MIGRATED_REVID=97828177
* Extract RecursiveDirectoryTraversalFunction from RecursivePkgFunctionGravatar Mark Schaller2015-06-30
RecursivePkgFunction has a nice framework for doing work across a directory structure that would be nice to have access to when writing other similar SkyFunctions. This extracts that general framework, and changes RecursivePkgFunction into a specialization of it. -- MOS_MIGRATED_REVID=97231974
generated by cgit v1.2.3 (git 2.39.1) at 2024-07-08 14:33:20 +0000