aboutsummaryrefslogtreecommitdiffhomepage
path: root/src/main/java/com/google/devtools/build/lib/skyframe
Commit message (Collapse)AuthorAge
* Make unpredictable action inputs and outputs available to Actions.Gravatar Michael Thvedt2016-02-10
| | | | | -- MOS_MIGRATED_REVID=114249806
* Split the execution of the WORKSPACE file after each load statementGravatar Damien Martin-Guillerez2016-02-10
| | | | | | | | | | | | This is the main step toward supporting load statement of remote repository in the WORKSPACE file (bug #824). Load statement that refers to a remote repository will depend on the previous fragment of the workspace file. Issue #824 Step 3. -- MOS_MIGRATED_REVID=114237027
* Skyframe-based loading phase runner: report errors if a package is in error.Gravatar Ulf Adams2016-02-10
| | | | | | | | | | | | | | | | | | | | | This is a bit odd - the legacy loading phase runner reports a loading error, but no target pattern error in keep_going mode, even though it's clearly an error in the referenced target itself, rather than in its transitive closure. This happens because the target pattern eval swallows such errors in keep_going mode, and doesn't even report the error. I tried changing that, but it's a fairly large refactoring, and that code path is dead if we switch to the new one. In the Skyframe-based implementation, both keep_going and nokeep_going paths now report a target pattern error. (Note that the new code can never report a loading error, because it doesn't perform transitive loading.) The corresponding test is moved from SkyframeLoadingAndAnalysisTest to LoadingPhaseRunnerTest - we don't need any integration test setup for that. -- MOS_MIGRATED_REVID=114236897
* Fix NPE in TestsInSuiteFunction.Gravatar Ulf Adams2016-02-10
| | | | | | | | Note that this only affects the new Skyframe-based implementation of the LoadingPhaseRunner, which isn't live yet. -- MOS_MIGRATED_REVID=114230017
* Properly report loading errors during configuration creation.Gravatar Ulf Adams2016-02-10
| | | | | | | | | | | | | | | | | This only applies to interleaved loading and analysis - the production code is fine. Add tests for the RedirectChaser, the fdoOptimize code path, the XcodeConfig, and the Jvm loader. Unfortunately, the configuration factory we internally create by default contains a mock Jvm loader implementation. Since that is one Yak too many right now, I'm adding a temporary method to the AnalysisMock. I added the tests to BuildViewTest for now; technically, they ought to go into the language-specific test cases, but that would require more refactoring as those don't currently run with interleaved loading and analysis. -- MOS_MIGRATED_REVID=114221476
* Make The Build Faster: Use AutoCloseable when dealing with resources.Gravatar Philipp Wollermann2016-02-10
| | | | | -- MOS_MIGRATED_REVID=114204404
* Update FileSystemValueChecker to handle TreeArtifact values.Gravatar Michael Thvedt2016-02-10
| | | | | -- MOS_MIGRATED_REVID=114202845
* Support for handling TreeArtifact metadata and returning TreeArtifacts from ↵Gravatar Michael Thvedt2016-02-09
| | | | | | | ArtifactFunction. -- MOS_MIGRATED_REVID=114174899
* Rename MiddlemanExpander to ArtifactExpander, and refactor it to yield ↵Gravatar Michael Thvedt2016-02-09
| | | | | | | ArtifactFiles. -- MOS_MIGRATED_REVID=114166208
* Tiny refactoring: extract shared code between -k and -nok.Gravatar Ulf Adams2016-02-08
| | | | | -- MOS_MIGRATED_REVID=114086842
* WorkspaceASTFunction returns a list of ASTs so we can split the AST before ↵Gravatar Damien Martin-Guillerez2016-02-07
| | | | | | | | | load statements Issue #824 Step 2. -- MOS_MIGRATED_REVID=113986176
* Add an intermediate SkyFunction for resolving the external packageGravatar Damien Martin-Guillerez2016-02-07
| | | | | | | | | | | | | | The WORKSPACE file parsing needs to be separated into several parts to enable load of labels in the WORKSPACE file. This change adds an intermediate SkyFunction, ExternalPackageFunction, that requires all the WORKSPACE file part to be parsed to resolve //external: labels. Issue #824 Step 1. -- MOS_MIGRATED_REVID=113984026
* Use a clever hybrid approach for evaluating globs during package loading: ↵Gravatar Nathan Harmata2016-02-05
| | | | | | | | | | | | | | | | | first try to get a skyframe cache-hit; otherwise, fall back to legacy globbing. This gives us the best of both worlds: no extra skyframe restarts on glob-dep-misses, and much better incremental performance in the common case that a package's globs haven't changed. See the class-comment for PackageFunction.SkyframeHybridGlobber for a detailed description and explanation. This CL has no impact on semantics and is a strict performance win. Bazel users: Here's an example benchmark (does an incremental loading phase of a target T but forces all packages to be reloaded): nharmata@nharmata:~/bazel$ N=10; B="output/bazel"; T=//src/main/java/com/google/devtools/build/lib:bazel/BazelServer_deploy.jar; CMD="build --noanalyze $T"; $B clean &> /dev/null; P=base_workspace/tools/build_rules/prelude_bazel; rm $P; touch base_workspace/tools/build_rules/BUILD; $B $CMD &> /dev/null; time for i in $(seq 1 $N); do echo "#hi" >> $P; $B $CMD &> /dev/null; done For a very large internal Google target, this CL improves the benchmark performance by ~6%. A more targeted benchmark would be for loading a single package that has lots of expensive globs. For example, the time to incrementally load a single pathological Google-internal package was reduced by ~36%. Alternatives considered: Introduce skyframe native globbing, gated by flags for both globbing during preprocessing and globbing during regular BUILD file evaluation. The approach in this CL is superior performance-wise. -- MOS_MIGRATED_REVID=113899687
* Move the Globber interface into its own file.Gravatar Nathan Harmata2016-02-05
| | | | | -- MOS_MIGRATED_REVID=113893917
* Fix configuration error handling for the interleaved case.Gravatar Ulf Adams2016-02-04
| | | | | | | | In the interleaved case, loading errors can occur during configuration creation and we need to correctly report such errors in that case. -- MOS_MIGRATED_REVID=113826273
* Have GlobFunction make use of the assumption that the glob's package exists ↵Gravatar Nathan Harmata2016-02-04
| | | | | | | | | by having it not declare a dep on the PackageLookupValue for the package. This optimization means that a BUILD file edit doesn't (necessarily) invalidate all the globs in the package; the PackageLookupValue node would get change-pruned but we still pay the very small cost of invalidating unnecessarily. Also slightly improve variable naming in GlobFunctionTest. -- MOS_MIGRATED_REVID=113799936
* Correctly flag loading errors in the interleaved case.Gravatar Ulf Adams2016-02-03
| | | | | | | | | | | | | | | In the interleaved case, loading errors can now also be discovered during the analysis phase. Add a boolean flag to the SkyframeAnalysisResult to indicate that such an error happened, and pass it through in BuildView. Also refactor BuildView to simplify the code a bit - simply pass the SkyframeAnalysisResult to the createResult method. There is already an integration test for this - I'm adding a faster unit test in BuildViewTest, as this is part of the BuildView contract. -- MOS_MIGRATED_REVID=113716870
* Trim the BuildOptions input of ConfigurationFragment.key to only thoseGravatar Greg Estren2016-02-02
| | | | | | | | | | | | | | | | | | | | | | | | | | options actually needed by the fragment. This protects against, e.g., unnecessarily duplicating CppConfiguration instances when only Java flags change. This is a recommit of ca1b21ac6d8a58041db822725b42de151b163dee which was rolled back because it broke LIPO. This change is particularly important for dynamic configurations, which may mix and match fragments arbitrarily throughout a build. This not only has performance implications, but also correctness implications: code that expects two configured targets to have the same fragment (value) shouldn't break just because the second CT's configuration is a trimmed version of the first's. The original change breaks FDO/LIPO because CppConfiguration can't be shared across configurations. That's because it mutates state when prepareHook() is called, and each configuration calls prepareHook. We should ultimately solve this by refactoring the FDO/LIPO implementation but don't want to block dynamic configuration progress on that. So this change only enables trimming for dynamic configurations. -- MOS_MIGRATED_REVID=113570250
* Move tests from SkyframeLoadingAndAnalysisTest to LoadingPhaseRunnerTest.Gravatar Ulf Adams2016-02-02
| | | | | | | | This exposes an issue with the new Skyframe-based implementation, which is fixed here as well. -- MOS_MIGRATED_REVID=113556169
* Fix "put the colon here" message for non-default repositories.Gravatar Brian Silverman2016-02-02
| | | | | | | -- Change-Id: I86b90ff6eda6a16c2efeb8fffca301e2594c254e Reviewed-on: https://bazel-review.googlesource.com/#/c/2784/ MOS_MIGRATED_REVID=113529297
* Implement proper error handling for interleaved loading and analysis.Gravatar Ulf Adams2016-02-02
| | | | | | | | Add test coverage by re-running BuildViewTest with the new Skyframe loading phase runner. -- MOS_MIGRATED_REVID=113517509
* Parse the workspace name when a repository is loadedGravatar Kristina Chodorow2016-01-29
| | | | | | | | | Moved RepositoryValue to RepositoryDirectoryValue so that it could be cached (and not re-downloaded) even if the WorkspaceAST caused a Skyframe restart (as mentioned in https://github.com/bazelbuild/bazel/blob/master/src/main/java/com/google/devtools/build/lib/rules/repository/RepositoryDelegatorFunction.java#L130-L133). -- MOS_MIGRATED_REVID=113358489
* Small refactoring of SkyframeBuildView to reduce the diff for the next change.Gravatar Ulf Adams2016-01-29
| | | | | -- MOS_MIGRATED_REVID=113338481
* Parallelize Package retrieval during Sky-query operation. To maintain ↵Gravatar Eric Fellheimer2016-01-29
| | | | | | | type-safety, we now must pass in the exception type of the callback. -- MOS_MIGRATED_REVID=113313312
* Clear up some confusion about glob prefetching (the old comment was wrong). ↵Gravatar Nathan Harmata2016-01-29
| | | | | | | | | Also add some TODOs for potentially improving package loading performance. This CL has no semantic impact. -- MOS_MIGRATED_REVID=113301656
* Refactor DependencyResolver to collect and return loading errors.Gravatar Ulf Adams2016-01-28
| | | | | | | | | This should never be triggered in production, where we always run a loading phase first and only analyze targets that load successfully. I.e., this is just plumbing which will be hooked up in a subsequent change. -- MOS_MIGRATED_REVID=113258593
* Fix: return an error from the analysis phase with keep_going in error cases.Gravatar Ulf Adams2016-01-27
| | | | | | | | | | | - if there are failed top-level aspects - if there are conflicting actions I ended up rewriting how errors are signaled from the SkyframeBuildView. I think this is safe, but please review carefully. -- MOS_MIGRATED_REVID=113150100
* Store data about aspect configurations on Dependencies.Gravatar Michael Staib2016-01-27
| | | | | | | | | | | | | | | | | | | | Dependencies are the data structure which needs to propagate the configuration for each aspect as created by trimConfigurations down to the point where it's actually used. We need this to store different configurations for different aspects in a world where aspects have their own configurations, which may have more fragments than the target they're attached to. That world is on its way. Also in this CL: * Refactor Dependency to be an abstract parent class with separate implementations for Attribute.Transitions and BuildConfigurations, as well as null configurations, to avoid having to check nullness in various places. Users of the API will not see this, but get factory methods instead of constructors. As a consequence of this, refactor Dependency to be its own top-level class instead of a nested class in DependencyResolver. -- MOS_MIGRATED_REVID=113109615
* Move RepositoryName to its own top-level classGravatar Kristina Chodorow2016-01-26
| | | | | | | | | | As it's about to get more important in its own right. The only change that isn't just moving code around is making RepositoryName.validate package-private. -- MOS_MIGRATED_REVID=112956571
* Share some error handling code between aspects and top-level aspects.Gravatar Ulf Adams2016-01-25
| | | | | -- MOS_MIGRATED_REVID=112948493
* Make AspectFunction error handling match TopLevelAspectFunction.Gravatar Ulf Adams2016-01-25
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Also match ConfiguredTargetFunction for target loading. It isn't currently possible to trigger either of these code paths - the loading phase ensures that we never attempt to analyze targets that fail to load - the Skylark import or conversion cannot fail, because Skylark checks during .bzl execution that all referenced symbols are Skylark aspects Therefore, the only way to trigger this would be if there was a native rule requesting a non-existent or broken Skylark aspect for its dependencies, but that is currently not possible - native rules can only request native aspects. However, for interleaved loading and analysis, we need to limit the set of exception classes that can be thrown from AspectFunction - we do that here by changing the constructor of AspectFunctionException to only accept either NoSuchThingException or AspectCreationException. That in turn requires that we re-throw the Skylark import and conversion exceptions as AspectCreationException, which is exactly what TopLevelAspectFunction is already doing, and necessary for correct error handling if we do ever support Skylark aspects in native rules. Alternatively, I could change the code path to crash Bazel, but that seems strictly worse. Even if we can't test this code, it's conceptually the right way to handle these errors. I'll move part of the error handling into loadSkylarkAspect in a subsequent change. -- MOS_MIGRATED_REVID=112938284
* Move analysis root cause tracking to the ConfiguredTargetFunction.Gravatar Ulf Adams2016-01-25
| | | | | | | | | | | | | | | | | | | | | | | | | The main remaining problem with interleaved loading and analysis is error handling. When interleaving, we don't run a real loading phase anymore, and loading errors can occur during the analysis phase, and need to be handled there. The plan is to have ConfiguredTargetFunction throw a ConfiguredValueCreationException with a list of all loading root causes, which requires that we also catch ConfiguredValueCreationException here, which in turn breaks analysis root cause handling, as that is currently relying on Skyframe root cause tracking. Moving analysis root cause handling into CTFunction makes it possible to subsequently also implement loading root cause handling here. This is also necessary if we want to have complete root cause handling in the general case: a target may have any number (and combination) of loading and analysis root causes at the same time. For now, we only pass a single analysis root cause, which mirrors the current Skyframe-based implementation. -- MOS_MIGRATED_REVID=112930871
* Use the existing exception instance, not a new one.Gravatar Ulf Adams2016-01-25
| | | | | | | | | | | | | Note that this can never happen in production - the loading phase ensures that we only attempt to analyze targets that load successfully. That's also why there are no tests for this. This code path will become live in a later change, with corresponding test coverage; the existing tests cover this in principle, we just need to make them run with the reduced loading phase. -- MOS_MIGRATED_REVID=112926211
* Allow an exit code to be specified in the ActionExecutionException when an ↵Gravatar Googler2016-01-25
| | | | | | | | | action fails, and propagate the exit code to BuildFailedException. And update all call sites that copy or wrap ActionExecutionExceptions. -- MOS_MIGRATED_REVID=112811857
* Extract parsing of the WORKSPACE file in two partsGravatar Damien Martin-Guillerez2016-01-22
| | | | | | | | | | | The WORKSPACE file AST is now parsed as a separate SkyFunction and this will be used to have multiple SkyValue for the same WORKSPACE file, splitting the execution of the AST after load statements to enable load statement of external dependencies in the WORKSPACE file. -- MOS_MIGRATED_REVID=112768897
* Properly filter out Aspect attributes when `blaze query` requests ↵Gravatar Dmitry Lomov2016-01-22
| | | | | | | | | --noimplicit_deps and/or --nohost_deps. RELNOTES: --noimplicit_deps and --nohost_deps work correctly for Aspect attributes. -- MOS_MIGRATED_REVID=112724917
* Refactor dependency filtering out of Rule class.Gravatar Dmitry Lomov2016-01-22
| | | | | -- MOS_MIGRATED_REVID=112717648
* Perform package loading in parallel with transitive target visitation. This ↵Gravatar Janak Ramakrishnan2016-01-21
| | | | | | | is a partial rollback of commit f1e257d because it turns out that loading sequentially can be a bottleneck. -- MOS_MIGRATED_REVID=112628616
* Fix a couple of bugs related to error handling for top-level aspects.Gravatar Ulf Adams2016-01-20
| | | | | -- MOS_MIGRATED_REVID=112561390
* Support unreadable filesGravatar Michajlo Matijkiw2016-01-20
| | | | | -- MOS_MIGRATED_REVID=112507181
* Allow opaque OptionsClassProvider to be passed around in BuildDriver#meta et ↵Gravatar Nathan Harmata2016-01-20
| | | | | | | all so that custom implementations can have custom options passed around. -- MOS_MIGRATED_REVID=112502778
* General cleanup for the configured target / aspect creation code.Gravatar Ulf Adams2016-01-19
| | | | | | | | | | | | | | - update some comments - add some comments to make it easier to follow - delete some dead code, in particular the SkyframeDependencyResolver can never be null; remove an non-applicable @Nullable annotation I'm trying to figure out how the error handling code works, in order to add support for interleaved loading+analysis, which requires handling loading errors in this code path. -- MOS_MIGRATED_REVID=112456674
* Disallow loads from external repos in WORKSPACE files. Currently, attempting ↵Gravatar John Field2016-01-18
| | | | | | | | | to do such a load will result in a skyframe circular dependency exception. As a side effect of this change, SkylarkImportFailedExceptions thrown by SkylarkImportLookupFunction are now caught by PackageFunction and wrapped and rethrown as a PackageFunctionException. Previously, the first exception wasn't caught, generating an uncaught exception error at top level. -- MOS_MIGRATED_REVID=112328755
* Refactor action output dumping to go through a single method.Gravatar Ulf Adams2016-01-15
| | | | | | | | | | | | | This is in preparation for moving the output dumping to the Reporter for two reasons: - we're currently relying on there being a Reporter which locks on itself; i.e., this code is tightly coupled with the Reporter code - the current Blaze output is confusing; we first print an info event, then the actual output, and then an error event - putting the Reporter in control allows us to fix that -- MOS_MIGRATED_REVID=112240684
* Display parsing errors for WORKSPACE filesGravatar Kristina Chodorow2016-01-15
| | | | | | | | | | | | | Previously, for some types of errors, you'd just see "WORKSPACE file could not be parsed" if there was a syntax error (instead of what the syntax error was). Change-Id: Ia278cf23b79f8daba02917cdfb4cc515a87080b1 -- Change-Id: Ia278cf23b79f8daba02917cdfb4cc515a87080b1 Reviewed-on: https://github.com/bazelbuild/bazel/pull/774 MOS_MIGRATED_REVID=112190322
* Add a few other Label.resolveRepositoryRelative() calls to make select() ↵Gravatar Lukacs Berki2016-01-15
| | | | | | | | | | | statements work in remote repositories. This work is somewhat sisyphean; the principled thing to do would be to just do this resolution when labels are parsed. Fixes #783. -- MOS_MIGRATED_REVID=112137996
* Remove some dead code.Gravatar Ulf Adams2016-01-14
| | | | | | | | | | | | The ConfiguredTargetFunction can only throw a NoSuch{Target,Package}Exception if it's run for a non-existent target. However, it will never request a CT for a non-existent target: all labels are run through DependencyResolver in order to determine the configuration transition, which requires loading the target. If it doesn't exist, the label is never returned from the DependencyResolver, and errors are swallowed (we rely on the loading phase to report the error). -- MOS_MIGRATED_REVID=112127012
* Use AutoProfiler in SkyQueryEnvironment#init. Also augment the ↵Gravatar Nathan Harmata2016-01-14
| | | | | | | WalkableGraphFactory interface with a hook to inform the factory when the WalkableGraph is done being used. -- MOS_MIGRATED_REVID=112074284
* Store excluded subdirectories as PathFragment instead of String inside ↵Gravatar Janak Ramakrishnan2016-01-14
| | | | | | | | | TargetPatternKey, since that is what is needed by callers. Also, since the PathFragments come from packages, they are guaranteed to be well-formed, so the checks we were doing were unnecessary. -- MOS_MIGRATED_REVID=112059930
* Inside findTargetsBeneathDirectory(), invoke the callback with all targets ↵Gravatar Eric Fellheimer2016-01-11
| | | | | | | from the current package batch. -- MOS_MIGRATED_REVID=111858978
generated by cgit v1.2.3 (git 2.39.1) at 2024-08-25 05:49:27 +0000