aboutsummaryrefslogtreecommitdiffhomepage
path: root/src/main/java/com/google/devtools/build/lib/skyframe
Commit message (Collapse)AuthorAge
...
* Support declared providers for aspectsGravatar Vladimir Moskva2017-03-06
| | | | | | | | Fixes #2016 -- PiperOrigin-RevId: 149102037 MOS_MIGRATED_REVID=149102037
* Rationalize local resource acquisitionGravatar Ulf Adams2017-03-03
| | | | | | | | | | | | | | | | | | | | | | Move all local resource acquisition to where local execution actually happens. Don't attempt to acquire resources per action, but only for individual spawns. This significantly simplifies the code. The downside is that we don't account for action-level work anymore. In general, actions should not perform any process execution themselves, but always delegate such work to a SpawnStrategy implementation. This change makes sure that every Spawn has local resources set in a way that is consistent with the previous state. However, there are two actions - Fileset and FileWrite -, which are not spawns, and so we now don't limit their concurrent execution anymore. For Fileset, all work is done in a custom Fileset-specific thread pool, so this shouldn't be a problem. I'm not sure about FileWriteAction. -- PiperOrigin-RevId: 149012600 MOS_MIGRATED_REVID=149012600
* Rollback of commit 34ffc4cab1b6cdde54e5945427b79c24fff39aa5.Gravatar Dmitry Lomov2017-03-02
| | | | | | | | | | | | | | *** Reason for rollback *** Roll-forward of commit 01120026dc313ee7ad9ea95069a29252eb19173b with fix. *** Original change description *** Automated [] rollback of commit 01120026dc313ee7ad9ea95069a29252eb19173b. -- PiperOrigin-RevId: 148897534 MOS_MIGRATED_REVID=148897534
* BEP: Report pattern expansion failures Gravatar Klaus Aehlig2017-03-01
| | | | | | | | | | | If expanding a pattern fails, report this on the build event protocol; also include details of what happened. -- Change-Id: I2bc9caf7c085911b80551d7892cc34f5e9961c7b Reviewed-on: https://cr.bazel.build/8795 PiperOrigin-RevId: 148894326 MOS_MIGRATED_REVID=148894326
* Rollback of commit 01120026dc313ee7ad9ea95069a29252eb19173b.Gravatar Jon Brandvein2017-03-01
| | | | | | -- PiperOrigin-RevId: 148888469 MOS_MIGRATED_REVID=148888469
* Do not execute aspect implementation if target advertizes but fails to ↵Gravatar Dmitry Lomov2017-03-01
| | | | | | | | | | | provide a provider. Previously we always executed the function, but didn't add the aspect to the deps. -- PiperOrigin-RevId: 148887089 MOS_MIGRATED_REVID=148887089
* Export exportable values as we go instead of at the end of evaluation.Gravatar Dmitry Lomov2017-03-01
| | | | | | | | | This simplifies implementation of attr.* functions and helps make declared provider implementation simpler. -- PiperOrigin-RevId: 148867326 MOS_MIGRATED_REVID=148867326
* Rollback of commit 822c37816ac669e51bec3853b41849a19ec5e230.Gravatar Nathan Harmata2017-03-01
| | | | | | -- PiperOrigin-RevId: 148844518 MOS_MIGRATED_REVID=148844518
* Add check that the WORKSPACE found in LocalRepositoryLookupFunction isn't a ↵Gravatar John Cater2017-02-28
| | | | | | | | | | | | directory. Fixes #2580. -- Change-Id: I6e1271a9c1442819e956f7fcdcb5da535ed027db Reviewed-on: https://cr.bazel.build/9110 PiperOrigin-RevId: 148751072 MOS_MIGRATED_REVID=148751072
* Clean up the semantics of input discovering actions a bit by making ↵Gravatar Lukacs Berki2017-02-28
| | | | | | | | | | | | updateInputs() and inputsKnown() non-overridable and removing setInputs(). This comes at the cost of adding a flag to every action instance that's not used for non-input-discovering actions, but I think that's a deal. Simpler APIs are good, mmmmkay? Also fixed a few pre-existing issues in TestAction and ObjcCompileAction. -- PiperOrigin-RevId: 148749734 MOS_MIGRATED_REVID=148749734
* Reimplement blaze query using an async evaluation model. Use a concurrent ↵Gravatar Nathan Harmata2017-02-28
| | | | | | | | | | | | | | | | | | | backend for SkyQueryEnvironment's implementation in order to achieve parallelism. Advantages: -New design has no flaws that the old design had. -Code is structured so that deadlocks due to thread starvation are impossible (yup!). Disadvantages: -The meat of this change needs to all be in a single CL because every single QueryFunction and QueryExpression needs to be rewritten in the async style. Still TODO: -Fully embrace the async model in all QueryFunctions (e.g. 'rdeps', 'allpaths'). -Use concurrency in BlazeQueryEnvironment to achieve parallel evaluation for (non SkyQuery) 'blaze query' and genquery. -- PiperOrigin-RevId: 148690279 MOS_MIGRATED_REVID=148690279
* Clean up event busGravatar Klaus Aehlig2017-02-27
| | | | | | | | ...now that the ExtendedEventHandler can serve the same purpose. -- PiperOrigin-RevId: 148640618 MOS_MIGRATED_REVID=148640618
* Provide more reporting options to SkyFunctions Gravatar Klaus Aehlig2017-02-27
| | | | | | | | | | | | | With more specific information to be reported by Skyfunctions, e.g., to inform the build-event protocol on missing files, the EventHandler interface is no longer enough. Therefore, provide an enriched context for reporting events. -- Change-Id: I2d06166fe4d5b9054e24ad8c752fafc039e3f9f8 Reviewed-on: https://cr.bazel.build/8794 PiperOrigin-RevId: 148463437 MOS_MIGRATED_REVID=148463437
* Fix Cpp action cachingGravatar Ulf Adams2017-02-24
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This combines both previous changes and extends them to work both with and without kchodorow@'s rollout of the exec root rearrangement. Unfortunately, each of these changes individually breaks something somewhere, so they must all go into a single commit. Change 1: CppCompileAction must return false from inputsKnown for .d pruning This is necessary (but not sufficient) for the action cache to work correctly. Consider the following sequence of events: 1. action is executed 2. .d pruning is performed 3. action cache writes entry with post-pruning inputs list 4. action gets regenerated (e.g., due to server restart) 5. the action cache calls inputsKnown(), which returns true 6. action cache checks entry from step 3 against pre-pruning inputs list, which results in a cache miss The action cache needs to know that the current list is not the final list, so inputsKnown() in step 5 must return false if .d pruning is enabled. Change 2: Fix artifact root discovery for external artifacts The SkyframeExecutor was assuming that all exec paths were coming from the main repository. Instead, rely on external exec paths to start with "../". Additional change 3: In addition, update the PackageRootResolverWithEnvironment and the HeaderDiscovery to use the single unified repository name guessing implementation. Previously, the PackageRootResolverWithEnvironment was poisoning the source artifact cache, which then caused subsequent lookups to return a bad artifact. Add a precondition to double-check that artifacts looked up by exec path have the correct source root. For compatibility with kchodorow@'s upcoming exec root refactor, if the exec path starts with "external", then assume it's coming from an external repository. This must be removed when that change is successfully rolled out, or it will break if anyone creates a package called 'external'. Additional change 4: On top of all of that, PackageRootResolverWithEnvironment and SkyframeExecutor must use the same source root computation as the Package class itself. I extracted the corresponding code to Root, and added comments both there and in Package to clearly indicate that these methods have to always be modified in sync. Fixes #2490. -- PiperOrigin-RevId: 148439309 MOS_MIGRATED_REVID=148439309
* Report inconsistent aspect order error to the user.Gravatar Dmitry Lomov2017-02-24
| | | | | | -- PiperOrigin-RevId: 148342788 MOS_MIGRATED_REVID=148342788
* Fix broken javadoc link.Gravatar Shreya Bhattarai2017-02-23
| | | | | | -- PiperOrigin-RevId: 148253141 MOS_MIGRATED_REVID=148253141
* Allow absolute symlinks in TreeArtifacts. Let the downstream action (if ↵Gravatar Janak Ramakrishnan2017-02-23
| | | | | | | | there is one) worry about it. -- PiperOrigin-RevId: 148249223 MOS_MIGRATED_REVID=148249223
* Rollback of commit 4b73e972d909bcd533f2f9940f95a00b9b73bdde.Gravatar Dmitry Lomov2017-02-17
| | | | | | | | | | | | | | | | | | | | | | | *** Reason for rollback *** Broke tests on CI: http://ci.bazel.io/job/bazel-tests/570/ *** Original change description *** Roll forward execroot change RELNOTES[INC]: Previously, an external repository would be symlinked into the execution root at execroot/local_repo/external/remote_repo. This changes it to be at execroot/remote_repo. This may break genrules/Skylark actions that hardcode execution root paths. If this causes breakages for you, ensure that genrules are using $(location :target) to access files and Skylark rules are using http://bazel.io/docs/skylark/lib/File.html's path, dirname, etc. functions. Cust... -- PiperOrigin-RevId: 147833177 MOS_MIGRATED_REVID=147833177
* Make the interface dynamic library builder a regular label instead of a ↵Gravatar Lukacs Berki2017-02-17
| | | | | | | | magic artifact. -- PiperOrigin-RevId: 147830857 MOS_MIGRATED_REVID=147830857
* Roll forward execroot changeGravatar Kristina Chodorow2017-02-16
| | | | | | | | | | | | | | | | | RELNOTES[INC]: Previously, an external repository would be symlinked into the execution root at execroot/local_repo/external/remote_repo. This changes it to be at execroot/remote_repo. This may break genrules/Skylark actions that hardcode execution root paths. If this causes breakages for you, ensure that genrules are using $(location :target) to access files and Skylark rules are using http://bazel.io/docs/skylark/lib/File.html's path, dirname, etc. functions. Custom crosstools that hardcode external/<repo> paths will have to be updated. Issue #1262. -- PiperOrigin-RevId: 147726370 MOS_MIGRATED_REVID=147726370
* Fix a hypothetical issue with 'buildfiles' and 'loadfiles' with duplicate ↵Gravatar Nathan Harmata2017-02-16
| | | | | | | | | | | | | | | | | | targets. These two functions don't use a Uniquifier internally so if the same bzl file foo.bzl is loaded multiple ways in 'e' in 'loadfiles(e)' then 'f' in 'f(loadfiles(e))' will observe foo.bzl multiple times. This isn't observable in practice (i.e. I cannot write a black-box query test that would fail without this CL) because: (1) BlazeQueryEnvironment#eval(QueryExpression, VariableContext<Target>, Callback<Target>) uses an intermediate aggregating callback and thus doesn't use the streaming query evaluation model. This means that the internal deduping in BlazeQueryEnvironment#getBuildFiles is sufficient. (2) SkyQueryEnvironment uses an outer BatchStreamedCallback which internally uses a Uniquifier. Still, this CL is useful for SkyQuery because without we're doing wasted work (e.g. in my example above, 'f' will wastefully do duplicate work with the .bzl files from the inner loadfiles(e)) because the deduping happens on the final query result, not on the intermediate result from 'buildfiles'/'loadfiles'. For 'buildfiles', there's an additional wart with (1) above because FakeSubincludeTarget doesn't override Object#equals/hashCode. -- PiperOrigin-RevId: 147656183 MOS_MIGRATED_REVID=147656183
* Loosen visibility of ASTFileLookupValue.Gravatar Janak Ramakrishnan2017-02-15
| | | | | | -- PiperOrigin-RevId: 147559691 MOS_MIGRATED_REVID=147559691
* Restrict aspects visible to other aspects according to their advertised ↵Gravatar Dmitry Lomov2017-02-15
| | | | | | | | providers. -- PiperOrigin-RevId: 147526961 MOS_MIGRATED_REVID=147526961
* Some minor logging improvements related to diff awareness:Gravatar Nathan Harmata2017-02-15
| | | | | | | | | | | | (i) Log [the first 100] external [non external-repo] file paths when we encounter them in skyframe [when externalFileAction is not ASSUME_NON_EXISTENT_AND_IMMUTABLE_FOR_EXTERNAL_PATHS]. These caveats are to prevent log spam. (ii) Fix grammar when logging the results of handleDiffsWithMissingDiffInformation in the case when all package paths have known diff information and we're merely checking external/output/external-repo files. (iii) Log the purpose of each skyframe graph scan we do (see (ii)). -- PiperOrigin-RevId: 147508404 MOS_MIGRATED_REVID=147508404
* Fix IllegalStateException when loading from a non declared external repository Gravatar Damien Martin-Guillerez2017-02-14
| | | | | | | | | | | This is another instance of #1793 that happens when a WORKSPACE file exists in some path underneath your current workspace. -- Change-Id: Idb01cd643548a170a27c9103f1b5081b058cc005 Reviewed-on: https://cr.bazel.build/8143 PiperOrigin-RevId: 147353935 MOS_MIGRATED_REVID=147353935
* Fix PackageProgressReceiver to not call contains on a list.Gravatar Janak Ramakrishnan2017-02-13
| | | | | | -- PiperOrigin-RevId: 147323591 MOS_MIGRATED_REVID=147323591
* Add @Nullable annotations to PackageFunction#progressReceiver.Gravatar Janak Ramakrishnan2017-02-13
| | | | | | -- PiperOrigin-RevId: 147202629 MOS_MIGRATED_REVID=147202629
* Introduce CppCompileActionTemplate, which expands into a list of ↵Gravatar Rumou Duan2017-02-10
| | | | | | | | CppCompileActions that to be executed at execution time. -- PiperOrigin-RevId: 147163077 MOS_MIGRATED_REVID=147163077
* Tighten a precondition check during action execution.Gravatar Lukacs Berki2017-02-10
| | | | | | | | Input discovery is handled somewhere else, so the same Action instance should never be put twice into buildActionMap . -- PiperOrigin-RevId: 147132869 MOS_MIGRATED_REVID=147132869
* Remove dead code - unused method in PackageManager interfaceGravatar Ulf Adams2017-02-09
| | | | | | -- PiperOrigin-RevId: 147018240 MOS_MIGRATED_REVID=147018240
* Fix spelling mistake.Gravatar Nathan Harmata2017-02-09
| | | | | | -- PiperOrigin-RevId: 146949259 MOS_MIGRATED_REVID=146949259
* Introduce a new SkyValue that merely contains the workspace name. The ↵Gravatar Nathan Harmata2017-02-09
| | | | | | | | | | | | workspace name is needed for package loading, and so splitting out this computation into a separate skyframe node that can be change-pruned gives us better incrementality; previously we'd need to reload all packages on a WORKSPACE file change. N.B. (i) This CL doesn't solve all the other performance issues with //external in Bazel/Blaze since it's still inefficiently used for resolving labels like @foo//bar:baz. (ii) This CL doesn't address the wasteful invalidation + change pruning of all the packages. -- PiperOrigin-RevId: 146925369 MOS_MIGRATED_REVID=146925369
* Introduce a Skyframe function: ActionEnvironmentFunction Gravatar Damien Martin-Guillerez2017-02-08
| | | | | | | | | | | | | | | | | | ActionEnvironmentFunction returns the list of environment variable with the one overwritten by --action_env being replaced. This let other Skyframe function declares dependency to any value of the environment and being influenced by the --action_env flag. This will be used to declare dependency of remote repositories on environment variables (step 3 of https://bazel.build/designs/2016/10/18/repository-invalidation.html) -- Change-Id: I1ed3fb6f48e8e17d4d64c903fccecb6ed7596350 Reviewed-on: https://cr.bazel.build/7974 PiperOrigin-RevId: 146918603 MOS_MIGRATED_REVID=146918603
* Remove dead code in SkyframeLabelVisitorGravatar Ulf Adams2017-02-07
| | | | | | | | | | Two of the functions are never used, several of the parameter are not used or can be simplified (require callers to convert from Target -> Label). -- PiperOrigin-RevId: 146787380 MOS_MIGRATED_REVID=146787380
* Adds abstractions for include processing to CppSemanticsGravatar Googler2017-02-07
| | | | | | -- PiperOrigin-RevId: 146694092 MOS_MIGRATED_REVID=146694092
* Improve the error message added in commit ↵Gravatar Lukacs Berki2017-02-02
| | | | | | | | 41ab6db797bac30c8e39fc705254fef329b6fdbc . -- PiperOrigin-RevId: 146341606 MOS_MIGRATED_REVID=146341606
* Inject the changes in the client environment for the ClientEnvironmentFunction Gravatar Damien Martin-Guillerez2017-02-02
| | | | | | | | | | | | Using a precomputed values prevent us from being able to depends on any environment variable (or we would invalidate the whole graph each time the environment change, that is each time we run a command). -- Change-Id: If322df4240271d04b9278e1222f936156531580c Reviewed-on: https://cr.bazel.build/8400 PiperOrigin-RevId: 146298592 MOS_MIGRATED_REVID=146298592
* Simplify action input discovery by removing ↵Gravatar Lukacs Berki2017-01-31
| | | | | | | | | | Action#getInputsWhenSkippingInputDiscovery(). This has the side effect of always allowing new inputs to be discovered when Action#discoversInputs() return true, but since those actions are very few and have mostly to do with C++, I think it's an acceptable tradeoff for th ecode being that much simpler. -- PiperOrigin-RevId: 146098150 MOS_MIGRATED_REVID=146098150
* Fix error handling in WorkspaceFile/WorkspaceASTGravatar Kristina Chodorow2017-01-30
| | | | | | -- PiperOrigin-RevId: 145818492 MOS_MIGRATED_REVID=145818492
* Add dependency for external files to the corresponding RepositoryDirectoryValue Gravatar Damien Martin-Guillerez2017-01-26
| | | | | | | | | | | | | | | | Previously we were adding a dependency to the rule itself, however the repository itself can depends on further dependencies (environment, template files, ...) with Skylark repositories. Not invalidating when those change was causing weird invalidation issues that were shown with the invalidation on environment (basically a refetch of the repository on environment invalidation would not cause an invalidation of the corresponding package, the next call would not have invalidation of the repository but would see the change in the build files...). -- Change-Id: I8945d9885e6390734ba02ccc6c3c6ca639fcec35 Reviewed-on: https://cr.bazel.build/8137 PiperOrigin-RevId: 145675258 MOS_MIGRATED_REVID=145675258
* Support mapping of Paths to URIsGravatar Klaus Aehlig2017-01-18
| | | | | | | | | | | Bazel-created files (like log files of test runs) are internally reported as Paths. However, this is not always the most useful representation of the location of that artifact for a consumer of build events. Therefore, support a mapping of paths to more useful URIs. -- PiperOrigin-RevId: 144843525 MOS_MIGRATED_REVID=144843525
* Change TargetContext.findDirectPrerequisite from config1 == config2 to ↵Gravatar Greg Estren2017-01-18
| | | | | | | | | | config1.equals(c2). This fixes an obscure bug between dynamic configurations, host configuration caching, and Skyframe skyKey interning that makes Bazel crash under certain sequences of builds. See changes for details. -- PiperOrigin-RevId: 144766296 MOS_MIGRATED_REVID=144766296
* Be very deliberate about the concept of a TargetPattern's "directory".Gravatar Nathan Harmata2017-01-16
| | | | | | -- PiperOrigin-RevId: 144494739 MOS_MIGRATED_REVID=144494739
* Fix a null check in ActionMetadataHandler.Gravatar Kush Chakraborty2017-01-16
| | | | | | -- PiperOrigin-RevId: 144444493 MOS_MIGRATED_REVID=144444493
* Log the IOException when an output was not successfully created. It's ↵Gravatar Janak Ramakrishnan2017-01-13
| | | | | | | | possible the exception has information we've been suppressing. -- PiperOrigin-RevId: 144405370 MOS_MIGRATED_REVID=144405370
* Log if dangling symlink is found. This is an odd choice of thing to log, but ↵Gravatar Janak Ramakrishnan2017-01-13
| | | | | | | | shouldn't come up very often, except in the linked bug. -- PiperOrigin-RevId: 144367142 MOS_MIGRATED_REVID=144367142
* Encapsulate the required provider logic in RequiredProviders class.Gravatar Dmitry Lomov2017-01-10
| | | | | | | | | For now, only for aspects, but eventually expand to Attribute's mandatory providers as well. -- PiperOrigin-RevId: 144063369 MOS_MIGRATED_REVID=144063369
* Extract a set of advertised providers into a separate class.Gravatar Dmitry Lomov2017-01-10
| | | | | | -- PiperOrigin-RevId: 143991903 MOS_MIGRATED_REVID=143991903
* Provide Skylark imports, even if the package has some errors.Gravatar Laurent Le Brun2017-01-05
| | | | | | | | | | | Since we evaluate the BUILD file despite errors (e.g. parse errors), we should provide the .bzl dependencies to the BUILD file. This change removes some confusing/irrelevant error messages. -- PiperOrigin-RevId: 143696291 MOS_MIGRATED_REVID=143696291
* Fix cycle detection between .bzl files.Gravatar Laurent Le Brun2017-01-05
| | | | | | | | | The code assumed that the last element before a cycle was a BUILD file. It can also be a .bzl file. -- PiperOrigin-RevId: 143673940 MOS_MIGRATED_REVID=143673940
generated by cgit v1.2.3 (git 2.39.1) at 2024-07-08 15:48:27 +0000