aboutsummaryrefslogtreecommitdiffhomepage
path: root/src/main/java/com/google/devtools/build/lib/skyframe
Commit message (Collapse)AuthorAge
* Generalize ConfigSetting's special options handling.Gravatar mstaib2017-04-05
| | | | | | | | | | | | | | | | Currently ConfigSetting is treated specially; TransitiveTargetFunction identifies it by name, then runs a special function on it. This change makes ConfigSetting use a new options reference function in RuleClass, which TransitiveTargetFunction will evaluate over the rule and use with the options-to-fragments map to figure out which fragments are needed. Although this is still a hack and not really a new feature on RuleClass, this avoids a dependency on ConfigSettingRule from TransitiveTargetFunction, which is necessary for ConfigSettingRule to be moved to rules/config. RELNOTES: None. PiperOrigin-RevId: 152156905
* Refactor all ctor callsites of PathFragment to instead call a static ↵Gravatar nharmata2017-04-05
| | | | | | | | | | | | 'create' method. This paves the way for changing PathFragment to e.g. an abstract class with multiple subclasses. This way we can split out the windows-specific stuff into one of these concrete classes, making the code more readable and also saving memory (since the shallow heap size of the NonWindowsPathFragment subclass will hopefully be smaller than that of the current PathFragment). This also lets us pursue gc churn optimizations. We can now do interning in PathFragment#create and can also get rid of unnecessary intermediate PathFragment allocations. RELNOTES: None PiperOrigin-RevId: 152145768
* BEP: provide events a generic converter classGravatar aehlig2017-04-04
| | | | | | | | | | | Change the BuildEvent interface to accept a generic class of converters. In this way, we won't have to change it again in the future, once more converters are needed. In fact, a new converter is needed right now (will be added in a follow-up patch) to allow build events to know the name of named artifact groups already reported in the stream. Change-Id: Ibb32ea5fff361e21bcf2d34818d8351a1da7a2e3 PiperOrigin-RevId: 152131870
* Automated g4 rollback of commit 1d9e1ac90197b1d3d7b137ba3c1ada67bb9ba31b.Gravatar hlopko2017-04-04
| | | | | | | | | | | | | | | | | | *** Reason for rollback *** Breaks //src/test/shell/integration:force_delete_output_test *** Original change description *** Symlink output directories to the correct directory name If the workspace directory is /path/to/my/proj and the name in the WORKSPACE file is "floop", this will symlink the output directories to output_base/execroot/floop instead of output_base/execroot/proj. More prep for #1262, fixes #1681. PiperOrigin-RevId: 152126545
* Add "--build_manual_tests" options that forces manual tests to be built.Gravatar dslomov2017-04-04
| | | | | | | | | This is useful for IDEs and other tools utilizing command-line aspects for reflection over build graph. RELNOTES: None. PiperOrigin-RevId: 152038248
* If --batch, --keep_going, --discard_analysis_cache, and the new ↵Gravatar janakr2017-03-31
| | | | | | | | | | | | | | | --noexperimental_enable_critical_path_profiling flags are all specified, then Bazel will delete Actions from ActionLookupValues as they are executed in order to save memory. Because an already-run action may output an artifact that is only requested later in the build, we need to maintain a way for the artifact to look up the action. But in most cases we don't need to keep the action itself, just its output metadata. Some actions unfortunately are needed post-execution, and so we special-case them. Also includes dependency change with description: Move action out of key. This keeps action references from polluting the graph -- actions are just stored in one SkyValue, instead of being present in SkyKeys. This does mean additional memory used: we have a separate ActionLookupData object per Action executed. That may reach ~24M for million-action builds. PiperOrigin-RevId: 151756383
* Remove some unused bookkeeping things about the no-longer-existent PackageCache.Gravatar nharmata2017-03-31
| | | | | | RELNOTES: None PiperOrigin-RevId: 151729291
* Symlink output directories to the correct directory nameGravatar kchodorow2017-03-31
| | | | | | | | | | If the workspace directory is /path/to/my/proj and the name in the WORKSPACE file is "floop", this will symlink the output directories to output_base/execroot/floop instead of output_base/execroot/proj. More prep for #1262, fixes #1681. PiperOrigin-RevId: 151712384
* Drop loading-phase values if --discard_analysis_cache is true and we're not ↵Gravatar janakr2017-03-31
| | | | | | keeping incremental state. PiperOrigin-RevId: 151639711
* Actually clear the per-build syscall cache after loading. Setting it to null ↵Gravatar janakr2017-03-31
| | | | | | doesn't work because the PackageFactory keeps a reference to it. PiperOrigin-RevId: 151632473
* Clear AspectValues when discarding analysis cache, along with ↵Gravatar janakr2017-03-29
| | | | | | | | ConfiguredTargetValues. Also clear transitive packages for both, even for top-level targets. This is not expected to save significant memory, but is expected to reduce the number of references to Packages, allowing them to be dropped more easily when discarding analysis cache and running in batch mode. PiperOrigin-RevId: 151508877
* Unit tests respect top-level rule-class transitions.Gravatar Cal Peyser2017-03-27
| | | | | | -- PiperOrigin-RevId: 151129669 MOS_MIGRATED_REVID=151129669
* Remove provider safety check and make all old ctx objects featurelessGravatar Vladimir Moskva2017-03-23
| | | | | | | | | | | | It's now allowed to return anything from a rule implementation function. If a ctx object is returned it becomes featureless and cannot be used from anywhere else (i.e. from an implementation function of another rule). Fixes #2319 -- PiperOrigin-RevId: 151015919 MOS_MIGRATED_REVID=151015919
* Remove only-in-tests null checks in SkyframeActionExecutor.Gravatar Janak Ramakrishnan2017-03-22
| | | | | | | | Still on step -0.5, let's be honest. -- PiperOrigin-RevId: 150783638 MOS_MIGRATED_REVID=150783638
* Enforce that the SkyKey returned by ActionLookupKey#getSkyKey is an ↵Gravatar Janak Ramakrishnan2017-03-21
| | | | | | | | | | ActionLookupKey. It was only being violated for some singleton keys. Also do some drive-by cleanups of unused variables. Step approximately -2. -- PiperOrigin-RevId: 150668944 MOS_MIGRATED_REVID=150668944
* Make explicit that no I/O is done when constructing a FileArtifactValue for ↵Gravatar Janak Ramakrishnan2017-03-21
| | | | | | | | | | | | an ordinary Artifact in ArtifactFunction. If the data available in the FileValue is not sufficient to recreate the FileArtifactValue without disk access, then ActionMetadataHandler already stores the FileArtifactValue separately. This is approximately step -3 in a grand plan to save a bunch more memory when building with --batch --discard_analysis_cache --keep_going. -- PiperOrigin-RevId: 150646007 MOS_MIGRATED_REVID=150646007
* Fix missing type params for Java7Gravatar Mark Schaller2017-03-12
| | | | | | -- PiperOrigin-RevId: 149813927 MOS_MIGRATED_REVID=149813927
* Remove usage of SynchronizedBatchCallback findTargetsBeneathDirectoryAsync; ↵Gravatar Nathan Harmata2017-03-12
| | | | | | | | | | by design, the callback passed in there is thread safe. A previous change accidentally introduced this unnecessary synchronization. Also move SynchronizedBatchCallback to the file with its only current usage. -- PiperOrigin-RevId: 149809083 MOS_MIGRATED_REVID=149809083
* Improve query error msg when a package has a broken Skylark loadGravatar Mark Schaller2017-03-12
| | | | | | | | | | | | | | | | | | | | | | | | The error message logged during query (and build) when a package has a broken Skylark load statement was not specific. Previously, it said "package contains errors:" and then the package name. Also, this error message was not emitted when using SkyQueryEnvironment and evaluating a query containing a "TargetsBelowDirectory" pattern (such as //foo/...) when a package below the specified directory had such an error. The approach taken by this CL is to include any package loading error message in the SkyValue produced by CollectPackagesUnderDirectoryFunction, and report them during evaluation of a TargetsBelowDirectory pattern. RELNOTES: Evaluation of commands on TargetsBelowDirectory patterns (e.g. //foo/...) matching packages that fail to load now report more detailed error messages in keep_going mode. -- PiperOrigin-RevId: 149802362 MOS_MIGRATED_REVID=149802362
* Description redacted.Gravatar Nathan Harmata2017-03-09
| | | | | | -- PiperOrigin-RevId: 149585165 MOS_MIGRATED_REVID=149585165
* Pass the repository name through to build info factoriesGravatar Kristina Chodorow2017-03-06
| | | | | | | | | | | | I was fixing the Android tests and I noticed that the bazel_workspace_status_test was failing. It was like when you have a thread coming out of a sweater and you start pulling and pretty soon you have no sweater. Long story short, my "Android tests fix" cl ended up needing ~1k more lines changed (on top of the existing enormous CL). So, I'm creating some smaller CLs with the changes that I can extract from the mega CL. Again. Prep for #1681. -- PiperOrigin-RevId: 149106039 MOS_MIGRATED_REVID=149106039
* Support declared providers for aspectsGravatar Vladimir Moskva2017-03-06
| | | | | | | | Fixes #2016 -- PiperOrigin-RevId: 149102037 MOS_MIGRATED_REVID=149102037
* Rationalize local resource acquisitionGravatar Ulf Adams2017-03-03
| | | | | | | | | | | | | | | | | | | | | | Move all local resource acquisition to where local execution actually happens. Don't attempt to acquire resources per action, but only for individual spawns. This significantly simplifies the code. The downside is that we don't account for action-level work anymore. In general, actions should not perform any process execution themselves, but always delegate such work to a SpawnStrategy implementation. This change makes sure that every Spawn has local resources set in a way that is consistent with the previous state. However, there are two actions - Fileset and FileWrite -, which are not spawns, and so we now don't limit their concurrent execution anymore. For Fileset, all work is done in a custom Fileset-specific thread pool, so this shouldn't be a problem. I'm not sure about FileWriteAction. -- PiperOrigin-RevId: 149012600 MOS_MIGRATED_REVID=149012600
* Rollback of commit 34ffc4cab1b6cdde54e5945427b79c24fff39aa5.Gravatar Dmitry Lomov2017-03-02
| | | | | | | | | | | | | | *** Reason for rollback *** Roll-forward of commit 01120026dc313ee7ad9ea95069a29252eb19173b with fix. *** Original change description *** Automated [] rollback of commit 01120026dc313ee7ad9ea95069a29252eb19173b. -- PiperOrigin-RevId: 148897534 MOS_MIGRATED_REVID=148897534
* BEP: Report pattern expansion failures Gravatar Klaus Aehlig2017-03-01
| | | | | | | | | | | If expanding a pattern fails, report this on the build event protocol; also include details of what happened. -- Change-Id: I2bc9caf7c085911b80551d7892cc34f5e9961c7b Reviewed-on: https://cr.bazel.build/8795 PiperOrigin-RevId: 148894326 MOS_MIGRATED_REVID=148894326
* Rollback of commit 01120026dc313ee7ad9ea95069a29252eb19173b.Gravatar Jon Brandvein2017-03-01
| | | | | | -- PiperOrigin-RevId: 148888469 MOS_MIGRATED_REVID=148888469
* Do not execute aspect implementation if target advertizes but fails to ↵Gravatar Dmitry Lomov2017-03-01
| | | | | | | | | | | provide a provider. Previously we always executed the function, but didn't add the aspect to the deps. -- PiperOrigin-RevId: 148887089 MOS_MIGRATED_REVID=148887089
* Export exportable values as we go instead of at the end of evaluation.Gravatar Dmitry Lomov2017-03-01
| | | | | | | | | This simplifies implementation of attr.* functions and helps make declared provider implementation simpler. -- PiperOrigin-RevId: 148867326 MOS_MIGRATED_REVID=148867326
* Rollback of commit 822c37816ac669e51bec3853b41849a19ec5e230.Gravatar Nathan Harmata2017-03-01
| | | | | | -- PiperOrigin-RevId: 148844518 MOS_MIGRATED_REVID=148844518
* Add check that the WORKSPACE found in LocalRepositoryLookupFunction isn't a ↵Gravatar John Cater2017-02-28
| | | | | | | | | | | | directory. Fixes #2580. -- Change-Id: I6e1271a9c1442819e956f7fcdcb5da535ed027db Reviewed-on: https://cr.bazel.build/9110 PiperOrigin-RevId: 148751072 MOS_MIGRATED_REVID=148751072
* Clean up the semantics of input discovering actions a bit by making ↵Gravatar Lukacs Berki2017-02-28
| | | | | | | | | | | | updateInputs() and inputsKnown() non-overridable and removing setInputs(). This comes at the cost of adding a flag to every action instance that's not used for non-input-discovering actions, but I think that's a deal. Simpler APIs are good, mmmmkay? Also fixed a few pre-existing issues in TestAction and ObjcCompileAction. -- PiperOrigin-RevId: 148749734 MOS_MIGRATED_REVID=148749734
* Reimplement blaze query using an async evaluation model. Use a concurrent ↵Gravatar Nathan Harmata2017-02-28
| | | | | | | | | | | | | | | | | | | backend for SkyQueryEnvironment's implementation in order to achieve parallelism. Advantages: -New design has no flaws that the old design had. -Code is structured so that deadlocks due to thread starvation are impossible (yup!). Disadvantages: -The meat of this change needs to all be in a single CL because every single QueryFunction and QueryExpression needs to be rewritten in the async style. Still TODO: -Fully embrace the async model in all QueryFunctions (e.g. 'rdeps', 'allpaths'). -Use concurrency in BlazeQueryEnvironment to achieve parallel evaluation for (non SkyQuery) 'blaze query' and genquery. -- PiperOrigin-RevId: 148690279 MOS_MIGRATED_REVID=148690279
* Clean up event busGravatar Klaus Aehlig2017-02-27
| | | | | | | | ...now that the ExtendedEventHandler can serve the same purpose. -- PiperOrigin-RevId: 148640618 MOS_MIGRATED_REVID=148640618
* Provide more reporting options to SkyFunctions Gravatar Klaus Aehlig2017-02-27
| | | | | | | | | | | | | With more specific information to be reported by Skyfunctions, e.g., to inform the build-event protocol on missing files, the EventHandler interface is no longer enough. Therefore, provide an enriched context for reporting events. -- Change-Id: I2d06166fe4d5b9054e24ad8c752fafc039e3f9f8 Reviewed-on: https://cr.bazel.build/8794 PiperOrigin-RevId: 148463437 MOS_MIGRATED_REVID=148463437
* Fix Cpp action cachingGravatar Ulf Adams2017-02-24
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This combines both previous changes and extends them to work both with and without kchodorow@'s rollout of the exec root rearrangement. Unfortunately, each of these changes individually breaks something somewhere, so they must all go into a single commit. Change 1: CppCompileAction must return false from inputsKnown for .d pruning This is necessary (but not sufficient) for the action cache to work correctly. Consider the following sequence of events: 1. action is executed 2. .d pruning is performed 3. action cache writes entry with post-pruning inputs list 4. action gets regenerated (e.g., due to server restart) 5. the action cache calls inputsKnown(), which returns true 6. action cache checks entry from step 3 against pre-pruning inputs list, which results in a cache miss The action cache needs to know that the current list is not the final list, so inputsKnown() in step 5 must return false if .d pruning is enabled. Change 2: Fix artifact root discovery for external artifacts The SkyframeExecutor was assuming that all exec paths were coming from the main repository. Instead, rely on external exec paths to start with "../". Additional change 3: In addition, update the PackageRootResolverWithEnvironment and the HeaderDiscovery to use the single unified repository name guessing implementation. Previously, the PackageRootResolverWithEnvironment was poisoning the source artifact cache, which then caused subsequent lookups to return a bad artifact. Add a precondition to double-check that artifacts looked up by exec path have the correct source root. For compatibility with kchodorow@'s upcoming exec root refactor, if the exec path starts with "external", then assume it's coming from an external repository. This must be removed when that change is successfully rolled out, or it will break if anyone creates a package called 'external'. Additional change 4: On top of all of that, PackageRootResolverWithEnvironment and SkyframeExecutor must use the same source root computation as the Package class itself. I extracted the corresponding code to Root, and added comments both there and in Package to clearly indicate that these methods have to always be modified in sync. Fixes #2490. -- PiperOrigin-RevId: 148439309 MOS_MIGRATED_REVID=148439309
* Report inconsistent aspect order error to the user.Gravatar Dmitry Lomov2017-02-24
| | | | | | -- PiperOrigin-RevId: 148342788 MOS_MIGRATED_REVID=148342788
* Fix broken javadoc link.Gravatar Shreya Bhattarai2017-02-23
| | | | | | -- PiperOrigin-RevId: 148253141 MOS_MIGRATED_REVID=148253141
* Allow absolute symlinks in TreeArtifacts. Let the downstream action (if ↵Gravatar Janak Ramakrishnan2017-02-23
| | | | | | | | there is one) worry about it. -- PiperOrigin-RevId: 148249223 MOS_MIGRATED_REVID=148249223
* Rollback of commit 4b73e972d909bcd533f2f9940f95a00b9b73bdde.Gravatar Dmitry Lomov2017-02-17
| | | | | | | | | | | | | | | | | | | | | | | *** Reason for rollback *** Broke tests on CI: http://ci.bazel.io/job/bazel-tests/570/ *** Original change description *** Roll forward execroot change RELNOTES[INC]: Previously, an external repository would be symlinked into the execution root at execroot/local_repo/external/remote_repo. This changes it to be at execroot/remote_repo. This may break genrules/Skylark actions that hardcode execution root paths. If this causes breakages for you, ensure that genrules are using $(location :target) to access files and Skylark rules are using http://bazel.io/docs/skylark/lib/File.html's path, dirname, etc. functions. Cust... -- PiperOrigin-RevId: 147833177 MOS_MIGRATED_REVID=147833177
* Make the interface dynamic library builder a regular label instead of a ↵Gravatar Lukacs Berki2017-02-17
| | | | | | | | magic artifact. -- PiperOrigin-RevId: 147830857 MOS_MIGRATED_REVID=147830857
* Roll forward execroot changeGravatar Kristina Chodorow2017-02-16
| | | | | | | | | | | | | | | | | RELNOTES[INC]: Previously, an external repository would be symlinked into the execution root at execroot/local_repo/external/remote_repo. This changes it to be at execroot/remote_repo. This may break genrules/Skylark actions that hardcode execution root paths. If this causes breakages for you, ensure that genrules are using $(location :target) to access files and Skylark rules are using http://bazel.io/docs/skylark/lib/File.html's path, dirname, etc. functions. Custom crosstools that hardcode external/<repo> paths will have to be updated. Issue #1262. -- PiperOrigin-RevId: 147726370 MOS_MIGRATED_REVID=147726370
* Fix a hypothetical issue with 'buildfiles' and 'loadfiles' with duplicate ↵Gravatar Nathan Harmata2017-02-16
| | | | | | | | | | | | | | | | | | targets. These two functions don't use a Uniquifier internally so if the same bzl file foo.bzl is loaded multiple ways in 'e' in 'loadfiles(e)' then 'f' in 'f(loadfiles(e))' will observe foo.bzl multiple times. This isn't observable in practice (i.e. I cannot write a black-box query test that would fail without this CL) because: (1) BlazeQueryEnvironment#eval(QueryExpression, VariableContext<Target>, Callback<Target>) uses an intermediate aggregating callback and thus doesn't use the streaming query evaluation model. This means that the internal deduping in BlazeQueryEnvironment#getBuildFiles is sufficient. (2) SkyQueryEnvironment uses an outer BatchStreamedCallback which internally uses a Uniquifier. Still, this CL is useful for SkyQuery because without we're doing wasted work (e.g. in my example above, 'f' will wastefully do duplicate work with the .bzl files from the inner loadfiles(e)) because the deduping happens on the final query result, not on the intermediate result from 'buildfiles'/'loadfiles'. For 'buildfiles', there's an additional wart with (1) above because FakeSubincludeTarget doesn't override Object#equals/hashCode. -- PiperOrigin-RevId: 147656183 MOS_MIGRATED_REVID=147656183
* Loosen visibility of ASTFileLookupValue.Gravatar Janak Ramakrishnan2017-02-15
| | | | | | -- PiperOrigin-RevId: 147559691 MOS_MIGRATED_REVID=147559691
* Restrict aspects visible to other aspects according to their advertised ↵Gravatar Dmitry Lomov2017-02-15
| | | | | | | | providers. -- PiperOrigin-RevId: 147526961 MOS_MIGRATED_REVID=147526961
* Some minor logging improvements related to diff awareness:Gravatar Nathan Harmata2017-02-15
| | | | | | | | | | | | (i) Log [the first 100] external [non external-repo] file paths when we encounter them in skyframe [when externalFileAction is not ASSUME_NON_EXISTENT_AND_IMMUTABLE_FOR_EXTERNAL_PATHS]. These caveats are to prevent log spam. (ii) Fix grammar when logging the results of handleDiffsWithMissingDiffInformation in the case when all package paths have known diff information and we're merely checking external/output/external-repo files. (iii) Log the purpose of each skyframe graph scan we do (see (ii)). -- PiperOrigin-RevId: 147508404 MOS_MIGRATED_REVID=147508404
* Fix IllegalStateException when loading from a non declared external repository Gravatar Damien Martin-Guillerez2017-02-14
| | | | | | | | | | | This is another instance of #1793 that happens when a WORKSPACE file exists in some path underneath your current workspace. -- Change-Id: Idb01cd643548a170a27c9103f1b5081b058cc005 Reviewed-on: https://cr.bazel.build/8143 PiperOrigin-RevId: 147353935 MOS_MIGRATED_REVID=147353935
* Fix PackageProgressReceiver to not call contains on a list.Gravatar Janak Ramakrishnan2017-02-13
| | | | | | -- PiperOrigin-RevId: 147323591 MOS_MIGRATED_REVID=147323591
* Add @Nullable annotations to PackageFunction#progressReceiver.Gravatar Janak Ramakrishnan2017-02-13
| | | | | | -- PiperOrigin-RevId: 147202629 MOS_MIGRATED_REVID=147202629
* Introduce CppCompileActionTemplate, which expands into a list of ↵Gravatar Rumou Duan2017-02-10
| | | | | | | | CppCompileActions that to be executed at execution time. -- PiperOrigin-RevId: 147163077 MOS_MIGRATED_REVID=147163077
* Tighten a precondition check during action execution.Gravatar Lukacs Berki2017-02-10
| | | | | | | | Input discovery is handled somewhere else, so the same Action instance should never be put twice into buildActionMap . -- PiperOrigin-RevId: 147132869 MOS_MIGRATED_REVID=147132869
generated by cgit v1.2.3 (git 2.39.1) at 2024-07-04 03:42:05 +0000