aboutsummaryrefslogtreecommitdiffhomepage
path: root/src/main/java/com/google/devtools/build/lib/skyframe
Commit message (Collapse)AuthorAge
* Fix an analysis phase performance regression with dynamic configurations.Gravatar Greg Estren2016-12-14
| | | | | | | | | | | | | | | | | | | | | | | | | | | | The short story is that env.valuesMissing() returns true without regard for which Skyframe evaluation missed values. In other words, given: env.getValues(missingValues); // Not all values ready. env.getValues(presentValues); // All value ready. if (env.valuesMissing()) { return null; } this returns null even if "env.getValues(presentValues)" has all its results. This doesn't have correctness consequences, but it does have (major) performance consequences, particularly in ConfiguredTargetFunction. The quick reason is that the successful call can still do useful followup work, but that gets short-circuited if the function nulls out early. See the code changes for full details. This eliminates a 30% hit on analysis time with dynamic configurations. With this change, that goes down to 0. The takeaway: ConfiguredTargetFunction is both unintuitively complex and performance-critical. C'est la vie. -- PiperOrigin-RevId: 142044069 MOS_MIGRATED_REVID=142044069
* Fixed PackageLookupFunction to check for all possible build file names when ↵Gravatar John Cater2016-12-12
| | | | | | | | | | | | loading external repositories. Fixes #2198. -- Change-Id: Ife0232f8e4652a90bc3b7dceec6e67312a142879 Reviewed-on: https://cr.bazel.build/7691 PiperOrigin-RevId: 141771126 MOS_MIGRATED_REVID=141771126
* Description redacted.Gravatar Googler2016-12-12
| | | | | | -- PiperOrigin-RevId: 141561079 MOS_MIGRATED_REVID=141561079
* Provide deterministic order for split configured deps (roll forward part 2).Gravatar Greg Estren2016-12-07
| | | | | | | | | | | | | | | | | | | | | Also: - Make ConfiguredTargetFunction.getDynamicConfigurations more readable. - Add a bit more testing coverage for configured dep resolution. This is a roll forward of commit 7505d94c19727e3100ac5e16a960bff2cb324f23. The original changed failed for two reasons: 1) Windows-only: "ppc" wasn't recognized as a valid cpu: https://github.com/bazelbuild/bazel/issues/2191 2) Bazel requires android_binary's "manifest" attribute to be "AndroidManifest.xml": https://www.google.com/url?sa=D&q=http%3A%2F%2Fci.bazel.io%2Fjob%2Fbazel-tests%2FBAZEL_VERSION%3DHEAD%2CPLATFORM_NAME%3Dubuntu_15.10-x86_64%2FlastCompletedBuild%2FtestReport%2F This version uses "armeabi-v7a" instead of "ppc" and "AndroidManifest.xml" in the splitDeps test. -- PiperOrigin-RevId: 141313454 MOS_MIGRATED_REVID=141313454
* Rollback of commit 12d766df10fbc5eba16ec1e6c20c8cd85f9c616f.Gravatar Damien Martin-Guillerez2016-12-06
| | | | | | | | | | | | | | | | | | | | | | | | | | | *** Reason for rollback *** Still fails bazel-tests See http://ci.bazel.io/job/bazel-tests/BAZEL_VERSION=HEAD,PLATFORM_NAME=ubuntu_15.10-x86_64/lastCompletedBuild/testReport/ for instance *** Original change description *** Provide deterministic order for split configured deps (roll forward) Also: - Make ConfiguredTargetFunction.getDynamicConfigurations more readable. - Add a bit more testing coverage for configured dep resolution. This is a roll forward of commit 7505d94c19727e3100ac5e16a960bff2cb324f23. The original changed failed on Windows because "ppc" wasn't recognized as a valid cpu: https://github.com/bazelbuild/bazel/issues/2191 This version uses "armeabi-v7a" instead. -- PiperOrigin-RevId: 141212457 MOS_MIGRATED_REVID=141212457
* Provide deterministic order for split configured deps (roll forward)Gravatar Greg Estren2016-12-06
| | | | | | | | | | | | | | | | | Also: - Make ConfiguredTargetFunction.getDynamicConfigurations more readable. - Add a bit more testing coverage for configured dep resolution. This is a roll forward of commit 7505d94c19727e3100ac5e16a960bff2cb324f23. The original changed failed on Windows because "ppc" wasn't recognized as a valid cpu: https://github.com/bazelbuild/bazel/issues/2191 This version uses "armeabi-v7a" instead. -- PiperOrigin-RevId: 141185293 MOS_MIGRATED_REVID=141185293
* Rollback of commit 7505d94c19727e3100ac5e16a960bff2cb324f23.Gravatar Damien Martin-Guillerez2016-12-06
| | | | | | | | | | | | | | | | | | | | *** Reason for rollback *** Newly added test fail on Windows platform Fixes https://github.com/bazelbuild/bazel/issues/2191 *** Original change description *** Provide deterministic order for split configured deps. Also: - Make ConfiguredTargetFunction.getDynamicConfigurations more readable. - Add a bit more testing coverage for configured dep resolution. -- PiperOrigin-RevId: 141167110 MOS_MIGRATED_REVID=141167110
* Provide deterministic order for split configured deps.Gravatar Greg Estren2016-12-06
| | | | | | | | | | Also: - Make ConfiguredTargetFunction.getDynamicConfigurations more readable. - Add a bit more testing coverage for configured dep resolution. -- PiperOrigin-RevId: 141095973 MOS_MIGRATED_REVID=141095973
* Fix test configured target creation code that was dropping split deps.Gravatar Greg Estren2016-12-06
| | | | | | | | | | | | | | | Short story: Dependency -> BuildConfiguration maps can have multiple values because of split transitions. This is unfortunately one of those instances where the test logic is forked from production. So this doesn't actually demonstrate bugs in production code. We already have tests in ConfigurationsForTargetsTest that directly check the production logic. So this cl's primary value is to unbreak tests that depend on the forked logic. -- PiperOrigin-RevId: 141094830 MOS_MIGRATED_REVID=141094830
* Add 'aspect_id' for to identify aspects that has been applied to the target.Gravatar Dmitry Lomov2016-12-05
| | | | | | | | | | | | | | | | | | | Add a concept of aspect_id: an unique string that identifies a propagating aspect (aspect class + parameters). This string is designed to be: - Unique for each aspect - human-readable for debugging purposes - not easily parsable. Skylark API: - `ctx.aspect_id` returns an identifier of the current aspect inside aspect implementation function - `Target.aspect_ids` return a list of aspect ids for aspects applied to a given Target (https://www.bazel.io/versions/master/docs/skylark/lib/Target.html) -- PiperOrigin-RevId: 141057865 MOS_MIGRATED_REVID=141057865
* Make build events correctly report target expansionGravatar Klaus Aehlig2016-12-05
| | | | | | | | | | | In particular, correctly report the expansion of a test suite even if only requested to build (but not test). -- Change-Id: Ia25305a4f5ed53118cbb1970a8055d156e8e50be Reviewed-on: https://cr.bazel.build/7535 PiperOrigin-RevId: 141027655 MOS_MIGRATED_REVID=141027655
* Fix bug in configuration creation logic that wrongly trims notrim configs.Gravatar Greg Estren2016-12-05
| | | | | | | | Without this, many Bazel tests fail with --experimental_dynamic_configs=notrim_partial. -- PiperOrigin-RevId: 140873563 MOS_MIGRATED_REVID=140873563
* Null out perBuildSyscallCache after analysis is finished to save memory ↵Gravatar Janak Ramakrishnan2016-12-02
| | | | | | | | during execution. -- PiperOrigin-RevId: 140864295 MOS_MIGRATED_REVID=140864295
* Prune modules when building modules themselves to reduce build times shortenGravatar Googler2016-12-02
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | critical paths. When the inputs of a module M are reduced to a set S, then that same set S also needs to be supplied to compile something that uses M. To do this, input discovery is divided into two stages. For each CppCompileAction, the first stage discovers the necessary modules (M above). These are then added as inputs to ensure that they are built. The second stage then finds all the modules (S above) that are required to use those and also adds them as inputs. For now, the new behavior is guarded by a new flag --experimental_prune_more_modules. This is currently implemented by reading the .d files of used modules add adding all their module dependencies. There are two noteworthy alternatives: 1. Hack up input discovery to understand modules, e.g. if a modular header is hit, continue scanning from all it's headers. However, this seems very brittle and a lot of additional information would have to be passed to the input discovery. 2. Directly pass the results from input discovery of one CppCompileAction to another one. However, this seems to tightly couple the execution of different CppCompileActions and might lead to a mess of different states, more memory consumption, etc. With the current implementation, there is a bit of runtime overhead of reading the .d files (many times). This could potentially be improved by caching the results. However, even without this caching, the runtime overhead is limited (<10%) for all builds I have tried (I have tried with builds where all the compile results are already in the executor's cache. -- MOS_MIGRATED_REVID=140793217
* Allow aspects to specify multiple sets of required providers to match ↵Gravatar Rumou Duan2016-12-01
| | | | | | | against rules. Aspects can attach to a rule if at least one set of required providers are present on the rule. -- MOS_MIGRATED_REVID=140605023
* Remove isSkylark and eval from Environment.Gravatar Laurent Le Brun2016-11-29
| | | | | -- MOS_MIGRATED_REVID=140371603
* Provide AspectDescriptor to ConfiguredAspect.Gravatar Dmitry Lomov2016-11-29
| | | | | | | | Also clean up the setting of aspect name in ConfiguredAspect and AspectDefintion - it is now obtained from the AspectClass. -- MOS_MIGRATED_REVID=140357052
* Use BlazeInterner's chosen concurrency level in InternerWithPresenceCheck's ↵Gravatar Nathan Harmata2016-11-28
| | | | | | | internal ConcurrentMap. -- MOS_MIGRATED_REVID=140253038
* Refactor LegacyLoadingPhaseRunner to be more like SkyframeLoadingPhaseRunner.Gravatar Ulf Adams2016-11-22
| | | | | | | | This is in preparation for extracting some common reporting code, and also a step towards interleaving target pattern eval and config creation. -- MOS_MIGRATED_REVID=139890205
* Implement 'output_groups' provider.Gravatar Dmitry Lomov2016-11-21
| | | | | | | | | | This behavior - that 'output_groups' is a provider available on targets and aspects - has been accidental, but people already depend on it. This CL keeps that behavior, while fixing the bug that two aspects could not both provide output groups. -- MOS_MIGRATED_REVID=139578378
* Allow labels in the '--aspects' parameter.Gravatar Dmitry Lomov2016-11-21
| | | | | -- MOS_MIGRATED_REVID=139573590
* Run the analysis phase with as many threads as the user wants. In order to ↵Gravatar Janak Ramakrishnan2016-11-18
| | | | | | | | | avoid memory blow-up intra-configured-target analysis, use a semaphore to ensure that CPU-bound work only occurs on #CPU-many threads. RELNOTES: Use --loading_phase_threads to control the number of threads used during the loading/analysis phase. -- MOS_MIGRATED_REVID=139477645
* Cache all previously computed values in ↵Gravatar Janak Ramakrishnan2016-11-17
| | | | | | | | | SkyQueryEnvironment#beforeEvaluateQuery if possible to save on latency for small queries. This assumes that if the graph is up to date, then the data in SkyQueryEnvironment is also up to date. It also assumes that a ForkJoinPool remains usable until #shutdownNow is called. -- MOS_MIGRATED_REVID=139386363
* Re-use the precomputed universe key in WalkableGraphFactory#prepareAndGet.Gravatar Janak Ramakrishnan2016-11-17
| | | | | -- MOS_MIGRATED_REVID=139357569
* Rollback of commit a3f5f576cd35798140ba3e81d03d919dd4ecb847.Gravatar Alex Humesky2016-11-17
| | | | | | | | | | | | | | *** Reason for rollback *** Breaks targets in the depot: [] *** Original change description *** output_group is not a real Skylark provider for aspects, as well as for rules. -- MOS_MIGRATED_REVID=139354682
* Fixes incomplete support for dynamic split transitions in Bazel'sGravatar Greg Estren2016-11-17
| | | | | | | | | | | | | | | | | | | | | | test infrastructure. The small picture story is that SkyframeExecutor.getDynamicConfigOptions (which gets dynamic BuildOptions for tests) wasn't updated with dynamic split support when that was added to ConfiguredTargetFunction.getDynamicTransitionOptions (which does the same thing for production builds). This change plugs that gap. See 373e3e28274cca5b87f48abe33884edb84016dd3 for the original change. The bigger picture story is that Bazel's configured target creation logic is forked: test code goes down a similar but sadly not-quite-the-same path that results in tons of duplicated logic, spaghetti code mess, and risk of bugs like this one. We'd like to ultimately undo that fork. But unfortunately it's an involved effort that won't happen overnight. In the meantime, this change takes one small step by merging the two methods that caused this bug. -- MOS_MIGRATED_REVID=139342710
* Supports --nodistinct_host_configuration withGravatar Greg Estren2016-11-16
| | | | | | | | | | | --experimental_dynamic_configs=notrim. This is simply a matter of populating the "global" host config from the target config instead of applying the HostTransition patch. -- MOS_MIGRATED_REVID=139222963
* Code cleanup for interleaving target pattern eval and config creation.Gravatar Ulf Adams2016-11-16
| | | | | -- MOS_MIGRATED_REVID=139209942
* Aspects-on-aspects implementation.Gravatar Dmitry Lomov2016-11-15
| | | | | -- MOS_MIGRATED_REVID=139189444
* Make TargetPattern evaluation during query evaluation more parallel-friendly ↵Gravatar Nathan Harmata2016-11-15
| | | | | | | by introducing TargetPattern#parEval, which allows TargetPatterns' evaluations to explicitly have parallel implementations (no need to secretly use a FJP). -- MOS_MIGRATED_REVID=139101922
* Cache PerBuildSyscallCache if concurrency level is unchanged.Gravatar Janak Ramakrishnan2016-11-14
| | | | | -- MOS_MIGRATED_REVID=138893829
* output_group is not a real Skylark provider for aspects, as well as for rules.Gravatar Dmitry Lomov2016-11-11
| | | | | -- MOS_MIGRATED_REVID=138863855
* Do not crash when aspects provide duplicate things.Gravatar Dmitry Lomov2016-11-11
| | | | | -- MOS_MIGRATED_REVID=138860974
* Implement build tag filtering.Gravatar Lukacs Berki2016-11-11
| | | | | | | If the --build_tag_filters option is specified, targets built will be filtered according to their tags (at least one included, none excluded) -- MOS_MIGRATED_REVID=138856195
* Update package lookup to check for files named BUILD.bazel before files namedGravatar John Cater2016-11-11
| | | | | | | | | | | BUILD. Fixes #552. RELNOTES[NEW]: Packages are defined in BUILD.bazel as well as BUILD files. -- MOS_MIGRATED_REVID=138828981
* Add a mechanism for bounding the number of Packages SkyQueryEnvironment's ↵Gravatar Nathan Harmata2016-11-11
| | | | | | | expensive parallel operations can operate on at once. -- MOS_MIGRATED_REVID=138779172
* Add SkylarkAspect to SkylarkRuleContext for aspectsGravatar Dmitry Lomov2016-11-09
| | | | | -- MOS_MIGRATED_REVID=138556910
* Rollback of commit b91fac3b1f0f5142a906aa12280de936f29fb607.Gravatar Luis Fernando Pino Duque2016-11-09
| | | | | | | | | | | | | | | | *** Reason for rollback *** Breaks several tests under //devtools/ide/eclipse/aspects as shown by the nightly []: [] *** Original change description *** Restrict google_glob_information to eclipse_aspect. '--legacy_expose_globs' flag has no effect now. -- MOS_MIGRATED_REVID=138517258
* Restrict google_glob_information to eclipse_aspect.Gravatar Dmitry Lomov2016-11-08
| | | | | | | '--legacy_expose_globs' flag has no effect now. -- MOS_MIGRATED_REVID=138410651
* Add logging to TimestampGranularityMonitor.Gravatar Julio Merino2016-11-02
| | | | | | | | These new log statements help in understanding what files trigger the TimestampGranularityMonitor's wait logic and when the wait is performed. -- MOS_MIGRATED_REVID=137868235
* Do not tickle TimestampGranularityMonitor for CONSTANT_METADATA artifacts.Gravatar Julio Merino2016-11-02
| | | | | | | | | | | | | | | | | | | | "Constant metadata" artifacts represent real files whose changes should be ignored by the build system. However, these artifacts were triggering the timestamp granularity checks in TimestampGranularityMonitor because the fact that they were "constant metadata" was not respected. Avoid this so that their regeneration does not cause the build to unnecessarily stall. One of these artifacts is the volatile workspace status file, which is unconditionally updated on each build. Before this fix, "blaze build" would get stuck for up to a second waiting for file system timestamps to catch up. With this fix, the artifact is ignored and the wait is gone. This problem is magnified on macOS where the default HFS+ file system only has second-level granularity. (This also affects Linux, but because current Linux file systems have milli/nanosecond-level granularity, the wait imposed by TimestampGranularityMonitor is minimal and thus not generally noticeable.) -- MOS_MIGRATED_REVID=137867586
* Add experimental flag to stop requiring all transitive modules as inputs.Gravatar Googler2016-10-31
| | | | | | | | | Requiring all transitive modules to always be available can lead to long critical paths and even unnecessary compiles in combination with the prune_header_modules feature. -- MOS_MIGRATED_REVID=137696794
* Fix uses of LocalRepositoryLookupFunction to work with JDK7.Gravatar John Cater2016-10-26
| | | | | -- MOS_MIGRATED_REVID=137257525
* Add new skyframe function to lookup the repository given a path, and use thatGravatar John Cater2016-10-25
| | | | | | | to report invalid package references. Fixes #1592. -- MOS_MIGRATED_REVID=137164164
* Add a new concept of failure causesGravatar Klaus Aehlig2016-10-25
| | | | | | | | | | | | | | | | | | | | Not all possible reasons for failure are uniquely identified by a label. Therefore, add a new data type describing possible root causes of failures and use it. The new type is added in causes/*.java and coresponds to Haskell's one-line definition data Cause = LabelCause Label | ActionCause Path Label deriving Show With future clean up of other failure causes inadequately described by a label, we expect that type to be extended by new constructors (i.e., new classes implementing Cause). -- Change-Id: I6fec74c78cec6abb9c10e32743b05a792888fead Reviewed-on: https://bazel-review.googlesource.com/#/c/6617 MOS_MIGRATED_REVID=137156390
* Accept valid relative symlinks in TreeArtifacts.Gravatar Rumou Duan2016-10-25
| | | | | -- MOS_MIGRATED_REVID=137072310
* Proper action output checks for TreeArtifacts. Instead of crashing Bazel, we ↵Gravatar Rumou Duan2016-10-19
| | | | | | | now handle failed TreeArtifact output checks gracefully. -- MOS_MIGRATED_REVID=136627086
* Add type arguments for JDK7 build.Gravatar Janak Ramakrishnan2016-10-19
| | | | | -- MOS_MIGRATED_REVID=136575806
* Per action file caching for input TreeArtifact.Gravatar Rumou Duan2016-10-19
| | | | | -- MOS_MIGRATED_REVID=136475556
* Rollback of commit b776d6c12e952eb358c1a036cc9d93d8944c4c77.Gravatar Nathan Harmata2016-10-18
| | | | | | | *** Reason for rollback *** -- MOS_MIGRATED_REVID=136448178
generated by cgit v1.2.3 (git 2.39.1) at 2024-08-07 00:44:47 +0000