aboutsummaryrefslogtreecommitdiffhomepage
path: root/src/main/java/com/google/devtools/build/lib/buildtool
Commit message (Collapse)AuthorAge
* Move the instrumentation filter computation to BuildToolGravatar ulfjack2017-06-06
| | | | | | | | | | | | | | | | | | Previously, it was in CoverageCommand, which led to all sorts of special casing and workarounds, because it required options editing, error handling during options editing, as well as early package path setup. It also caused us to duplicate target pattern evaluation for the coverage command. A semantic change is that the TestCommand can now also trigger the heuristic instrumentation filter computation, if collect_code_coverage is enabled; we might consider also enabling this for the BuildCommand, which can also have collect_code_coverage enabled (e.g., when a user wants to build a non-test binary with coverage instrumentation). Also remove a bunch of unnecessary AbruptExitException declarations. RELNOTES: bazel test now also computes a default instrumentation filter if --collect_code_coverage is enabled PiperOrigin-RevId: 158129953
* Make --aspects a repeatable flag.Gravatar dslomov2017-05-31
| | | | | RELNOTES: '--aspects' can occur more than once on the command line. PiperOrigin-RevId: 157568229
* BEP: Correctly report test exit code.Gravatar buchgr2017-05-30
| | | | | | | | | | | | | When a test failure occurred the BEP would still report a build_finished event with exit code SUCCESS, even though bazel's exit code was TESTS_FAILED. This is because we would rely on the exit code reported by the BuildCompleteEvent. However, this exit code contains only the build status without taking into account the test status. In this change we introduce the TestingCompleteEvent that reports bazel's final exit code in case of "bazel test". RELNOTES: None. PiperOrigin-RevId: 157445808
* Remove workspace tree under correct nameGravatar kchodorow2017-05-26
| | | | | | Actual fix for #2819. PiperOrigin-RevId: 157142420
* Automated g4 rollback of commit c4134802dd15d6ef5cca6521f6bf6aac395ee2ad.Gravatar kchodorow2017-05-23
| | | | | | | | | | | | | | | | | | | | | | | | | | *** Reason for rollback *** Roll forward of directory name change *** Original change description *** Automated g4 rollback of commit 1d9e1ac90197b1d3d7b137ba3c1ada67bb9ba31b. *** Reason for rollback *** Breaks //src/test/shell/integration:force_delete_output_test *** Original change description *** Symlink output directories to the correct directory name If the workspace directory is /path/to/my/proj and the name in the WORKSPACE file is "floop", this will symlink the output directories to output_base/execroot/floop instead of output_base/execroot/proj. More prep for #1262, fixes #1681. PiperOrigin-RevId: 156892980
* Fix crash when building output files with --target_environment.Gravatar gregce2017-05-15
| | | | PiperOrigin-RevId: 155871926
* Introduce an EvaluationProgressReceiver implementation that does nothing.Gravatar nharmata2017-05-10
| | | | | | | | This is useful for dealing with all the existing implementations in the face of interface changes that are irrelevant. RELNOTES: None PiperOrigin-RevId: 155525021
* Make symlinks consistentGravatar kchodorow2017-05-04
| | | | PiperOrigin-RevId: 155105523
* Document output groups.Gravatar Dmitry Lomov2017-05-04
| | | | | | | Work towards #2880. Change-Id: I7b661e368c0bd60fc6bcc10c7c1d63b82ba9702e PiperOrigin-RevId: 154957882
* BEP: Add WorkspaceStatusGravatar Klaus Aehlig2017-05-03
| | | | | | | | Add an event reporting the workspace status as key-value pairs reported via the workspace_status_command. Change-Id: I5791551798a594bc2465f483eb97f9d4fd4c7cfd PiperOrigin-RevId: 154845224
* Implement OutputGroupInfo provider.Gravatar dslomov2017-05-03
| | | | | | | Work towards #2894. RELNOTES: None. PiperOrigin-RevId: 154829065
* Re-introduce the overall_success field in BEP.Gravatar buchgr2017-04-28
| | | | | | | | Removal of BuildFinished.overall_success broke internal tests and thus we decided to reintroduce the field and deprecate it. RELNOTES: None. PiperOrigin-RevId: 154432961
* Store Skylark command-line flags in SkyframeGravatar brandjon2017-04-28
| | | | | | | | | This is the first of two CLs for making command line options able to affect the Skylark interpreter. It introduces SkylarkSemanticsOptions, and stores it as a precomputed (injected) value in Skyframe. The next CL will read these options from Skyframe when constructing the Skylark environment. This CL affects the dataflow from command/test initialization to Skyframe. Some code paths, like those used for testing, use the default SkylarkSemanticsOptions and therefore won't be able to use (for example) --incompatible_* flags. The call sites to update were found by searching for uses of defaultVisibility and working upward from there. RELNOTES: None PiperOrigin-RevId: 154432058
* Allow bypassing the automated detection of jobs when --jobs=auto.Gravatar jmmv2017-04-28
| | | | | | | | | | | | | | | | | | This is useful for the Blaze case where builds are cloud-backed. In that case, we need to use a very large number of jobs to handle the remote tasks, as each task is handled by a blocking thread controlled by jobs. We were previously doing this using an invocation policy, but the UI was extremely confusing: "blaze help build" would claim that the default for --jobs was "auto" when, internally, "auto" was being converted to the hardcoded number pretty much silently. However, explicitly specifying --jobs=auto on the command line yielded different behaviors. With this change, --jobs=auto works for both Bazel and Blaze and does sensible things in each case without the user having to know. RELNOTES: None. PiperOrigin-RevId: 154418572
* BEP: Report exit code in BuildCompleteEventGravatar Jakob Buchgraber2017-04-27
| | | | | | | | | | | | | We decided to use the exit code as opposed to a status enum as bazel reports the build status as an exit code internally and the number of exit codes is dynamic. That is, a bazel module might add additional exit codes and thus we need to support reporting these. The overall_success field is now redundant and has thus been removed. A build was successfull iff ExitCode.code equals zero. Change-Id: I268db972b09d983f6cd07df34e3a384efb607358 PiperOrigin-RevId: 154413356
* Split out option usage restriction from option category.Gravatar ccalvarin2017-04-18
| | | | | | | | | These are two different concepts. Do not remove category overload compatibility in this CL, to keep this change limited to converting the current uses of category. With some flyby formatting fixes on affected OptionsBases. RELNOTES: None. PiperOrigin-RevId: 153390002
* Move action_outs directory to under the output path.Gravatar janakr2017-04-06
| | | | PiperOrigin-RevId: 152307322
* Refactor all ctor callsites of PathFragment to instead call a static ↵Gravatar nharmata2017-04-05
| | | | | | | | | | | | 'create' method. This paves the way for changing PathFragment to e.g. an abstract class with multiple subclasses. This way we can split out the windows-specific stuff into one of these concrete classes, making the code more readable and also saving memory (since the shallow heap size of the NonWindowsPathFragment subclass will hopefully be smaller than that of the current PathFragment). This also lets us pursue gc churn optimizations. We can now do interning in PathFragment#create and can also get rid of unnecessary intermediate PathFragment allocations. RELNOTES: None PiperOrigin-RevId: 152145768
* BEP: provide events a generic converter classGravatar aehlig2017-04-04
| | | | | | | | | | | Change the BuildEvent interface to accept a generic class of converters. In this way, we won't have to change it again in the future, once more converters are needed. In fact, a new converter is needed right now (will be added in a follow-up patch) to allow build events to know the name of named artifact groups already reported in the stream. Change-Id: Ibb32ea5fff361e21bcf2d34818d8351a1da7a2e3 PiperOrigin-RevId: 152131870
* Automated g4 rollback of commit 1d9e1ac90197b1d3d7b137ba3c1ada67bb9ba31b.Gravatar hlopko2017-04-04
| | | | | | | | | | | | | | | | | | *** Reason for rollback *** Breaks //src/test/shell/integration:force_delete_output_test *** Original change description *** Symlink output directories to the correct directory name If the workspace directory is /path/to/my/proj and the name in the WORKSPACE file is "floop", this will symlink the output directories to output_base/execroot/floop instead of output_base/execroot/proj. More prep for #1262, fixes #1681. PiperOrigin-RevId: 152126545
* If --batch, --keep_going, --discard_analysis_cache, and the new ↵Gravatar janakr2017-03-31
| | | | | | | | | | | | | | | --noexperimental_enable_critical_path_profiling flags are all specified, then Bazel will delete Actions from ActionLookupValues as they are executed in order to save memory. Because an already-run action may output an artifact that is only requested later in the build, we need to maintain a way for the artifact to look up the action. But in most cases we don't need to keep the action itself, just its output metadata. Some actions unfortunately are needed post-execution, and so we special-case them. Also includes dependency change with description: Move action out of key. This keeps action references from polluting the graph -- actions are just stored in one SkyValue, instead of being present in SkyKeys. This does mean additional memory used: we have a separate ActionLookupData object per Action executed. That may reach ~24M for million-action builds. PiperOrigin-RevId: 151756383
* Symlink output directories to the correct directory nameGravatar kchodorow2017-03-31
| | | | | | | | | | If the workspace directory is /path/to/my/proj and the name in the WORKSPACE file is "floop", this will symlink the output directories to output_base/execroot/floop instead of output_base/execroot/proj. More prep for #1262, fixes #1681. PiperOrigin-RevId: 151712384
* Clear AspectValues when discarding analysis cache, along with ↵Gravatar janakr2017-03-29
| | | | | | | | ConfiguredTargetValues. Also clear transitive packages for both, even for top-level targets. This is not expected to save significant memory, but is expected to reduce the number of references to Packages, allowing them to be dropped more easily when discarding analysis cache and running in batch mode. PiperOrigin-RevId: 151508877
* BEP: Also report options Gravatar Klaus Aehlig2017-03-27
| | | | | | | | | | | | Additionally to reporting on the originally received command line, also report it in a parsed form. This, in particular, includes reconstructing the options originally provided by the user. -- Change-Id: I5a1c6ed57874f1d37d10bf11d597256d63bc5516 Reviewed-on: https://cr.bazel.build/9459 PiperOrigin-RevId: 151288656 MOS_MIGRATED_REVID=151288656
* BEP: Also report the original command line Gravatar Klaus Aehlig2017-03-23
| | | | | | | | | | | In the protocol, also report the command line as received by the server, before and option handlers modify it. -- Change-Id: If5bab172944679752477836a5f499d2837201888 Reviewed-on: https://cr.bazel.build/9453 PiperOrigin-RevId: 150894322 MOS_MIGRATED_REVID=150894322
* Add Action context mnemonic to imprecise spawn strategy error message.Gravatar Chloe Calvarin2017-03-22
| | | | | | -- PiperOrigin-RevId: 150750767 MOS_MIGRATED_REVID=150750767
* Automatically set --jobs based on the number of CPU threads.Gravatar Julio Merino2017-03-20
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This change adds the new magic value "auto" to --jobs and makes this the default. When --jobs=auto, we determine the number of available CPU threads and set a reasonable value for --jobs based on this number. I'm explicitly not defining what "reasonable" means because we may want to change the heuristics later on. The goal here is to reduce the load on the system when running Bazel while not adversely affecting build times significantly. Previous versions of Bazel defaulted --jobs to 200, which could easily overload the local machine with a lot of processes. This value was derived from Blaze's default, which makes sense because most jobs are network-bound due to distributed execution; however, in the Bazel case, this never made sense and is actually harmful. This change was initiated by problems observed on Macs where Bazel would bring machines to their knees due to system resource overload. It's likely that the overload is caused by too much RAM usage rather than CPU, but both of these should go down with a more limited jobs value. Should help alleviate issue #1160. RELNOTES: The --jobs flag now defaults to "auto", which causes Bazel to use a reasonable degree of parallelism based on the local machine's capacity. -- PiperOrigin-RevId: 150466088 MOS_MIGRATED_REVID=150466088
* Make AspectCompleteEvent an instance of BuildEvent Gravatar Klaus Aehlig2017-03-16
| | | | | | | | | | | | | | | In this way, we can also report the artifacts generated by aspects in the build event protocol. For the time being, those events are reported unsolicitly (i.e., as children of progress events), but they may be linked to pattern expansion later. -- Change-Id: I7fb83088d7fdb5424f77bfb78700e8371231b13c Reviewed-on: https://cr.bazel.build/9370 PiperOrigin-RevId: 150181433 MOS_MIGRATED_REVID=150181433
* Global cleanup change.Gravatar Googler2017-03-03
| | | | | | -- PiperOrigin-RevId: 149089903 MOS_MIGRATED_REVID=149089903
* Rationalize local resource acquisitionGravatar Ulf Adams2017-03-03
| | | | | | | | | | | | | | | | | | | | | | Move all local resource acquisition to where local execution actually happens. Don't attempt to acquire resources per action, but only for individual spawns. This significantly simplifies the code. The downside is that we don't account for action-level work anymore. In general, actions should not perform any process execution themselves, but always delegate such work to a SpawnStrategy implementation. This change makes sure that every Spawn has local resources set in a way that is consistent with the previous state. However, there are two actions - Fileset and FileWrite -, which are not spawns, and so we now don't limit their concurrent execution anymore. For Fileset, all work is done in a custom Fileset-specific thread pool, so this shouldn't be a problem. I'm not sure about FileWriteAction. -- PiperOrigin-RevId: 149012600 MOS_MIGRATED_REVID=149012600
* Convert --use_action_cache to a regular optionGravatar Googler2017-03-01
| | | | | | | | RELNOTES: Convert --use_action_cache to a regular option -- PiperOrigin-RevId: 148804881 MOS_MIGRATED_REVID=148804881
* BEP: Add BuildFinished event. Gravatar Jakob Buchgraber2017-02-28
| | | | | | | | | | | | The build event protocol now emits a BuildFinished event at the end of a build. The event is a child of the BuildStarted event. The code changes were larger than expected, due to some refactoring in BuildEventStreamer. This was necessary as the BuildCompleteEvent now implements the BuildEvent interface and Guava's EventBus always invokes the most specialized subscriber method. Thus, the buildEvent(BuildEvent) and buildCompleted(BuildCompletedEvent) methods had to be merged. -- Change-Id: Id0c2bd7220dc8ce03128b7126587e212ee8ce836 Reviewed-on: https://cr.bazel.build/9053 PiperOrigin-RevId: 148788582 MOS_MIGRATED_REVID=148788582
* BEP: Add tags and output files to TargetComplete event Gravatar Jakob Buchgraber2017-02-28
| | | | | | | | | | | | | | | A target's tags and output files are reported as part of the TargetComplete event of the build event protocol (BEP). The output files are grouped by their corresponding output group. If an output file belongs to more than one output group it appears once in each output group. -- Change-Id: Ia37db68709850d8550e478dcc30064dc7366bd1b Reviewed-on: https://cr.bazel.build/8955 PiperOrigin-RevId: 148667599 MOS_MIGRATED_REVID=148667599
* Clean up event busGravatar Klaus Aehlig2017-02-27
| | | | | | | | ...now that the ExtendedEventHandler can serve the same purpose. -- PiperOrigin-RevId: 148640618 MOS_MIGRATED_REVID=148640618
* Provide more reporting options to SkyFunctions Gravatar Klaus Aehlig2017-02-27
| | | | | | | | | | | | | With more specific information to be reported by Skyfunctions, e.g., to inform the build-event protocol on missing files, the EventHandler interface is no longer enough. Therefore, provide an enriched context for reporting events. -- Change-Id: I2d06166fe4d5b9054e24ad8c752fafc039e3f9f8 Reviewed-on: https://cr.bazel.build/8794 PiperOrigin-RevId: 148463437 MOS_MIGRATED_REVID=148463437
* Fix spelling of millisGravatar Klaus Aehlig2017-02-21
| | | | | | | | -- Change-Id: I9e1483b9e99751b03d332110db023d99c045cb52 Reviewed-on: https://cr.bazel.build/9012 PiperOrigin-RevId: 148069886 MOS_MIGRATED_REVID=148069886
* Rollback of commit 4b73e972d909bcd533f2f9940f95a00b9b73bdde.Gravatar Dmitry Lomov2017-02-17
| | | | | | | | | | | | | | | | | | | | | | | *** Reason for rollback *** Broke tests on CI: http://ci.bazel.io/job/bazel-tests/570/ *** Original change description *** Roll forward execroot change RELNOTES[INC]: Previously, an external repository would be symlinked into the execution root at execroot/local_repo/external/remote_repo. This changes it to be at execroot/remote_repo. This may break genrules/Skylark actions that hardcode execution root paths. If this causes breakages for you, ensure that genrules are using $(location :target) to access files and Skylark rules are using http://bazel.io/docs/skylark/lib/File.html's path, dirname, etc. functions. Cust... -- PiperOrigin-RevId: 147833177 MOS_MIGRATED_REVID=147833177
* Roll forward execroot changeGravatar Kristina Chodorow2017-02-16
| | | | | | | | | | | | | | | | | RELNOTES[INC]: Previously, an external repository would be symlinked into the execution root at execroot/local_repo/external/remote_repo. This changes it to be at execroot/remote_repo. This may break genrules/Skylark actions that hardcode execution root paths. If this causes breakages for you, ensure that genrules are using $(location :target) to access files and Skylark rules are using http://bazel.io/docs/skylark/lib/File.html's path, dirname, etc. functions. Custom crosstools that hardcode external/<repo> paths will have to be updated. Issue #1262. -- PiperOrigin-RevId: 147726370 MOS_MIGRATED_REVID=147726370
* Apply --symlink_prefix to the output and exec root symlinks.Gravatar Luis Fernando Pino Duque2017-02-10
| | | | | | | | | | | | | | | | | | | Turns out --symlink_prefix is not applied to the bazel-{out,workspace} unlike bazel-{bin,genfiles,testlogs} thus breaking the semantics of --symlink_prefix. This CL fixes the problem by using the symlink prefix instead of the product name when building the output and exec root symlinks. The bazel-out symlink is still created for backwards compatiblity but will be removed eventually. RELNOTES[INC]: Using --symlink_prefix is now applied to the output symlink (e.g. bazel-out) and the exec root symlink (e.g. bazel-workspace). -- PiperOrigin-RevId: 147161547 MOS_MIGRATED_REVID=147161547
* Rollback of commit 49d97f9bb4dd04ef62198ded7b53b2a8f8c603db.Gravatar Luis Fernando Pino Duque2017-02-10
| | | | | | | | *** Reason for rollback *** -- PiperOrigin-RevId: 147146321 MOS_MIGRATED_REVID=147146321
* Apply --symlink_prefix to the output and exec root symlinks.Gravatar Luis Fernando Pino Duque2017-02-08
| | | | | | | | | | | | | | | | Turns out --symlink_prefix is not applied to the bazel-{out,workspace} unlike bazel-{bin,genfiles,testlogs} thus breaking the semantics of --symlink_prefix. This CL fixes the problem by using the symlink prefix instead of the product name when building the output and exec root symlinks. RELNOTES[INC]: Using --symlink_prefix is now applied to the output symlink (e.g. bazel-out) and the exec root symlink (e.g. bazel-workspace). -- PiperOrigin-RevId: 146902534 MOS_MIGRATED_REVID=146902534
* Treat configuration errors as failures in the master log.Gravatar Peter Schmitt2017-02-08
| | | | | | -- PiperOrigin-RevId: 146820813 MOS_MIGRATED_REVID=146820813
* Add a flag to print the effective (Spawn)ActionContexts for debugging.Gravatar Philipp Wollermann2017-02-01
| | | | | | | | | | | | | | | | | | | | | | | | | By specifying the flag "--debug_print_action_contexts", Bazel will print the contents of the internal SpawnActionContext and ContextMap maps, which allows developers to see which kind of actions are run using which strategy. Example output of Bazel at HEAD: $ ./output/bazel build --debug_print_action_contexts INFO: SpawnActionContextMap: "" = LinuxSandboxedStrategy INFO: SpawnActionContextMap: "Closure" = WorkerSpawnStrategy INFO: SpawnActionContextMap: "Javac" = WorkerSpawnStrategy INFO: ContextMap: Context = BazelWorkspaceStatusActionContext INFO: ContextMap: CppCompileActionContext = SpawnGccStrategy INFO: ContextMap: CppLinkActionContext = SpawnLinkStrategy INFO: ContextMap: FileWriteActionContext = FileWriteStrategy INFO: ContextMap: FilesetActionContext = FilesetActionContextImpl INFO: ContextMap: IncludeScanningContext = DummyIncludeScanningContext INFO: ContextMap: SpawnActionContext = LinuxSandboxedStrategy INFO: ContextMap: SymlinkTreeActionContext = SymlinkTreeStrategy INFO: ContextMap: TestActionContext = ExclusiveTestStrategy (Can you spot the bug found by this feature here? The default TestActionContext is ExclusiveTestStrategy, which is probably not what we want.) -- PiperOrigin-RevId: 146233390 MOS_MIGRATED_REVID=146233390
* Tiny cleanup: narrow the scope of two variable declarations.Gravatar Greg Estren2017-01-30
| | | | | | -- PiperOrigin-RevId: 145827720 MOS_MIGRATED_REVID=145827720
* Support mapping of Paths to URIsGravatar Klaus Aehlig2017-01-18
| | | | | | | | | | | Bazel-created files (like log files of test runs) are internally reported as Paths. However, this is not always the most useful representation of the location of that artifact for a consumer of build events. Therefore, support a mapping of paths to more useful URIs. -- PiperOrigin-RevId: 144843525 MOS_MIGRATED_REVID=144843525
* Merge the action input prefetcher into the ExecutorBuilder.Gravatar Ulf Adams2017-01-13
| | | | | | | | | This should be a no-op change, primarily intended to improve the BlazeModule API. The code simplification in the distributor code path is incidental. -- PiperOrigin-RevId: 144441458 MOS_MIGRATED_REVID=144441458
* Move some classes to lib.exec in preparation for layering enforcementGravatar Ulf Adams2016-12-21
| | | | | | | | | | | | This change is part of the mu-bazel effort, which aims to build a minimally useful Bazel binary with most extraneous functionality removed. As part of that, we want to enforce layering of packages. In particular, lib.actions must not depend on lib.rules or lib.exec. lib.rules must not depend on lib.exec. Moving these classes is a necessary step to enforce that layering. -- PiperOrigin-RevId: 142668172 MOS_MIGRATED_REVID=142668172
* Only announce test summaries in BEP if expectedGravatar Klaus Aehlig2016-11-30
| | | | | | | | | | | In some situations, we do not even expect a test summary to appear in the BEP, e.g., if testing is not asked for. Do not announce a test summary in this case. -- Change-Id: Ifd46e3582292b087bb1d37f255a140f631854830 Reviewed-on: https://cr.bazel.build/7373 MOS_MIGRATED_REVID=140589645
* Move some actions to language-specific packages, where they belong.Gravatar Lukacs Berki2016-11-23
| | | | | -- MOS_MIGRATED_REVID=140024976
* Code cleanup for interleaving target pattern eval and config creation.Gravatar Ulf Adams2016-11-16
| | | | | -- MOS_MIGRATED_REVID=139209942
generated by cgit v1.2.3 (git 2.39.1) at 2024-08-24 07:31:24 +0000