aboutsummaryrefslogtreecommitdiffhomepage
path: root/src/main/java/com/google/devtools/build/lib/runtime/CommandEnvironment.java
Commit message (Collapse)AuthorAge
* Store Skylark command-line flags in SkyframeGravatar brandjon2017-04-28
| | | | | | | | | This is the first of two CLs for making command line options able to affect the Skylark interpreter. It introduces SkylarkSemanticsOptions, and stores it as a precomputed (injected) value in Skyframe. The next CL will read these options from Skyframe when constructing the Skylark environment. This CL affects the dataflow from command/test initialization to Skyframe. Some code paths, like those used for testing, use the default SkylarkSemanticsOptions and therefore won't be able to use (for example) --incompatible_* flags. The call sites to update were found by searching for uses of defaultVisibility and working upward from there. RELNOTES: None PiperOrigin-RevId: 154432058
* Add repository override optionGravatar kchodorow2017-04-20
| | | | | | | | | | | RELNOTES: Adds a --override_repository option that takes a repository name and path. This forces Bazel to use the directory at that path for the repository. Example usage: `--override_repository=foo=/home/user/gitroot/foo`. Fixes #1266 PiperOrigin-RevId: 153599291
* PiperOrigin-RevId: 152483983Gravatar lpino2017-04-07
|
* When tracking the critical path, if not keeping incremental state, don't ↵Gravatar janakr2017-04-06
| | | | | | | | keep references to actions indefinitely. Instead, once an action is finished executing, keep just some metadata about it. This allows actions to be unconditionally dropped when running with --batch, --discard_analysis_cache, and --keep_going, even if profiling is enabled. The additional fields here add between 8 and 12 bytes per component, and we have one component per action. This additional penalty is only incurred when we are already saving memory, so I think it's ok. The full penalty will be realized only towards the end of the build, when every action has started executing at least once. Users can still specify --noexperimental_enable_critical_path_profiling if they want to squeeze even more memory out. PiperOrigin-RevId: 152328870
* Move action_outs directory to under the output path.Gravatar janakr2017-04-06
| | | | PiperOrigin-RevId: 152307322
* Automated g4 rollback of commit 1d9e1ac90197b1d3d7b137ba3c1ada67bb9ba31b.Gravatar hlopko2017-04-04
| | | | | | | | | | | | | | | | | | *** Reason for rollback *** Breaks //src/test/shell/integration:force_delete_output_test *** Original change description *** Symlink output directories to the correct directory name If the workspace directory is /path/to/my/proj and the name in the WORKSPACE file is "floop", this will symlink the output directories to output_base/execroot/floop instead of output_base/execroot/proj. More prep for #1262, fixes #1681. PiperOrigin-RevId: 152126545
* If --batch, --keep_going, --discard_analysis_cache, and the new ↵Gravatar janakr2017-03-31
| | | | | | | | | | | | | | | --noexperimental_enable_critical_path_profiling flags are all specified, then Bazel will delete Actions from ActionLookupValues as they are executed in order to save memory. Because an already-run action may output an artifact that is only requested later in the build, we need to maintain a way for the artifact to look up the action. But in most cases we don't need to keep the action itself, just its output metadata. Some actions unfortunately are needed post-execution, and so we special-case them. Also includes dependency change with description: Move action out of key. This keeps action references from polluting the graph -- actions are just stored in one SkyValue, instead of being present in SkyKeys. This does mean additional memory used: we have a separate ActionLookupData object per Action executed. That may reach ~24M for million-action builds. PiperOrigin-RevId: 151756383
* Symlink output directories to the correct directory nameGravatar kchodorow2017-03-31
| | | | | | | | | | If the workspace directory is /path/to/my/proj and the name in the WORKSPACE file is "floop", this will symlink the output directories to output_base/execroot/floop instead of output_base/execroot/proj. More prep for #1262, fixes #1681. PiperOrigin-RevId: 151712384
* Env.vars: server won't ignore the client envGravatar Dmitry Lomov2017-03-08
| | | | | | | | | | | | | | | | | | Make the --ignore_client_env flag a no-op. The client will pass --client_env flags to the server even in --batch mode. This simplifies the code as well as ensuring that the server always uses the up-do-date client environment. We'll gradually get rid of all System.getenv calls in the server, because the server should always respect the client env. Roll forward of 149403129 with fixes. -- PiperOrigin-RevId: 149435060 MOS_MIGRATED_REVID=149435060
* Rollback of commit 94d8f4e9c8c8757aa7ab7c1a7c3e9afb34039127.Gravatar Klaus Aehlig2017-03-07
| | | | | | | | | | | | | | | | | | | | | | | | *** Reason for rollback *** broke //src/test/shell/bazel:bazel_bootstrap_distfile_test *** Original change description *** Env.vars: server won't ignore the client env Make the --ignore_client_env flag a no-op. The client will pass --client_env flags to the server even in --batch mode. This simplifies the code as well as ensuring that the server always uses the up-do-date client environment. We'll gradually get rid of all System.getenv calls in the server, because the server should always respect the client env. -- PiperOrigin-RevId: 149416602 MOS_MIGRATED_REVID=149416602
* Env.vars: server won't ignore the client envGravatar Laszlo Csomor2017-03-07
| | | | | | | | | | | | | | | | Make the --ignore_client_env flag a no-op. The client will pass --client_env flags to the server even in --batch mode. This simplifies the code as well as ensuring that the server always uses the up-do-date client environment. We'll gradually get rid of all System.getenv calls in the server, because the server should always respect the client env. -- PiperOrigin-RevId: 149403129 MOS_MIGRATED_REVID=149403129
* Provide more reporting options to SkyFunctions Gravatar Klaus Aehlig2017-02-27
| | | | | | | | | | | | | With more specific information to be reported by Skyfunctions, e.g., to inform the build-event protocol on missing files, the EventHandler interface is no longer enough. Therefore, provide an enriched context for reporting events. -- Change-Id: I2d06166fe4d5b9054e24ad8c752fafc039e3f9f8 Reviewed-on: https://cr.bazel.build/8794 PiperOrigin-RevId: 148463437 MOS_MIGRATED_REVID=148463437
* Introduce a Skyframe function: ActionEnvironmentFunction Gravatar Damien Martin-Guillerez2017-02-08
| | | | | | | | | | | | | | | | | | ActionEnvironmentFunction returns the list of environment variable with the one overwritten by --action_env being replaced. This let other Skyframe function declares dependency to any value of the environment and being influenced by the --action_env flag. This will be used to declare dependency of remote repositories on environment variables (step 3 of https://bazel.build/designs/2016/10/18/repository-invalidation.html) -- Change-Id: I1ed3fb6f48e8e17d4d64c903fccecb6ed7596350 Reviewed-on: https://cr.bazel.build/7974 PiperOrigin-RevId: 146918603 MOS_MIGRATED_REVID=146918603
* Merge the action input prefetcher into the ExecutorBuilder.Gravatar Ulf Adams2017-01-13
| | | | | | | | | This should be a no-op change, primarily intended to improve the BlazeModule API. The code simplification in the distributor code path is incidental. -- PiperOrigin-RevId: 144441458 MOS_MIGRATED_REVID=144441458
* Skylark repositories: propage value from --action_env to repository_ctx.environ Gravatar Damien Martin-Guillerez2017-01-10
| | | | | | | | | | | | | | | | | | | | | | The environment is now computed with a mixture of the client environment and the values specified by the --action_env flag. If a user want to overwrite its environment for skylark repository, they can do `--action_env FOO=BAR` and the repository will see FOO as having the value BAR, whichever value is set in the client environment. Also propagate it to all repository functions, and deduplicate the way the client environment is passed to repository functions. Design doc: https://bazel.build/designs/2016/10/18/repository-invalidation.html [step 1] RELNOTES[INC]: repository_ctx environment is now affected by --action_env flag (value from the client environment will be replaced by value given on the command line through --action_env). -- Change-Id: I131a9695439aa9949d5001f820e2ae450e41332f Reviewed-on: https://cr.bazel.build/7971 PiperOrigin-RevId: 144091492 MOS_MIGRATED_REVID=144091492
* Don't get output service or action input prefetcher if not running a "build" ↵Gravatar Janak Ramakrishnan2016-11-18
| | | | | | | command, to save time on the critical path of a build. -- MOS_MIGRATED_REVID=139477157
* Stop including the size of the action cache file in our crash message. I've ↵Gravatar Janak Ramakrishnan2016-11-17
| | | | | | | never seen a case it was useful, and it adds to startup latency by putting a file stat on the critical path of every command. -- MOS_MIGRATED_REVID=139467038
* Merge all action input prefetchers into a single one.Gravatar Ulf Adams2016-11-16
| | | | | | | | | It seems unnecessary to pass these lists around, so this should simplify the APIs. Also, I want to move the action input prefetcher setup to the serverInit call. -- MOS_MIGRATED_REVID=139304928
* Add a GenSignatureActionContext to inject the command line info.Gravatar Ulf Adams2016-11-15
| | | | | | | | | This is needed to fulfill the contract of the gensignature rule / the corresponding audit trail protobuf. Note that the action is uncachable and unconditional to execute. -- MOS_MIGRATED_REVID=139178114
* Refactor the module API to use the builder pattern for executor creation.Gravatar Ulf Adams2016-11-02
| | | | | | | This significantly simplifies several of our modules. -- MOS_MIGRATED_REVID=137713119
* Make --watchfs a common command option.Gravatar Ulf Adams2016-10-14
| | | | | | | | | | | | | | | | | | | | | | | | | Adding an options parameter to DiffAwareness#getCurrentView seems like the simplest way to achieve that. Alternatives considered: 1. Making the diff awareness modules stateful. However, I did not want to do so as I've also been working on improving the module API to reduce state, or at least to have a proper lifecycle management for any necessary state. 2. Making the watchFs flag a constructor parameter. However, that would also invalidate any implementations that don't use the flag (of which we have several). 3. Only passing in a single boolean flag instead of an options class provider; however, this is a more principled, futureproof API, which allows other modules / awareness implementations to use their own options. RELNOTES: --watchfs is now a command option; the startup option of the same name is deprecated. I.e., use bazel build --watchfs, not blaze --watchfs build. -- MOS_MIGRATED_REVID=136154395
* Rollback of commit 2891ec527eed27d0c6460d66f51cb66a43373b6a.Gravatar Damien Martin-Guillerez2016-10-14
| | | | | | | | | | | | | | | | | | | | | | | | | | *** Reason for rollback *** Causes our integration tests on Darwin to time out *** Original change description *** Make --watchfs a common command option. Adding an options parameter to DiffAwareness#getCurrentView seems like the simplest way to achieve that. Alternatives considered: 1. Making the diff awareness modules stateful. However, I did not want to do so as I've also been working on improving the module API to reduce state, or at least to have a proper lifecycle management for any necessary state. 2. Making the watchFs flag a constructor parameter. However, that would also invalidate any implement... *** -- MOS_MIGRATED_REVID=136070807
* Make --watchfs a common command option.Gravatar Ulf Adams2016-10-13
| | | | | | | | | | | | | | | | | | | | | | | | | Adding an options parameter to DiffAwareness#getCurrentView seems like the simplest way to achieve that. Alternatives considered: 1. Making the diff awareness modules stateful. However, I did not want to do so as I've also been working on improving the module API to reduce state, or at least to have a proper lifecycle management for any necessary state. 2. Making the watchFs flag a constructor parameter. However, that would also invalidate any implementations that don't use the flag (of which we have several). 3. Only passing in a single boolean flag instead of an options class provider; however, this is a more principled, futureproof API, which allows other modules / awareness implementations to use their own options. RELNOTES: --watchfs is now a command option; the startup option of the same name is deprecated. I.e., use bazel build --watchfs, not blaze --watchfs build. -- MOS_MIGRATED_REVID=136026835
* Enable Bazel commands to exit at any time.Gravatar Michael Staib2016-10-05
| | | | | | | | | | | | | This is the first step on a journey toward allowing commands to AbruptExit wherever they please, similar to how the user can press Ctrl+C at any time and we (should) bail out as fast as we can. By interrupting the command's main thread, we at least offer the command the ability to see that an error requiring a bail has happened, and it should trigger at potentially more locations, rather than just between phases. -- MOS_MIGRATED_REVID=135152330
* Add an info item to show the currently inherited client environmentGravatar Klaus Aehlig2016-09-21
| | | | | | | | | | | | | Add 'bazel info client-env' which outputs entries for the configuration file that would freeze the current client environment. The main intended use case is to use 'bazel info client-env >> .bazelrc' to keep the project reproducible once a suitable value for the environment variables that used to be taken from the client environment has been found. -- Change-Id: Ib4d14dd824d223f335a4d4de04ee21c4a3ec4d83 Reviewed-on: https://bazel-review.googlesource.com/#/c/6112 MOS_MIGRATED_REVID=133699234
* Remove some more occurences of deprecated getExecRoot() callsGravatar Kristina Chodorow2016-09-15
| | | | | -- MOS_MIGRATED_REVID=133257532
* Track client environment in SkyframeGravatar Klaus Aehlig2016-09-14
| | | | | | | | | | | | | ...to determine which actions have to be recomputed based on changes to the client environment. Note that this change does it the simple way and reconsideres all actions on a changed client environment, while still only reexecuting those, where the part that was inherited from the environment actually did change. -- Change-Id: Ie1116d094642165e5e959447a6fcf49d19b37d6e Reviewed-on: https://bazel-review.googlesource.com/#/c/5431 MOS_MIGRATED_REVID=133010705
* Only call BlazeModule.handleOptions when we're done editing them.Gravatar Ulf Adams2016-09-06
| | | | | | | | | Previously, it was possible for us to edit the options in command.editOptions, after reporting them to the modules, though I don't have any examples of that happening. I found this just looking through the code. -- MOS_MIGRATED_REVID=132310873
* Add a framework for prefetching input files (in case they come from remote ↵Gravatar Philipp Wollermann2016-08-25
| | | | | | | file systems). -- MOS_MIGRATED_REVID=131280794
* Support setting the build id via the client environmentGravatar Klaus Aehlig2016-08-16
| | | | | | | | | | | | | | | | Normally, Bazel generates a new UUID for every build invocation. In some use cases, however, the invocation of Bazel is part of a larger build process or otherwise controlled by a different tool. In this case, it is useful to tell Bazel the identifier of the build to make it fit with other identifiers generated by the controlling process. To achieve this, the variable BAZEL_INTERNAL_INVOCATION_ID from the client environment is inspected and if it is set and its value is a syntactically correct UUID, this UUID will be used as the identifier for the build. It is in the responsibility of the caller to ensure the id is sufficiently unique if that environment variable is set. -- MOS_MIGRATED_REVID=130079446
* Remove unused methodGravatar Kristina Chodorow2016-07-21
| | | | | -- MOS_MIGRATED_REVID=128080879
* Various fixes for gRPC mode:Gravatar Lukacs Berki2016-07-07
| | | | | | | | - Use the abrupt exit protocol to return the correct exit code when the server dies unexpectedly in gRPC mode - Report the command waiting time correctly even if the waiting is done within the server -- MOS_MIGRATED_REVID=126780186
* Post BlazeDirectories as part of CommandStartedEvent.Gravatar Ulf Adams2016-07-06
| | | | | | | | We have some modules that assume 1:1 correspondence between servers and workspaces, which I'd like to fix. -- MOS_MIGRATED_REVID=126628292
* Make "bazel info" load only the packages that are necessary for its job.Gravatar Lukacs Berki2016-06-01
| | | | | -- MOS_MIGRATED_REVID=123723401
* When we don't have an OutputService, precompute and cache the ↵Gravatar Nathan Harmata2016-05-02
| | | | | | | | | | | assumed-to-be-immutable filesystem type of output_base. When we do have an OutputService, defer computation of the OutputService's dynamic filesystem type until we actually need it at the start of a "build" or "test" command. This saves ~450us on each invocation, according to my benchmarks. -- MOS_MIGRATED_REVID=121156323
* Inject the BlazeDirectories in SkyframeExecutor.Gravatar Ulf Adams2016-04-15
| | | | | | | | | The BlazeDirectories are also needed for loading the WORKSPACE file, so inject them as part of preparePackageLoading rather than in createConfigurations, which is too late. -- MOS_MIGRATED_REVID=119931633
* Move BlazeRuntime.initCommand to BlazeWorkspace.Gravatar Ulf Adams2016-03-30
| | | | | -- MOS_MIGRATED_REVID=118568284
* Move most of the workspace-handling code from BlazeRuntime to a new class.Gravatar Ulf Adams2016-03-30
| | | | | -- MOS_MIGRATED_REVID=118563271
* Move most BlazeRuntime methods related to workspace directories to CommandEnv.Gravatar Ulf Adams2016-03-30
| | | | | -- MOS_MIGRATED_REVID=118561661
* Move the BuildView creation to BuildTool, the only remaining user.Gravatar Ulf Adams2016-03-30
| | | | | -- MOS_MIGRATED_REVID=118547283
* Delay BuildView creation in CommandEnvironment until beforeCommand().Gravatar Ulf Adams2016-03-30
| | | | | | | | This is so that we can access the build options during creation, which is a step towards making the coverage module stateless. -- MOS_MIGRATED_REVID=118475354
* Remove calls of CommandEnvironment.getView; remove ide-info from BuildView.Gravatar Ulf Adams2016-03-30
| | | | | | | This makes the late initialization of BuildView more obviously safe. -- MOS_MIGRATED_REVID=118469655
* Move TimestampGranularityMonitor to CommandEnvironment.Gravatar Ulf Adams2016-03-23
| | | | | | | | This is one of the last pieces of state in BlazeRuntime that isn't safe to share across command invocations. -- MOS_MIGRATED_REVID=117910631
* Add an experimental flag to enable the Skyframe-based loading phase runner.Gravatar Ulf Adams2016-02-12
| | | | | | | | | | | | | | Refactor the CommandEnvironment to no longer create a LoadingPhaseRunner eagerly. Unfortunately, that means we can't reuse the TargetPatternEvaluator either. Fortunately, the SkyframeTargetPatternEvaluator is a very lightweight class, which only contains the offset as state, so we just create a new instance whenever we need one. On the plus side, the LoadingPhaseRunner API is now stateless, and the QueryCommand no longer creates a unnecessary LoadingPhaseRunner instance. -- MOS_MIGRATED_REVID=114519731
* Use Bazel Preconditions variant which avoids varargs array creationGravatar Mark Schaller2015-12-10
| | | | | | | Reduces garbage. -- MOS_MIGRATED_REVID=109914243
* Track the entire OutputService instead of just the BatchStatter.Gravatar Eric Fellheimer2015-11-16
| | | | | -- MOS_MIGRATED_REVID=107800790
* Refactor the loading phase runner creation and remove the test duplication.Gravatar Ulf Adams2015-11-06
| | | | | | | | | The loading phase runner is now always created by the SkyframeExecutor, and the duplicate test is dropped in favor of subclassing and overriding one method. -- MOS_MIGRATED_REVID=107188756
* Add a skyframe-based loading phase runner hidden behind a compile-time const.Gravatar Ulf Adams2015-11-03
| | | | | | | More testing required before we even add a command-line option. -- MOS_MIGRATED_REVID=106922350
* Split the LoadingPhaseRunner into interface and implementation.Gravatar Ulf Adams2015-11-03
| | | | | | | | This is in preparation for adding a Skyframe-based implementation that uses the recently added TargetPatternPhaseFunction. -- MOS_MIGRATED_REVID=106920157
* Move LoadingPhaseRunner.loadForConfigurations to CommandEnvironment.Gravatar Ulf Adams2015-11-02
| | | | | | | | It's unclear if the method is still necessary with Skyframe, but I don't want to investigate in detail right now. -- MOS_MIGRATED_REVID=106838896
generated by cgit v1.2.3 (git 2.39.1) at 2024-08-06 12:04:47 +0000