aboutsummaryrefslogtreecommitdiffhomepage
path: root/src/main/java/com/google/devtools/build/lib/sandbox/ProcessWrapperSandboxedSpawnRunner.java
Commit message (Collapse)AuthorAge
* Add param file support to sandboxed spawn runners.Gravatar tomlu2018-05-01
| | | | | | | Spawns can contain param files in the form of virtual action inputs. Sandboxed runners must write these prior to execution of the action. RELNOTES: None PiperOrigin-RevId: 194951208
* Remove the on-disk caching of Xcode locations.Gravatar jmmv2018-04-28
| | | | | | | | | | | | | | | | | | | | | | | There is no need for the cache to be on disk. Originally, there was a desire to share this cache with other tools... but this never happened. And, actually, because Bazel is in control of what it runs, it can just inject the "cached" values into those tools via flags. Instead, just store the cache in-memory. This avoids having to open and read the cache on every single action that is locally executed on a Mac. Results when building a large iOS app from a clean slate show up to a 1% wall time improvement on my Mac Pro 2013 and a reduction in the variance of the measurements. This change also gets rid of the OS check from the action execution's critical path. There is not much use in checking this: if we instantiate this by mistake, the actual calls will fail. But sometimes we want to actually run this code on non-macOS systems (e.g. for unit-testing with mocked tools), so we should allow that. And this change also ensures that XcodeLocalEnvProviderTest builds and runs... RELNOTES: None. PiperOrigin-RevId: 194681802
* sandbox: Generate actually unique sandbox paths.Gravatar philwo2018-04-25
| | | | | | | | | | | | | Turns out the "unique" id returned by SpawnExecutionContext is only unique within one SpawnRunner, but not across multiple SpawnRunner classes. We have to prefix the generated path with the name of the current strategy in order to not run into conflicts when a build happens to use multiple different sandbox strategies. RELNOTES: None. PiperOrigin-RevId: 194230475
* Rename SpawnExecutionPolicy -> SpawnExecutionContext.Gravatar tomlu2018-04-19
| | | | | | | | This class will be used to tie a Spawn to a SpawnRunner, and isn't really a policy object. It will carry state such as the expanded inputs and expanded command line. Currently a context can be passed between different SpawnRunners. This will be addressed independently, so a context is tied to a particular spawn runner. PiperOrigin-RevId: 193501918
* Roll forward of ↵Gravatar philwo2018-04-16
| | | | | | | | | | | | | | | | | | | | https://github.com/bazelbuild/bazel/commit/656a0bab1e025ff3c27d595284a4bf1c5a8d8028 with test (unknown commit) and fix. Big round of sandbox fixes / performance improvements. - The number of stat() syscalls in the SymlinkedSandboxedSpawn was way too high. Do less, feel better. - When using --experimental_sandbox_base, ensure that symlinks in the path are resolved. Before this, you had to check whether on your system /dev/shm is a symlink to /run/shm and then use that instead. Now it no longer matters, as symlinks are resolved. - Remove an unnecessary directory creation from each sandboxed invocation. Turns out that the "tmpdir" that we created was no longer used after some changes to Bazel's TMPDIR handling. - Use simpler sandbox paths, by using the unique ID for each Spawn provided by SpawnExecutionPolicy instead of a randomly generated temp folder name. This also saves a round-trip from our VFS to NIO and back. Clean up the sandbox base before each build to ensure that the unique IDs are actually unique. ;) - Use Java 8's Process#isAlive to check whether a process is alive instead of trying to get the exitcode and catching an exception. Closes #4913. PiperOrigin-RevId: 193031017
* Internal changeGravatar dannark2018-04-04
| | | | PiperOrigin-RevId: 191642942
* Big round of sandbox fixes / performance improvements.Gravatar Philipp Wollermann2018-03-26
| | | | | | | | | | | | | | | | - The number of stat() syscalls in the SymlinkedSandboxedSpawn was way too high. Do less, feel better. - When using --experimental_sandbox_base, ensure that symlinks in the path are resolved. Before this, you had to check whether on your system /dev/shm is a symlink to /run/shm and then use that instead. Now it no longer matters, as symlinks are resolved. - Remove an unnecessary directory creation from each sandboxed invocation. Turns out that the "tmpdir" that we created was no longer used after some changes to Bazel's TMPDIR handling. - Use simpler sandbox paths, by using the unique ID for each Spawn provided by SpawnExecutionPolicy instead of a randomly generated temp folder name. This also saves a round-trip from our VFS to NIO and back. Clean up the sandbox base before each build to ensure that the unique IDs are actually unique. ;) - Use Java 8's Process#isAlive to check whether a process is alive instead of trying to get the exitcode and catching an exception. Closes #4913. PiperOrigin-RevId: 190472170
* Refactor and cleanup the sandboxing code.Gravatar Philipp Wollermann2018-03-23
| | | | | | | | | | | | | - Remove Optional<> where it's not needed. It's nice for return values, but IMHO it was overused in this code (e.g. Optional<List<X>> is an anti-pattern, as the list itself can already signal that it is empty). - Use Bazel's own Path class when dealing with paths, not String or java.io.File. - Move LinuxSandboxUtil into the "sandbox" package. - Remove dead code and unused fields. - Migrate deprecated VFS method calls to their replacements. - Fix a bug in ExecutionStatistics where a FileInputStream was not closed. Closes #4868. PiperOrigin-RevId: 190217476
* Remove unnecessary product name and constructors from sandboxed spawn runners.Gravatar jmmv2018-03-23
| | | | | | | | | | The product name was previously propagated to all sandboxed spawn runners unconditionally, even when the majority did not need it. Also, these same spawn runners had obsolete constructors that were not used at all or were only used in tests. Remove both for simplicity. RELNOTES: None. PiperOrigin-RevId: 190195305
* Replace instances of XCode with Xcode which is the proper casing for the name.Gravatar Sergio Campama2018-03-06
| | | | | | Closes #4640. PiperOrigin-RevId: 188022228
* Adding a property name to the SpawnRunner. Most runners already had it, I ↵Gravatar olaola2018-02-22
| | | | | | | | | | just add it to the interface, and include it in the SpawnResult. This will be used to categorize/aggregate spawns executed by various runners. Also, minor refinement to the cacheHit property of the SpawnResult with remote execution. RELNOTES: None TESTED=presubmit, next cl PiperOrigin-RevId: 186637978
* Inject the product name in the constructor where neededGravatar ulfjack2018-02-20
| | | | | | ...instead of passing it everywhere so we can rewrite the local env. PiperOrigin-RevId: 186316859
* Automated rollback of commit 04757dba0174d22c0a695a7ed5fe511fd13df008.Gravatar laszlocsomor2018-02-16
| | | | | | | | | | | | | | | | | | | | | | *** Reason for rollback *** There's already a --test_tmpdir flag, and Java tests don't pick up this new one. More info: https://github.com/bazelbuild/bazel/issues/4621#issuecomment-366217321 *** Original change description *** tmpdir,local-exec: implement --local_tmp_root Add new flag called `--local_tmp_root`, which (if specified) tells Bazel what temp directory should locally executed actions use. Fixes https://github.com/bazelbuild/bazel/issues/4621 Related to https://github.com/bazelbuild/bazel/issues/3215 RELNOTES[NEW]: The new "--local_tmp_root=<path>" flag allows specifying the temp directory for locally executed actions. Change-Id: Ice69a5e63d0bf4d3b5c9ef4dbdd1ed1c5025f85e PiperOrigin-RevId: 185982705
* tmpdir,local-exec: implement --local_tmp_rootGravatar Laszlo Csomor2018-02-13
| | | | | | | | | | | | | | Add new flag called `--local_tmp_root`, which (if specified) tells Bazel what temp directory should locally executed actions use. Fixes https://github.com/bazelbuild/bazel/issues/4621 Related to https://github.com/bazelbuild/bazel/issues/3215 RELNOTES[NEW]: The new "--local_tmp_root=<path>" flag allows specifying the temp directory for locally executed actions. Change-Id: Ice69a5e63d0bf4d3b5c9ef4dbdd1ed1c5025f85e PiperOrigin-RevId: 185509555
* actions,temp: respect TMPDIR envvarGravatar Laszlo Csomor2018-01-15
| | | | | | | Fixes https://github.com/bazelbuild/bazel/issues/4376 Change-Id: Id78bb0930044626304e54f07735db4d4b2c84720 PiperOrigin-RevId: 181959528
* sandbox: add env[TMPDIR] instead of `tmpDir`Gravatar Laszlo Csomor2018-01-11
| | | | | | | | | | | | | | | | Bazel now adds env[TMPDIR] to the set of sandbox-writable paths, instead of adding the caller-defined `tmpDir` as it used to. Since every caller of getWritableDirs passes the LocalEnvProvider-processed environment to getWritableDirs, and because all such callers use either PosixLocalEnvProvider or XCodeLocalEnvProvider, we can be sure that the environment has an entry for TMPDIR. Change-Id: Ia89544a009e56d9cc922ab56823d16d20465545e PiperOrigin-RevId: 181595606
* sandbox runners: always use updated environmentGravatar Laszlo Csomor2018-01-08
| | | | | | | | | | | The sandboxed Spawn runners already used to call LocalEnvProvider.rewriteEnvironment in order to add TMPDIR for example, but didn't consistently use this rewritten environment everywhere, and used the Spawn's original environment instead. Change-Id: Ifbede6ac0f14b83c617dcbcd85edfae88718b157 PiperOrigin-RevId: 181130506
* Extract LocalEnvProvider members to classes.Gravatar Laszlo Csomor2018-01-04
| | | | | | | | | | | | | | | | | | Create a PosixLocalEnvProvider and WindowsLocalEnvProvider class, with singleton instances for now. This refactoring should not change functionality, it's just a requirement for an upcoming change. That upcoming change is for these classes to respect the client environment's TMPDIR or TMP/TEMP envvars. See https://github.com/bazelbuild/bazel/issues/4376 Change-Id: I032bb6f18adf8af9e43e6bc543c09c58adae3863 PiperOrigin-RevId: 180799936
* Enable local action execution statistics collection for sandboxed actions ↵Gravatar ruperts2017-12-22
| | | | | | | | | | | | | that use either the LinuxSandboxedSpawnRunner or the ProcessWrapperSandboxedSpawnRunner. In particular, record metrics for user and system CPU execution time, block I/O and involuntary context switches. This feature is guarded behind a new option, --experimental_collect_local_sandbox_action_metrics. Note: We still need to enable execution statistics for the DarwinSandboxedSpawnRunner in a later change. RELNOTES: None. PiperOrigin-RevId: 179976217
* Move compulsory ProcessWrapperUtil parameters into its builder() method, to ↵Gravatar ruperts2017-12-20
| | | | | | | be consistent with LinuxSandboxUtil. RELNOTES: None. PiperOrigin-RevId: 179758847
* Add a CommandLineBuilder for the process-wrapper embedded tool, and use it ↵Gravatar ruperts2017-11-30
| | | | | | | | | everywhere instead of duplicating process-wrapper --shell_arguments in Blaze. To avoid a cyclic dependency, I broke up exec/local:local into exec/local:local and exec/local:options. RELNOTES: None. PiperOrigin-RevId: 177419268
* Remove process-wrapper duplication from RunCommand.Gravatar ruperts2017-11-28
| | | | | | | Also rename ProcessWrapperRunner to ProcessWrapperUtil, since it isn't really a Runner (it doesn't run anything). RELNOTES: None. PiperOrigin-RevId: 177260951
* Actions now have a temp envvar.Gravatar László Csomor2017-10-16
| | | | | | | | | | | | | | | | | | | | | Every build and test action that creates a Spawn will now have platform-specific environment variables for temp directories: - on Windows: TMP and TEMP - on Linux/Darwin: TMPDIR This is particularly important on Windows where e.g. Java programs cannot create temp directories unless there's a valid TMP or TEMP environment variable set. Fixes: - https://github.com/bazelbuild/bazel/issues/1590 - https://github.com/bazelbuild/bazel/issues/2349 - https://github.com/bazelbuild/bazel/issues/2870 Change-Id: Ib758307daf6b3a51b0f71ae5e65e5bb564dad643 PiperOrigin-RevId: 172326371
* Move SpawnResult from build.lib.exec into build.lib.actions so that e.g. ↵Gravatar ruperts2017-09-22
| | | | | | | build.lib.actions.SpawnActionContext can import SpawnResult without creating a cyclic dependency. RELNOTES: None. PiperOrigin-RevId: 169642267
* Use java.time.Duration for timeoutsGravatar ulfjack2017-08-09
| | | | PiperOrigin-RevId: 164577062
* Refactor BuildRequest out of sandboxed spawn runner initialization.Gravatar Benjamin Peterson2017-08-04
| | | | | | | | | | | | | Remove BuildRequest as parameter to sandboxed spawn runner constructors. Previously, the build request was used to obtain some options, but those can be extricated from a CommandEnvironment, which is passed in, too. Also, remove LinuxSandboxedSpawnRunner's aliased sandboxOptions member variable. It can just use the superclass's. Change-Id: I1ef1a45cbf7e800d0809f05673f097a148289740 PiperOrigin-RevId: 164257471
* Fix #1027: XML test output missing for tests that timeoutGravatar philwo2017-07-19
| | | | | | The sandbox execution strategies were not passing the timeout grace period specified via --local_termination_grace_seconds, resulting in process-wrapper defaulting to just 5 seconds and linux-sandbox not using any grace period at all, instead directly SIGKILLing the child on timeout. PiperOrigin-RevId: 162349919
* Split the sandbox strategies into SpawnRunner implementations and strategiesGravatar ulfjack2017-07-13
This adds a bunch of classes that only implement the SpawnRunner interface, and will allow us to support remote caching in combination with local sandboxed execution in a subsequent change. PiperOrigin-RevId: 161664556
generated by cgit v1.2.3 (git 2.39.1) at 2024-07-07 07:41:05 +0000