| Commit message (Collapse) | Author | Age |
|---|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
https://github.com/bazelbuild/bazel/commit/656a0bab1e025ff3c27d595284a4bf1c5a8d8028 with test (unknown commit) and fix.
Big round of sandbox fixes / performance improvements.
- The number of stat() syscalls in the SymlinkedSandboxedSpawn was way too high. Do less, feel better.
- When using --experimental_sandbox_base, ensure that symlinks in the path are resolved. Before this, you had to check whether on your system /dev/shm is a symlink to /run/shm and then use that instead. Now it no longer matters, as symlinks are resolved.
- Remove an unnecessary directory creation from each sandboxed invocation. Turns out that the "tmpdir" that we created was no longer used after some changes to Bazel's TMPDIR handling.
- Use simpler sandbox paths, by using the unique ID for each Spawn provided by SpawnExecutionPolicy instead of a randomly generated temp folder name. This also saves a round-trip from our VFS to NIO and back. Clean up the sandbox base before each build to ensure that the unique IDs are actually unique. ;)
- Use Java 8's Process#isAlive to check whether a process is alive instead of trying to get the exitcode and catching an exception.
Closes #4913.
PiperOrigin-RevId: 193031017
|
| |
|
|
| |
PiperOrigin-RevId: 191642942
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- The number of stat() syscalls in the SymlinkedSandboxedSpawn was way too high. Do less, feel better.
- When using --experimental_sandbox_base, ensure that symlinks in the path are resolved. Before this, you had to check whether on your system /dev/shm is a symlink to /run/shm and then use that instead. Now it no longer matters, as symlinks are resolved.
- Remove an unnecessary directory creation from each sandboxed invocation. Turns out that the "tmpdir" that we created was no longer used after some changes to Bazel's TMPDIR handling.
- Use simpler sandbox paths, by using the unique ID for each Spawn provided by SpawnExecutionPolicy instead of a randomly generated temp folder name. This also saves a round-trip from our VFS to NIO and back. Clean up the sandbox base before each build to ensure that the unique IDs are actually unique. ;)
- Use Java 8's Process#isAlive to check whether a process is alive instead of trying to get the exitcode and catching an exception.
Closes #4913.
PiperOrigin-RevId: 190472170
|
| |
|
|
|
|
| |
They need this to parse input manifests. Previously we would grab the exec root from the Root, but wish to unsupport this.
PiperOrigin-RevId: 181669143
|
| |
|
|
|
|
|
| |
...instead of injecting it through the constructor. Simplify all the callers
accordingly.
PiperOrigin-RevId: 164955391
|
| |
|
|
|
|
| |
This removes a bunch of code duplication that I previously introduced.
PiperOrigin-RevId: 162909430
|
| |
|
|
|
|
| |
The sandbox execution strategies were not passing the timeout grace period specified via --local_termination_grace_seconds, resulting in process-wrapper defaulting to just 5 seconds and linux-sandbox not using any grace period at all, instead directly SIGKILLing the child on timeout.
PiperOrigin-RevId: 162349919
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The main change here is to only catch SpawnExecException in
StandaloneTestStrategy, so all other exceptions simplify propagate up. As a
result, Bazel no longer retries tests that fail with an exception, we only
retry tests that actually ran, had a spawn result, and resulted in a
UserExecException. That is probably what we want.
Also do some cleanup:
- Remove ExecException.timedOut; nobody was calling it (but there's still
SpawnExecException.timedOut)
- Remove SpawnActionContext.shouldPropagateExecException; all exceptions
(except SpawnExecException) are now propagated by default
- Remote the SandboxOptions from the SandboxStrategies; all sandboxing options
are now handled by the underlying SpawnRunner implementations
I'll send a followup CL to remove the UserExecException and
EnvironmentalExecException types; the types don't do anything special, and
there are no catch blocks in production code that catch one of these more
specific types.
This should fix #3322 by removing a bunch of special handling.
PiperOrigin-RevId: 161960919
|
| |
|
|
|
|
|
|
| |
This adds a bunch of classes that only implement the SpawnRunner interface, and
will allow us to support remote caching in combination with local sandboxed
execution in a subsequent change.
PiperOrigin-RevId: 161664556
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
- Make use of existing abstractions like SpawnRunner and SpawnExecutionPolicy.
- Instead of having the *Strategy create a *Runner, and then call back into
SandboxStrategy, create a single SandboxContainer which contains the full
command line, environment, and everything needed to create and delete the
sandbox directory.
- Do all the work in SandboxStrategy, including creation and deletion of the
sandbox directory.
- Use SpawnResult instead of throwing, catching, and rethrowing.
- Simplify the control flow a bit.
PiperOrigin-RevId: 161644979
|
| |
|
|
|
|
|
|
| |
This will allow us to add new and optional flags like selecting a strategy used to spawn / wait for the child process.
No one except Bazel should be calling "process-wrapper" and I couldn't find any references, so this breaking change should be fine.
PiperOrigin-RevId: 159685867
|
| |
|
|
|
|
|
| |
Move everything to ActionExecutionContext, and drop Executor whereever possible.
This clarifies the API, makes it simpler to test, and simplifies the code.
PiperOrigin-RevId: 159414816
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Also add an interface to allow injecting that logic into LocalSpawnRunner; this
is in preparation for rewriting StandaloneSpawnStrategy to use
LocalSpawnRunner.
At the same time, this reduces the dependencies from exec / standalone to
rules.apple, which is a prerequisite for micro-Bazel.
There's a small semantic change hidden here - we now only set the new
XCodeLocalEnvProvider if we're actually running on Darwin, so we no longer
fail execution on non-Darwin platforms if XCODE_VERSION_OVERRIDE or
APPLE_SDK_VERSION_OVERRIDE is set. As a result, I moved the corresponding test
from StandaloneSpawnStrategyTest to the new XCodeLocalEnvProviderTest.
While I'm at it, also open source DottedVersionTest and CacheManagerTest.
PiperOrigin-RevId: 158829077
|
| |
|
|
|
|
|
|
|
| |
This unifies our code to use just one standard implementation to get the
entire expanded input files for a Spawn, including from Filesets and
Runfiles.
Change-Id: I1e286508adf0a9aeddf70934b010e6fcc144c4a7
PiperOrigin-RevId: 155497273
|
| |
|
|
|
| |
Change-Id: I1bc1901ea7cd9a5b93c280ec0ff8ac0d10959a09
PiperOrigin-RevId: 155381163
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
(fallback only).
On macOS the processwrapper-sandbox will be used when the darwin-sandbox doesn't
work. Most notably this is the case for nested sandboxing, e.g. Bazel running
Bazel inside an integration test.
Also includes a fix to pull in some extra environment vars on macOS, similar to
what DarwinSandboxedStrategy and StandaloneSpawnStrategy already do. Without
this the processwrapper-sandbox seems to occasionally cause ObjC builds (and two
of our tests) to fail.
Change-Id: Ic7462080caf56d9bb98e2f3765bd37853b01632b
RELNOTES: Sandboxing is now enabled by default on FreeBSD (via processwrapper-sandbox).
PiperOrigin-RevId: 155366728
|
| |
|
|
|
|
|
|
|
|
|
| |
selected even if they're not the preferred one on a platform.
Simplify the SandboxActionContextProvider and remove the warning about
sandboxing being unsupported. With the ProcessWrapperSandboxedStrategy
now being reliable enough and the strategies printing their real name in
the UI, this is overall a better UX.
PiperOrigin-RevId: 153825986
|
| |
|
|
|
|
| |
With the process-wrapper improvements and the additional deletion of the sandbox base in the SandboxModule in, this should be reliable enough. The warning was also not actionable for users and annoyed them, so let's get rid of it.
PiperOrigin-RevId: 153823045
|
| |
|
|
|
|
|
|
| |
Usually, Bazel creates the sandbox directories underneath its
output_base. With --experimental_sandbox_base you can specify a
different parent directory for this, e.g. /dev/shm to run all sandboxed
actions on a memory-backed filesystem.
PiperOrigin-RevId: 152490815
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
*** Reason for rollback ***
Breaks //src/test/shell/integration:force_delete_output_test
*** Original change description ***
Symlink output directories to the correct directory name
If the workspace directory is /path/to/my/proj and the name in the WORKSPACE
file is "floop", this will symlink the output directories to
output_base/execroot/floop instead of output_base/execroot/proj.
More prep for #1262, fixes #1681.
PiperOrigin-RevId: 152126545
|
| |
|
|
|
|
|
|
|
|
| |
If the workspace directory is /path/to/my/proj and the name in the WORKSPACE
file is "floop", this will symlink the output directories to
output_base/execroot/floop instead of output_base/execroot/proj.
More prep for #1262, fixes #1681.
PiperOrigin-RevId: 151712384
|
|
|
Extract the process-wrapper + symlink tree sandbox strategy into its own
class and allow its use in FreeBSD.
RELNOTES: Bazel can now use the process-wrapper + symlink tree based sandbox implementation in FreeBSD.
--
PiperOrigin-RevId: 151171652
MOS_MIGRATED_REVID=151171652
|
philwo
dannark
tomlu
ulfjack
hlopko
kchodorow