| Commit message (Collapse) | Author | Age |
|---|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
Remove BuildRequest as parameter to sandboxed spawn runner
constructors. Previously, the build request was used to obtain some
options, but those can be extricated from a CommandEnvironment, which
is passed in, too.
Also, remove LinuxSandboxedSpawnRunner's aliased sandboxOptions member
variable. It can just use the superclass's.
Change-Id: I1ef1a45cbf7e800d0809f05673f097a148289740
PiperOrigin-RevId: 164257471
|
| |
|
|
|
|
| |
This removes a bunch of code duplication that I previously introduced.
PiperOrigin-RevId: 162909430
|
| |
|
|
|
|
| |
The sandbox execution strategies were not passing the timeout grace period specified via --local_termination_grace_seconds, resulting in process-wrapper defaulting to just 5 seconds and linux-sandbox not using any grace period at all, instead directly SIGKILLing the child on timeout.
PiperOrigin-RevId: 162349919
|
| |
|
|
|
|
|
|
| |
This adds a bunch of classes that only implement the SpawnRunner interface, and
will allow us to support remote caching in combination with local sandboxed
execution in a subsequent change.
PiperOrigin-RevId: 161664556
|
| |
|
|
|
|
|
|
| |
This will allow us to add new and optional flags like selecting a strategy used to spawn / wait for the child process.
No one except Bazel should be calling "process-wrapper" and I couldn't find any references, so this breaking change should be fine.
PiperOrigin-RevId: 159685867
|
| |
|
|
| |
PiperOrigin-RevId: 159423459
|
| |
|
|
| |
PiperOrigin-RevId: 159221067
|
| |
|
|
|
|
|
|
|
|
|
|
| |
This is basically a rollback of https://github.com/bazelbuild/bazel/commit/3e2329a73ffd5d60e5e2babe60ebe5bf322c07da, except this solves the
reason why the feature was removed in the first place. We now create
the helper files necessary to make files unreadable in Linux in Bazel's
Java code and manage their lifetime there.
Request was filed by a user here:
http://stackoverflow.com/questions/43849651/how-to-lock-down-the-bazel-filesystem-sandbox
PiperOrigin-RevId: 155913246
|
| |
|
|
|
|
|
|
|
| |
This gives us much improved process management, because Bazel can now
reliably kill child processes of spawns via their process group and wait
for them to exit.
Change-Id: Ib3cb20725b3c569aa5b317a69d7682f5774707b0
PiperOrigin-RevId: 155493511
|
| |
|
|
|
| |
Change-Id: I1bc1901ea7cd9a5b93c280ec0ff8ac0d10959a09
PiperOrigin-RevId: 155381163
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
(fallback only).
On macOS the processwrapper-sandbox will be used when the darwin-sandbox doesn't
work. Most notably this is the case for nested sandboxing, e.g. Bazel running
Bazel inside an integration test.
Also includes a fix to pull in some extra environment vars on macOS, similar to
what DarwinSandboxedStrategy and StandaloneSpawnStrategy already do. Without
this the processwrapper-sandbox seems to occasionally cause ObjC builds (and two
of our tests) to fail.
Change-Id: Ic7462080caf56d9bb98e2f3765bd37853b01632b
RELNOTES: Sandboxing is now enabled by default on FreeBSD (via processwrapper-sandbox).
PiperOrigin-RevId: 155366728
|
| |
|
|
|
|
|
|
|
|
|
| |
selected even if they're not the preferred one on a platform.
Simplify the SandboxActionContextProvider and remove the warning about
sandboxing being unsupported. With the ProcessWrapperSandboxedStrategy
now being reliable enough and the strategies printing their real name in
the UI, this is overall a better UX.
PiperOrigin-RevId: 153825986
|
| |
|
|
|
|
|
|
| |
Usually, Bazel creates the sandbox directories underneath its
output_base. With --experimental_sandbox_base you can specify a
different parent directory for this, e.g. /dev/shm to run all sandboxed
actions on a memory-backed filesystem.
PiperOrigin-RevId: 152490815
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
*** Reason for rollback ***
Breaks //src/test/shell/integration:force_delete_output_test
*** Original change description ***
Symlink output directories to the correct directory name
If the workspace directory is /path/to/my/proj and the name in the WORKSPACE
file is "floop", this will symlink the output directories to
output_base/execroot/floop instead of output_base/execroot/proj.
More prep for #1262, fixes #1681.
PiperOrigin-RevId: 152126545
|
| |
|
|
|
|
|
|
|
|
| |
If the workspace directory is /path/to/my/proj and the name in the WORKSPACE
file is "floop", this will symlink the output directories to
output_base/execroot/floop instead of output_base/execroot/proj.
More prep for #1262, fixes #1681.
PiperOrigin-RevId: 151712384
|
| |
|
|
|
|
|
|
|
|
|
| |
Extract the process-wrapper + symlink tree sandbox strategy into its own
class and allow its use in FreeBSD.
RELNOTES: Bazel can now use the process-wrapper + symlink tree based sandbox implementation in FreeBSD.
--
PiperOrigin-RevId: 151171652
MOS_MIGRATED_REVID=151171652
|
| |
|
|
|
|
|
|
|
|
|
|
| |
This change is part of the mu-bazel effort, which aims to build a minimally
useful Bazel binary with most extraneous functionality removed. As part of
that, we want to enforce layering of packages. In particular, lib.actions must
not depend on lib.rules or lib.exec. lib.rules must not depend on lib.exec.
Moving these classes is a necessary step to enforce that layering.
--
PiperOrigin-RevId: 142668172
MOS_MIGRATED_REVID=142668172
|
| |
|
|
|
| |
--
MOS_MIGRATED_REVID=137500893
|
| |
|
|
|
| |
--
MOS_MIGRATED_REVID=136818366
|
| |
|
|
|
|
|
|
|
| |
simple synchronous cleanup.
Tested with bazel building itself that this does not result in a performance degradation.
--
MOS_MIGRATED_REVID=134766597
|
| |
|
|
|
| |
--
MOS_MIGRATED_REVID=131817068
|
| |
|
|
|
|
|
|
|
| |
linux-sandbox in the same sandbox execution environment.
--
Change-Id: I51a875a87d92ae13ad575eb41026ce5d3db94f8b
Reviewed-on: https://bazel-review.googlesource.com/#/c/5611/
MOS_MIGRATED_REVID=131578077
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
This solves a performance issue that slowed down builds by about 40% at least on Linux, due to clone() with CLONE_NEWNET becoming extremely slow (>1 second) for highly parallel builds. See this thread for a discussion: https://lkml.org/lkml/2014/8/20/40
For the sake of consistency, we apply the same policy on OS X, too.
If we find a better way to block network access for processes on Linux that doesn't have this performance hit, we will revisit this.
RELNOTES: Sandboxed builds allow network access for builds by default. Tests will still be run without networking, unless "requires-network" is specified as a tag.
--
MOS_MIGRATED_REVID=131393514
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This has the following improvements upon the older one:
- Uses PID namespaces, PR_SET_PDEATHSIG and a number of other tricks for
further process isolation and 100% reliable killing of child processes.
- Uses clone() instead of unshare() to work around a Linux kernel bug that
made creating a sandbox unreliable.
- Instead of mounting a hardcoded list of paths + whatever you add with
--sandbox_add_path, this sandbox instead mounts all of /, except for what
you make inaccessible via --sandbox_block_path. This should solve the
majority of "Sandboxing breaks my build, because my compiler is installed
in /opt or /usr/local" issues that users have seen.
- Instead of doing magic with bind mounts, we create a separate execroot for
each process containing symlinks to the input files. This is simpler and
gives more predictable performance.
- Actually makes everything except the working directory read-only
(fixes #1364). This means that a running process can no longer accidentally
modify your source code (yay!).
- Prevents a number of additional "attacks" or leaks, like accidentally
inheriting file handles from the parent.
- Simpler command-line interface.
- We can provide the same semantics in a Mac OS X sandbox, which will come in
a separate code review from yueg@.
It has the following caveats / known issues:
- The "fallback to /bin/bash on error" feature is gone, but now that the
sandbox mounts everything by default, the main use-case for this is no
longer needed.
The following improvements are planned:
- Use a FUSE filesystem if possible for the new execroot, instead of creating
symlinks.
- Mount a base image instead of "/".
FAQ:
Q: Why is mounting all of "/" okay, doesn't this make the whole sandbox
useless?
A: This is still a reasonable behavior, because the sandbox never tried to
isolate your build from the operating system it runs in. Instead it is
supposed to protect your data from a test running "rm -rf $HOME" and to
make it difficult / impossible for actions to use input files that are not
declared dependencies. For even more isolation the sandbox will support
mounting a base image as its root in a future version (similar to Docker
images).
Q: Let's say my process-specific execroot contains a symlink to an input file
"good.h", can't the process just resolve the symlink, strip off the file
name and then look around in the workspace?
A: Yes. Unfortunately we could not find any way on Linux to make a file appear
in a different directory with *all* of the semantics we would like. The
options investigated were:
1) Copying input files, which is much too slow.
2) Hard linking input files, which is fast, but doesn't work cross-
filesystems and it's also not possible to make them read-only.
3) Bind mounts, which don't scale once you're up in the thousands of input
files (across all actions) - it seems like the kernel has some
non-linear performance behavior when the mount table grows too much,
resulting in the mount syscall taking more time the more mounts you
have.
4) FUSE filesystem, good in theory, but wasn't ready for the first
iteration.
RELNOTES: New sandboxing implementation for Linux in which all actions run in a separate execroot that contains input files as symlinks back to the originals in the workspace. The running action now has read-write access to its execroot and /tmp only and can no longer write in arbitrary other places in the file system.
--
Change-Id: Ic91386fc92f8eef727ed6d22e6bd0f357d145063
Reviewed-on: https://bazel-review.googlesource.com/#/c/4053
MOS_MIGRATED_REVID=130638204
|
| |
|
|
|
|
|
| |
--
Change-Id: Idf232f3dce3a3221d9a35c89dcef13437b0c25ba
Reviewed-on: https://bazel-review.googlesource.com/#/c/3905/
MOS_MIGRATED_REVID=129620348
|
| |
|
|
|
|
|
| |
(But this will make it easier to try out the overlayfs-based sandbox later.)
--
MOS_MIGRATED_REVID=125320914
|
| |
|
|
|
|
|
|
|
| |
BlazeRuntime#getProductName() or a reference to TestConstants.PRODUCT_NAME for tests.
This CL prepares the codebase in order to delete the constant.
--
MOS_MIGRATED_REVID=122993568
|
| |
|
|
|
| |
--
MOS_MIGRATED_REVID=121002371
|
| |
|
|
|
|
|
|
|
|
|
|
| |
- add flag --sandbox_add_path, which takes a list of additional paths as argument and mount these paths to sandbox. Fixes #884.
- mount target of /etc/resolv.conf if it is a symlink. Fixes #738.
RELNOTES:
- add flag --sandbox_add_path, which takes a list of additional paths as argument and mount these paths to sandbox.
- mount target of /etc/resolv.conf if it is a symlink.
--
MOS_MIGRATED_REVID=117364211
|
| |
|
|
|
|
|
|
|
|
|
| |
The headers were modified with
`find . -type f -exec 'sed' '-Ei' 's|Copyright 201([45]) Google|Copyright 201\1 The Bazel Authors|' '{}' ';'`
And manual edit for not Google owned copyright. Because of the nature of ijar, I did not modified the header of file owned by Alan Donovan.
The list of authors were extracted from the git log. It is missing older Google contributors that can be added on-demand.
--
MOS_MIGRATED_REVID=103938715
|
| |
|
|
|
| |
--
MOS_MIGRATED_REVID=103269584
|
| |
|
|
|
|
|
|
|
| |
namespace-runner now have to be explicitly activated via --sandbox_debug.
Fixes #424.
--
MOS_MIGRATED_REVID=102566625
|
| |
|
|
|
|
|
|
|
|
|
|
| |
use non-declared inputs) and safety (spawns can no longer affect the host system, e.g. accidentally wipe your home directory). This implementation works on Linux only and uses Linux containers ("namespaces").
The strategy works with all actions that Bazel supports (C++ / Java compilation, genrules, test execution, Skylark-based rules, ...) and in tests, Bazel could successfully bootstrap itself and pass the whole test suite using sandboxed execution.
This is not the default behavior yet, but can be activated explicitly by using:
bazel build --genrule_strategy=sandboxed --spawn_strategy=sandboxed //my:stuff
--
MOS_MIGRATED_REVID=101457297
|
|
|
--
MOS_MIGRATED_REVID=97126283
|
Benjamin Peterson
ulfjack
philwo
hlopko
kchodorow
Damien Martin-Guillerez
Yue Gan
Luis Fernando Pino Duque