| Commit message (Collapse) | Author | Age |
|---|
| |
|
|
|
|
|
| |
For example, it now outputs resource usage statistics like the amount of user time and system time used.
RELNOTES: None
PiperOrigin-RevId: 178948958
|
| |
|
|
|
| |
RELNOTES: None
PiperOrigin-RevId: 175697300
|
| |
|
|
|
|
|
|
|
| |
- Refactoring to share more code between the two programs.
- Remove setuid() call in linux-sandbox. It was added due to a wrong understanding of what process-wrapper did in the beginning and unless someone installed linux-sandbox as a setuid binary, it was a no-op.
- Switch to a new process group in linux-sandbox to avoid accidentally killing our parent.
RELNOTES: None.
PiperOrigin-RevId: 156332503
|
| |
|
|
|
|
|
| |
No functional changes.
Change-Id: Ia87c19b70dd1ff8fa7465ad90c499cf351b9687b
PiperOrigin-RevId: 156188343
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
*** Reason for rollback ***
Likely cause for b/38172480 ("blaze now waits for all processes spawned by local tests to terminate") and b/38194553 ("Server terminated abruptly (error code: 14, error message: 'Endpoint read failed'").
I have a fix almost ready, but it consists of many lines of new code - we shouldn't rush that into Bazel's 0.5.0 release. Instead, let's roll this back, do a release using the known good older process-wrapper and then go forward in 0.5.1 with a better and well tested new version of this.
*** Original change description ***
process-wrapper: Wait for all (grand)children before exiting.
This uses Linux's PR_SET_CHILD_SUBREAPER and FreeBSD's PROC_REAP_ACQUIRE features to become an init-like process for all (grand)children spawned by process-wrapper, which allows us to a) kill them reliably and then b) wait for them reliably. Before this change, we only killed the main child, waited for it, then fired off a kill -9 on the process group, without waiting for it. This led to a race condition where Bazel would try to use...
***
PiperOrigin-RevId: 156068188
|
| |
|
|
|
|
|
|
|
|
|
| |
if (!source_specified) {
^
src/main/tools/linux-sandbox-options.cc:126:8: note: 'source_specified' was declared here bool source_specified;
^
Closes #2735.
PiperOrigin-RevId: 155067584
|
| |
|
|
|
|
|
|
| |
This uses Linux's PR_SET_CHILD_SUBREAPER and FreeBSD's PROC_REAP_ACQUIRE features to become an init-like process for all (grand)children spawned by process-wrapper, which allows us to a) kill them reliably and then b) wait for them reliably. Before this change, we only killed the main child, waited for it, then fired off a kill -9 on the process group, without waiting for it. This led to a race condition where Bazel would try to use or delete files that were still helt open by children of the main child and thus to bugs like #2371.
This means we now have reliable process management on Linux, FreeBSD and Windows. Unfortunately I couldn't find any feature like this on macOS, so this is the only OS that will still have this race condition.
PiperOrigin-RevId: 153817210
|
| |
|
|
|
|
|
|
|
|
|
| |
If needed you can restore the old behavior by passing the flag
--sandbox_tmpfs_path=/tmp to bazel.
Fixes #2508.
--
PiperOrigin-RevId: 151127924
MOS_MIGRATED_REVID=151127924
|
| |
|
|
|
|
| |
--
PiperOrigin-RevId: 151120717
MOS_MIGRATED_REVID=151120717
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Try to run /bin/true as a test of whether the Linux sandbox works,
instead of just trying to create a bunch of namespaces as a proxy.
This helps resolve issues on Linux distros where the earlier check
worked, but then the sandbox ultimately failed due to other operations
being unsupported.
As an example, Debian Jessie and certain Docker versions seem to allow
the creation of PID namespaces, but forbid mounting a new proc on top of
/proc (see #1972). This resulted in Bazel thinking that sandboxing works
fine, when it actually didn't. The improved check correctly catches this
situation and disabled sandboxing.
--
PiperOrigin-RevId: 151116894
MOS_MIGRATED_REVID=151116894
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This can be reactivated by passing the --sandbox_fake_username flag
to Bazel.
Reasoning: 'nobody' has a non-existent home directory on many Linux
distros, leading to issues when tools try to stat / read / write to the
home directory.
Related to #2688.
RELNOTES: The Linux sandbox no longer changes the user to 'nobody' by
default, instead the current user is used as is. The old behavior can be
restored via the --sandbox_fake_username flag.
--
PiperOrigin-RevId: 151115218
MOS_MIGRATED_REVID=151115218
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
By removing the --sandbox_block_path feature in an earlier change and
taking advantage of the fact that in a mount namespace we can actually
"remount" mount points to be read-only without bind mounting them to
some other place beforehand, this is no longer necessary. The code
becomes much simpler due to this, for example we no longer need to
chroot.
--
PiperOrigin-RevId: 151111360
MOS_MIGRATED_REVID=151111360
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
It is in the way of optimizing the performance of the sandbox, because
it requires us to create two helper files (an unreadable file and an
unreadable directory) which are bind-mounted on top of paths specified
via this flag. These two helper files were created on a tmpfs mounted by
the sandbox until now, which ensured that they were automatically
deleted on exit. However, mounting tmpfs on /dev/shm or /tmp causes
issues like #2686 or #1882.
By removing this flag, we can get rid of the two helper files, which
means we can also remove the reliance on a "sandbox temp directory"
completely in the next change.
--
PiperOrigin-RevId: 151107496
MOS_MIGRATED_REVID=151107496
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
The behavior of the Linux sandbox was changed to not hide the local hostname by default.
It is now only hidden when the --sandbox_fake_hostname flag is specified.
Also, instead of using the hostname "sandbox" in this case, it now uses "localhost", which fixes the issue of sandboxed processes not being able to resolve their local hostname.
RELNOTES: For increased compatibility with environments where UTS namespaces are not available, the Linux sandbox no longer hides the hostname of the local machine by default. Use --sandbox_fake_hostname to re-enable this feature.
--
PiperOrigin-RevId: 146244268
MOS_MIGRATED_REVID=146244268
|
| |
|
|
|
|
|
|
|
|
| |
RELNOTES: New flag --sandbox_add_mount_pair to specify customized source:target path pairs to bind mount inside the sandbox.
--
Change-Id: Ifbacfc0e16bbaedcf5b6d3937799710f2cfa3d58
Reviewed-on: https://cr.bazel.build/7150
PiperOrigin-RevId: 142542381
MOS_MIGRATED_REVID=142542381
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
/tmp.
Add "-b" option to linux-sandbox to explicitly bind mount files / directories into the sandbox. This is used to pull in the workspace and output base of Bazel even when they're located in /tmp and would thus be hidden by the tmpfs we mount on the /tmp directory in the sandbox.
Add "-S" option to linux-sandbox to explicitly specify a temporary directory to be used to contain the sandbox. This can be created by Bazel and then removed more reliably, compared to the earlier behavior where the sandbox would create its own temporary root directory in /tmp/sandbox.XXXXXX (and fail to delete it in case it gets killed by a signal).
Fix spurious empty.XXXXXX files and directories not being deleted from /tmp.
--
MOS_MIGRATED_REVID=133695992
|
|
|
This has the following improvements upon the older one:
- Uses PID namespaces, PR_SET_PDEATHSIG and a number of other tricks for
further process isolation and 100% reliable killing of child processes.
- Uses clone() instead of unshare() to work around a Linux kernel bug that
made creating a sandbox unreliable.
- Instead of mounting a hardcoded list of paths + whatever you add with
--sandbox_add_path, this sandbox instead mounts all of /, except for what
you make inaccessible via --sandbox_block_path. This should solve the
majority of "Sandboxing breaks my build, because my compiler is installed
in /opt or /usr/local" issues that users have seen.
- Instead of doing magic with bind mounts, we create a separate execroot for
each process containing symlinks to the input files. This is simpler and
gives more predictable performance.
- Actually makes everything except the working directory read-only
(fixes #1364). This means that a running process can no longer accidentally
modify your source code (yay!).
- Prevents a number of additional "attacks" or leaks, like accidentally
inheriting file handles from the parent.
- Simpler command-line interface.
- We can provide the same semantics in a Mac OS X sandbox, which will come in
a separate code review from yueg@.
It has the following caveats / known issues:
- The "fallback to /bin/bash on error" feature is gone, but now that the
sandbox mounts everything by default, the main use-case for this is no
longer needed.
The following improvements are planned:
- Use a FUSE filesystem if possible for the new execroot, instead of creating
symlinks.
- Mount a base image instead of "/".
FAQ:
Q: Why is mounting all of "/" okay, doesn't this make the whole sandbox
useless?
A: This is still a reasonable behavior, because the sandbox never tried to
isolate your build from the operating system it runs in. Instead it is
supposed to protect your data from a test running "rm -rf $HOME" and to
make it difficult / impossible for actions to use input files that are not
declared dependencies. For even more isolation the sandbox will support
mounting a base image as its root in a future version (similar to Docker
images).
Q: Let's say my process-specific execroot contains a symlink to an input file
"good.h", can't the process just resolve the symlink, strip off the file
name and then look around in the workspace?
A: Yes. Unfortunately we could not find any way on Linux to make a file appear
in a different directory with *all* of the semantics we would like. The
options investigated were:
1) Copying input files, which is much too slow.
2) Hard linking input files, which is fast, but doesn't work cross-
filesystems and it's also not possible to make them read-only.
3) Bind mounts, which don't scale once you're up in the thousands of input
files (across all actions) - it seems like the kernel has some
non-linear performance behavior when the mount table grows too much,
resulting in the mount syscall taking more time the more mounts you
have.
4) FUSE filesystem, good in theory, but wasn't ready for the first
iteration.
RELNOTES: New sandboxing implementation for Linux in which all actions run in a separate execroot that contains input files as symlinks back to the originals in the workspace. The running action now has read-write access to its execroot and /tmp only and can no longer write in arbitrary other places in the file system.
--
Change-Id: Ic91386fc92f8eef727ed6d22e6bd0f357d145063
Reviewed-on: https://bazel-review.googlesource.com/#/c/4053
MOS_MIGRATED_REVID=130638204
|
ruperts
philwo
Peter Foley
Xin Gao