| Commit message (Collapse) | Author | Age |
|
|
|
|
| |
Change-Id: Ib0c957de68a8a8035f2e13d7c9fe8d1549a3744d
PiperOrigin-RevId: 185645675
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
1.Deleted config_setting for --cpu=x64_windows_msys, because we don't build
Bazel with MSYS gcc anymore.
2.Deleted config_setting for --cpu=x64_windows_msvc, because it uses exactly
the same toolchain as --cpu=x64_windows, it'll be removed in the future.
This change reduces the complexity of our BUILD files and make them less
confusing.
Change-Id: I939831a6861413b0f745fb1be98aacd4fb780e0a
PiperOrigin-RevId: 181751853
|
|
|
|
|
|
|
| |
This will enable an easier transition from checked-in BUILD files to ones generated by copybara.
RELNOTES: None
PiperOrigin-RevId: 177514519
|
|
|
|
|
|
|
| |
For example, it now outputs resource usage statistics like the amount of user time and system time used.
RELNOTES: None
PiperOrigin-RevId: 177263221
|
|
|
|
|
|
|
|
| |
This will allow us to add new and optional flags like selecting a strategy used to spawn / wait for the child process.
No one except Bazel should be calling "process-wrapper" and I couldn't find any references, so this breaking change should be fine.
PiperOrigin-RevId: 159685867
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
*** Reason for rollback ***
Rollforward with fix for FreeBSD. I added the missing headers and verified that it now builds fine on FreeBSD 11, too.
*** Original change description ***
Automated g4 rollback of commit 7f520a8286c39c5145b6d816cd0be5a6b7b18250.
*** Reason for rollback ***
This broke Bazel CI on freebsd:
http://ci.bazel.io/view/Dashboard/job/Bazel/JAVA_VERSION=1.8,PLATFORM_NAME=freebsd-11/1516/console#
*** Original change description ***
Refactor process-wrapper code so the spawn/wait code is pluggable.
In an upcoming change I'll reintroduce the new platform-specific implementations that can kill and wait for all descendant processes spawned by the wrapped process.
This has no functional changes.
PiperOrigin-RevId: 158133159
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
After this change, with a Bazel having MSVC as default toolchain,
the command to build a MSVC Bazel on Windows is simply:
bazel build //src:bazel
although bazel build --cpu=x64_windows_msvc //src:bazel is still supported.
And the command to build a MSYS Bazel on Windows is:
bazel build --cpu=x64_windows_msys //src:bazel
Also made //src/test/cpp:blaze_util_test pass without
--cpu=x64_windows_msvc
Change-Id: Iaf37513c778768d06fb5700442d5229a5f348964
PiperOrigin-RevId: 157446905
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
*** Reason for rollback ***
This broke Bazel CI on freebsd:
http://ci.bazel.io/view/Dashboard/job/Bazel/JAVA_VERSION=1.8,PLATFORM_NAME=freebsd-11/1516/console#
*** Original change description ***
Refactor process-wrapper code so the spawn/wait code is pluggable.
In an upcoming change I'll reintroduce the new platform-specific implementations that can kill and wait for all descendant processes spawned by the wrapped process.
This has no functional changes.
PiperOrigin-RevId: 156884488
|
|
|
|
|
|
|
|
| |
In an upcoming change I'll reintroduce the new platform-specific implementations that can kill and wait for all descendant processes spawned by the wrapped process.
This has no functional changes.
PiperOrigin-RevId: 156849610
|
|
|
|
|
|
|
|
|
| |
- Refactoring to share more code between the two programs.
- Remove setuid() call in linux-sandbox. It was added due to a wrong understanding of what process-wrapper did in the beginning and unless someone installed linux-sandbox as a setuid binary, it was a no-op.
- Switch to a new process group in linux-sandbox to avoid accidentally killing our parent.
RELNOTES: None.
PiperOrigin-RevId: 156332503
|
|
|
|
|
|
|
| |
No functional changes.
Change-Id: Ia87c19b70dd1ff8fa7465ad90c499cf351b9687b
PiperOrigin-RevId: 156188343
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
*** Reason for rollback ***
Likely cause for b/38172480 ("blaze now waits for all processes spawned by local tests to terminate") and b/38194553 ("Server terminated abruptly (error code: 14, error message: 'Endpoint read failed'").
I have a fix almost ready, but it consists of many lines of new code - we shouldn't rush that into Bazel's 0.5.0 release. Instead, let's roll this back, do a release using the known good older process-wrapper and then go forward in 0.5.1 with a better and well tested new version of this.
*** Original change description ***
process-wrapper: Wait for all (grand)children before exiting.
This uses Linux's PR_SET_CHILD_SUBREAPER and FreeBSD's PROC_REAP_ACQUIRE features to become an init-like process for all (grand)children spawned by process-wrapper, which allows us to a) kill them reliably and then b) wait for them reliably. Before this change, we only killed the main child, waited for it, then fired off a kill -9 on the process group, without waiting for it. This led to a race condition where Bazel would try to use...
***
PiperOrigin-RevId: 156068188
|
|
|
|
|
|
|
|
| |
This uses Linux's PR_SET_CHILD_SUBREAPER and FreeBSD's PROC_REAP_ACQUIRE features to become an init-like process for all (grand)children spawned by process-wrapper, which allows us to a) kill them reliably and then b) wait for them reliably. Before this change, we only killed the main child, waited for it, then fired off a kill -9 on the process group, without waiting for it. This led to a race condition where Bazel would try to use or delete files that were still helt open by children of the main child and thus to bugs like #2371.
This means we now have reliable process management on Linux, FreeBSD and Windows. Unfortunately I couldn't find any feature like this on macOS, so this is the only OS that will still have this race condition.
PiperOrigin-RevId: 153817210
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
After this change, a msys bazel can be built with
a MSVC-default Bazel by adding --cpu=x64_windows_msys --host=x64_windows_msys
See https://github.com/bazelbuild/bazel/issues/2627
--
Change-Id: Iaa82bf4dd911c5740b98d3b2739dfccca6203f79
Reviewed-on: https://cr.bazel.build/9293
PiperOrigin-RevId: 149532274
MOS_MIGRATED_REVID=149532274
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
bazel build //src:bazel --cpu=x64_windows_msvc
now succeeds, hooray!
This change adds empty implementations for
build-runfiles and process-wrapper to make that
possible.
This means we can now build a bazel binary that
doesn't depend on MSYS.
The resulting binary is not yet functional because
many methods are still to be implemented, and they
just write "TODO: implement" or something similar.
But still this is great news, because now we can
add compile tests to the CI for MSVC!
See https://github.com/bazelbuild/bazel/issues/2107
--
PiperOrigin-RevId: 146106178
MOS_MIGRATED_REVID=146106178
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Update `select` statements in BUILD files with the
new config_setting.
This is a first step on a long path that leads to
us being able to compile bazel on Windows with
--cpu=x64_windows_msvc. Needless to say, we're not
there yet.
Tested: on Linux, Darwin, Windows/MSYS
--
MOS_MIGRATED_REVID=134534613
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This has the following improvements upon the older one:
- Uses PID namespaces, PR_SET_PDEATHSIG and a number of other tricks for
further process isolation and 100% reliable killing of child processes.
- Uses clone() instead of unshare() to work around a Linux kernel bug that
made creating a sandbox unreliable.
- Instead of mounting a hardcoded list of paths + whatever you add with
--sandbox_add_path, this sandbox instead mounts all of /, except for what
you make inaccessible via --sandbox_block_path. This should solve the
majority of "Sandboxing breaks my build, because my compiler is installed
in /opt or /usr/local" issues that users have seen.
- Instead of doing magic with bind mounts, we create a separate execroot for
each process containing symlinks to the input files. This is simpler and
gives more predictable performance.
- Actually makes everything except the working directory read-only
(fixes #1364). This means that a running process can no longer accidentally
modify your source code (yay!).
- Prevents a number of additional "attacks" or leaks, like accidentally
inheriting file handles from the parent.
- Simpler command-line interface.
- We can provide the same semantics in a Mac OS X sandbox, which will come in
a separate code review from yueg@.
It has the following caveats / known issues:
- The "fallback to /bin/bash on error" feature is gone, but now that the
sandbox mounts everything by default, the main use-case for this is no
longer needed.
The following improvements are planned:
- Use a FUSE filesystem if possible for the new execroot, instead of creating
symlinks.
- Mount a base image instead of "/".
FAQ:
Q: Why is mounting all of "/" okay, doesn't this make the whole sandbox
useless?
A: This is still a reasonable behavior, because the sandbox never tried to
isolate your build from the operating system it runs in. Instead it is
supposed to protect your data from a test running "rm -rf $HOME" and to
make it difficult / impossible for actions to use input files that are not
declared dependencies. For even more isolation the sandbox will support
mounting a base image as its root in a future version (similar to Docker
images).
Q: Let's say my process-specific execroot contains a symlink to an input file
"good.h", can't the process just resolve the symlink, strip off the file
name and then look around in the workspace?
A: Yes. Unfortunately we could not find any way on Linux to make a file appear
in a different directory with *all* of the semantics we would like. The
options investigated were:
1) Copying input files, which is much too slow.
2) Hard linking input files, which is fast, but doesn't work cross-
filesystems and it's also not possible to make them read-only.
3) Bind mounts, which don't scale once you're up in the thousands of input
files (across all actions) - it seems like the kernel has some
non-linear performance behavior when the mount table grows too much,
resulting in the mount syscall taking more time the more mounts you
have.
4) FUSE filesystem, good in theory, but wasn't ready for the first
iteration.
RELNOTES: New sandboxing implementation for Linux in which all actions run in a separate execroot that contains input files as symlinks back to the originals in the workspace. The running action now has read-write access to its execroot and /tmp only and can no longer write in arbitrary other places in the file system.
--
Change-Id: Ic91386fc92f8eef727ed6d22e6bd0f357d145063
Reviewed-on: https://bazel-review.googlesource.com/#/c/4053
MOS_MIGRATED_REVID=130638204
|
|
|
|
|
| |
--
MOS_MIGRATED_REVID=127538990
|
|
|
|
|
|
|
| |
@bazel_tools. Currently the tool still remains in embedded_binaries, but we will migrate away from that: Eventually it can simply live just under @bazel_tools.
--
MOS_MIGRATED_REVID=123436822
|
|
|
|
|
|
|
|
| |
This target include all non tests targets of Bazel to do integration tests of
bootstrapping.
--
MOS_MIGRATED_REVID=115830741
|
|
|
|
|
|
|
|
| |
We might revisit what the default toolchain for Windows should be, but
this CL uses msys to simplify bootstrapping.
--
MOS_MIGRATED_REVID=113665255
|
|
|
|
|
|
|
| |
For bazel on non-darwin architectures, this will simply be a stub, and should never be invoked. On darwin arcitectures, the tool will map xcode version to xcode path on the host system.
--
MOS_MIGRATED_REVID=111651147
|
|
|
|
|
|
|
|
|
| |
The code changes are mostly due to dslomov, not me, although I refactored
the Jvm class a bit based on his changes. I set dslomov as the author.
--
Reviewed-on: https://github.com/bazelbuild/bazel/pull/688
MOS_MIGRATED_REVID=109536553
|
|
|
|
|
| |
--
MOS_MIGRATED_REVID=106605250
|
|
|
|
|
|
|
|
|
|
|
| |
* -n: Create a new network namespace with only loopback interface.
* -r: set the uid/gid inside the sandbox to be root (instead of nobody)
so that setuid programs like ping can still run when needed.
--
Change-Id: I8ab434e47e0f6933ee9de02e135c8daec39fe73f
Reviewed-on: https://bazel-review.googlesource.com/#/c/2101/
MOS_MIGRATED_REVID=104858163
|
|
|
|
|
|
|
| |
--
Change-Id: I4e65cc583e758d2f7e45209ffcb37f6a871e2ed7
Reviewed-on: https://bazel-review.git.corp.google.com/#/c/1840
MOS_MIGRATED_REVID=101462155
|
|
|
|
|
|
|
|
|
|
|
|
| |
use non-declared inputs) and safety (spawns can no longer affect the host system, e.g. accidentally wipe your home directory). This implementation works on Linux only and uses Linux containers ("namespaces").
The strategy works with all actions that Bazel supports (C++ / Java compilation, genrules, test execution, Skylark-based rules, ...) and in tests, Bazel could successfully bootstrap itself and pass the whole test suite using sandboxed execution.
This is not the default behavior yet, but can be activated explicitly by using:
bazel build --genrule_strategy=sandboxed --spawn_strategy=sandboxed //my:stuff
--
MOS_MIGRATED_REVID=101457297
|
|
|
|
|
|
|
|
|
|
|
| |
resource, then passing it to the parser as a string instead of putting it into embedded_binaries then passing a Path to it to the parser.
This makes the upcoming default WORKSPACE rules for Android much more palatable. In particular, Android rules won't need to be special cased when building the Bazel binary because the contents are self-contained in BazelRuleClassProvider (and the jdk.WORKSPACE file, which is a simple Java resource)
Even better would be not to use a string, but some kind of structured data, but that's probably more effort than it's worth.
--
MOS_MIGRATED_REVID=95983199
|
|
|
|
|
| |
--
MOS_MIGRATED_REVID=88742425
|
|
|
|
|
|
|
| |
This is needed for taking the runfiles prefix from the WORKSPACE file instead of hardcoding it.
--
MOS_MIGRATED_REVID=87347883
|
|
--
MOE_MIGRATED_REVID=85702957
|