| Commit message (Collapse) | Author | Age |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Add jdk as runfiles for java, jar, and javac in jdk.BUILD. Done via data
attribute of filegroup.
This should 'mostly' work if you handwave around the differences between
default_runfiles and data_runfiles. The jdk runfiles will always be
accessed via data_runfiles, and thus not platform-targeted. Therefore, my
expectation is the target machine will be sent the host JDK.
However, since the jdk is itself composed of binary imports anyway, it
seems like the JDK already has issues being platform-targeted.
To reviewers:
I didn't do a thorough audit of the jdk.BUILD file. More specifically,
unsure if xjc or wsimport need a jdk as well.
Still investigating a good testing strategy.
Change-Id: I138b95b8cee2808c89e4202b822ada6a6577acce
PiperOrigin-RevId: 152290935
|
|
|
|
|
|
|
|
|
|
|
| |
If needed you can restore the old behavior by passing the flag
--sandbox_tmpfs_path=/tmp to bazel.
Fixes #2508.
--
PiperOrigin-RevId: 151127924
MOS_MIGRATED_REVID=151127924
|
|
|
|
|
|
| |
--
PiperOrigin-RevId: 151120717
MOS_MIGRATED_REVID=151120717
|
|
|
|
|
|
|
|
|
|
|
|
| |
Instead of trying to detect overlapping mount points, just ignore any
ENOENT errors during remount. If this error happens, the mount point
wouldn't be accessible anyway, so there's no harm if the remount fails.
Fixes #1948.
--
PiperOrigin-RevId: 151118726
MOS_MIGRATED_REVID=151118726
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Try to run /bin/true as a test of whether the Linux sandbox works,
instead of just trying to create a bunch of namespaces as a proxy.
This helps resolve issues on Linux distros where the earlier check
worked, but then the sandbox ultimately failed due to other operations
being unsupported.
As an example, Debian Jessie and certain Docker versions seem to allow
the creation of PID namespaces, but forbid mounting a new proc on top of
/proc (see #1972). This resulted in Bazel thinking that sandboxing works
fine, when it actually didn't. The improved check correctly catches this
situation and disabled sandboxing.
--
PiperOrigin-RevId: 151116894
MOS_MIGRATED_REVID=151116894
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This can be reactivated by passing the --sandbox_fake_username flag
to Bazel.
Reasoning: 'nobody' has a non-existent home directory on many Linux
distros, leading to issues when tools try to stat / read / write to the
home directory.
Related to #2688.
RELNOTES: The Linux sandbox no longer changes the user to 'nobody' by
default, instead the current user is used as is. The old behavior can be
restored via the --sandbox_fake_username flag.
--
PiperOrigin-RevId: 151115218
MOS_MIGRATED_REVID=151115218
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
By removing the --sandbox_block_path feature in an earlier change and
taking advantage of the fact that in a mount namespace we can actually
"remount" mount points to be read-only without bind mounting them to
some other place beforehand, this is no longer necessary. The code
becomes much simpler due to this, for example we no longer need to
chroot.
--
PiperOrigin-RevId: 151111360
MOS_MIGRATED_REVID=151111360
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
It is in the way of optimizing the performance of the sandbox, because
it requires us to create two helper files (an unreadable file and an
unreadable directory) which are bind-mounted on top of paths specified
via this flag. These two helper files were created on a tmpfs mounted by
the sandbox until now, which ensured that they were automatically
deleted on exit. However, mounting tmpfs on /dev/shm or /tmp causes
issues like #2686 or #1882.
By removing this flag, we can get rid of the two helper files, which
means we can also remove the reliance on a "sandbox temp directory"
completely in the next change.
--
PiperOrigin-RevId: 151107496
MOS_MIGRATED_REVID=151107496
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
After this change, a msys bazel can be built with
a MSVC-default Bazel by adding --cpu=x64_windows_msys --host=x64_windows_msys
See https://github.com/bazelbuild/bazel/issues/2627
--
Change-Id: Iaa82bf4dd911c5740b98d3b2739dfccca6203f79
Reviewed-on: https://cr.bazel.build/9293
PiperOrigin-RevId: 149532274
MOS_MIGRATED_REVID=149532274
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The default C++ CROSSTOOL on Windows now becomes MSVC,
--cpu=x64_windows_msvc is still supported
To use MSYS toolcahin, add --cpu=x64_windows_msys and
--host_cpu=x64_windows_msys for host compilation
See https://github.com/bazelbuild/bazel/issues/2627
--
Change-Id: Ie788a39cb5ffbc9fc956ccfd51a3cc816c74543a
Reviewed-on: https://cr.bazel.build/9292
PiperOrigin-RevId: 149530250
MOS_MIGRATED_REVID=149530250
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This rule can be used in building JNI shared libraries for Windows.
For example, see TensorFlow usage of these targets in jdk.BUILD:
https://github.com/tensorflow/tensorflow/blo[]a98083a6c16f263d668271889863596efbeb84/tensorflow/java/src/main/native/BUILD#L68
Closes #2599.
--
Reviewed-on: https://github.com/bazelbuild/bazel/pull/2599
PiperOrigin-RevId: 149527656
MOS_MIGRATED_REVID=149527656
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
*** Reason for rollback ***
Broke tests on CI: http://ci.bazel.io/job/bazel-tests/570/
*** Original change description ***
Roll forward execroot change
RELNOTES[INC]: Previously, an external repository would be symlinked into the
execution root at execroot/local_repo/external/remote_repo. This changes it to
be at execroot/remote_repo. This may break genrules/Skylark actions that
hardcode execution root paths. If this causes breakages for you, ensure that
genrules are using $(location :target) to access files and Skylark rules are
using http://bazel.io/docs/skylark/lib/File.html's path, dirname, etc.
functions. Cust...
--
PiperOrigin-RevId: 147833177
MOS_MIGRATED_REVID=147833177
|
|
|
|
|
|
|
|
| |
magic artifact.
--
PiperOrigin-RevId: 147830857
MOS_MIGRATED_REVID=147830857
|
|
|
|
|
|
|
|
|
| |
The targets contain the entries in the extdir and are used to
construct a classpath, so the label is not actually a directory.
--
PiperOrigin-RevId: 147798672
MOS_MIGRATED_REVID=147798672
|
|
|
|
|
|
| |
--
PiperOrigin-RevId: 147727032
MOS_MIGRATED_REVID=147727032
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
RELNOTES[INC]: Previously, an external repository would be symlinked into the
execution root at execroot/local_repo/external/remote_repo. This changes it to
be at execroot/remote_repo. This may break genrules/Skylark actions that
hardcode execution root paths. If this causes breakages for you, ensure that
genrules are using $(location :target) to access files and Skylark rules are
using http://bazel.io/docs/skylark/lib/File.html's path, dirname, etc.
functions. Custom crosstools that hardcode external/<repo> paths will have to
be updated.
Issue #1262.
--
PiperOrigin-RevId: 147726370
MOS_MIGRATED_REVID=147726370
|
|
|
|
|
|
| |
--
PiperOrigin-RevId: 146469548
MOS_MIGRATED_REVID=146469548
|
|
|
|
|
|
|
|
| |
Seems like on Ubuntu 16.04 we have to ignore EPERM on failed remounts, too.
--
PiperOrigin-RevId: 146354561
MOS_MIGRATED_REVID=146354561
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The behavior of the Linux sandbox was changed to not hide the local hostname by default.
It is now only hidden when the --sandbox_fake_hostname flag is specified.
Also, instead of using the hostname "sandbox" in this case, it now uses "localhost", which fixes the issue of sandboxed processes not being able to resolve their local hostname.
RELNOTES: For increased compatibility with environments where UTS namespaces are not available, the Linux sandbox no longer hides the hostname of the local machine by default. Use --sandbox_fake_hostname to re-enable this feature.
--
PiperOrigin-RevId: 146244268
MOS_MIGRATED_REVID=146244268
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
bazel build //src:bazel --cpu=x64_windows_msvc
now succeeds, hooray!
This change adds empty implementations for
build-runfiles and process-wrapper to make that
possible.
This means we can now build a bazel binary that
doesn't depend on MSYS.
The resulting binary is not yet functional because
many methods are still to be implemented, and they
just write "TODO: implement" or something similar.
But still this is great news, because now we can
add compile tests to the CI for MSVC!
See https://github.com/bazelbuild/bazel/issues/2107
--
PiperOrigin-RevId: 146106178
MOS_MIGRATED_REVID=146106178
|
|
|
|
|
|
|
|
|
|
| |
RELNOTES: New flag --sandbox_add_mount_pair to specify customized source:target path pairs to bind mount inside the sandbox.
--
Change-Id: Ifbacfc0e16bbaedcf5b6d3937799710f2cfa3d58
Reviewed-on: https://cr.bazel.build/7150
PiperOrigin-RevId: 142542381
MOS_MIGRATED_REVID=142542381
|
|
|
|
|
|
|
| |
These may happen when a broken NFS mount is present on the user system. Ideally, that mount should just be fixed or removed, but where this is not possible, it's probably safe to ignore it.
--
MOS_MIGRATED_REVID=140476478
|
|
|
|
|
|
|
|
|
|
|
|
| |
Skip remounting such mount points because they are not actually
visible in the sandbox after mounting new tmpfs.
Fixes https://github.com/bazelbuild/bazel/issues/1959
--
Change-Id: Ia1361559966ffb05ea1ddbeaee1ed7d3ebdb9e15
Reviewed-on: https://bazel-review.googlesource.com/#/c/6970/
MOS_MIGRATED_REVID=137397312
|
|
|
|
|
| |
--
MOS_MIGRATED_REVID=136338300
|
|
|
|
|
|
|
|
|
| |
Tested with the alpine:3.4 Docker image.
--
Change-Id: I8f3e585051988d8fd437ccd69c9c7bd009fd45d2
Reviewed-on: https://bazel-review.googlesource.com/#/c/5790/
MOS_MIGRATED_REVID=135468656
|
|
|
|
|
|
|
|
|
|
|
|
| |
This is a semantic change for Bazel, which now observes the --cpu flag when
looking up a Jvm. Use "-default" as a fallback to keep the change backwards
compatible.
RELNOTES[INC]: Bazel now uses the --cpu flag to look up Jvms; it falls back
to "default" if it can't find a Jvm matching the CPU value.
--
MOS_MIGRATED_REVID=135333759
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Update `select` statements in BUILD files with the
new config_setting.
This is a first step on a long path that leads to
us being able to compile bazel on Windows with
--cpu=x64_windows_msvc. Needless to say, we're not
there yet.
Tested: on Linux, Darwin, Windows/MSYS
--
MOS_MIGRATED_REVID=134534613
|
|
|
|
|
|
|
| |
occasional hangs seen during builds and on Bazel CI.
--
MOS_MIGRATED_REVID=134279208
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
/tmp.
Add "-b" option to linux-sandbox to explicitly bind mount files / directories into the sandbox. This is used to pull in the workspace and output base of Bazel even when they're located in /tmp and would thus be hidden by the tmpfs we mount on the /tmp directory in the sandbox.
Add "-S" option to linux-sandbox to explicitly specify a temporary directory to be used to contain the sandbox. This can be created by Bazel and then removed more reliably, compared to the earlier behavior where the sandbox would create its own temporary root directory in /tmp/sandbox.XXXXXX (and fail to delete it in case it gets killed by a signal).
Fix spurious empty.XXXXXX files and directories not being deleted from /tmp.
--
MOS_MIGRATED_REVID=133695992
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This has the following improvements upon the older one:
- Uses PID namespaces, PR_SET_PDEATHSIG and a number of other tricks for
further process isolation and 100% reliable killing of child processes.
- Uses clone() instead of unshare() to work around a Linux kernel bug that
made creating a sandbox unreliable.
- Instead of mounting a hardcoded list of paths + whatever you add with
--sandbox_add_path, this sandbox instead mounts all of /, except for what
you make inaccessible via --sandbox_block_path. This should solve the
majority of "Sandboxing breaks my build, because my compiler is installed
in /opt or /usr/local" issues that users have seen.
- Instead of doing magic with bind mounts, we create a separate execroot for
each process containing symlinks to the input files. This is simpler and
gives more predictable performance.
- Actually makes everything except the working directory read-only
(fixes #1364). This means that a running process can no longer accidentally
modify your source code (yay!).
- Prevents a number of additional "attacks" or leaks, like accidentally
inheriting file handles from the parent.
- Simpler command-line interface.
- We can provide the same semantics in a Mac OS X sandbox, which will come in
a separate code review from yueg@.
It has the following caveats / known issues:
- The "fallback to /bin/bash on error" feature is gone, but now that the
sandbox mounts everything by default, the main use-case for this is no
longer needed.
The following improvements are planned:
- Use a FUSE filesystem if possible for the new execroot, instead of creating
symlinks.
- Mount a base image instead of "/".
FAQ:
Q: Why is mounting all of "/" okay, doesn't this make the whole sandbox
useless?
A: This is still a reasonable behavior, because the sandbox never tried to
isolate your build from the operating system it runs in. Instead it is
supposed to protect your data from a test running "rm -rf $HOME" and to
make it difficult / impossible for actions to use input files that are not
declared dependencies. For even more isolation the sandbox will support
mounting a base image as its root in a future version (similar to Docker
images).
Q: Let's say my process-specific execroot contains a symlink to an input file
"good.h", can't the process just resolve the symlink, strip off the file
name and then look around in the workspace?
A: Yes. Unfortunately we could not find any way on Linux to make a file appear
in a different directory with *all* of the semantics we would like. The
options investigated were:
1) Copying input files, which is much too slow.
2) Hard linking input files, which is fast, but doesn't work cross-
filesystems and it's also not possible to make them read-only.
3) Bind mounts, which don't scale once you're up in the thousands of input
files (across all actions) - it seems like the kernel has some
non-linear performance behavior when the mount table grows too much,
resulting in the mount syscall taking more time the more mounts you
have.
4) FUSE filesystem, good in theory, but wasn't ready for the first
iteration.
RELNOTES: New sandboxing implementation for Linux in which all actions run in a separate execroot that contains input files as symlinks back to the originals in the workspace. The running action now has read-write access to its execroot and /tmp only and can no longer write in arbitrary other places in the file system.
--
Change-Id: Ic91386fc92f8eef727ed6d22e6bd0f357d145063
Reviewed-on: https://bazel-review.googlesource.com/#/c/4053
MOS_MIGRATED_REVID=130638204
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Please don't merge before I8b8c3596fd2a4c125071406eefd407ca402099dd. The
test will fail then.
I've seen two issues with this functionality breaking: #481 and #1637.
Seems like it's time to add a test which catches the issue on the
Debian/Ubuntu machines people commonly test on. The test fails on my
Debian system before I8b8c3596fd2a4c125071406eefd407ca402099dd, and
passes with that change applied.
--
Change-Id: Ib785c874cdb9192920f9935b696bfd6c9c0e5f4f
Reviewed-on: https://bazel-review.git.corp.google.com/#/c/5451/
MOS_MIGRATED_REVID=130635565
|
|
|
|
|
|
|
|
|
| |
fixes #1637 (https://github.com/bazelbuild/bazel/issues/1637)
--
Change-Id: I8b8c3596fd2a4c125071406eefd407ca402099dd
Reviewed-on: https://bazel-review.git.corp.google.com/#/c/5410/2
MOS_MIGRATED_REVID=130633667
|
|
|
|
|
|
|
| |
on Windows by default)
--
MOS_MIGRATED_REVID=129319018
|
|
|
|
|
| |
--
MOS_MIGRATED_REVID=127538990
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
*** Reason for rollback ***
Apparently we now try to open output files for the process twice: once when we are constructing the output streams, and the second time when we tell the process to redirect its outputs. This causes the outputs to be empty on Windows
*** Original change description ***
Do redirection of stdout / stderr in Java instead of reimplementing it in every process wrapper again.
--
MOS_MIGRATED_REVID=126801016
|
|
|
|
|
|
|
| |
every process wrapper again.
--
MOS_MIGRATED_REVID=126279021
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
*** Reason for rollback ***
Breaks Bazel build on Ubuntu 15.10. Manually tested that rolling this back fixes the build.
*** Original change description ***
sandbox: Mount input files and directories actually read-only.
Fixes #1364.
RELNOTES[INC]: Bazel's sandbox mounts input files read-only in this release. If your build suddenly fails due to tools not being able to write to files, then this is probably working as intended (you should never modify input files in your build), but please feel free to provide feedback.
--
MOS_MIGRATED_REVID=125439581
|
|
|
|
|
|
|
|
|
|
|
|
| |
This adds a new configuration option that allows disabling the creation of symlink forest for runfiles.
On Windows, symlink forest is disabled by default; only the runfiles manifest is created.
For shell tests, a function 'rlocation' is provided that converts from runfiles location to a real location.
Work towards #1212.
--
MOS_MIGRATED_REVID=125439553
|
|
|
|
|
|
|
|
|
| |
Fixes #1364.
RELNOTES[INC]: Bazel's sandbox mounts input files read-only in this release. If your build suddenly fails due to tools not being able to write to files, then this is probably working as intended (you should never modify input files in your build), but please feel free to provide feedback.
--
MOS_MIGRATED_REVID=125324318
|
|
|
|
|
|
|
| |
(But this will make it easier to try out the overlayfs-based sandbox later.)
--
MOS_MIGRATED_REVID=125320914
|
|
|
|
|
|
|
| |
Fixes #1330.
--
MOS_MIGRATED_REVID=124705354
|
|
|
|
|
|
|
| |
@bazel_tools. Currently the tool still remains in embedded_binaries, but we will migrate away from that: Eventually it can simply live just under @bazel_tools.
--
MOS_MIGRATED_REVID=123436822
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
On Windows, we use hard links in runfiles tree, and we need
to delete and recreate all of them on every runfiles tree update
(otherwise the links might still point to outdated files).
Occasionally the hard link cannot be unlinked (due to permissions
or file being busy). This CL just ignores the error (and hopes for the best).
This will allow us to make progress on Windows.
Work towards #1212.
--
MOS_MIGRATED_REVID=121949474
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
In a mailing list discussion [1], somebody reported namespace-sandbox
segfaulting, which was traced down to something with getpwuid. This
debugging would be a lot easier if bazel printed out the error instead
of segfaulting.
[1] https://groups.google.com/d/topic/bazel-discuss/FR949mCW9cA/discussion
--
Change-Id: I96320287b1886347343c1a50d660c097534d91d2
Reviewed-on: https://bazel-review.googlesource.com/#/c/3400
MOS_MIGRATED_REVID=120803905
|
|
|
|
|
|
|
|
|
|
|
|
| |
It looks like the fix for #481 wasn't quite complete. Somebody on the
mailing list reported namespace-sandbox failing due to this [1].
[1] https://groups.google.com/d/topic/bazel-discuss/FR949mCW9cA/discussion
--
Change-Id: I52dbe4c9639c9df282a035d9acc9b8bff0c5004b
Reviewed-on: https://bazel-review.googlesource.com/#/c/3401
MOS_MIGRATED_REVID=120102039
|
|
|
|
|
|
|
|
|
| |
on Windows
--
Change-Id: I0048202b431ca05b88f67153389ca40c1542b1d5
Reviewed-on: https://bazel-review.googlesource.com/#/c/3371
MOS_MIGRATED_REVID=119861292
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
*** Reason for rollback ***
Contributor finds some bugs and after fixing some bugs there are more bugs to fix now.
*** Original change description ***
Mount whole directories into the sandbox when possible
This halves the overhead with sandboxing enabled vs disabled for a test
that basically only mounts a bunch of files out of a directory, and
slows that same test with a single extra file added to the directory
(but not mounted) by only ~4%.
The test is <https://gist.github.com/bsilver8192/10527a862ce16bb7f79a>
with 30000 inputs moved to a subdirectory and on...
***
ROLLBACK_OF=119138157
--
MOS_MIGRATED_REVID=119828267
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
src/main/java/com/google/devtools/build/lib/exec/SymlinkTreeHelper.java:
enable --windows_compatible flag on Windows to make build-runfiles.exe work.
scritps/bootstrap/compile.sh:
--windows_compatible will also be passed to a dummy build-runfiles.exe defined in complie.sh.
Which is actully a batch script, modify it to make it work.
-----
With the changes above, we are able to build bazel with bazel. But when you try to
run ./compile.sh compile /path/to/bazel again without clean up tmp directory, it will fail with
a permission deny error. The reason seems to be that you can't use build-runfile.exe to build the
same hard link twice, still trying to solve that.
--
Change-Id: I93340b1ba9fa415f6db963b106e264799e33ede3
Reviewed-on: https://bazel-review.googlesource.com/#/c/3334
MOS_MIGRATED_REVID=119751076
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This halves the overhead with sandboxing enabled vs disabled for a test
that basically only mounts a bunch of files out of a directory, and
slows that same test with a single extra file added to the directory
(but not mounted) by only ~4%.
The test is <https://gist.github.com/bsilver8192/10527a862ce16bb7f79a>
with 30000 inputs moved to a subdirectory and only 10 genrules.
This change means symlinks will be mounted directly as their target
rather than as a symlink, but this solves some weird behavior with
multi-level symlinks and will only break things which don't declare all
of their dependencies.
--
Change-Id: I1aa39dccb2e5fca2893bdab9065ee043d34019b2
Reviewed-on: https://bazel-review.googlesource.com/#/c/3220/
MOS_MIGRATED_REVID=119138157
|
|
|
|
|
|
|
|
| |
This target include all non tests targets of Bazel to do integration tests of
bootstrapping.
--
MOS_MIGRATED_REVID=115830741
|