| Commit message (Collapse) | Author | Age |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This has the following improvements upon the older one:
- Uses PID namespaces, PR_SET_PDEATHSIG and a number of other tricks for
further process isolation and 100% reliable killing of child processes.
- Uses clone() instead of unshare() to work around a Linux kernel bug that
made creating a sandbox unreliable.
- Instead of mounting a hardcoded list of paths + whatever you add with
--sandbox_add_path, this sandbox instead mounts all of /, except for what
you make inaccessible via --sandbox_block_path. This should solve the
majority of "Sandboxing breaks my build, because my compiler is installed
in /opt or /usr/local" issues that users have seen.
- Instead of doing magic with bind mounts, we create a separate execroot for
each process containing symlinks to the input files. This is simpler and
gives more predictable performance.
- Actually makes everything except the working directory read-only
(fixes #1364). This means that a running process can no longer accidentally
modify your source code (yay!).
- Prevents a number of additional "attacks" or leaks, like accidentally
inheriting file handles from the parent.
- Simpler command-line interface.
- We can provide the same semantics in a Mac OS X sandbox, which will come in
a separate code review from yueg@.
It has the following caveats / known issues:
- The "fallback to /bin/bash on error" feature is gone, but now that the
sandbox mounts everything by default, the main use-case for this is no
longer needed.
The following improvements are planned:
- Use a FUSE filesystem if possible for the new execroot, instead of creating
symlinks.
- Mount a base image instead of "/".
FAQ:
Q: Why is mounting all of "/" okay, doesn't this make the whole sandbox
useless?
A: This is still a reasonable behavior, because the sandbox never tried to
isolate your build from the operating system it runs in. Instead it is
supposed to protect your data from a test running "rm -rf $HOME" and to
make it difficult / impossible for actions to use input files that are not
declared dependencies. For even more isolation the sandbox will support
mounting a base image as its root in a future version (similar to Docker
images).
Q: Let's say my process-specific execroot contains a symlink to an input file
"good.h", can't the process just resolve the symlink, strip off the file
name and then look around in the workspace?
A: Yes. Unfortunately we could not find any way on Linux to make a file appear
in a different directory with *all* of the semantics we would like. The
options investigated were:
1) Copying input files, which is much too slow.
2) Hard linking input files, which is fast, but doesn't work cross-
filesystems and it's also not possible to make them read-only.
3) Bind mounts, which don't scale once you're up in the thousands of input
files (across all actions) - it seems like the kernel has some
non-linear performance behavior when the mount table grows too much,
resulting in the mount syscall taking more time the more mounts you
have.
4) FUSE filesystem, good in theory, but wasn't ready for the first
iteration.
RELNOTES: New sandboxing implementation for Linux in which all actions run in a separate execroot that contains input files as symlinks back to the originals in the workspace. The running action now has read-write access to its execroot and /tmp only and can no longer write in arbitrary other places in the file system.
--
Change-Id: Ic91386fc92f8eef727ed6d22e6bd0f357d145063
Reviewed-on: https://bazel-review.googlesource.com/#/c/4053
MOS_MIGRATED_REVID=130638204
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
If the tmpdir wasn't below the execroot, Bazel would crash and print a stack
trace.
Also, the test was incorrect because the EOF wasn't quoted, so it was actually
just executing "echo TEST_TMPDIR=/path/to/tmpdir" which (surprise surprise)
matched TEST_TMPDIR=/path/to/tmpdir.
Finally, the test was _also_ incorrect because it was using the cached test
result for the second case, since changing the tmpdir doesn't invalidate the
test result. So it not only was comparing a constant string to a constant
string, but it wasn't even re-evaluating the constant string.
--
MOS_MIGRATED_REVID=130637221
|
|
|
|
|
|
|
|
|
| |
method cannot be found when called from Skylark.
PAIR=laurentlb
--
MOS_MIGRATED_REVID=130636387
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Please don't merge before I8b8c3596fd2a4c125071406eefd407ca402099dd. The
test will fail then.
I've seen two issues with this functionality breaking: #481 and #1637.
Seems like it's time to add a test which catches the issue on the
Debian/Ubuntu machines people commonly test on. The test fails on my
Debian system before I8b8c3596fd2a4c125071406eefd407ca402099dd, and
passes with that change applied.
--
Change-Id: Ib785c874cdb9192920f9935b696bfd6c9c0e5f4f
Reviewed-on: https://bazel-review.git.corp.google.com/#/c/5451/
MOS_MIGRATED_REVID=130635565
|
|
|
|
|
|
|
|
|
| |
fixes #1637 (https://github.com/bazelbuild/bazel/issues/1637)
--
Change-Id: I8b8c3596fd2a4c125071406eefd407ca402099dd
Reviewed-on: https://bazel-review.git.corp.google.com/#/c/5410/2
MOS_MIGRATED_REVID=130633667
|
|
|
|
|
|
|
|
|
|
|
|
| |
If the user interrupts the build, they probably don't want local
fallback to happen.
Also clean up an unused method and parameter.
--
Change-Id: I6bf80fa110bbba911b0743f24c25240c208c98d1
Reviewed-on: https://bazel-review.googlesource.com/5470
MOS_MIGRATED_REVID=130612791
|
|
|
|
|
| |
--
MOS_MIGRATED_REVID=130609319
|
|
|
|
|
| |
--
MOS_MIGRATED_REVID=130576075
|
|
|
|
|
| |
--
MOS_MIGRATED_REVID=130553300
|
|
|
|
|
| |
--
MOS_MIGRATED_REVID=130547971
|
|
|
|
|
| |
--
MOS_MIGRATED_REVID=130546999
|
|
|
|
|
|
|
| |
instead of hard-coded values.
--
MOS_MIGRATED_REVID=130543727
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
sorted. Previously, it would return a list formed by concatenating the sorted results of each pattern in the 'includes' list.
A bunch of cleanups and one bug fix:
-Remove the unused-except-for tests GlobCache#globsUpToDate. This code has been dead for a very very long time, ever since we switched to using Skyframe.
-Change the semantics of the 'glob' function as described above.
-Change UnixGlob to return unsorted results. Document this in UnixGlob and GlobCache.
-Change LegacyGlobber to conditionally return sorted results. Have users other than PackageFunction get sorted results (as described above). Have PackageFunction's use case get completely unsorted results, and have PackageFunction do the sorting itself.
-Have PackageFunction's HybridGlobber unconditionally sort the glob result list. This ensure deterministic glob results, fixing a bug where the order of the elements of the result depended on the contents of the Skyframe graph, which of course depends on the sequence of incremental Blaze commands.
--
MOS_MIGRATED_REVID=130540152
|
|
|
|
|
| |
--
MOS_MIGRATED_REVID=130535008
|
|
|
|
|
|
|
| |
Otherwise, we'll get an NPE in build(), which doesn't help in finding the place where the null was added.
--
MOS_MIGRATED_REVID=130531765
|
|
|
|
|
|
|
| |
codesign action.
--
MOS_MIGRATED_REVID=130530871
|
|
|
|
|
|
|
| |
the interface allows it.
--
MOS_MIGRATED_REVID=130530262
|
|
|
|
|
|
|
|
|
|
| |
Otherwise we fail the Precondition check added at the start of CppLinkAction
build() for a shared library target's indexing action.
Added a test which reproduces the issue without the fix.
--
MOS_MIGRATED_REVID=130513291
|
|
|
|
|
|
|
| |
Also allow AbruptExitException from all server startup hooks.
--
MOS_MIGRATED_REVID=130513167
|
|
|
|
|
|
|
| |
line option to determine if it should be a whole archive one and use the artifact category in LinkerInput to make that decision instead.
--
MOS_MIGRATED_REVID=130508699
|
|
|
|
|
|
|
| |
0978a8ea1593ef204ea97155014f76baba2508c5 broke it.
--
MOS_MIGRATED_REVID=130496094
|
|
|
|
|
| |
--
MOS_MIGRATED_REVID=130469884
|
|
|
|
|
|
|
|
|
|
|
| |
InetAddress.getLocalHost().getHostName() can take seconds to complete as it
performs reverse DNS lookup. Prior to this cl hostname lookup was performed on
every build, noticeably slowing down null builds especially. This cl caches
computed hostname so null builds are faster for the lifetime of the server.
--
Reviewed-on: https://bazel-review.googlesource.com/#/c/5432/
MOS_MIGRATED_REVID=130441617
|
|
|
|
|
|
|
| |
The new rule doesn't currently function - it requires adding a JavaLite protoc plugin to Bazel, and an additional change to this code to load and use it.
--
MOS_MIGRATED_REVID=130440693
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
*** Reason for rollback ***
Broke exoblaze build
This is a partial rollback: changes to mocks, and to allow running java tests
with the launcher enabled and disabled were left in.
*** Original change description ***
Roll forward commit 94c86135d05a1844263c59f3ce6b1c1917e0f4c8
And don't provide a default value for :java_launcher
--
MOS_MIGRATED_REVID=130429620
|
|
|
|
|
|
|
| |
now to forbid it, since Skyframe lookups are interruptible.
--
MOS_MIGRATED_REVID=130429286
|
|
|
|
|
| |
--
MOS_MIGRATED_REVID=130424806
|
|
|
|
|
|
|
| |
throw InterruptedException.
--
MOS_MIGRATED_REVID=130424634
|
|
|
|
|
|
|
| |
experimental_objc_library, which may export cc providers.
--
MOS_MIGRATED_REVID=130415669
|
|
|
|
|
| |
--
MOS_MIGRATED_REVID=130406840
|
|
|
|
|
|
|
|
|
| |
QueryExceptions now that all relevant methods declare that they throw InterruptedException.
Small side benefit of commit 3c0adb26bac6d756fb97e4bcc6d4e5b2cefa5eeb.
--
MOS_MIGRATED_REVID=130402917
|
|
|
|
|
|
|
| |
LinkerInput#getArtifactCategory() and use this information to remove one use of LINK_LIBRARY_FILETYPES.
--
MOS_MIGRATED_REVID=130400793
|
|
|
|
|
|
|
| |
our C++ rules (except from precondition checks)
--
MOS_MIGRATED_REVID=130396421
|
|
|
|
|
|
|
| |
library.
--
MOS_MIGRATED_REVID=130394540
|
|
|
|
|
| |
--
MOS_MIGRATED_REVID=130374987
|
|
|
|
|
| |
--
MOS_MIGRATED_REVID=130371455
|
|
|
|
|
|
|
| |
memory leak in gRPC (or Netty).
--
MOS_MIGRATED_REVID=130369785
|
|
|
|
|
|
|
| |
much smaller. This adds more granularity and helps prevent excessive compilation by minimizing the number of inputs for each action.
--
MOS_MIGRATED_REVID=130359288
|
|
|
|
|
| |
--
MOS_MIGRATED_REVID=130330900
|
|
|
|
|
|
|
| |
The only place we now don't handle InterruptedException is in the action graph created after analysis, since I'm not sure that will be around for that much longer.
--
MOS_MIGRATED_REVID=130327770
|
|
|
|
|
| |
--
MOS_MIGRATED_REVID=130326607
|
|
|
|
|
|
|
| |
I must have been between "similar to" and "same as" and chose both.
--
MOS_MIGRATED_REVID=130281024
|
|
|
|
|
|
|
|
| |
A small cleanup and refactoring in advance of heavier work to be done
for Skylark computed defaults.
--
MOS_MIGRATED_REVID=130140706
|
|
|
|
|
| |
--
MOS_MIGRATED_REVID=130123926
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Increase the ping timeout on connect from five to ten seconds. This gives
servers which may be suffering from gc pressure or other ailments extra time to
respond.
On the other end, wait for orphaned servers to really die before proceeding.
This prevents race conditions around the delivery of SIGKILL and the starting
of the new server.
This may make us fail slower when the server is having hard times, however it
should give us better determinism, and these conditions should be rare.
--
MOS_MIGRATED_REVID=130118918
|
|
|
|
|
| |
--
MOS_MIGRATED_REVID=130114142
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Normally, Bazel generates a new UUID for every build invocation. In some
use cases, however, the invocation of Bazel is part of a larger build
process or otherwise controlled by a different tool. In this case, it
is useful to tell Bazel the identifier of the build to make it fit with
other identifiers generated by the controlling process. To achieve this,
the variable BAZEL_INTERNAL_INVOCATION_ID from the client environment
is inspected and if it is set and its value is a syntactically correct
UUID, this UUID will be used as the identifier for the build. It is in
the responsibility of the caller to ensure the id is sufficiently unique
if that environment variable is set.
--
MOS_MIGRATED_REVID=130079446
|
|
|
|
|
|
|
|
|
|
| |
The processed manifest has applicationId subbing
while the merged one is before substitution (if
the developer hasn't migrated to the new gradle-
style manifest merger).
--
MOS_MIGRATED_REVID=130046777
|
|
|
|
|
|
|
|
|
| |
Make sure that ParallelEvaluator treats SchedulerExceptions as less severe than other RuntimeExceptions (it already did, but add a comment emphasizing this).
The combination of the above means that we don't ignore hypothetical crashes in ParallelEvaluator worker threads _after_ a SchedulerException (e.g. due to an error in nokeep_going evaluation).
--
MOS_MIGRATED_REVID=130044230
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
OrderedSetMultimap. This maintains insertion order while eliminating duplicates.
Certain rules, in particular, otherwise break this invariant:
https://github.com/bazelbuild/bazel/blo[]e3e28274cca5b87f48abe33884edb84016dd3/src/main/java/com/google/devtools/build/lib/skyframe/ConfiguredTargetFunction.java#L403
There's no reason (to my knowledge) to need multiple instances of the same <Attribute, Dependency> pair.
More context from Google code review:
(Michael Staib):
> There are many things which pass around a dependentNodeMap or help construct one or modify one. We want an interface which has the right guarantees.
> ListMultimap is not the right interface because it has no guarantee of unique elements, which we want - we don't want the problem that this CL ran into, and there's no reason (that we know of, to be confirmed) that anyone would want multiple identical Dependencies.
> SetMultimap is not the right interface because it has no guarantee of deterministic iteration order or efficient iteration, which we want - dependency order sometimes matters (e.g., Java classpath or C++ link order).
> We agreed that the best way to get what we want is to define our own interface with its own simultaneous uniqueness and iterability guarantees. Unspoken in the discussion was why we wouldn't just use LinkedHashMultimap as the thing we pass around. IMO the reason for that is that we don't care that it be a LinkedHashMultimap specifically; if tomorrow Guava comes out with a faster cooler map that has deterministic and efficient iteration and guarantees element uniqueness, we want it.
> In this case we're going to make the "interface" be a (final?) class: OrderedSetMultimap, an extension of ForwardingSetMultimap which delegates to LinkedHashMultimap, an implementation which does support both of those guarantees.
> I had mentioned in the conversation that none of the Multimap implementations make guarantees about key iteration order, but this is not true - LinkedHashMultimap preserves key insertion order. We should perhaps declare this as part of the OrderedSetMultimap contract as well.
--
MOS_MIGRATED_REVID=130037643
|