| Commit message (Collapse) | Author | Age |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The RuleContext object is not available when creating dynamic configuration
transitions. Removing it from ResourceFilter's state allows us to work with
ResourceFilter objects while creating those transitions. If we didn't do this,
we'd need to seperate the rest of ResourceFilter's state into a seperate class
so that we could work with it as part of doing dynamic configurations.
In the next reviews, I'll start actually creating dynamic configurations based
on ResourceFilter state.
Also, create a withAttrsFrom method that can be used in dynamic configuration
transitions, and generally migrate methods that work with attributes from
RuleContext to AttributeMap when practical.
To support these changes:
No longer keep the parsed lists of FolderConfiguration and Density objects as
fields of the ResourceFilter, instead, write functions that get them when
needed. We want to have access to a RuleContext when we initialize them to avoid
errors, and we don't have one in the withAttrsFrom method which will be called
as part of transitioning with dynamic configurations.
We no longer have those parsed lists to represent whether the object filters
during execution or analysis, so replace them with a seperate enum field for
filter behavior. Include a FILTER_IN_ANALYSIS_WITH_DYNAMIC_CONFIGURATION
option, even though it won't fully be used until the dynamic configuration
transition is taken advantage of in the next few reviews.
RELNOTES: none
PiperOrigin-RevId: 151715400
|
|
|
|
|
|
|
|
|
|
| |
If the workspace directory is /path/to/my/proj and the name in the WORKSPACE
file is "floop", this will symlink the output directories to
output_base/execroot/floop instead of output_base/execroot/proj.
More prep for #1262, fixes #1681.
PiperOrigin-RevId: 151712384
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The test fails in the last action, where it tries to run the bazel-bin/hello-world binary with the linux-sandbox while mounting "source" to "target":
philwo@philwo-test:~/test$ $linux_sandbox -D \
> -M ${source} \
> -m ${target} \
> -- bazel-bin/hello-world
src/main/tools/linux-sandbox.cc:169: linux-sandbox-pid1 has PID 28912
[...]
src/main/tools/linux-sandbox-pid1.cc:260: remount ro: /home/philwo/test/mount_targets/x86_64-unknown-linux-gnu/sysroot/lib64/ld-2.19.so
src/main/tools/linux-sandbox-pid1.cc:349: sigaction(32, &sa, NULL) failed
src/main/tools/linux-sandbox-pid1.cc:349: sigaction(33, &sa, NULL) failed
src/main/tools/linux-sandbox-pid1.cc:459: waitpid returned 2
src/main/tools/linux-sandbox-pid1.cc:476: child died due to signal 11
PiperOrigin-RevId: 151693791
|
|
|
|
|
|
| |
RELNOTES: None.
PiperOrigin-RevId: 151688820
|
|
|
|
|
|
| |
keeping incremental state.
PiperOrigin-RevId: 151639711
|
|
|
|
|
|
|
|
|
|
| |
*** Reason for rollback ***
Breaks tests that expected previous id generation order.
RELNOTES: None.
PiperOrigin-RevId: 151638886
|
|
|
|
|
|
|
|
| |
RuleTransitionFactory#buildTransitionFor is @Nullable, but the
SkyframeExecutor expects a non-null Transition. If the factory
returns null, the SkyframeExecutor should be passed NONE, not null.
PiperOrigin-RevId: 151615724
|
|
|
|
|
|
| |
RELNOTES: None
PiperOrigin-RevId: 151602497
|
|
|
|
|
|
|
|
| |
Fixes https://github.com/bazelbuild/bazel/issues/2739.
RELNOTES: None
PiperOrigin-RevId: 151592983
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The new RemoteExecutionClient class only performs remote execution, and
nothing else; all higher-level functions, local rety, etc. will live outside
of the client.
In order to add unit tests, I had to add another layer of indirection between
the Grpc{RemoteExecutor,ActionCache} and GRPC, since GRPC generates final,
non-mockable classes. While a testing approach that uses a fake server can
also get some test coverage (as in GrpcActionCacheTest), it doesn't allow us
to test the full range of bad things that can happen at the GRPC layer.
The cloned implementation uses a single GRPC channel, as was recommended to
me by Jakob, who worked on GRPC. A single channel should be sufficiently
scalable, it's thread-safe, and it performs chunking internally. On the
server-side, the requests from a single channel can be dispatched to a thread
pool, so this should not be a blocker for server-side parallelism.
I also changed it to throw an exception whenever anything bad happens - this
makes it much more obvious if there's still bug in this code; the old code
silently swallows many errors, falling back to local execution, which papers
over many issues.
Furthermore, we now return a RemoteExecutionResult to indicate whether the
action ran at all (regardless of exit code), as well as the exit code.
All in all, this implementation is closer to the production code we're using
internally, although quite a few things are still missing.
The cloned implementation is not hooked up to RemoteSpawnStrategy yet. It
also does not support combining remote caching with local execution, but note
that RemoteSpawnStrategy regressed in that respect and currently also does
not support that mode.
PiperOrigin-RevId: 151578409
|
|
|
|
|
|
|
|
|
|
|
| |
So far only link actions were initialized in CppLinkActionConfigs. This cl
changes this class to also initialize CppCompile actions. This is needed for
our ongoing work removing hard-coded flags from Bazel and moving them into
Crosstool.
RELNOTES: None.
PiperOrigin-RevId: 151575045
|
|
|
|
|
|
|
|
|
|
| |
If a rule requires a special provider that's (maybe) not provided explicitly
but nevertheless available from a target, i.e. "files", the mandatory providers
check should pass.
Fixes #1490
PiperOrigin-RevId: 151571187
|
|
|
|
|
|
|
|
|
| |
RELNOTES[INC]: Converting artifacts to strings and printing them now return
"File" instead of "Artifact" to be consistent with the type name.
Fixes #2715
PiperOrigin-RevId: 151563252
|
|
|
|
|
|
| |
since it's already returned via JavaProvider.
PiperOrigin-RevId: 151550286
|
|
|
|
|
|
|
|
| |
ConfiguredTargetValues. Also clear transitive packages for both, even for top-level targets.
This is not expected to save significant memory, but is expected to reduce the number of references to Packages, allowing them to be dropped more easily when discarding analysis cache and running in batch mode.
PiperOrigin-RevId: 151508877
|
|
|
|
|
|
| |
RELNOTES: A downloader bug was fixed that prevented RFC 7233 Range connection resumes from working with certain HTTP servers
PiperOrigin-RevId: 151508000
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
It's possible for cases to happen where the BlazeCommandDispatcher
fails before starting the WaitForCompletionCommand. In these cases,
the test's command state handoff will never take place because no
command state will be created to be handed off (obviously).
This means that runIn will wait forever.
Instead, if the command completes and the command state has still
not been handed off, set an exception into the command state so
that the test will fail without further troubles.
RELNOTES: None.
PiperOrigin-RevId: 151495494
|
|
|
|
| |
PiperOrigin-RevId: 151486661
|
|
|
|
|
|
| |
The encapsulation was off, since the format in which we accept policy has nothing to do with how we then enforce the policy.
PiperOrigin-RevId: 151471747
|
|
|
|
|
|
| |
--
PiperOrigin-RevId: 151456059
MOS_MIGRATED_REVID=151456059
|
|
|
|
|
|
|
|
|
|
| |
file, so that each test run can have the latest version of the Classpaths.
This does not tackle the case where runfiles are not present, such as windows.
--
PiperOrigin-RevId: 151440999
MOS_MIGRATED_REVID=151440999
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The simple distributed caching in Bazel used ConcurrentMap as a blob store. It is incorrect to use an overloaded interface for this purpose.
This change defines a SimpleBlobStore interface that only has put(), get() and containsKey()
methods and allows a simple implementation of a blob store as remote cache for Bazel.
Also updated documentation to summarize the options available in the remote spwan strategy.
There is no functional change.
TESTED=shell integration tests
--
Change-Id: Iedff0bc4f06c4a93c398c53801014d998c3df13b
Reviewed-on: https://cr.bazel.build/9330
PiperOrigin-RevId: 151439467
MOS_MIGRATED_REVID=151439467
|
|
|
|
|
|
|
|
|
|
| |
* Fix the overwrite algorithm to be handled entirely in the accumulators.
* Improve the conflict error messages for attributes and simple values.
* Refactor the UnwrittenDataSubject to top level for reuse.
--
PiperOrigin-RevId: 151439084
MOS_MIGRATED_REVID=151439084
|
|
|
|
|
|
| |
--
PiperOrigin-RevId: 151381769
MOS_MIGRATED_REVID=151381769
|
|
|
|
|
|
|
|
|
|
| |
e.g.: cmd = "$(CC) foo.cc > $@"
Fixes #2729
--
PiperOrigin-RevId: 151369889
MOS_MIGRATED_REVID=151369889
|
|
|
|
|
|
|
|
|
|
|
|
| |
...irrespective of their success status. While a build typically
contains too many successful actions to report them all, extra actions
included in a build are worth reporting.
--
Change-Id: I6b20935895aa7b16836d6271f456176a7113317e
Reviewed-on: https://cr.bazel.build/9519
PiperOrigin-RevId: 151328633
MOS_MIGRATED_REVID=151328633
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
the rule.
Example:
tools/newlang/BUILD:
toolchain_type(name = "toolchain_type")
Allows users to refer to //tools/newlang:toolchain_type when discussing the
newlang toolchain.
The previous usage:
tools/newlang/BUILD:
toolchain_lookup(name = "lookup")
Lead to labels like //tools/newlang:lookup, which was unclear that each
language's toolchain needs one unique instance of toolchain_(lookup|type).
--
PiperOrigin-RevId: 151326827
MOS_MIGRATED_REVID=151326827
|
|
|
|
|
|
|
|
| |
http://ci.bazel.io/job/bazel-tests/BAZEL_VERSION=HEAD,PLATFORM_NAME=linux-x86_64/lastCompletedBuild/testReport/(root)/(empty)/test_sandbox_mount_customized_path/
--
PiperOrigin-RevId: 151306949
MOS_MIGRATED_REVID=151306949
|
|
|
|
|
|
| |
--
PiperOrigin-RevId: 151302418
MOS_MIGRATED_REVID=151302418
|
|
|
|
|
|
| |
--
PiperOrigin-RevId: 151170448
MOS_MIGRATED_REVID=151170448
|
|
|
|
|
|
| |
--
PiperOrigin-RevId: 151145409
MOS_MIGRATED_REVID=151145409
|
|
|
|
|
|
|
|
|
|
|
| |
Note that this is not quite a no-op. Setting -isystem puts these directories at
the beginning of the search list, whereas not setting them results in them
being near the end. However, this code is unnecessary and confusing and caused
a bug with the clang toolchains so it is best to remove it.
--
PiperOrigin-RevId: 151140445
MOS_MIGRATED_REVID=151140445
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
execution.
If you're feeling like you've already seen this, that's correct, these were the exact contents of commit e860316559eac366d47923a8eb4b5489a661aa35... and then, on Nov 15, something unclear happened and the code disappeared! Perhaps it was the result of a faulty sync. In any case, nobody noticed, and the CL went in. It was later rolloed back and resubmitted, but the crucial code changes were gone.
TESTED=local server with profiling for SHA1 specifically
RELNOTES: n/a
--
PiperOrigin-RevId: 151139685
MOS_MIGRATED_REVID=151139685
|
|
|
|
|
|
|
|
| |
case-insensitive paths.
--
PiperOrigin-RevId: 151137308
MOS_MIGRATED_REVID=151137308
|
|
|
|
|
|
| |
--
PiperOrigin-RevId: 151129669
MOS_MIGRATED_REVID=151129669
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Rollback of commit df366408188f0451bae9b2ed079c795a4beb2e2b.
In addition to undoing the rollback of my previous change, fix the bugs it
introduced and add tests for those bugs.
Always ignore empty filters. Empty filters are always useless or
counterproductive. Before the original change, empty filters as a single item
within the list of filters (e.g., ["en", ""]) were ignored, but empty filters
as a portion of a string in the list (e.g., ["en,"]) were not. I can't imagine
any reason people would actually want the empty filter (if it were handled
correctly, it would effectively tell Bazel to just ignore every other filter
the user passed in).
Since it makes more sense with the new code, represent the stringified filters
as a list of strings, rather than a single string of comma-seperated values.
Manually trim whitespace from each token. Before the original change, the code
trimmed whitespace following commas (e.g., ["en, es"] -> ["en,es"]) but not
otherwise. If we're allowing whitespace in filter strings anyway, there doesn't
seem to be any reason to allow it in some places but not others.
--
PiperOrigin-RevId: 151128685
MOS_MIGRATED_REVID=151128685
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Also refactor the way we compute writable dirs so that they're computed
only once per running action, not twice.
Fixes #2056, fixes #1973, fixes #1460.
RELNOTES: /tmp and /dev/shm are now writable by default inside the
Linux sandbox.
--
PiperOrigin-RevId: 151123543
MOS_MIGRATED_REVID=151123543
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Try to run /bin/true as a test of whether the Linux sandbox works,
instead of just trying to create a bunch of namespaces as a proxy.
This helps resolve issues on Linux distros where the earlier check
worked, but then the sandbox ultimately failed due to other operations
being unsupported.
As an example, Debian Jessie and certain Docker versions seem to allow
the creation of PID namespaces, but forbid mounting a new proc on top of
/proc (see #1972). This resulted in Bazel thinking that sandboxing works
fine, when it actually didn't. The improved check correctly catches this
situation and disabled sandboxing.
--
PiperOrigin-RevId: 151116894
MOS_MIGRATED_REVID=151116894
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This can be reactivated by passing the --sandbox_fake_username flag
to Bazel.
Reasoning: 'nobody' has a non-existent home directory on many Linux
distros, leading to issues when tools try to stat / read / write to the
home directory.
Related to #2688.
RELNOTES: The Linux sandbox no longer changes the user to 'nobody' by
default, instead the current user is used as is. The old behavior can be
restored via the --sandbox_fake_username flag.
--
PiperOrigin-RevId: 151115218
MOS_MIGRATED_REVID=151115218
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
By removing the --sandbox_block_path feature in an earlier change and
taking advantage of the fact that in a mount namespace we can actually
"remount" mount points to be read-only without bind mounting them to
some other place beforehand, this is no longer necessary. The code
becomes much simpler due to this, for example we no longer need to
chroot.
--
PiperOrigin-RevId: 151111360
MOS_MIGRATED_REVID=151111360
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
It is in the way of optimizing the performance of the sandbox, because
it requires us to create two helper files (an unreadable file and an
unreadable directory) which are bind-mounted on top of paths specified
via this flag. These two helper files were created on a tmpfs mounted by
the sandbox until now, which ensured that they were automatically
deleted on exit. However, mounting tmpfs on /dev/shm or /tmp causes
issues like #2686 or #1882.
By removing this flag, we can get rid of the two helper files, which
means we can also remove the reliance on a "sandbox temp directory"
completely in the next change.
--
PiperOrigin-RevId: 151107496
MOS_MIGRATED_REVID=151107496
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
*** Reason for rollback ***
Rolling forward with the correct changes to the AndroidResourceMergingAction. Tested manually.
*** Original change description ***
Automated [] rollback of commit a58f245a4b40c0ef961b1f30d96b16a9349711c3.
*** Reason for rollback ***
broke over 100k targets, in the depot, see []
*** Original change description ***
Move library R generation to a separate action, ensuring the merging happens
off the java critical path.
--
PiperOrigin-RevId: 151087737
MOS_MIGRATED_REVID=151087737
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
classloader. Currently this funcionality is hidden behind the "experimental_testrunner" flag.
Original description (from commit 786cfa2ed980e278c42ee474408844f7e3720385):
Separate the classpaths of the TestRunner with the test target, and use a separate Classloader to load the test target's classes. This enables a clean separation of the classes of the TestRunner with the target under test.
This is achieved with the following steps:
1. Start the test runner with only the bare bones classpaths to the Test Runner's classes which are used by the system ClassLoader.
2. Have all the classpaths required to load the test target's classes in a TEST_TARGET_CLASSPATH environment variable exported by the stub script.
3. Use a new classloader to load all the test target's classes using the paths in TEST_TARGET_CLASSPATH.
This additionally enables the persistent test runner (currently experimental), to reload all the target's classes for every subsequent test run, so it can pick up any changes to the classes in between runs.
The persistent test runner can be used by adding the argument
--test_strategy=experimental_worker to the bazel test command (and having the tag "experimental_testrunner" in the java_test rule).
Tested this against:
1. gerrit/gerrit-common:client_tests: Dismal avg. improvement of 580ms to 557ms (just 23ms)
2. intellij/intellij/base:unit_tests: Somewhat modest avg. improvement 1661ms to 913ms (748 ms)
--
PiperOrigin-RevId: 151065529
MOS_MIGRATED_REVID=151065529
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
reverseDepsToConsolidate field in InMemoryNodeEntry. As part of that, revamp our logic of how we store pending operations: store adds bare on initial evaluations, and checks bare on incremental evaluations and operations on done nodes.
This should improve performance in two ways: BuildingState loses two fields, saving working memory intra-build. Storing pending reverse dep operations bare also saves memory intra-build. Note that neither of these changes helps resting memory state, only while a node is still evaluating.
Because of this, we can simplify ReverseDepsUtil a bit, making ReverseDepsUtilImpl a static class, which it always wanted to be (what it really wants to be is a superclass of InMemoryNodeEntry, but I don't want to spend the object alignment bits).
Finally, this makes it pretty tempting to get rid of BuildingState altogether on initial evaluations. We'd still keep DirtyBuildingState, but we could save another ~24 bytes by storing BuildingState's one remaining field, signaledDeps, directly inside InMemoryNodeEntry.
--
PiperOrigin-RevId: 151048879
MOS_MIGRATED_REVID=151048879
|
|
|
|
|
|
|
|
| |
on the presence of features.
--
PiperOrigin-RevId: 151038811
MOS_MIGRATED_REVID=151038811
|
|
|
|
|
|
|
|
|
|
|
|
| |
As of Bazel 0.4.5, the android_sdk_for_test and android_ndk_for_test are bound in android.WORKSPACE, so we do not need to include them in the main Bazel WORKSPACE file. I've update the comments to reflect that all that is needed to run the tests is android_sdk_repository and android_ndk_repository and the environment variables that they read.
Also, delete scripts/workspace_user.sh. All of its functionality (reading environment variables, detecting api levels, build tools versions) is now part of android_{s,n}dk_repository.
Fixes https://github.com/bazelbuild/bazel/issues/2284.
--
PiperOrigin-RevId: 151032551
MOS_MIGRATED_REVID=151032551
|
|
|
|
|
|
|
|
|
|
| |
This fixes remote test execution.
Fixes #1593.
--
PiperOrigin-RevId: 151030133
MOS_MIGRATED_REVID=151030133
|
|
|
|
|
|
|
|
|
|
|
|
| |
experimental_testrunner is invoked. This has the unintended side effect of accidentally supplying dependencies such as JUnit and hamcrest, when bazel should enforce the target to specify those dependencies explicitly.
We should eventually make this the default behaviour of bazel instead of hiding it behind the experimental_testrunner flag.
This is another partial rollforward of commit 786cfa2ed980e278c42ee474408844f7e3720385
--
PiperOrigin-RevId: 151022781
MOS_MIGRATED_REVID=151022781
|
|
|
|
|
|
|
|
|
|
|
| |
...in a way preserving the list structure and without escaping. In other
words, return the options as needed for an execve(3) call.
--
Change-Id: Ifb168d2e720392fb3a97d557960e0c1f1a83f543
Reviewed-on: https://cr.bazel.build/9458
PiperOrigin-RevId: 151019830
MOS_MIGRATED_REVID=151019830
|
|
|
|
|
|
|
|
| |
It turns out expect_log doesn't work as I expected..
--
PiperOrigin-RevId: 151016907
MOS_MIGRATED_REVID=151016907
|