| Commit message (Collapse) | Author | Age |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
*** Reason for rollback ***
The no-cache tag is not respected (see b/77857812) and thus this breaks remote caching for all projects with symlink outputs.
*** Original change description ***
Only allow regular files and directories spawn outputs to be uploaded to a remote cache.
The remote cache protocol only knows about regular files and
directories. Currently, during action output upload, symlinks are
resolved into regular files. This means cached "executions" of an
action may have different output file types than the original
execution, which can be a footgun. This CL bans symlinks from cachable
spawn outputs and fixes http...
***
PiperOrigin-RevId: 193338629
|
|
|
|
|
|
|
| |
It breaks downstream rules_nodejs. See https://github.com/bazelbuild/bazel/issues/5028 for details.
RELNOTES: None.
PiperOrigin-RevId: 193074798
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bazel automatically detects the local Bash and
creates a custom toolchain rule for it.
Later, rules that use Bash will require this
toolchain and retrieve Bash's path from it instead
of relying on hardcoded paths or the
`--shell_executable` flag.
See https://github.com/bazelbuild/bazel/issues/4319
Change-Id: Idd8242a20d202b1f5a56cddac95b625c6c08ede9
Closes #4980.
Change-Id: Ic2406a4da260b284e15852070d58472ca18340af
PiperOrigin-RevId: 193022708
|
|
|
|
|
|
|
|
|
| |
The flag --experimental_repository_cache has been renamed to --repository_cache
(with the old name still usable for some releases). Adapt the shell tests to
use the preferred name of that flag.
Change-Id: I88ba70d386d5e8bb1bb60bcac58b2f1cc325ff17
PiperOrigin-RevId: 193003482
|
|
|
|
|
|
|
|
|
|
|
|
| |
... in register_toolchains and register_execution_platforms.
Fixes #5000.
Change-Id: Iac06b3cb79746ea720de8835b7bd1abb19149526
Closes #5001.
PiperOrigin-RevId: 192615586
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Remove the .useShell method, expect callers to
just pass the shell interpreter if they need it.
This removes the argument vector transformation
heuristic, and stops shelling out to cmd.exe on
Windows.
Also remove the .setWorkingDir method because
callers always had to set the working directory.
Instead, the CommandBuilder constructor takes the
working directory.
Closes #2190. (Test code still shells out to
cmd.exe though.)
Closes #5007.
Change-Id: I545e01c811daaf34913cb585492923da81aa02ee
PiperOrigin-RevId: 192611370
|
|
|
|
|
|
|
|
|
| |
BuildConfiguration.Fragment#compatibleWithStrategy().
The experiment did not work out.
RELNOTES: None.
PiperOrigin-RevId: 192567832
|
|
|
|
|
|
|
|
|
|
|
| |
...so that a repository rule can access them in any order without being
restarted. Restarting a repository rule can be expensive, if it already
accessed the network or executed expensive commands.
Improves on #4533.
Change-Id: I618c25322511dab42a80c1dddb0798fc9aea3106
PiperOrigin-RevId: 192420257
|
|
|
|
|
| |
RELNOTES: None.
PiperOrigin-RevId: 192313667
|
|
|
|
|
|
|
|
|
|
|
|
| |
While we still recommend to name repositories in canonical way, a canonical way
can not always be ensured, e.g., if different versions of the same external
repository have to be used. Our design for upcoming renaming repositories honors
this observation. So remove the old warning about the name in an external
repository's WORKSPACE file not matching the name given to it in the global
WORKSPACE file; we certainly will not make this an error in the future.
Change-Id: I5eef92ec61dc81d25734d71187a635024630322c
PiperOrigin-RevId: 192114195
|
|
|
|
|
|
|
|
| |
Fixes https://github.com/bazelbuild/bazel/issues/4409
Closes #4522.
PiperOrigin-RevId: 192112573
|
|
|
|
|
|
|
| |
This will mean the messages will make it to the right output stream.
RELNOTES:
PiperOrigin-RevId: 191925662
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Also include the test target in the test log for easier debugging of tests.
Fixes #4960.
Change-Id: Ic79bf21e8eeba765b607755c1d290ee4e910d01f
Closes #4972.
Sample output:
```
exec ${PAGER:-/usr/bin/less} "$0" || exit 1
Executing tests from //src/test/shell/bazel:bazel_example_test
-----------------------------------------------------------------------------
```
Change-Id: I433cd9251a564b269da5566bb8401151ae4c99b7
PiperOrigin-RevId: 191891278
|
|
|
|
|
|
|
|
|
|
|
| |
...so that a repository rule can access them in any order without being
restarted. Restarting a repository rule can be expensive, if it already
accessed the network or executed expensive commands.
Improves on #4533.
Change-Id: I1d27f88492e4e51fb955f3f066a28a75c5016169
PiperOrigin-RevId: 191874363
|
|
|
|
|
|
|
| |
Fixes: 2846
RELNOTES: Bazel now displays information about remote cache hits and execution strategies used in its UI after every build and test, and adds a corresponding line "process stats" to BuildToolLogs in BEP.
PiperOrigin-RevId: 191441770
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Don't ask me how so many things can be wrong in a single test...
Progress towards #4770.
FYI @rupertks @buchgr @ulfjack
## Replace 25000 invocations of perl with a single "sha256sum"
This speeds up the test by a factor 2x on my iMac (before: 1200s, now: 600s).
On macOS, "shasum" is a Perl script. Instead of simply passing all input files to the thing at once, we were invoking it once per file. This means roughly 25,000 invocations of Perl per test run. And it's even worse - it wasn't just a call to that Perl script, it was wrapped in a "cat | shasum | cut" pipeline, resulting in silent data loss when you accidentally passed multiple input files to the thing, 75,000 processes being spawned just to compute hashes and losing the file name of what was actually hashed. WTF.
Also, we were using SHA256 to essentially verify that two directory trees are equal. For this purpose, relying on SHA1 should be absolutely fine - and that is, provided by a good native implementation, four times faster than `shasum`. It saves another 10 seconds of the overall run.
With this change, the test also prints the result of a failed determinism check in an easier to read format "filename hash" instead of "hash filename" and on top of that, it also prints the filenames in the diff on macOS, which was missing formerly. Without this, it was basically impossible to debug failures of this test on macOS, as you couldn't see *which files were different*. You had *one* job, bazel_determinism_test.
Before:
```
-- Test log: -----------------------------------------------------------
--- /private/var/tmp/_bazel_buildkite/30004132848cb6cbb0d8bc124cd9712b/bazel-sandbox/8820973750646175047/execroot/io_bazel/_tmp/e503f3f3df14b71e247bc3d7d9bf3608/sum1 2018-03-28 18:00:43.000000000 +0000
+++ /private/var/tmp/_bazel_buildkite/30004132848cb6cbb0d8bc124cd9712b/bazel-sandbox/8820973750646175047/execroot/io_bazel/_tmp/e503f3f3df14b71e247bc3d7d9bf3608/sum2 2018-03-28 18:10:34.000000000 +0000
@@ -10417,0 +10418 @@
+ecd53ba69a8d479d3fa4234e959f869cd10f7ebc68860d2b7915879f8b8b2c54
@@ -10605 +10605,0 @@
-f1954b59039b74d0a0ee3b2bced748604b95b8455a5bf80489296bd81878a5c8
------------------------------------------------------------------------
```
Now (I artificially introduced non-hermeticism to show how a failure would look like):
```
-- Test log: -----------------------------------------------------------
--- /private/var/tmp/_bazel_philwo/7a01905b4627ca044e5e3f5ad5b14d26/bazel-sandbox/5464595340038418595/execroot/io_bazel/_tmp/e503f3f3df14b71e247bc3d7d9bf3608/sum1 2018-03-30 17:12:39.000000000 +0000
+++ /private/var/tmp/_bazel_philwo/7a01905b4627ca044e5e3f5ad5b14d26/bazel-sandbox/5464595340038418595/execroot/io_bazel/_tmp/e503f3f3df14b71e247bc3d7d9bf3608/sum2 2018-03-30 17:17:27.000000000 +0000
@@ -903 +903 @@
-bazel-bin/src/bazel 31d811338ca364f0631560dd4d29406dd6a778ce
+bazel-bin/src/bazel 8f009173894730b00a1d1d6349af7d10f4d21cf3
@@ -5656 +5656 @@
-bazel-bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar f5ec8c4415ad8ecdc0385affc68f2dd4dbf241ef
+bazel-bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar 9899ae35cf431087a34a830bfdaf19d99616689c
@@ -8343 +8343 @@
-bazel-bin/src/main/java/com/google/devtools/build/lib/worker/_javac/worker/libworker_classes/com/google/devtools/build/lib/worker/WorkerFactory.class 780baa17c19ef99ef0b9291db1791ed8e0f1b231
+bazel-bin/src/main/java/com/google/devtools/build/lib/worker/_javac/worker/libworker_classes/com/google/devtools/build/lib/worker/WorkerFactory.class d45c14f09e73e7fcdf01f96aa32646c87b704bc2
@@ -8359 +8359 @@
-bazel-bin/src/main/java/com/google/devtools/build/lib/worker/libworker.jar 60e3afbfec17da7e44c1f0f61cf2a446196717be
+bazel-bin/src/main/java/com/google/devtools/build/lib/worker/libworker.jar 70f557e87d1b32b2e46c79554fe6bf3b89aeaf6e
@@ -11343 +11343 @@
-bazel-genfiles/src/install_base_key 3fad754e4ea19bd1120df5bf16e1f39372e6b9fe
+bazel-genfiles/src/install_base_key 7d7e8b62493912c5ec153032e104640e3980e6b3
@@ -11376 +11376 @@
-bazel-genfiles/src/package.zip 1ce3431b021ca338806162eca72ff84118001df5
+bazel-genfiles/src/package.zip 65f4801d91bbe10cba0d2d4d55c7cf319cd6722d
------------------------------------------------------------------------
test_determinism FAILED: Non-deterministic outputs found! .
```
## Remove obsolete check for BAZEL_TEST_XTRACE
That string does not appear anywhere in our repo, except for these two lines in the test, so there's no point in checking for it.
## Remove obsolete check for Java 7
That was about time.
## Performance improvements and usability fixes
- There's no need to use mktemp to create a unique directory under TEST_TMPDIR, as every test suite has its own TEST_TMPDIR.
- There's no need to remove stuff, as this will just degrade performance and make debugging harder. The surrounding Bazel or system will clean up later.
- There's no need to copy bazel-bin/src/bazel to ./bazel1 before calling it, as you can just call the built bazel from its original location.
- There's no need to run "bazel clean" before the second "bazel build" invocation - it's better to just use two separate output_bases. This is faster and also makes debugging easier, as you can compare the two output_bases in case of a test failure.
- There's no need to call "diff" twice - we can just save the output immediately in the `if` block.
Closes #4945.
PiperOrigin-RevId: 191118833
|
|
|
|
|
|
|
|
|
|
|
| |
We expect that the client passes all startup options to the server, default or explicit. The server's listing of default values should not matter. Yet for a number of these options, the default value in the server was relied upon, because the server command line was not constructed with the client's default value included. Fix visible cases of this, long term this should be tested for, so the invariant is not broken again.
This has been the documented expectation for a long time, but a number of violations have crept up over time. Update the comments that lead to this expectation to be more realistic.
Add debug statement that shows which options are changed when startup options cause the server to be restarted. The detailed logs will only be seen if --client_debug is set to TRUE.
RELNOTES: None.
PiperOrigin-RevId: 191066983
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
We suggest our users to regularly test their code base
with --all_incompatible_chages to have time to adapt their
code base to upcoming incompatible changes. When doing so,
it is annoying if that breaks due to the Skylark code embedded
in bazel not being ready for the upcoming incompatible changes.
Therefore, add tests exercising the embedded code enabling
all incompatible changes. This way, hopefully, changes like
f1ad0e6df1528894b will come before the deprecation is added.
Change-Id: Iea60ff6a4019db310d1f1a1ad6a7b2bb53101c6d
PiperOrigin-RevId: 190766449
|
|
|
|
|
| |
RELNOTES: None.
PiperOrigin-RevId: 190759949
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
remote cache.
The remote cache protocol only knows about regular files and
directories. Currently, during action output upload, symlinks are
resolved into regular files. This means cached "executions" of an
action may have different output file types than the original
execution, which can be a footgun. This CL bans symlinks from cachable
spawn outputs and fixes https://github.com/bazelbuild/bazel/issues/4840.
The interface of SpawnCache.CacheHandle.store is refactored:
1. The outputs parameter is removed, since that can be retrieved from the underlying Spawn.
2. It can now throw ExecException in order to fail actions.
Closes #4902.
Change-Id: I0d1d94d48779b970bb5d0840c66a14c189ab0091
PiperOrigin-RevId: 190608852
|
|
|
|
|
|
|
|
| |
If a test target is built and testing is requested, nevertheless report what
was built.
Change-Id: Ida25f5f73fd4b5e102f2f4923acc555088495a8a
PiperOrigin-RevId: 190431506
|
|
|
|
|
|
|
| |
To replace blaze_util::die and blaze_util::pdie as well, FATAL statements need to accept blaze exit codes.
RELNOTES: None.
PiperOrigin-RevId: 190285798
|
|
|
|
|
|
| |
in either a FULL or LITE version. Trigger new output with the new --transitions cquery flag in the new CqueryOptions class.
PiperOrigin-RevId: 190278664
|
|
|
|
|
|
| |
TESTED=presubmit
RELNOTES: None
PiperOrigin-RevId: 190056498
|
|
|
|
|
|
| |
Fixes #4897.
PiperOrigin-RevId: 190045930
|
|
|
|
|
|
|
|
| |
This is technically an incompatible change, but I think it's unlikely to affect a lot of users. Note that this change leaves out Windows, where we set the PATH to the server env PATH plus the MSYS root determined from the shell path.
This is a cleanup, and it also makes unknown commit slightly safer.
PiperOrigin-RevId: 189981959
|
|
|
|
|
|
|
|
|
| |
We are still unable to turn this on to write to files, but there are currently 2 logging systems in use in the client: the inactive one, and the print-to-stderr option triggered by --client_debug. Combine these, so we can use the same logging format for both.
Also combine it with the VerboseLogging functionality - it was not documented anywhere and seems redundant.
RELNOTES: None.
PiperOrigin-RevId: 189979051
|
|
|
|
|
| |
RELNOTES: None
PiperOrigin-RevId: 189973158
|
|
|
|
|
| |
RELNOTES: None.
PiperOrigin-RevId: 189938838
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
To simplify the transition for existing repositories, we keep the property
that paths to true source files in your own repository may be hard-coded.
Add a test verifying that we keep this property.
Also add a test checking that this property is true, if the same repository
is built as a remote repository. But disable this test, as this property
does not hold yet; we plan to make it true with a change of the execroot
layout.
Change-Id: I0fb961f8ddb8feb98a6309533fc82f2acd9afd0d
PiperOrigin-RevId: 189898773
|
|
|
|
|
|
|
|
| |
Explicitly set local test jobs to 0 (= default, use local resources instead),
as otherwise this test will fail if --local_test_jobs is set in a system-wide
bazelrc.
PiperOrigin-RevId: 189887918
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This introduces user-facing options to enable the experimental sandboxfs
support and, when enabled, mounts a sandboxfs instance throughout the
build. The sandboxfs' process handle is passed to the
SandboxActionContextProvider so that the SpawnRunners can later consume it.
Note that this does NOT yet provide sandboxfs support for the builds as the
SpawnRunners are untouched.
RELNOTES: None.
PiperOrigin-RevId: 189678732
|
|
|
|
|
|
|
|
| |
https://github.com/bazelbuild/bazel/commit/2732df0202499aff75de38c012fec57ed53a5a30
SKIP_KOKORO: BazelCI
RELNOTES: None
PiperOrigin-RevId: 189378309
|
|
|
|
|
|
|
|
|
| |
timeouts.
The refactoring to have an Exception that contains partial results will also be used in the next CL, in order to propagate and save remote server logs.
RELNOTES: None
PiperOrigin-RevId: 189344465
|
|
|
|
| |
PiperOrigin-RevId: 189225279
|
|
|
|
|
|
|
|
| |
These are mostly tests. This is a step toward making the
ar_files/as_files attributes mandatory.
RELNOTES: None.
PiperOrigin-RevId: 189052327
|
|
|
|
| |
PiperOrigin-RevId: 189023695
|
|
|
|
|
|
|
|
| |
This allows more flexibility in registering toolchains and execution
platforms, both in the WORKSPACE and from the command-line.
Change-Id: I6fe75507d1a74de74085b7c927fdf093c152b894
PiperOrigin-RevId: 188813688
|
|
|
|
|
| |
RELNOTES: None.
PiperOrigin-RevId: 188744724
|
|
|
|
|
|
| |
relevant and only trigger when the implicit or explicit max depth > 20 which is confusing.
PiperOrigin-RevId: 188559702
|
|
|
|
| |
PiperOrigin-RevId: 188459395
|
|
|
|
|
|
|
|
| |
(fixes performance regression #4749). Also adding Skylark tests for input/output directories.
TESTED=locally
RELNOTES: Fix performance regression
PiperOrigin-RevId: 188346410
|
|
|
|
|
| |
RELNOTES:none
PiperOrigin-RevId: 188026038
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
It shouldn't be open sourced at all, since it solves a very specific internal
use case, and for external use cc_library with its includes, include_prefix, and
strip_include_prefix attributes can be used to perfectly replace cc_inc_library.
RELNOTES: Removed cc_inc_library, please use cc_library instead
Use includes, include_prefix, and strip_include_prefix attributes of cc_library
to replace the behavior of cc_inc_library with cc_library.
PiperOrigin-RevId: 187991313
|
|
|
|
|
|
|
|
|
|
|
|
| |
We pass the clientEnv to the workspace status command, and it may use
environment variables to influence status. We must therefore reevaluate
the workspace status command whenever the clientEnv changes.
Fixes #4222
Closes #4705.
PiperOrigin-RevId: 187988847
|
|
|
|
|
|
|
|
|
|
|
| |
//src/test/shell/bazel/android:android_ndk_integration_test
and
//src/test/shell/bazel:bazel_test_test
They're timing out on Kokoro recently.
RELNOTES: None
PiperOrigin-RevId: 187853786
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
NDK15 changelog:
https://android.googlesource.com/platform/ndk/+/ndk-r15-release/CHANGELOG.md
Notable changes for this CL:
- Clang has been updated to v5.0.300080.
- android-9 is not supported anymore. Anything lower than android-14 will default to android-14.
- NDK sysroot has been changed to %ndk%/sysroot
- Requires additional compilation flags because the headers are now unified in a single location across archs and API levels
Also removed NDK 10 checks in integration tests because it is old and we don't test against that anymore.
Fixes https://github.com/bazelbuild/bazel/issues/3137
Paves the way for r16: https://github.com/bazelbuild/bazel/issues/4068
RELNOTES[NEW]: Added Android NDK r15 support, including compatibility with Unified Headers.
PiperOrigin-RevId: 187648715
|
|
|
|
|
|
|
|
|
|
|
| |
The test_errmsg test tested the behaviour when workspace_status_command
was set to a file that did not exist, but other tests created a file
with that name. If test_errmsg did not run first, the
workspace_status_command would be valid and the test would fail.
Closes #4704.
PiperOrigin-RevId: 187470549
|
|
|
|
|
|
|
|
|
|
| |
By simply sharing the utility function. In this way, we get
feature parity between git_repository and http_archive.
Fixes #4676.
Change-Id: I99b300e42b2f267d8d04fd965f09c24f3ae54f10
PiperOrigin-RevId: 187450644
|
|
|
|
|
|
|
| |
Fixed https://github.com/bazelbuild/bazel/issues/4722
RELNOTES: None
PiperOrigin-RevId: 187443457
|