| Commit message (Collapse) | Author | Age |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
After having open sourced the Build Event Service code, there is no more need
to have two separate bazel modules that both create a BuildEventStreamer.
This change merges the BuildEventStreamerModule logic into the
BuildEventServiceModule.
DELTA=677 (330 added, 316 deleted, 31 changed)
DELTA_BY_EXTENSION=java=293,oss=32
RELNOTES: None.
PiperOrigin-RevId: 158960687
|
|
|
|
|
|
|
| |
...instead of relying on all the methods to call printErrLn with exactly the
right format string.
PiperOrigin-RevId: 158951236
|
|
|
|
|
|
|
|
| |
In order for BlazeModule.workspaceInit to be self-contained, also pass in the
BlazeRuntime; we have use cases where this context is relevant, and there's
currently no other way to get a reference to the BlazeRuntime.
PiperOrigin-RevId: 158861142
|
|
|
|
|
|
|
|
|
|
|
| |
Don't pass the Command annotation explicitly, but add it to CommandEnvironment
instead; most modules don't need it in the first place, so it was a lot of
boilerplate for not much. Also change it so that the command is passed to the
constructor.
Add some documentation to the beforeCommand method.
PiperOrigin-RevId: 158847128
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Fixes https://github.com/bazelbuild/bazel/issues/1586
Fixes https://github.com/bazelbuild/bazel/issues/2326
This change also undoes https://github.com/bazelbuild/bazel/commit/956810b6ee24289e457a4b8d0a84ff56eb32c264 -- since
Bazel now closes its stdout/stderr before
cleaning, jvm.out is also closed so we don't need
to open it with deletion sharing.
RELNOTES: Windows: bazel clean --expunge works
Change-Id: I692f2e86a5983cb472a142a093611fd1c694cd3b
PiperOrigin-RevId: 158682987
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The experimental UI uses a thread to regularly update itself to reflect time-based
changes. As that that thread has to call synchronized methods any waiting for it to
finish has to happen outside any synchronized block. Unfortunately, 9e0308e0f7 accidentally
moved the stopUpdateThread() in buildComplete() into the synchronized block, thus giving
an opportunity for deadlocks. Move it out again.
Also, as the accounting for pending transports now happens in synchronized methods in
the state tracker, the buildEventTransportClosed() method does not have to be synchronized
any more---thus eliminating the second deadlock opportunity.
Change-Id: Icacb2ee70f4bedde1c1aac2bcfefc6fabf42fdd3
PiperOrigin-RevId: 158537844
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
It was originally included in runtime due to external dependencies, and
a desire to keep the options parser a general options library. These
dependencies have been or will be removed, and there are plenty of other
general flag libraries.
InvocationPolicy is fundamentally acting on the properties of this
specific OptionsParser and needs proper access to it for the proper
solution to a number of existing bugs, which means having access to
things that should be package private.
PiperOrigin-RevId: 158523111
|
|
|
|
|
|
|
|
| |
The base names are not necessarily unique enough to identify which
artifact is being talked about.
Change-Id: Ic8ff78c8f26f98e0e9a2d558d03f4cf9ae9111c8
PiperOrigin-RevId: 158499210
|
|
|
|
| |
PiperOrigin-RevId: 158473075
|
|
|
|
|
|
|
|
| |
Instead of passing a client env into the test strategies, use the same
mechansim as --action_env, by depending on the right set of Skyframe nodes that
correspond to client env entries.
PiperOrigin-RevId: 158401670
|
|
|
|
|
|
|
| |
This creates a new class BlazeWorkspaceTest, and removes the initCommand method
from BlazeRuntime.
PiperOrigin-RevId: 158396032
|
|
|
|
|
| |
RELNOTES: None
PiperOrigin-RevId: 158279811
|
|
|
|
|
|
|
|
|
|
|
| |
BlazeCommand.editOptions is currently called fairly late in the startup
process, so it must be restrictive in what it does, as any change to the
options can potentially introduce inconsistencies between different parts of
Bazel. Removing the CommandEnvironment reduces the amount of damage it can do,
and may allow us to move the call earlier in the startup process (maybe even
to a point where the CommandEnvironment does not exist yet).
PiperOrigin-RevId: 158133862
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Previously, it was in CoverageCommand, which led to all sorts of special casing
and workarounds, because it required options editing, error handling during
options editing, as well as early package path setup. It also caused us to
duplicate target pattern evaluation for the coverage command.
A semantic change is that the TestCommand can now also trigger the heuristic
instrumentation filter computation, if collect_code_coverage is enabled; we
might consider also enabling this for the BuildCommand, which can also have
collect_code_coverage enabled (e.g., when a user wants to build a non-test
binary with coverage instrumentation).
Also remove a bunch of unnecessary AbruptExitException declarations.
RELNOTES: bazel test now also computes a default instrumentation filter if --collect_code_coverage is enabled
PiperOrigin-RevId: 158129953
|
|
|
|
|
|
|
|
|
|
| |
The help message previously said that `--batch` is strongly discouraged, but in fact it's only discouraged when using bazel interactively. Since the tradeoffs of `--batch` are fully explained in the online docs, this commit changes the help message to explain what the option does without getting into the tradeoffs.
Fixes #3051
Closes #3118.
PiperOrigin-RevId: 158112611
|
|
|
|
|
|
|
|
|
|
|
| |
If console output limiting is enabled in the actual output stream
is wrapped in a CountingOutputStream to hard-limit the number of
bytes written. As on the console, the two streams, stdout and stderr,
might interleave, proper flushing of writes is important. Therefore,
make sure flushing is propagated through the CountingOutputStream.
Change-Id: I591a2a1ae798a9d8ef704118b22960ff9773a59e
PiperOrigin-RevId: 157707049
|
|
|
|
|
|
|
|
|
|
|
| |
When building with timestamps enabled, it is useful to know the date
as well, e.g., when later looking at logs. This, however, is not the
case if the command does not build (e.g., "bazel help", "bazel query"),
or, in general, if the first output is only produced after the command
is completed.
Change-Id: I75ef38fbb98e886b1dc38899efa10188055f87e2
PiperOrigin-RevId: 157700578
|
|
|
|
|
|
|
|
|
| |
Setting this flag to false makes Bazel exit with an error on undefined configs. The default value is true, so this change has no impact unless users explicitly pass --noallow_undefined_configs.
This change supersedes an unmerged one, I'm taking the problem from fkp@.
Credit to him for the original change.
PiperOrigin-RevId: 157593338
|
|
|
|
|
|
|
|
| |
Move AuthAndTLSOptions to its own package, so that tests/remote no longer
depends on lib:runtime.
RELNOTES: None.
PiperOrigin-RevId: 157469629
|
|
|
|
|
|
|
| |
Using boolean options with expansions on them meant they were still used if in the --no(flag) case.
RELNOTES: Clean command no longer uses boolean values for --async, --expunge, and --expunge_async options.
PiperOrigin-RevId: 157465859
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When a test failure occurred the BEP would still report a build_finished event
with exit code SUCCESS, even though bazel's exit code was TESTS_FAILED. This is
because we would rely on the exit code reported by the BuildCompleteEvent. However,
this exit code contains only the build status without taking into account the test status.
In this change we introduce the TestingCompleteEvent that reports bazel's final
exit code in case of "bazel test".
RELNOTES: None.
PiperOrigin-RevId: 157445808
|
|
|
|
|
|
|
|
| |
The BEP protocol currently does not include stdout/stderr when sent over BES.
This change makes the BuildEventStreamer created by the BuildEventServiceModule listen in on stdout/stderr.
RELNOTES: None.
PiperOrigin-RevId: 157439952
|
|
|
|
|
|
|
|
|
|
| |
As the state tracker keeps track of which transports we still have to
wait for, make the event handler just ask the state tracker. In this way,
we also handle gracefully if the closing of a transport is reported more
than once.
Change-Id: I0e1959d827268319ec00541994314c9325ef2307
PiperOrigin-RevId: 157395608
|
|
|
|
|
|
|
|
| |
We initially didn't consider that multiple BuildEventStreamer instances might
exist during a build.
RELNOTES: None.
PiperOrigin-RevId: 157385069
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
*** Reason for rollback ***
Rollforward with fix.
*** Original change description ***
Automated g4 rollback of commit c78c947e6a8cbb323304f872a3dcabb989a3d76b.
*** Reason for rollback ***
Breaks android targets in the nightly - see []
*** Original change description ***
Do not retain transitive data in AndroidLocalTestBase...
***
ROLLBACK_OF=156745610
RELNOTES: None
PiperOrigin-RevId: 157028029
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
*** Reason for rollback ***
Roll forward of directory name change
*** Original change description ***
Automated g4 rollback of commit 1d9e1ac90197b1d3d7b137ba3c1ada67bb9ba31b.
*** Reason for rollback ***
Breaks //src/test/shell/integration:force_delete_output_test
*** Original change description ***
Symlink output directories to the correct directory name
If the workspace directory is /path/to/my/proj and the name in the WORKSPACE
file is "floop", this will symlink the output directories to
output_base/execroot/floop instead of output_base/execroot/proj.
More prep for #1262, fixes #1681.
PiperOrigin-RevId: 156892980
|
|
|
|
|
|
|
|
|
|
|
|
| |
Add an option allowing to set a hard limit on the number of characters
bazel will write to stdout/stderr (combined). In this way, it can be
avoided to overwhelm the user with information (especially, if the
invocation of bazel is wrapped in some way). Once the limit is approaching,
bazel will try hard to meaningfully reduce the output, but will ultimately
resort to just dropping output completely.
Change-Id: I49cce96cc6a025c9753632dd489021766df81077
PiperOrigin-RevId: 156849105
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
*** Reason for rollback ***
Breaks android targets in the nightly - see []
*** Original change description ***
Do not retain transitive data in AndroidLocalTestBase.
The argument strings and artifacts were both transitive and flattened, causing O(N^2) memory consumption.
PiperOrigin-RevId: 156745610
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Update the command line flags used by remote execution/caching as well as the
build event service (BES).
Major changes:
- Remote execution/caching and BES share flags for authentication and TLS.
- Removed API Key authentication from BES, as it's not being used.
- Add TLS support to BES upload.
- Add --bes_project_id flag. If set, the value is propagated as part of BES
lifecycle events.
For reviewers:
Start your review at CommonRemoteAndBesOptions, BuildEventServiceOptions and
RemoteOptions. The other changes are mostly automatic IDE renames of fields and
flag updates in shell script tests.
RELNOTES: None.
PiperOrigin-RevId: 156553857
|
|
|
|
|
|
| |
as part of the startup arguments.
PiperOrigin-RevId: 156090009
|
|
|
|
|
|
| |
The argument strings and artifacts were both transitive and flattened, causing O(N^2) memory consumption.
PiperOrigin-RevId: 156083738
|
|
|
|
|
|
|
| |
Add a flag that determines the aspects use for MIv2.
RELNOTES:None.
PiperOrigin-RevId: 155899777
|
|
|
|
|
|
|
|
|
|
|
| |
While builds usually finish in a single day, for console logs (especially
those collected on a CI system) it is still useful to know the date they
where collected on. So, if --show_timestamps is given, show a line with the
current date at the beginning of the build to not lose the information due to
the shorter time stamps, compare #2989.
Change-Id: Ief2ae011cdb4de7ac5d5a9acd4d200913220b2a3
PiperOrigin-RevId: 155859302
|
|
|
|
|
|
|
|
|
|
|
| |
While sending chunks of stdout/stderr when a progress event is sent anyway
is a good idea, we cannot entirely rely on this, as the amount of information
buffered might grow too big. So set a fixed limit after which we flush out
stdout and stderr; nevertheless, we still make sure we send buffers that were
given to us in a single call to write in one go.
Change-Id: Ie27bcf7d50671e003babd13cdb1d3f7fc1cb232f
PiperOrigin-RevId: 155736641
|
|
|
|
|
|
|
|
| |
In multi-configuration builds, test can be run for different configurations as well. So
we have to report the configuration for each test as well.
Change-Id: I939cce7464823fbcd3c7161a50b41b94bbed8812
PiperOrigin-RevId: 155493397
|
|
|
|
|
|
| |
failures.
PiperOrigin-RevId: 155425839
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
*** Reason for rollback ***
Rollforward with fix
To have stdout/stderr in the BuildEventStream, the BuildEventStreamerModule
registers a listener that accumulates the values for the streamer to
collect it and regularly send it over the stream. As we have to also catch
stdout/stderr that happens early in the build, we have to register the listener
before(!) the options are parsed; therefore it is registered unconditionally.
Now, if there is no streamer created after parsing the options there is no
one to collect the data and it grows indefinitely. Fix this, by disabling
the collection in this case.
*** Original change description ***
Automated g4 rollback of commit 9e37b2e52d6e42eec15712942c7f208b64c651e5.
*** Reason for rollback ***
Results in NegativeArraySizeExceptions when there's a high volume of data.
*** Original change description ***
BEP: Report stdout/stderr
By recording registering a properly synchronized OutErr as listener
and providing it as OutErrProvider to the BuildEventStreamer.
Change-Id: Id553fcdb85327be28561634268511304fcc2ad3f
PiperOrigin-RevId: 155374710
|
|
|
|
|
|
|
|
|
| |
This is possibly a nit, but we don't want to reuse flag names in the future, so it's a good idea to include what the flag *does* in the name as opposed to just the feature it affects, in case the same feature is changed multiple times. Updated javadoc for incompatible change system to say so.
This rename is ok because these flags have only been submitted over the past couple days.
RELNOTES: None
PiperOrigin-RevId: 155371363
|
|
|
|
|
|
|
|
|
|
| |
In preparation to support multi-configuration builds, provide infrastructure
allowing build events to reference BuildConfigurations. The streaming mechanism
will ensure that build configurations are introduced in the stream before being
referenced for the first time.
Change-Id: I6b96fbebc76a05eff4f75a07e8a9cfbcd57f9c22
PiperOrigin-RevId: 155368666
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
*** Reason for rollback ***
Results in NegativeArraySizeExceptions when there's a high volume of data.
*** Original change description ***
BEP: Report stdout/stderr
By recording registering a properly synchronized OutErr as listener
and providing it as OutErrProvider to the BuildEventStreamer.
Change-Id: Id553fcdb85327be28561634268511304fcc2ad3f
PiperOrigin-RevId: 155252872
|
|
|
|
|
|
|
|
|
|
| |
Move the position of the timestamp to the beginning of the line to
have a more readable log. Also, show the timestamp for progress as
well. While there, reduce timestamp to second precision, to reduce
noise.
Change-Id: Ibfa6caca2e0d207f54e3660bccbf894bba3c5ae3
PiperOrigin-RevId: 155181731
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
*** Reason for rollback ***
Filter out conflicting flag policies for applicable commands. Only enforce the filtered, applicable policy.
*** Original change description ***
Automated g4 rollback of commit aa7f9307636d38cbb93a03acac8f4c59adfa0ee8.
*** Reason for rollback ***
Broke --experimental_inmemory_dotd_files
*** Original change description ***
Filter out conflicting flag policies before invocation policy enforcement.
This is to minimize the likelihood of obscure policy conflict. Now, the last policy on a flag (after policy expansion) will be the only one in the "canonical" invocation policy. There should be no reason for explicitly setting multiple policies on a single flag, but if an expansion flag is policy'd and one of its children has a more specific policy on it, make sure that the policy on the child flag is after the policy on the expansion flag.
Note that this restriction (only the last policy gets applied) also applies for repeatable flags. Make sure all values being set to a repeatable flag are set in a single SetValue operation, with multiple flagValues set.
PiperOrigin-RevId: 154969189
|
|
|
|
|
|
|
|
| |
By recording registering a properly synchronized OutErr as listener
and providing it as OutErrProvider to the BuildEventStreamer.
Change-Id: Id553fcdb85327be28561634268511304fcc2ad3f
PiperOrigin-RevId: 154943162
|
|
|
|
|
|
|
|
| |
Add an event reporting the workspace status as key-value pairs reported
via the workspace_status_command.
Change-Id: I5791551798a594bc2465f483eb97f9d4fd4c7cfd
PiperOrigin-RevId: 154845224
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Extend the build-event streamer to also report about stdout/stderr,
if provided. This information is reported in the progress events. At
the moment, we only report stdout/stderr in progress events we send
anyway, but the interface is generic enough that we could add time-based
reporting later, if needed. Also note, that at the end of the build, we
report the final progress event, so that all stdout/stderr generated before
the build-complete event get also reported in the build-event protocol.
Change-Id: If5dbd59c151edbce02d0a9b2e5938b63c0a5dc58
PiperOrigin-RevId: 154811110
|
|
|
|
|
|
|
|
|
|
| |
Besides writing console output to the console and the command log, allow
modules to register other places where the output should be recorded to.
This will allow the build-event protocol to also report on the console
output.
Change-Id: Ie700243120b0db7c3c68d192abeb0ab7033dc175
PiperOrigin-RevId: 154528369
|
|
|
|
|
|
|
|
|
| |
This is the first of two CLs for making command line options able to affect the Skylark interpreter. It introduces SkylarkSemanticsOptions, and stores it as a precomputed (injected) value in Skyframe. The next CL will read these options from Skyframe when constructing the Skylark environment.
This CL affects the dataflow from command/test initialization to Skyframe. Some code paths, like those used for testing, use the default SkylarkSemanticsOptions and therefore won't be able to use (for example) --incompatible_* flags. The call sites to update were found by searching for uses of defaultVisibility and working upward from there.
RELNOTES: None
PiperOrigin-RevId: 154432058
|
|
|
|
|
|
|
|
|
| |
Add an order constraint the the command-line related events: the commandline
should always be reported before the event reporting the completion of the
option parsing.
Change-Id: Ief1b822e2a8ce2f7d8f63c3b6182832d2bd102ef
PiperOrigin-RevId: 154397674
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When unregistering an event handler, that uses colors, we need to be sure
to set the terminal in a normal state after unregistering that event handler.
The ExperimentalEventHandler does use curses, however only if options allow
to do so (otherwise, curses will be filtered out by the underlying terminal).
So, we cannot conclude the use of color from the fact that the
ExperimentalEventHandler was in use. Hence when releasing it, only reset the
the terminal, if we did use color and avoid the unnecessary ^[[0m; otherwise.
RELNOTES: None.
PiperOrigin-RevId: 154177220
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
*** Reason for rollback ***
Rollforward with fix for test flakiness. BEP transport closed
events are delivered via their own threadpool and thus might
not have been sent immediately. BuildEventStreamerTest#testSimpleStream
now waits for a bit until the event has been delivered. I ran the test
with --runs_per_test=1000 several times and had no further failures.
*** Original change description ***
Automated g4 rollback of commit 3d596d63f883fff56001ed7b2e5cf51dba45f082.
*** Reason for rollback ***
Made BuildEventStreamerTest#testSimpleStream 3% flaky based on --runs_per_test=1000.
RELNOTES: None
PiperOrigin-RevId: 154170833
|