| Commit message (Collapse) | Author | Age |
|
|
|
|
|
|
| |
It is possible to create duplicate artifacts with different owners, so we have to tolerate this when uploading files.
RELNOTES: None
PiperOrigin-RevId: 207302014
|
|
|
|
|
|
| |
Similar to the sequence number, the timestamp should be immutable and computed when the event is enqueued (not at serialization time).
PiperOrigin-RevId: 206186036
|
|
|
|
|
|
|
|
| |
don't retry precondition_failed and
invalid_argument status codes.
RELNOTES: None
PiperOrigin-RevId: 205566423
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- refactor the BuildEventServiceClient interface to
report errors via StatusException and InterruptException.
- do the groundwork necessary to do retries based on
rpc status codes.
- improve the execution speed of the
BuildEventServiceStubbyClientTest from 1m5s to 5s.
RELNOTES: None
PiperOrigin-RevId: 205563431
|
|
|
|
|
|
|
|
|
|
|
| |
The synchronized block in sendBuildEvent is
not needed at all, since all the logic in
there is local to the method and thread safe.
See the issue for more details.
RELNOTES: None
PiperOrigin-RevId: 204915541
|
|
|
|
|
|
|
|
|
|
|
|
| |
If a local file upload fails, then the build event
gets lost. This was the case because of a bug in the
implementation where local file upload errors are
retried by the BES retrier. This is the root cause
for the "Non-consecutive sequence number" errors we
have been seeing recently.
RELNOTES: None
PiperOrigin-RevId: 204906550
|
|
|
|
| |
PiperOrigin-RevId: 204882046
|
|
|
|
|
|
|
| |
The artifact uploaders may need command-level options.
RELNOTES: None
PiperOrigin-RevId: 204151808
|
|
|
|
|
|
| |
Fixes #5439.
PiperOrigin-RevId: 203917717
|
|
|
|
|
|
|
|
|
|
| |
Sources:
https://github.com/bazelbuild/bazel-blog/commit/846478d6943162f4c4d7d50001069e0ca7b2ec28
https://github.com/bazelbuild/bazel-blog/commit/aacaa25314678c08772372b3d46697f7963bb201
RELNOTES: None.
PiperOrigin-RevId: 203763253
|
| |
|
|
|
|
| |
This reverts commit a2cac548616e6e6f433df27146c2971f352a4041.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
There can be multiple BuildEventTransports active at
the same time and we need to ensure that each transport
gets its own BuildEventArtifactUploader as these transports
might have different lifecycles.
We do that by introducing another level of indirection via
the BuildEventArtifactUploaderFactory. BlazeModules now
register a factory object instead of an uploader.
In addition, the BuildEventArtifactUploader gets a shutdown()
method that allows to free any resources associated with it.
PiperOrigin-RevId: 203752092
|
|
|
|
|
|
|
| |
Instead of just a path, events now include information about the type of file (output, source file, stdout/stderr, test logs, etc.). This information can be used by the uploaders to determine a) whether to upload, b) what kind of lease to give the files.
RELNOTES: None
PiperOrigin-RevId: 203285549
|
|
|
|
|
|
|
|
|
|
| |
This changes the BuildEventArtifactUploader to an async interface,
thereby no longer potentially delaying event delivery over the
eventbus. Additionally, the BES transport is changed to start
uploading local files immediately as the events are delivered.
RELNOTES: None
PiperOrigin-RevId: 202694121
|
|
|
|
|
|
|
|
| |
This should be a no-op, mostly replacing PathConverter with
BuildEventArtifactUploader, since none of the implementations perform any
upload yet.
PiperOrigin-RevId: 200685325
|
|
|
|
|
|
|
|
|
| |
- use an internal event wrapper to unify handling
- rewrite the test to be more explicit about event names and ordering
This a part split out of unknown commit.
PiperOrigin-RevId: 200015904
|
|
|
|
|
|
| |
Event Protocol is being streamed.
PiperOrigin-RevId: 199790757
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The current semantics of the flag is to allow BES upload to continue
past the nominal end of the build invocation, and possibly overlapping
a subsequent build invocation. This conflicts with file uploads, which
must read the file before it is removed or modified by the subsequent
build invocation. On Linux, we could just open a file handle, but this
isn't possible on Windows.
We decided to make the flag a no-op for now. Note that the default is
already set to false. We may resurrect this option in the future if
there's a strong use case for it, possibly with slightly different
semantics.
PiperOrigin-RevId: 199620392
|
|
|
|
|
|
|
| |
The important_outputs field is deprecated, and this adds a flag to disable
its generation entirely.
PiperOrigin-RevId: 197186530
|
|
|
|
|
|
|
|
|
|
|
|
| |
Disabling best effort upload means that a build will
wait for the BES upload to finish and also that it
will fail if the BES upload fails.
We are planning on removing/deprecating the best effort
upload functionality eventually.
RELNOTES: None
PiperOrigin-RevId: 197148222
|
|
|
|
|
|
|
| |
This is in preparation for adding options to the class, which can be used
to modify the behavior of build events posted to the BEP.
PiperOrigin-RevId: 196997573
|
|
|
|
|
|
|
|
|
| |
This constructor was creating an Exception with a null message leading to possible
NullPointerExceptions in a few places in our codebase. The call sites have
been replaced with calls to AbruptException(String message, ExitCode exitCode) with
a meaningful message.
PiperOrigin-RevId: 196973540
|
|
|
|
|
|
|
| |
Nobody reads the documentation, apparently. It also clearly states that
getCommonCommandOptions should be used to add options to all commands.
PiperOrigin-RevId: 196965665
|
|
|
|
|
|
|
| |
channel used to connect with BES.
RELNOTES: none.
PiperOrigin-RevId: 196845127
|
|
|
|
|
|
|
| |
Closes #4628
Closes #5155.
PiperOrigin-RevId: 195662967
|
| |
|
|
|
|
|
|
|
| |
published via the BEP, instead of only publishing failed actions and extra actions.
RELNOTES: Add a --build_event_publish_all_actions flag to allow all actions to be published via the BEP.
PiperOrigin-RevId: 187683799
|
|
|
|
|
|
|
| |
Logs whenever BuildEvenServiceTransport starts a PublishBuildToolEventStream
streaming RPC.
PiperOrigin-RevId: 187374228
|
|
|
|
|
|
|
|
|
| |
Change-Id: Ib8dd9265b18fa0915f52427567845105fcdfa295
Closes #4447.
Change-Id: Ib8dd9265b18fa0915f52427567845105fcdfa295
PiperOrigin-RevId: 181943004
|
|
|
|
|
|
|
|
|
|
| |
For different applications, different size of buffered stdout/stderr might be
acceptable; essentially it is a trade off between latency and number of messages
generated. Put this trade off into the control of the user by adding an appropriate
flag.
Change-Id: I8fb4d19a336205fa28d01340f2f0b2be9b4a24f3
PiperOrigin-RevId: 181570242
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Add support for Google Cloud Storage (GCS) as a HTTP caching backend.
This commit mainly adds the infrastructure necessary to authenticate
to GCS.
Using GCS as a caching backend works as follows:
1) Create a new GCS bucket.
2) Create a service account that can read and write to the GCS bucket
and generate a JSON credentials file for it.
3) Invoke Bazel as follows:
bazel build
--remote_rest_cache=https://storage.googleapis.com/<bucket>
--auth_enabled
--auth_scope=https://www.googleapis.com/auth/devstorage.read_write
--auth_credentials=</path/to/creds.json>
I'll add simplification's and docs in a subsequent commit.
Change-Id: Ie827d7946a2193b97ea7d9aa72eb15f09de2164d
PiperOrigin-RevId: 179406380
|
|
|
|
|
|
|
|
| |
Blaze had its own class to avoid GC from varargs array creation for the precondition happy path. Guava now (mostly) implements these, making it unnecessary to maintain our own.
This change was almost entirely automated by search-and-replace. A few BUILD files needed fixing up since I removed an export of preconditions from lib:util, which was all done by add_deps. There was one incorrect usage of Preconditions that was caught by error prone (which checks Guava's version of Preconditions) that I had to change manually.
PiperOrigin-RevId: 175033526
|
|
|
|
|
|
|
|
|
|
| |
BES.
- Add a prefix to user provided keywords, so it can be distinguished from
keywords provided directly by Bazel.
- Keywords are also stored in a Set to avoid duplicates.
PiperOrigin-RevId: 174872442
|
|
|
|
|
|
|
|
|
|
| |
active, restarting the RPC if required.
- BuildEventServiceTransport sender thread checks the RPC is still active every 1s, failing the RPC if it was ended before we reached the end of the BEP stream.
- "fail_fast" is the default behavior on gRPC (it needs to be disabled using .withWaitForReady).
PiperOrigin-RevId: 173921837
|
|
|
|
|
|
|
|
|
|
| |
The flakiness was caused by waiting for the server to receive the second
life-cycle event, but not waiting for the ack to get to the client. In that
case, the interruption would happen before the life-cycle call returned, in
which case it threw the InterruptedException, and then the stream doesn't
exist yet, and so there's no mechanism to send anything to the server.
PiperOrigin-RevId: 170472383
|
|
|
|
| |
PiperOrigin-RevId: 170121049
|
|
|
|
|
|
|
|
|
|
|
| |
- The upload of build events is retried infinitely often as long as
between two failures progress was made, that is ACKs were received.
- Set the initial backoff after a failure to 0, and every further
backoff to 1s. This aligns relatively well with the backoff times
gRPC connections are retried [1].
PiperOrigin-RevId: 170022707
|
|
|
|
|
|
| |
TESTED=unit tests
RELNOTES: none
PiperOrigin-RevId: 169395919
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Introduce a new method BuildEventTransport#closeNow() which forcefully
closes the BES upload even if there are still events to upload. This
is executed if the bazel server is being shutdown.
The implementation in the BuildEventServiceTransport works by shutting
down the ExecutorService handling the upload, which in turn will
interrupt the upload thread, which in turn will cancel the streaming RPC.
The BuildEventServiceTransport will wait up to 100ms for that
cancellation to happen.
Also fix a bug where the BES transport would not wait for ACKS.
PiperOrigin-RevId: 168359947
|
|
|
|
|
|
|
|
|
| |
Split collect, concurrent, vfs, windows into package-level BUILD files.
Move clock classes out of "util", into their own Java package.
Move CompactHashSet into its own Java package to break a dependency cycle.
Give nestedset and inmemoryfs their own package-level BUILD files.
PiperOrigin-RevId: 167702127
|
|
|
|
|
|
|
| |
This removes our dependency on third_party/joda_time, which can be removed in the next commit.
Change-Id: Ibda131d34d0abdc2d675db4bfbd2e99480c055ee
PiperOrigin-RevId: 167515260
|
|
|
|
|
|
|
|
| |
BUILD files.
Replace all ":relative" labels with "//absolute:path" labels for easier search & replace.
PiperOrigin-RevId: 167500985
|
|
|
|
|
|
|
| |
This change handling for when BuildEventServiceTransport would timesout waiting for pendingAcks.
RELNOTES: None.
PiperOrigin-RevId: 166375844
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Notification keywords published BES include:command_name and protocol_name.
These keywords can be used by subscribers to filter BES streams of interest.
This change also cleans up use of deprecated fields in in
PublishBuildToolEventStreamRequest.
RELEASE_NOTES: None.
Change-Id: I331fc9a818728ad6986230ebdea7d4019e8e49fc
PiperOrigin-RevId: 164285745
|
|
|
|
|
|
|
|
|
|
|
| |
BES and Remote Execution have separate implementations of gRPC channel
creation, authentication and TLS. We should merge them, to avoid
duplication and bugs. One such bug is #3640, where the BES code had a
different implementation for Google Application Default Credentials.
RELNOTES: The Build Event Service (BES) client now properly supports
Google Applicaton Default Credentials.
PiperOrigin-RevId: 164253879
|
|
|
|
|
|
|
| |
keywords.
Change-Id: I8421e9d2d8a5e6720d7b8d6de4417bee71c0fa68
PiperOrigin-RevId: 162966141
|
|
|
|
|
|
|
|
| |
Incorrect instruction for OSX (apt-get instead of brew)
Closes #3437.
PiperOrigin-RevId: 162923095
|
|
|
|
|
|
| |
Change the return type for RPC PublishBuildToolEventStream from OrderedBuildEvent to PublishBuildToolEventStreamRequest.
Change-Id: I8421e9d2d8a5e6720d7b8d6de4417bee71c0fa68
|