aboutsummaryrefslogtreecommitdiffhomepage
path: root/src/main/java/com/google/devtools/build/lib/buildeventservice
Commit message (Collapse)AuthorAge
* Make BES tolerant to duplicate artifacts in output groups.Gravatar tomlu2018-08-03
| | | | | | | It is possible to create duplicate artifacts with different owners, so we have to tolerate this when uploading files. RELNOTES: None PiperOrigin-RevId: 207302014
* Make timestamps of InternalOrderedBuildEvent immutable.Gravatar Googler2018-07-26
| | | | | | Similar to the sequence number, the timestamp should be immutable and computed when the event is enqueued (not at serialization time). PiperOrigin-RevId: 206186036
* bes: don't retry all status codesGravatar buchgr2018-07-22
| | | | | | | | don't retry precondition_failed and invalid_argument status codes. RELNOTES: None PiperOrigin-RevId: 205566423
* bes: make error handling saneGravatar buchgr2018-07-22
| | | | | | | | | | | | | | - refactor the BuildEventServiceClient interface to report errors via StatusException and InterruptException. - do the groundwork necessary to do retries based on rpc status codes. - improve the execution speed of the BuildEventServiceStubbyClientTest from 1m5s to 5s. RELNOTES: None PiperOrigin-RevId: 205563431
* fix deadlock in the bes upload. Fixes #5402Gravatar buchgr2018-07-17
| | | | | | | | | | | The synchronized block in sendBuildEvent is not needed at all, since all the logic in there is local to the method and thread safe. See the issue for more details. RELNOTES: None PiperOrigin-RevId: 204915541
* bes: fix bug where the bes upload would lose eventsGravatar buchgr2018-07-17
| | | | | | | | | | | | If a local file upload fails, then the build event gets lost. This was the case because of a bug in the implementation where local file upload errors are retried by the BES retrier. This is the root cause for the "Non-consecutive sequence number" errors we have been seeing recently. RELNOTES: None PiperOrigin-RevId: 204906550
* Improve error reporting for BuildEventService{Module,Transport}Gravatar lpino2018-07-17
| | | | PiperOrigin-RevId: 204882046
* Pass options to BuildEventArtifactUploaderFactory.Gravatar tomlu2018-07-11
| | | | | | | The artifact uploaders may need command-level options. RELNOTES: None PiperOrigin-RevId: 204151808
* Update third_party/protobuf to 3.6.0Gravatar Loo Rong Jie2018-07-10
| | | | | | Fixes #5439. PiperOrigin-RevId: 203917717
* Sync jmmv@'s table styles for the docs.bazel.build site.Gravatar jingwen2018-07-09
| | | | | | | | | | Sources: https://github.com/bazelbuild/bazel-blog/commit/846478d6943162f4c4d7d50001069e0ca7b2ec28 https://github.com/bazelbuild/bazel-blog/commit/aacaa25314678c08772372b3d46697f7963bb201 RELNOTES: None. PiperOrigin-RevId: 203763253
* Update protobuf to 3.6.0. Fixes #5439Gravatar Loo Rong Jie2018-07-09
|
* Revert "Update protobuf to 3.6.0. Fixes #5439"Gravatar Jakob Buchgraber2018-07-09
| | | | This reverts commit a2cac548616e6e6f433df27146c2971f352a4041.
* Update protobuf to 3.6.0. Fixes #5439Gravatar Loo Rong Jie2018-07-09
|
* bep: introduce BuildEventArtifactUploaderFactoryGravatar buchgr2018-07-09
| | | | | | | | | | | | | | | | There can be multiple BuildEventTransports active at the same time and we need to ensure that each transport gets its own BuildEventArtifactUploader as these transports might have different lifecycles. We do that by introducing another level of indirection via the BuildEventArtifactUploaderFactory. BlazeModules now register a factory object instead of an uploader. In addition, the BuildEventArtifactUploader gets a shutdown() method that allows to free any resources associated with it. PiperOrigin-RevId: 203752092
* Include more information about build events' referenced local files.Gravatar tomlu2018-07-04
| | | | | | | Instead of just a path, events now include information about the type of file (output, source file, stdout/stderr, test logs, etc.). This information can be used by the uploaders to determine a) whether to upload, b) what kind of lease to give the files. RELNOTES: None PiperOrigin-RevId: 203285549
* bep: Make the BuildEventArtifactUploader async.Gravatar buchgr2018-06-29
| | | | | | | | | | This changes the BuildEventArtifactUploader to an async interface, thereby no longer potentially delaying event delivery over the eventbus. Additionally, the BES transport is changed to start uploading local files immediately as the events are delivered. RELNOTES: None PiperOrigin-RevId: 202694121
* Add a mechanism for build event protocol events to upload filesGravatar ulfjack2018-06-15
| | | | | | | | This should be a no-op, mostly replacing PathConverter with BuildEventArtifactUploader, since none of the implementations perform any upload yet. PiperOrigin-RevId: 200685325
* Refactor the build event service transportGravatar ulfjack2018-06-11
| | | | | | | | | - use an internal event wrapper to unify handling - rewrite the test to be more explicit about event names and ordering This a part split out of unknown commit. PiperOrigin-RevId: 200015904
* Print an additional link at the end of the build pointing where the Build ↵Gravatar lpino2018-06-08
| | | | | | Event Protocol is being streamed. PiperOrigin-RevId: 199790757
* Make bes_best_effort a no-opGravatar ulfjack2018-06-07
| | | | | | | | | | | | | | | | The current semantics of the flag is to allow BES upload to continue past the nominal end of the build invocation, and possibly overlapping a subsequent build invocation. This conflicts with file uploads, which must read the file before it is removed or modified by the subsequent build invocation. On Linux, we could just open a file handle, but this isn't possible on Windows. We decided to make the flag a no-op for now. Note that the default is already set to false. We may resurrect this option in the future if there's a strong use case for it, possibly with slightly different semantics. PiperOrigin-RevId: 199620392
* Add BuildEventProtocolOptions; use to enable/disable important_outputsGravatar ulfjack2018-05-18
| | | | | | | The important_outputs field is deprecated, and this adds a flag to disable its generation entirely. PiperOrigin-RevId: 197186530
* bes: disable best effort upload by default.Gravatar buchgr2018-05-18
| | | | | | | | | | | | Disabling best effort upload means that a build will wait for the BES upload to finish and also that it will fail if the BES upload fails. We are planning on removing/deprecating the best effort upload functionality eventually. RELNOTES: None PiperOrigin-RevId: 197148222
* Rename BuildEventConverters to BuildEventContextGravatar ulfjack2018-05-17
| | | | | | | This is in preparation for adding options to the class, which can be used to modify the behavior of build events posted to the BEP. PiperOrigin-RevId: 196997573
* Remove AbruptException(ExitCode exitCode) constructor.Gravatar lpino2018-05-17
| | | | | | | | | This constructor was creating an Exception with a null message leading to possible NullPointerExceptions in a few places in our codebase. The call sites have been replaced with calls to AbruptException(String message, ExitCode exitCode) with a meaningful message. PiperOrigin-RevId: 196973540
* Fix modules returning command optionsGravatar ulfjack2018-05-17
| | | | | | | Nobody reads the documentation, apparently. It also clearly states that getCommonCommandOptions should be used to add options to all commands. PiperOrigin-RevId: 196965665
* Create {Un}ManagedBuildEventServiceGrpcClient to allow customization of the ↵Gravatar lpino2018-05-16
| | | | | | | channel used to connect with BES. RELNOTES: none. PiperOrigin-RevId: 196845127
* bes: add --bes_results_urlGravatar Jakob Buchgraber2018-05-07
| | | | | | | Closes #4628 Closes #5155. PiperOrigin-RevId: 195662967
* PiperOrigin-RevId: 188862123Gravatar lpino2018-03-13
|
* BEP: Add a --build_event_publish_all_actions flag to allow all actions to be ↵Gravatar ruperts2018-03-02
| | | | | | | published via the BEP, instead of only publishing failed actions and extra actions. RELNOTES: Add a --build_event_publish_all_actions flag to allow all actions to be published via the BEP. PiperOrigin-RevId: 187683799
* Add logging message when a PublishBuildToolEventStream() RPCs start.Gravatar Googler2018-02-28
| | | | | | | Logs whenever BuildEvenServiceTransport starts a PublishBuildToolEventStream streaming RPC. PiperOrigin-RevId: 187374228
* Remove some unnecessary depsGravatar Ulf Adams2018-01-15
| | | | | | | | | Change-Id: Ib8dd9265b18fa0915f52427567845105fcdfa295 Closes #4447. Change-Id: Ib8dd9265b18fa0915f52427567845105fcdfa295 PiperOrigin-RevId: 181943004
* BEP: make stdout/stderr buffering configurableGravatar Klaus Aehlig2018-01-11
| | | | | | | | | | For different applications, different size of buffered stdout/stderr might be acceptable; essentially it is a trade off between latency and number of messages generated. Put this trade off into the control of the user by adding an appropriate flag. Change-Id: I8fb4d19a336205fa28d01340f2f0b2be9b4a24f3 PiperOrigin-RevId: 181570242
* remote: Add support for Google Cloud Storage.Gravatar Jakob Buchgraber2017-12-18
| | | | | | | | | | | | | | | | | | | | | | Add support for Google Cloud Storage (GCS) as a HTTP caching backend. This commit mainly adds the infrastructure necessary to authenticate to GCS. Using GCS as a caching backend works as follows: 1) Create a new GCS bucket. 2) Create a service account that can read and write to the GCS bucket and generate a JSON credentials file for it. 3) Invoke Bazel as follows: bazel build --remote_rest_cache=https://storage.googleapis.com/<bucket> --auth_enabled --auth_scope=https://www.googleapis.com/auth/devstorage.read_write --auth_credentials=</path/to/creds.json> I'll add simplification's and docs in a subsequent commit. Change-Id: Ie827d7946a2193b97ea7d9aa72eb15f09de2164d PiperOrigin-RevId: 179406380
* Replace all usages of Blaze's Preconditions class with guava.Gravatar tomlu2017-11-09
| | | | | | | | Blaze had its own class to avoid GC from varargs array creation for the precondition happy path. Guava now (mostly) implements these, making it unnecessary to maintain our own. This change was almost entirely automated by search-and-replace. A few BUILD files needed fixing up since I removed an export of preconditions from lib:util, which was all done by add_deps. There was one incorrect usage of Preconditions that was caught by error prone (which checks Guava's version of Preconditions) that I had to change manually. PiperOrigin-RevId: 175033526
* Prefix all --bes_keywords with "user_keyword=<keyword>" before publishing to ↵Gravatar Googler2017-11-07
| | | | | | | | | | BES. - Add a prefix to user provided keywords, so it can be distinguished from keywords provided directly by Bazel. - Keywords are also stored in a Set to avoid duplicates. PiperOrigin-RevId: 174872442
* Make BuildEventServiceTransport sender thread verify the streaming RPC is ↵Gravatar Googler2017-10-31
| | | | | | | | | | active, restarting the RPC if required. - BuildEventServiceTransport sender thread checks the RPC is still active every 1s, failing the RPC if it was ended before we reached the end of the BEP stream. - "fail_fast" is the default behavior on gRPC (it needs to be disabled using .withWaitForReady). PiperOrigin-RevId: 173921837
* Fix flakiness of BuildEventServiceTestsGravatar ulfjack2017-09-29
| | | | | | | | | | The flakiness was caused by waiting for the server to receive the second life-cycle event, but not waiting for the ack to get to the client. In that case, the interruption would happen before the life-cycle call returned, in which case it threw the InterruptedException, and then the stream doesn't exist yet, and so there's no mechanism to send anything to the server. PiperOrigin-RevId: 170472383
* Adds option bes_keywords BEP publishing.Gravatar Googler2017-09-27
| | | | PiperOrigin-RevId: 170121049
* Improve BES retries.Gravatar buchgr2017-09-26
| | | | | | | | | | | - The upload of build events is retried infinitely often as long as between two failures progress was made, that is ACKs were received. - Set the initial backoff after a failure to 0, and every further backoff to 1s. This aligns relatively well with the backoff times gRPC connections are retried [1]. PiperOrigin-RevId: 170022707
* Passing Bazel metadata in gRPC headers.Gravatar olaola2017-09-21
| | | | | | TESTED=unit tests RELNOTES: none PiperOrigin-RevId: 169395919
* bes: cancel stream on bazel shutdownGravatar buchgr2017-09-12
| | | | | | | | | | | | | | | | | Introduce a new method BuildEventTransport#closeNow() which forcefully closes the BES upload even if there are still events to upload. This is executed if the bazel server is being shutdown. The implementation in the BuildEventServiceTransport works by shutting down the ExecutorService handling the upload, which in turn will interrupt the upload thread, which in turn will cancel the streaming RPC. The BuildEventServiceTransport will wait up to 100ms for that cancellation to happen. Also fix a bug where the BES transport would not wait for ACKS. PiperOrigin-RevId: 168359947
* More BUILD file refactorings.Gravatar philwo2017-09-06
| | | | | | | | | Split collect, concurrent, vfs, windows into package-level BUILD files. Move clock classes out of "util", into their own Java package. Move CompactHashSet into its own Java package to break a dependency cycle. Give nestedset and inmemoryfs their own package-level BUILD files. PiperOrigin-RevId: 167702127
* Rewrite all code to use the new Java 8 java.time classes.Gravatar Philipp Wollermann2017-09-05
| | | | | | | This removes our dependency on third_party/joda_time, which can be removed in the next commit. Change-Id: Ibda131d34d0abdc2d675db4bfbd2e99480c055ee PiperOrigin-RevId: 167515260
* Extract authandtls, buildeventservice, buildeventstream into package-level ↵Gravatar philwo2017-09-04
| | | | | | | | BUILD files. Replace all ":relative" labels with "//absolute:path" labels for easier search & replace. PiperOrigin-RevId: 167500985
* Abort stream if BuildEventServiceTransport does not wait for all ACKs.Gravatar Googler2017-08-25
| | | | | | | This change handling for when BuildEventServiceTransport would timesout waiting for pendingAcks. RELNOTES: None. PiperOrigin-RevId: 166375844
* Add keywords to BES PublishBuildToolEventStreamRequest.Gravatar Googler2017-08-07
| | | | | | | | | | | | | Notification keywords published BES include:command_name and protocol_name. These keywords can be used by subscribers to filter BES streams of interest. This change also cleans up use of deprecated fields in in PublishBuildToolEventStreamRequest. RELEASE_NOTES: None. Change-Id: I331fc9a818728ad6986230ebdea7d4019e8e49fc PiperOrigin-RevId: 164285745
* grpc: Consolidate gRPC code from BES and Remote Execution. Fixes #3460, #3486Gravatar buchgr2017-08-04
| | | | | | | | | | | BES and Remote Execution have separate implementations of gRPC channel creation, authentication and TLS. We should merge them, to avoid duplication and bugs. One such bug is #3640, where the BES code had a different implementation for Google Application Default Credentials. RELNOTES: The Build Event Service (BES) client now properly supports Google Applicaton Default Credentials. PiperOrigin-RevId: 164253879
* The first step to allow non-lifecycle build stream to specify notification ↵Gravatar Kai Xu2017-07-24
| | | | | | | keywords. Change-Id: I8421e9d2d8a5e6720d7b8d6de4417bee71c0fa68 PiperOrigin-RevId: 162966141
* Update install-os-x.mdGravatar Oleg Tsarev2017-07-24
| | | | | | | | Incorrect instruction for OSX (apt-get instead of brew) Closes #3437. PiperOrigin-RevId: 162923095
* bes: Update RPC PublishBuildToolEventStream.Gravatar Kai Xu2017-07-24
| | | | | | Change the return type for RPC PublishBuildToolEventStream from OrderedBuildEvent to PublishBuildToolEventStreamRequest. Change-Id: I8421e9d2d8a5e6720d7b8d6de4417bee71c0fa68
generated by cgit v1.2.3 (git 2.39.1) at 2024-06-20 07:38:14 +0000