aboutsummaryrefslogtreecommitdiffhomepage
path: root/src/main/java/com/google/devtools/build/lib/buildeventstream
Commit message (Collapse)AuthorAge
* Do not crash when attempting to upload a tree artifact.Gravatar tomlu2018-08-10
| | | | | | | These show up as directories. Filter these out and return null from the path converter, which should cause omission of those files from any build events. RELNOTES: None PiperOrigin-RevId: 208244910
* Make BES tolerant to duplicate artifacts in output groups.Gravatar tomlu2018-08-03
| | | | | | | It is possible to create duplicate artifacts with different owners, so we have to tolerate this when uploading files. RELNOTES: None PiperOrigin-RevId: 207302014
* Fix TargetCompleteEvent.referencedLocalFilesGravatar ulfjack2018-07-23
| | | | | | | | It was missing the baseline coverage files, if any. This is safe even if unknown commit is rolled back. PiperOrigin-RevId: 205626149
* Ensure BEP file output stream is flushed promptly.Gravatar Googler2018-07-16
| | | | | | | | | | | | | | | | | | Tulsi uses BEP json output in its UI, to simulate Bazel terminal outptut. This means we have to promptly flush the stream. It's sufficient to do flushing at the granularity of whole events, not any specific count of bytes, since whole events are what's being consumed by the user. To balance IO throughput and interactivity, let's flush at a regular sub-second interval. (The alternative solution of using a stream with smaller buffer could still end up with small-sized event descriptions buffered arbitrarily long; and abandoning buffering altogether would be suboptimal for throughput when writing a long sequence of small build events.) RELNOTES: None. PiperOrigin-RevId: 204790794
* remote: add a ByteStreamBuildEventArtifactUploaderGravatar buchgr2018-07-12
| | | | | | | | | | | | | | | This change allows local files referenced by the BEP/BES protocol to be uploaded to a ByteStream gRPC service. The ByteStreamUploader is now implicitly also used by the BES module which has a different lifecycle than the remote module. We introduce reference counting to ensure that the channel is closed after its no longer needed. This also fixes a bug where we currently leak one socket per remote build until the Bazel server is shut down. RELNOTES: None PiperOrigin-RevId: 204275316
* Omit parts of BEP events (like stdout, stderr) when the associated artifact ↵Gravatar tomlu2018-07-11
| | | | | | | uploader cannot upload a particular file. RELNOTES: None PiperOrigin-RevId: 204167372
* Pass options to BuildEventArtifactUploaderFactory.Gravatar tomlu2018-07-11
| | | | | | | The artifact uploaders may need command-level options. RELNOTES: None PiperOrigin-RevId: 204151808
* bep: introduce BuildEventArtifactUploaderFactoryGravatar buchgr2018-07-09
| | | | | | | | | | | | | | | | There can be multiple BuildEventTransports active at the same time and we need to ensure that each transport gets its own BuildEventArtifactUploader as these transports might have different lifecycles. We do that by introducing another level of indirection via the BuildEventArtifactUploaderFactory. BlazeModules now register a factory object instead of an uploader. In addition, the BuildEventArtifactUploader gets a shutdown() method that allows to free any resources associated with it. PiperOrigin-RevId: 203752092
* Publish used_heap_size_post_build metric to BEP.Gravatar tomlu2018-07-06
| | | | | | | This metric measures the used heap size at the end of the build, after a GC. Because this is expensive it is only collected if a flag is passed. RELNOTES: PiperOrigin-RevId: 203492365
* bep: guarantee event ordering in file transports.Gravatar buchgr2018-07-06
| | | | | | | | | | | Commit d3f7f7ae introduced a bug in the FileTransport implementations where events might not be written to file in the correct order. This change fixes this. Fixes #5515 RELNOTES: None PiperOrigin-RevId: 203457636
* Include more information about build events' referenced local files.Gravatar tomlu2018-07-04
| | | | | | | Instead of just a path, events now include information about the type of file (output, source file, stdout/stderr, test logs, etc.). This information can be used by the uploaders to determine a) whether to upload, b) what kind of lease to give the files. RELNOTES: None PiperOrigin-RevId: 203285549
* bep: Make the BuildEventArtifactUploader async.Gravatar buchgr2018-06-29
| | | | | | | | | | This changes the BuildEventArtifactUploader to an async interface, thereby no longer potentially delaying event delivery over the eventbus. Additionally, the BES transport is changed to start uploading local files immediately as the events are delivered. RELNOTES: None PiperOrigin-RevId: 202694121
* Fix NoSuchElementException when BuildEventArtifactUploaderMap has no elements.Gravatar lpino2018-06-21
| | | | | | When there are no uploaders in BuildEventArtifactUploaderMap then we should use the BuildEventArtifactUploader.LOCAL_FILES_UPLOADER as a fallback. Added some unit tests as regression tests for the underlying issue. PiperOrigin-RevId: 201529578
* Add new BuildMetrics event to BEP.Gravatar tomlu2018-06-19
| | | | | | | To start we add just a single metric, the number of actions constructed in the current build. RELNOTES: None PiperOrigin-RevId: 201248490
* Add a mechanism for build event protocol events to upload filesGravatar ulfjack2018-06-15
| | | | | | | | This should be a no-op, mostly replacing PathConverter with BuildEventArtifactUploader, since none of the implementations perform any upload yet. PiperOrigin-RevId: 200685325
* Add status details to the protosGravatar ulfjack2018-06-08
| | | | | | | | Both for the build event stream proto, which contains the TestResult event, and to the test status proto which is used for caching. The new field may be populated in the future. PiperOrigin-RevId: 199846232
* Cleanup BuildEventStreamOptionsGravatar ulfjack2018-06-08
| | | | PiperOrigin-RevId: 199795284
* Refactor analysis failure reporting, report loading failuresGravatar ulfjack2018-05-24
| | | | | | | | Split the AnalysisFailureEvent into a legacy and a new variant; always post an AnalysisFailureEvent, even if the legacy variant is not posted - this gives us some coverage for loading failures. PiperOrigin-RevId: 197862257
* Add BuildEventProtocolOptions; use to enable/disable important_outputsGravatar ulfjack2018-05-18
| | | | | | | The important_outputs field is deprecated, and this adds a flag to disable its generation entirely. PiperOrigin-RevId: 197186530
* Rename BuildEventConverters to BuildEventContextGravatar ulfjack2018-05-17
| | | | | | | This is in preparation for adding options to the class, which can be used to modify the behavior of build events posted to the BEP. PiperOrigin-RevId: 196997573
* Implement a limit for named_set_of_file eventsGravatar ulfjack2018-05-15
| | | | | | | | | | | This adds a command-line option which can be used to force Bazel to split up very large events, e.g., events with hundreds of thousands of entries. This is rare, but happens occasionally. It would be better to control the maximum event size directly, but that is significantly more difficult to do here. PiperOrigin-RevId: 196690600
* Remove use of bare Immutable{List,Map,Set} Builder classes.Gravatar jcater2018-04-20
| | | | | | Always use the more-qualified class name for clarity at the site of use. There are too many classes named Builder. PiperOrigin-RevId: 193649193
* Move test_timeout to BuildConfiguration and TargetCompleteEvent.Gravatar Googler2018-04-17
| | | | | | | | - Move test_timeout to BuildConfiguration. - Add test_timeout_seconds field to BEP TargetCompleteEvent. - Deprecate test_timeout field in ExecutionInfo. Data is still written to deprecated field to allow consumer transition. PiperOrigin-RevId: 193192636
* Fix some typos.Gravatar Googler2018-04-16
| | | | PiperOrigin-RevId: 193003375
* Make build_event_stream_java_proto public so it can be used by the IntelliJ ↵Gravatar Googler2018-04-12
| | | | | | | plugin. RELNOTES: None PiperOrigin-RevId: 192592698
* Fixes a broken @link.Gravatar shahan2018-04-09
| | | | | | @link doesn't support generics, so this is converted instead to @code. PiperOrigin-RevId: 192094248
* Makes leaf level AutoCodec marshallers into runtime codecs.Gravatar shahan2018-03-15
| | | | | | Next change will convert containers. PiperOrigin-RevId: 189261293
* ActionOwner references only BuildEvent instead of full BuildConfigurationGravatar shahan2018-03-10
| | | | | | This reduces the size of its serialized representation. PiperOrigin-RevId: 188597127
* Refactoring FileTransport to use the AsynchronousFileOutputStream.Gravatar olaola2018-03-07
| | | | | | | | Very thin wrapper, nothing except swallow+log all errors. TESTED=presubmit RELNOTES: None PiperOrigin-RevId: 188177872
* BEP: Add a --build_event_publish_all_actions flag to allow all actions to be ↵Gravatar ruperts2018-03-02
| | | | | | | published via the BEP, instead of only publishing failed actions and extra actions. RELNOTES: Add a --build_event_publish_all_actions flag to allow all actions to be published via the BEP. PiperOrigin-RevId: 187683799
* Replace path implementation.Gravatar tomlu2018-02-08
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Path and PathFragment have been replaced with String-based implementations. They are pretty similar, but each method is dissimilar enough that I did not feel sharing code was appropriate. A summary of changes: PATH ==== * Subsumes LocalPath (deleted, its tests repurposed) * Use a simple string to back Path * Path instances are no longer interned; Reference equality will no longer work * Always normalized (same as before) * Some operations will now be slower, like instance compares (which were previously just a reference check) * Multiple identical paths will now consume more memory since they are not interned PATH FRAGMENT ============= * Use a simple string to back PathFragment * No more segment arrays with interned strings * Always normalized * Remove isNormalized * Replace some isNormalizied uses with containsUpLevelReferences() to check if path fragments try to escape their scope * To check if user input is normalized, supply static methods on PathFragment to validate the string before constructing a PathFragment * Because PathFragments are always normalized, we have to replace checks for literal "." from PathFragment#getPathString to PathFragment#getSafePathString. The latter returns "." for the empty string. * The previous implementation supported efficient segment semantics (segment count, iterating over segments). This is now expensive since we do longer have a segment array. ARTIFACT ======== * Remove Path instance. It is instead dynamically constructed on request. This is necessary to avoid this CL becoming a memory regression. RELNOTES: None PiperOrigin-RevId: 185062932
* BEP: report analysis failure eventsGravatar Klaus Aehlig2018-01-26
| | | | | | | | | | | | | | ...as completion of the respective top-level targets. In this way, a failure is associated to its root cause, even if the cause is at analysis phase; in particular, visibility errors are correctly associated. For the time beeing, we associate visibility root causes only with labels; it is planned to change that to the more accurate configured labels in a follow-up change. Change-Id: I04121a7cd2099fc65171eae0719fd77b98aef09b PiperOrigin-RevId: 183359798
* BEP: add a separate id for visibility errorsGravatar Klaus Aehlig2018-01-10
| | | | | | | | | | A build might fail because of a visibility violation that does not happen at a top-level target. To avoid confusion, add a separate namespace for configured targets that are just mentioned to report the details of an error. Change-Id: I86587f7489500f1d888bae6ce3d6f4bd79ea1609 PiperOrigin-RevId: 181448003
* BEP: In the TestResult meta data, allow reporting remote cache hitsGravatar aehlig2017-12-06
| | | | | | | | | Extend the optional meta data that TestResults may report by a flag indicating that the test was taken from a remote cache. Also, allow reporting the precise exit code of the test action. PiperOrigin-RevId: 178075896
* BEP: Extend TestResult to optionally contain meta dataGravatar aehlig2017-11-29
| | | | | | | | Extend the protocol to support also showing meta data about an action execution, if the underlying test action is able to report it. PiperOrigin-RevId: 177294531
* BuildToolLogs: clean up interfaceGravatar aehlig2017-11-14
| | | | | | | | For build log files that are inlined, we eventually have to report ByteStrings, so avoid the detour through strings, potentially causing encoding issues. PiperOrigin-RevId: 175647576
* BEP: add commandline for reported actionsGravatar Klaus Aehlig2017-11-10
| | | | | | | | | | The build event protocol reports certain actions that are particularly important to the user, in particular failed ones. Also report the precise command line of those actions, if available. This is useful for IDEs wanting to report errors in a more precise way. Change-Id: I04224185ea46c608ad0dfcdb2e772071ca0736b3 PiperOrigin-RevId: 175203502
* BEP: also report the mnemonic of reported actionsGravatar Klaus Aehlig2017-11-09
| | | | | | | | | Knowing the kind of action that was reported (and hence likely failed) helps users quickly understand what possibly went wrong. So report this information. Change-Id: Ie4869234df895f404ac5d2ff6f05026e4e82ef7e PiperOrigin-RevId: 175043006
* BEP: Add an additional event with tool statisticsGravatar aehlig2017-10-27
| | | | | | | | | | After the completion of the build, also report useful logs/statistics. Do so in a separate event, so that the protocol allows reporting build completion early and purely staticial information later, maybe event after some computations. Change-Id: Ibd221546de76fcffcda7819c300187eac45ebd68 PiperOrigin-RevId: 173548733
* BuildEventStreamer: allow final messagesGravatar aehlig2017-10-26
| | | | | | | | | | After receiving a BuildCompletingEvent, allow some final late messages, but only those not yet seen and announced by that BuildCompletingEvent. This mechanism will be used in a follow-up change to add a message with tool statistics. Change-Id: I979bec5bd946208068faff9a4ddd5245a769f096 PiperOrigin-RevId: 173514552
* BEP: correctly report unbuilt targets if --nobuild is givenGravatar Klaus Aehlig2017-10-25
| | | | | | | | | If bazel is asked to only analyze a target, but not do any builds, correctly report this as the cause why the obtained configured targets were never completed. Change-Id: I6308f8c222861dc0205a63664d5ea9861dd38b6a PiperOrigin-RevId: 173364084
* BEP: correctly report the cause of unbuilt targets if --noanalyze is givenGravatar Klaus Aehlig2017-10-20
| | | | | | | | | If bazel is asked to only load target, but not perform an analysis phase, correctly report this as the cause why the obtained targets wer never configured. Change-Id: Ib630a6dc1b955b810a6cc40254c0ae746e2eca1e PiperOrigin-RevId: 172787777
* BuildEventStreamer: clear a pending BuildStarted event firstGravatar Klaus Aehlig2017-10-16
| | | | | | | | | | | | | | | | | | | | If we receive an event indicating that the build is over, we first post that event and then clear up all pending event by stating that their prerequisite event was aborted (which we can safely assert, as we know we will not process any further events). Now, if a build is aborted (e.g., user interruption) before the build starting event is generated, the streamer can receive a build-finished event while still having an event (e.g., the raw command line) blocked on the build-starting event. So the canonical order of clearing the stream would send a build-finished event before the build-starting event, which can be confusing to consumers of the stream. Therefore, if have to generate an artificial aborted build-starting event, do so first (including clearing the events blocked on the build-starting event) and only afterwards post the build-finished event in the stream. Change-Id: Ib33f16f74b7bee7a963df94bbcad7a56db9f07e3 PiperOrigin-RevId: 172305114
* BEP: explicitly report failures associated with a label as suchGravatar Klaus Aehlig2017-10-16
| | | | | | | | | | There is a conceptual difference between the (maybe unsuccessful) completion of a top-level target and a label as the root cause for a failure (i.e., a missing source file). Indicate that difference as such, by having a separate message for failures associated with an unconfigured label. Change-Id: I3f2e20d4dc85782eb11b104a7baf089e66d972e7 PiperOrigin-RevId: 172299938
* Report the structured Bazel command line via the BEP.Gravatar ccalvarin2017-10-10
| | | | | | | | | This is part of the effort outlined in https://bazel.build/designs/2017/07/13/improved-command-line-reporting.html. The refactoring of the options parser is not yet complete, so we still do not have complete & correct information about the canonical command line. Where the information is blatantly incorrect, a best approximation was made, with comments and tests documenting the deficiencies. Change the names of the initial CommandLine fields in the BEP to be explicitly identified as unstructured. RELNOTES: None. PiperOrigin-RevId: 171625377
* BEP: also report baseline coverageGravatar Klaus Aehlig2017-09-27
| | | | | | | ...as part of the TargetComplete message. Change-Id: I8aa321a2810c3926db943f32cb1811a169e34dd7 PiperOrigin-RevId: 170171458
* BEP: Add coverage data to test_resultGravatar Klaus Aehlig2017-09-26
| | | | | | | | Include the coverage data file into the list of files repeorted in the test_result event in the BEP. Change-Id: Ia7b653addd5589268ce919b3e0349371dbc18bf2 PiperOrigin-RevId: 169956864
* Unify buildevnts indicating build completionGravatar Klaus Aehlig2017-09-19
| | | | | | | ...by factoring out the common code parts. Change-Id: I2ead39ea218dd535af9684a81a7bbe29b558aa27 PiperOrigin-RevId: 169106854
* bep: Properly URI encode file pathsGravatar buchgr2017-09-14
| | | | | | RELNOTES: The Build Event Protocol's File.uri field is now properly encoded according to RFC2396. PiperOrigin-RevId: 168674865
* bes: cancel stream on bazel shutdownGravatar buchgr2017-09-12
| | | | | | | | | | | | | | | | | Introduce a new method BuildEventTransport#closeNow() which forcefully closes the BES upload even if there are still events to upload. This is executed if the bazel server is being shutdown. The implementation in the BuildEventServiceTransport works by shutting down the ExecutorService handling the upload, which in turn will interrupt the upload thread, which in turn will cancel the streaming RPC. The BuildEventServiceTransport will wait up to 100ms for that cancellation to happen. Also fix a bug where the BES transport would not wait for ACKS. PiperOrigin-RevId: 168359947
generated by cgit v1.2.3 (git 2.39.1) at 2024-06-23 03:17:58 +0000