aboutsummaryrefslogtreecommitdiffhomepage
path: root/src/test/java/com/google/devtools/build/lib/runtime
Commit message (Collapse)AuthorAge
* Automated g4 rollback of commit c4134802dd15d6ef5cca6521f6bf6aac395ee2ad.Gravatar kchodorow2017-05-23
| | | | | | | | | | | | | | | | | | | | | | | | | | *** Reason for rollback *** Roll forward of directory name change *** Original change description *** Automated g4 rollback of commit 1d9e1ac90197b1d3d7b137ba3c1ada67bb9ba31b. *** Reason for rollback *** Breaks //src/test/shell/integration:force_delete_output_test *** Original change description *** Symlink output directories to the correct directory name If the workspace directory is /path/to/my/proj and the name in the WORKSPACE file is "floop", this will symlink the output directories to output_base/execroot/floop instead of output_base/execroot/proj. More prep for #1262, fixes #1681. PiperOrigin-RevId: 156892980
* BEP: Provide infrastructure to report configurationsGravatar Klaus Aehlig2017-05-08
| | | | | | | | | | In preparation to support multi-configuration builds, provide infrastructure allowing build events to reference BuildConfigurations. The streaming mechanism will ensure that build configurations are introduced in the stream before being referenced for the first time. Change-Id: I6b96fbebc76a05eff4f75a07e8a9cfbcd57f9c22 PiperOrigin-RevId: 155368666
* Rollforward of commit aa7f9307636d38cbb93a03acac8f4c59adfa0ee8.Gravatar ccalvarin2017-05-04
| | | | | | | | | | | | | | | | | | | | | | | | *** Reason for rollback *** Filter out conflicting flag policies for applicable commands. Only enforce the filtered, applicable policy. *** Original change description *** Automated g4 rollback of commit aa7f9307636d38cbb93a03acac8f4c59adfa0ee8. *** Reason for rollback *** Broke --experimental_inmemory_dotd_files *** Original change description *** Filter out conflicting flag policies before invocation policy enforcement. This is to minimize the likelihood of obscure policy conflict. Now, the last policy on a flag (after policy expansion) will be the only one in the "canonical" invocation policy. There should be no reason for explicitly setting multiple policies on a single flag, but if an expansion flag is policy'd and one of its children has a more specific policy on it, make sure that the policy on the child flag is after the policy on the expansion flag. Note that this restriction (only the last policy gets applied) also applies for repeatable flags. Make sure all values being set to a repeatable flag are set in a single SetValue operation, with multiple flagValues set. PiperOrigin-RevId: 154969189
* BuildEventStreamer: support also providing stdout/stderrGravatar Klaus Aehlig2017-05-02
| | | | | | | | | | | | | Extend the build-event streamer to also report about stdout/stderr, if provided. This information is reported in the progress events. At the moment, we only report stdout/stderr in progress events we send anyway, but the interface is generic enough that we could add time-based reporting later, if needed. Also note, that at the end of the build, we report the final progress event, so that all stdout/stderr generated before the build-complete event get also reported in the build-event protocol. Change-Id: If5dbd59c151edbce02d0a9b2e5938b63c0a5dc58 PiperOrigin-RevId: 154811110
* Automated g4 rollback of commit aa7f9307636d38cbb93a03acac8f4c59adfa0ee8.Gravatar ajmichael2017-04-28
| | | | | | | | | *** Reason for rollback *** Broke --experimental_inmemory_dotd_files RELNOTES: None PiperOrigin-RevId: 154477949
* Automated g4 rollback of commit d9fea57268ff01c001fbcbdc2bd057c86c362e6f.Gravatar buchgr2017-04-25
| | | | | | | | | | | | | | | | | | | | *** Reason for rollback *** Rollforward with fix for test flakiness. BEP transport closed events are delivered via their own threadpool and thus might not have been sent immediately. BuildEventStreamerTest#testSimpleStream now waits for a bit until the event has been delivered. I ran the test with --runs_per_test=1000 several times and had no further failures. *** Original change description *** Automated g4 rollback of commit 3d596d63f883fff56001ed7b2e5cf51dba45f082. *** Reason for rollback *** Made BuildEventStreamerTest#testSimpleStream 3% flaky based on --runs_per_test=1000. RELNOTES: None PiperOrigin-RevId: 154170833
* Automated g4 rollback of commit 3d596d63f883fff56001ed7b2e5cf51dba45f082.Gravatar ajmichael2017-04-25
| | | | | | | | *** Reason for rollback *** Made BuildEventStreamerTest#testSimpleStream 3% flaky based on --runs_per_test=1000. RELNOTES: None PiperOrigin-RevId: 154090559
* BEP: Show protocol upload in the UIGravatar buchgr2017-04-24
| | | | | | | | | | | | | | | | | | | | | | The BEP (build event protocol) upload may at times take longer than the build itself. This is especially true for cached builds, where there is little build work, but the protocol still needs to be uploaded. In this case the bazel UI will now display that it's waiting for BEP upload, both in the current and the new experimental UI (--experimental_ui). When executing a run command, blaze waits for the BEP upload to finish before it runs the target. Major Modifications: - The BuildEventTransport interface now also has a name() method that returns a string. When waiting for a transport to finish the BEP upload, this string is displayed in the UI. - The BuildEventStreamer now emits two new events AnnounceBuildEventTransportsEvent and BuildEventTransportClosed on the event bus. This is how the experimental UI is informed about BEP upload progress. RELNOTES: None PiperOrigin-RevId: 154052401
* Filter out conflicting flag policies before invocation policy enforcement.Gravatar ccalvarin2017-04-20
| | | | | | | | | | | | | This is to minimize the likelihood of obscure policy conflict. Now, the last policy on a flag (after policy expansion) will be the only one in the "canonical" invocation policy. There should be no reason for explicitly setting multiple policies on a single flag, but if an expansion flag is policy'd and one of its children has a more specific policy on it, make sure that the policy on the child flag is after the policy on the expansion flag. Note that this restriction (only the last policy gets applied) also applies for repeatable flags. Make sure all values being set to a repeatable flag are set in a single SetValue operation, with multiple flagValues set. PiperOrigin-RevId: 153584034
* Fix an expansion flag issue where it expands to a repeatable flag.Gravatar ccalvarin2017-04-18
| | | | | | | | There's no reason an expansion flag should not expand to multiple values for a repeatable flag (a flag with allowMultiple set to true.) If this expansion flag is set in a SetValue policy, group its repeatable subflags into a single SetValue per subflag. For an overridable SetValue policy on an expansion, any repeatable flag that it expands to should append its value, and not override the user's original values. PiperOrigin-RevId: 153233784
* Add --all_incompatible_changes invocation policy test.Gravatar ccalvarin2017-04-11
| | | | | | By the time the invocation policy gets enforced, the expansion flag should have been expanded, and policy should act as on any other expansion flag. PiperOrigin-RevId: 152692831
* BEP: Extend infrastructure to allow reporting artifacts only onceGravatar Klaus Aehlig2017-04-07
| | | | | | | | | | Extend the functionality of the BuildEventStreamer to report those parts of NestedSets of Artifacts not reported earlier. In this way, duplicate reporting can be avoided, without the events themselves having to know which artifacts are known already. Change-Id: Ia959c28c440301860eac57ea5d9a712c0d49ebdf PiperOrigin-RevId: 152497672
* BEP: Improve the BuildEventTransport interface.Gravatar buchgr2017-04-07
| | | | | | | | | | | | | | | | Better specify the BuildEventTransport interface. Besides clarifying threading and blocking issues, this change also clarifies error handling/reporting. After several discussions we concluded that the BuildEventTransport interface should not provide error reporting / handling facilities, as there is not much bazel could do with this information. Instead, a transport may decide for itself if an error is fatal and abort the build or if an error should be logged to the user's terminal or if it should be ignored. Furthermore, changing the close() method lays the groundwork for an upcoming change that will report the transport shutdown status to the user command line. PiperOrigin-RevId: 152408938
* Refactor all ctor callsites of PathFragment to instead call a static ↵Gravatar nharmata2017-04-05
| | | | | | | | | | | | 'create' method. This paves the way for changing PathFragment to e.g. an abstract class with multiple subclasses. This way we can split out the windows-specific stuff into one of these concrete classes, making the code more readable and also saving memory (since the shallow heap size of the NonWindowsPathFragment subclass will hopefully be smaller than that of the current PathFragment). This also lets us pursue gc churn optimizations. We can now do interning in PathFragment#create and can also get rid of unnecessary intermediate PathFragment allocations. RELNOTES: None PiperOrigin-RevId: 152145768
* BEP: provide events a generic converter classGravatar aehlig2017-04-04
| | | | | | | | | | | Change the BuildEvent interface to accept a generic class of converters. In this way, we won't have to change it again in the future, once more converters are needed. In fact, a new converter is needed right now (will be added in a follow-up patch) to allow build events to know the name of named artifact groups already reported in the stream. Change-Id: Ibb32ea5fff361e21bcf2d34818d8351a1da7a2e3 PiperOrigin-RevId: 152131870
* Automated g4 rollback of commit 1d9e1ac90197b1d3d7b137ba3c1ada67bb9ba31b.Gravatar hlopko2017-04-04
| | | | | | | | | | | | | | | | | | *** Reason for rollback *** Breaks //src/test/shell/integration:force_delete_output_test *** Original change description *** Symlink output directories to the correct directory name If the workspace directory is /path/to/my/proj and the name in the WORKSPACE file is "floop", this will symlink the output directories to output_base/execroot/floop instead of output_base/execroot/proj. More prep for #1262, fixes #1681. PiperOrigin-RevId: 152126545
* Add --all_incompatible_changes, the user's shorthand for turning on all ↵Gravatar brandjon2017-04-03
| | | | | | | | | | | | --incompatible_* flags Note that if a developer adds a poorly-formatted incompatible change @Option, constructing an OptionsParser will now fail with an unchecked exception. This can cause some unit tests to fail in unexpected ways, but the developer should see an appropriate error message when the server starts up. To be added: A separate integration test that ensures the expansions of --all_incompatible_changes don't clobber each other. RELNOTES: None PiperOrigin-RevId: 151858287
* If --batch, --keep_going, --discard_analysis_cache, and the new ↵Gravatar janakr2017-03-31
| | | | | | | | | | | | | | | --noexperimental_enable_critical_path_profiling flags are all specified, then Bazel will delete Actions from ActionLookupValues as they are executed in order to save memory. Because an already-run action may output an artifact that is only requested later in the build, we need to maintain a way for the artifact to look up the action. But in most cases we don't need to keep the action itself, just its output metadata. Some actions unfortunately are needed post-execution, and so we special-case them. Also includes dependency change with description: Move action out of key. This keeps action references from polluting the graph -- actions are just stored in one SkyValue, instead of being present in SkyKeys. This does mean additional memory used: we have a separate ActionLookupData object per Action executed. That may reach ~24M for million-action builds. PiperOrigin-RevId: 151756383
* Expand Invocation FlagPolicies on expansion flags.Gravatar ccalvarin2017-03-31
| | | | | | | | For SetValue and UseDefault policies on expansion flags or flags with implicitRequirements, expand the policy into policy for each of its sub-flags. For SetValue, this addresses the issue with policies on expansion flags with overridable=true not actually letting user flags overrride the expansion. For UseDefault, this formalizes the behavior where UseDefault will wipe all user-provided flags that expand from a banned expansion flag, and will allow later work to guarantee that a later policy can override the expansion policy's subflags. Since expansion flags do not have value, break if the invocation policy uses AllowValue or DisallowValue on an expansion flag. PiperOrigin-RevId: 151718539
* Symlink output directories to the correct directory nameGravatar kchodorow2017-03-31
| | | | | | | | | | If the workspace directory is /path/to/my/proj and the name in the WORKSPACE file is "floop", this will symlink the output directories to output_base/execroot/floop instead of output_base/execroot/proj. More prep for #1262, fixes #1681. PiperOrigin-RevId: 151712384
* Catch cases where exec fails before WaitForCompletionCommand starts.Gravatar mstaib2017-03-29
| | | | | | | | | | | | | | | | | It's possible for cases to happen where the BlazeCommandDispatcher fails before starting the WaitForCompletionCommand. In these cases, the test's command state handoff will never take place because no command state will be created to be handed off (obviously). This means that runIn will wait forever. Instead, if the command completes and the command state has still not been handed off, set an exception into the command state so that the test will fail without further troubles. RELNOTES: None. PiperOrigin-RevId: 151495494
* Split out the InvocationPolicy parser from the enforcer.Gravatar ccalvarin2017-03-29
| | | | | | The encapsulation was off, since the format in which we accept policy has nothing to do with how we then enforce the policy. PiperOrigin-RevId: 151471747
* Split up InvocationPolicyEnforcerTest.Gravatar Chloe Calvarin2017-03-22
| | | | | | | | It was large and unwieldy - keep common logic in the original file but split out the tests into groups by invocation policy operation type. -- PiperOrigin-RevId: 150759733 MOS_MIGRATED_REVID=150759733
* Clean up InvocationPolicyEnforcerTest.Gravatar Chloe Calvarin2017-03-21
| | | | | | | | Remove magic constants and convert to Truth. -- PiperOrigin-RevId: 150692720 MOS_MIGRATED_REVID=150692720
* BuildEventStreamer: fix handling of postedEvents Gravatar Klaus Aehlig2017-03-21
| | | | | | | | | | | | | Always initialize the set of postedEvents. Also, record the initial event as posted. In this way, the build event streamer can also handle events that should be posted after the BuildStartedEvent but are told the streamer about earlier. -- Change-Id: Ic796f2434434819e6c62633755411527a9dcb7e0 Reviewed-on: https://cr.bazel.build/9451 PiperOrigin-RevId: 150620387 MOS_MIGRATED_REVID=150620387
* Global cleanup change.Gravatar Googler2017-03-14
| | | | | | -- PiperOrigin-RevId: 150051360 MOS_MIGRATED_REVID=150051360
* Don't update the action locality for actions with a null ownerGravatar Ulf Adams2017-03-07
| | | | | | -- PiperOrigin-RevId: 149418372 MOS_MIGRATED_REVID=149418372
* Make ExperimentalStateTracker aware of DownloadProgressEvents Gravatar Klaus Aehlig2017-03-02
| | | | | | | | | | | | In the experimental UI, to be able to report appropriately about ongoing downloads, we need to track their state, as updated by the DownloadProgressEvents. Do so. Also report on ongoing downloads in the progress bar. -- Change-Id: I668e963cd6da85ec598b23724066d366d465271f Reviewed-on: https://cr.bazel.build/9114 PiperOrigin-RevId: 148899297 MOS_MIGRATED_REVID=148899297
* BEP: Add BuildFinished event. Gravatar Jakob Buchgraber2017-02-28
| | | | | | | | | | | | The build event protocol now emits a BuildFinished event at the end of a build. The event is a child of the BuildStarted event. The code changes were larger than expected, due to some refactoring in BuildEventStreamer. This was necessary as the BuildCompleteEvent now implements the BuildEvent interface and Guava's EventBus always invokes the most specialized subscriber method. Thus, the buildEvent(BuildEvent) and buildCompleted(BuildCompletedEvent) methods had to be merged. -- Change-Id: Id0c2bd7220dc8ce03128b7126587e212ee8ce836 Reviewed-on: https://cr.bazel.build/9053 PiperOrigin-RevId: 148788582 MOS_MIGRATED_REVID=148788582
* Support mapping of Paths to URIsGravatar Klaus Aehlig2017-01-18
| | | | | | | | | | | Bazel-created files (like log files of test runs) are internally reported as Paths. However, this is not always the most useful representation of the location of that artifact for a consumer of build events. Therefore, support a mapping of paths to more useful URIs. -- PiperOrigin-RevId: 144843525 MOS_MIGRATED_REVID=144843525
* Fixes invocation policy to correctly apply allow/disallow policies to flagsGravatar Alex Humesky2016-12-16
| | | | | | | | | | | | | which have converters that return lists. The problem was that the policy might, say, disallow values ["a", "b"], and a flag --foo might have a converter which takes a string and splits it by commas to produce a list. The options parser would apply the converter to --foo=a,b to produce ["a", "b"], but invocation policy would compare each element of the policy to the list itself, which will never work. -- PiperOrigin-RevId: 142297177 MOS_MIGRATED_REVID=142297177
* Names of extra-action protos now take into account all aspect names.Gravatar Dmitry Lomov2016-12-16
| | | | | | | | | | | | | If an Aspect registered an action that an extra-action is shadowing, its name is used when creating the extra-action's ID and name. Since recently, an aspect can see other aspects applied to the same target. This CL record the names of other aspects applied to the target as well, disambiguating the action owners. -- PiperOrigin-RevId: 142264153 MOS_MIGRATED_REVID=142264153
* Support ordering constraints on build eventsGravatar Klaus Aehlig2016-11-21
| | | | | | | | | | | | | | In the build event protocol, we promise to respect certain ordering constraints that come naturally by causality anyway (e.g., the root cause of a failure has to come before the failure). However, in the way the call structure is organized, events might occur on the event bus in wrong order. So allow events to specify that order and make the streamer honor it. -- Change-Id: I1fbe9b93681f0ddfa35d9d00d5d1b02103da38a5 Reviewed-on: https://bazel-review.googlesource.com/#/c/7370 MOS_MIGRATED_REVID=139774946
* Make LineBufferedOutputStream resistant to exceptions thrown while flushing ↵Gravatar Lukacs Berki2016-11-21
| | | | | | | its buffer. -- MOS_MIGRATED_REVID=139771073
* Expose aspect-related information in the extra-action proto that Bazel hands ↵Gravatar Carmi Grushko2016-11-11
| | | | | | | | | to action_listener() rules. RELNOTES: Extra actions now contain aspect-related information. -- MOS_MIGRATED_REVID=138832922
* Migrate ActionOwner to @AutoValue.Gravatar Carmi Grushko2016-11-10
| | | | | -- MOS_MIGRATED_REVID=138680612
* Writes a machine readable representation of BuildEvents to a file (varint ↵Gravatar Eduardo Colaco2016-11-03
| | | | | | | | | | | | | | | | | | | delimited). Adds --experimental_build_event_binary_file option that enables varint delimited proto loggging to the specified file path Adds varint delimited BuildEventStreamTransport and BuildEventStreamerModule Adds BuildEventStreamerModule for configuring and setting up BuildEventStreamer and its associated BuildEventTransports. Adds BuildEventTransportFactory which creates a Set of transports from command options. Moves BuildEventStreamer configuration from BlazeCommandDispatcher and BuildEventStreamerModule -- Change-Id: If71f2b58654879c2509206da47e6d1a846bf397f Reviewed-on: https://bazel-review.googlesource.com/#/c/7010/ MOS_MIGRATED_REVID=138073726
* Enables AllowValues and DisallowValues invocation policy operations to takeGravatar Alex Humesky2016-10-25
| | | | | | | | | a value to use instead of the given flag value if it is disallowed by the policy. This is a generalization of the existing new_default_value behavior to cover any value of the flag, not just the default value if the flag is unset. -- MOS_MIGRATED_REVID=137104944
* Line buffer stderr.Gravatar Lukacs Berki2016-10-05
| | | | | | | Ideally, stdout would be line buffered, too, except that we sometimes pump large amounts of binary data through it, so line buffering would result in performance loss. -- MOS_MIGRATED_REVID=135237450
* Add some type annotations for the benefit of Java 7Gravatar Klaus Aehlig2016-10-05
| | | | | | | -- Change-Id: If2cd06ba19e1035275f30c445a99af7a7c81e1d8 Reviewed-on: https://bazel-review.googlesource.com/#/c/6430 MOS_MIGRATED_REVID=135210119
* Enable Bazel commands to exit at any time.Gravatar Michael Staib2016-10-05
| | | | | | | | | | | | | This is the first step on a journey toward allowing commands to AbruptExit wherever they please, similar to how the user can press Ctrl+C at any time and we (should) bail out as fast as we can. By interrupting the command's main thread, we at least offer the command the ability to see that an error requiring a bail has happened, and it should trigger at potentially more locations, rather than just between phases. -- MOS_MIGRATED_REVID=135152330
* Add a build-event streamer classGravatar Klaus Aehlig2016-10-04
| | | | | | | | | | | | | The BuildEventStreamer will listen for BuildEvents and stream them to the provided transports. It also ensures events are properly chained: for unsolicited events, it will add progress events to chain them and at the end of a build it closes all announced but not produced events as aborted. -- Change-Id: I623b582657573fe1288821c96f084e0ab0bca4d4 Reviewed-on: https://bazel-review.googlesource.com/#/c/6275 MOS_MIGRATED_REVID=134787541
* Move workspace initialization out of BlazeRuntime.Builder.Gravatar Ulf Adams2016-08-10
| | | | | | | Require all callers to call initWorkspace after BlazeRuntime construction. -- MOS_MIGRATED_REVID=129842292
* Split BlazeDirectories into two classes.Gravatar Ulf Adams2016-08-09
| | | | | | | | | | | | | This is done in preparation for allowing multiple workspaces / commands in a single server. That requires changes to the module API, which currently hard-codes a 1:1 correspondence between workspaces and servers. Note how BlazeDirectories exists even when it runs outside a workspace. It's not ideal that the output base is in ServerDirectories, and the BlazeRuntime creation also still requires a BlazeDirectories instance. -- MOS_MIGRATED_REVID=129736613
* ExperimentalStateTracker: make suffix gracefully handle negative lengthGravatar Klaus Aehlig2016-08-02
| | | | | | | | | | | | | | When requested to produce a suffix of a string of a string of a given length, gracefully handle the case where the requested length is negative---simply return the empty string in this case. While there, mark this static method as such; also increase visibility to default as it is generally useful and should be tested as an interface of this class. -- Change-Id: I821966f7ba3828809bc6d000358803c131740ec9 Reviewed-on: https://bazel-review.googlesource.com/#/c/4223 MOS_MIGRATED_REVID=129080284
* Add a ServerBuilder, and use that in the module API.Gravatar Ulf Adams2016-07-07
| | | | | | | This change is similar to a previous change that introduced WorkspaceBuilder. -- MOS_MIGRATED_REVID=126799657
* Move LoggingTerminalWriter out of testGravatar Klaus Aehlig2016-06-27
| | | | | | | | | ...so that it can be used by the main code base. -- Change-Id: Ibf59d74ab1b0cca9bd9406bb76ee6e71189653bf Reviewed-on: https://bazel-review.googlesource.com/#/c/3903 MOS_MIGRATED_REVID=125950656
* Add an option for the number of actions shown in the experimental UIGravatar Klaus Aehlig2016-06-21
| | | | | | | | | | | | | | Currently, the experimental UI always shows the 3 oldest, still running actions. While this seems a reasonable default, some users requested to be able to change the number of actions shown. Hence replace the hard-coded value by a flag. While there, also fix an off-by-one error in deciding when to put the ellipsis symbol. -- Change-Id: I037d208360fa1d3f100c99ab1ab1f5fc140138ac Reviewed-on: https://bazel-review.googlesource.com/#/c/3811 MOS_MIGRATED_REVID=125427168
* experimental UI: also report the strategy of running actionsGravatar Klaus Aehlig2016-06-20
| | | | | | | | | | In the experimental UI, for the running actions also report their strategy. This will give a more complete picture of what Bazel is currently doing. -- Change-Id: I9553c952ed494e0db225b2a1ae5e8eba00f68617 Reviewed-on: https://bazel-review.googlesource.com/#/c/3820 MOS_MIGRATED_REVID=125162808
* experimental UI: when showing the last test, color-code its stateGravatar Klaus Aehlig2016-06-03
| | | | | | | | | | | The experimental UI also keeps track, in the progress bar, of the last test that completed. When using curses, use colors to indicate whether the test passed or not. -- Change-Id: Iaa01a773c3bbf534692ed21dd420596cb63e2585 Reviewed-on: https://bazel-review.googlesource.com/#/c/3752 MOS_MIGRATED_REVID=123871492
generated by cgit v1.2.3 (git 2.39.1) at 2024-08-26 04:40:58 +0000