aboutsummaryrefslogtreecommitdiffhomepage
path: root/src/test/java/com/google/devtools/build/lib/runtime
Commit message (Collapse)AuthorAge
* UI: reset timer on every status/strategy changeGravatar Klaus Aehlig2017-11-21
| | | | | | | | | | | | | For an action, whenever the status or strategy changes, reset the timer. In this way, we show the time an action spent in the current state, rather than the total time the action spent since started. This might give a better picture of what is currently going on in the system. Fixes #4089. Change-Id: I441337cf2eec58702889ac7a6a9ed150708e8c9d PiperOrigin-RevId: 176497486
* UI: distinguish executing actionsGravatar Klaus Aehlig2017-11-21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When showing the current actions, the most interesting are those that are currently being executed, not those that are waiting for resources or otherwise in the process of being scheduled. Therefore, show the executing actions at the top of the list. Also add more visual hints to distinguish executing from non executing actions. Screenshot (14:51:21) [160 / 230] 32 actions, 11 running Compiling lotsofwork/true986.c; 0s linux-sandbox Compiling lotsofwork/true982.c; 0s linux-sandbox Compiling lotsofwork/true983.c; 0s linux-sandbox Executing genrule //lotsofwork:true975_c; 0s linux-sandbox Compiling lotsofwork/true981.c; 0s linux-sandbox Linking lotsofwork/true_999; 0s linux-sandbox Linking lotsofwork/true_996; 0s linux-sandbox Linking lotsofwork/true_998; 0s linux-sandbox Compiling lotsofwork/true980.c; 0s linux-sandbox Compiling lotsofwork/true98.c; 0s linux-sandbox Executing genrule //lotsofwork:true974_c; 0s linux-sandbox [Sched] Creating runfiles tree bazel-out/k8-fastbuild/bin/lotsofwork/true_974.runfiles [Sched] Linking lotsofwork/true_997 [Sched] Linking lotsofwork/true_995 [Sched] Linking lotsofwork/true_99 [Sched] Linking lotsofwork/true_991 ... Improves on #4089. Change-Id: I1bc6636d5e53a16151023bba595e38259eb1ac88 PiperOrigin-RevId: 176483192
* Add warning for configs that are expanded multiple times.Gravatar ccalvarin2017-11-20
| | | | | | | Some flags are relatively immune to repeated mentions, since only the last mention ends up mattering, but in some cases, the values are accumulated. Expanding the same config twice may lead to surprising results, so provide a friendly warning. RELNOTES: None PiperOrigin-RevId: 176422758
* Add warning for long chain of recursive --configs.Gravatar ccalvarin2017-11-20
| | | | | | | Configs are recursive, but let's be reasonable. If there's an absurdly long list of configs inheritance, warn about it. It is probably unnecessary, and might very well be unintentional and surprising to the user. RELNOTES: None. PiperOrigin-RevId: 176405183
* Change config expansion application order, gated by startup flag ↵Gravatar ccalvarin2017-11-20
| | | | | | | | | | | | | --expand_configs_in_place. --config options were expanded in a fix-point expansion, where in practice, the flags that --config values expanded to ended up between the normal bazelrc options and the command line's explicit options. Since the options parser has an order-based priority scheme and it accepts multiple mentions of a single-valued option, this conflicts with users' expectations of being able to override these config expansions by using the order in which they are mentioned. This change makes it possible to expand the config values defined in your bazelrc (or blazerc) files to occur in-place: --stuff --config=something --laterstuff will interpret the options that --config=something expands to as if they had been mentioned explicitly between --stuff and --laterstuff. In order to not break users relying on complex flag combinations to configure their builds, this behavior will not yet be turned on by default. Instead, use --expand_configs_in_place as a startup flag to test this feature. --announce_rc may be helpful for debugging any differences between the fixed point and in-place expansions. Once you've debugged your problems, add "startup --expand_configs_in_place" to your blazerc to stick to the new behavior. RELNOTES: Use --expand_configs_in_place as a startup argument to change the order in which --config expansions are interpreted. PiperOrigin-RevId: 176371289
* Add option-specified warning.Gravatar ccalvarin2017-11-10
| | | | | | | Accept warnings specified by flag. This flag will cause a warning to be printed like a standard Option deprecation warning or any other warning from options parsing. This can be used to deprecate definitions of config expansion or bazelrc files. This should not be used to deprecate internal option specifications. RELNOTES: None. PiperOrigin-RevId: 175231963
* Make the config expansion behavior modular.Gravatar ccalvarin2017-11-10
| | | | | | | | | | | Reorganize the BlazeOptionHandler to make it easy to exchange out how the config options are expanded using the definitions in the blazerc. Also change the getOptionsMap call to actually structure the rc options in the order that we parse them: we expand them in command order (for the test command, first add all "common" options, then "build," then "test") and then within each command, we expand the options in the rc order. This somewhat simplifies the rc expansion code, and avoids the two-phase regrouping that used to happen. This change should not change the semantics of rc option ordering. A followup change will add an alternative implementation for --config's expansion. RELNOTES: None. PiperOrigin-RevId: 175208971
* Avoid crash when a test and its alias are specified on the command line for ↵Gravatar lberki2017-11-03
| | | | | | | "blaze test". RELNOTES: None. PiperOrigin-RevId: 174386473
* Compute canonical list of options using OptionValueDescription's tracking of ↵Gravatar ccalvarin2017-10-30
| | | | | | | | | | | | | instances. Stop storing the canonical list of arguments separately. For the canonicalize-flags command, we want to avoid showing options that either have no values in their own right (such as expansion options and wrapper options) and instances of options that did not make it to the final value. This is work we already do in OptionValueDescription, so we can generate the canonical form from the values tracked there, instead of tracking it separately. This means the canonical list is more correct where implicit requirements are concerned: implicit requirements are not listed in the canonical form, but now the values they overwrote will be correctly absent as well. Use this improved list for the effective command line published to the BEP. RELNOTES: Published command lines should have improved lists of effective options. PiperOrigin-RevId: 173873154
* Thread FileSystem through to a single Path#delete call site.Gravatar tomlu2017-10-30
| | | | | | This requires a fairly large amount of changes to fundamental objects like BlazeRuntime, Executor, and so on, as well as changing a lot of test code to thread the file system through. I expect future CLs to be much smaller. PiperOrigin-RevId: 173678144
* Split apart options that should only be set by the client from ↵Gravatar ccalvarin2017-10-27
| | | | | | | | | CommonCommandOptions. client_env and default_override stand apart from the other common options. They can be very long, and should only be used by the environment setup. They should ideally not be passed by option at all, but in the RunRequest proto message, but for now, isolate them so that they don't clutter the common command options, which are passed all over the place. RELNOTES: None. PiperOrigin-RevId: 173592980
* BuildEventStreamer: allow final messagesGravatar aehlig2017-10-26
| | | | | | | | | | After receiving a BuildCompletingEvent, allow some final late messages, but only those not yet seen and announced by that BuildCompletingEvent. This mechanism will be used in a follow-up change to add a message with tool statistics. Change-Id: I979bec5bd946208068faff9a4ddd5245a769f096 PiperOrigin-RevId: 173514552
* Remove feature to allow expansion flags to have values.Gravatar ccalvarin2017-10-18
| | | | | | | | | It was added as a potential fix for --config (an expansion flag with values), but this would have required forcing the parser to know the config's expansions at parsing time, which is not currently possible. Instead, we will use the new addition of option-location tracking to make sure we expand options at a the correct place, even if the expansion is triggered after the fact. This is mostly a straight forward undoing of https://github.com/bazelbuild/bazel/commit/7c7255ec8d6da20526c2c4078c57aadaf3dd3612, except where the context has changed. Notably, implicit requirements are effectively treated like expansion flags, so special casing in OptionDescription could be removed. RELNOTES: None. PiperOrigin-RevId: 172514997
* Track Option placement within a priority category.Gravatar ccalvarin2017-10-18
| | | | | | | | | An option has precedence over previous options at the same enum-valued priority. Track its placement in this ordering explicitly. This will allow after-the-fact expansion of expansion options such that they correctly take precedence or not compared to other mentions of the same flag. This is needed to fix --config's expansion. RELNOTES: None. PiperOrigin-RevId: 172367996
* Accept command lines from tools invoking Bazel.Gravatar ccalvarin2017-10-18
| | | | | | | | | | | For tools that wrap Bazel in some way, the original way that the tool was invoked can be a useful piece of information to track when logging what Bazel did and why. In order to output this information in the same way that Bazel outputs its command lines, we accept --tool_command_line in the structure command line format that Bazel uses in the BEP. These structured command lines are protos that we expect as a base64 encoded byte array. For simple scripts that wish to use this feature without compiling the proto, we will also accept any old string (that cannot be interpreted as a base64 encoding) as a single "chunk" in a structured command line. This is experimental for now and users should not get attached to the format. We will remove the experimental_ prefix when it is stable. RELNOTES: None. PiperOrigin-RevId: 172341216
* BuildEventStreamer: clear a pending BuildStarted event firstGravatar Klaus Aehlig2017-10-16
| | | | | | | | | | | | | | | | | | | | If we receive an event indicating that the build is over, we first post that event and then clear up all pending event by stating that their prerequisite event was aborted (which we can safely assert, as we know we will not process any further events). Now, if a build is aborted (e.g., user interruption) before the build starting event is generated, the streamer can receive a build-finished event while still having an event (e.g., the raw command line) blocked on the build-starting event. So the canonical order of clearing the stream would send a build-finished event before the build-starting event, which can be confusing to consumers of the stream. Therefore, if have to generate an artificial aborted build-starting event, do so first (including clearing the events blocked on the build-starting event) and only afterwards post the build-finished event in the stream. Change-Id: Ib33f16f74b7bee7a963df94bbcad7a56db9f07e3 PiperOrigin-RevId: 172305114
* Report the structured Bazel command line via the BEP.Gravatar ccalvarin2017-10-10
| | | | | | | | | This is part of the effort outlined in https://bazel.build/designs/2017/07/13/improved-command-line-reporting.html. The refactoring of the options parser is not yet complete, so we still do not have complete & correct information about the canonical command line. Where the information is blatantly incorrect, a best approximation was made, with comments and tests documenting the deficiencies. Change the names of the initial CommandLine fields in the BEP to be explicitly identified as unstructured. RELNOTES: None. PiperOrigin-RevId: 171625377
* Move SkylarkSemanticsOptions to the packages/ directory, alongside ↵Gravatar brandjon2017-10-07
| | | | | | | | | SkylarkSemanticsCodec Note that the syntax package and its test package still depend indirectly on the options parser via other Bazel-specific packages. RELNOTES: None PiperOrigin-RevId: 171342823
* Deprecate wrapperOptions.Gravatar ccalvarin2017-09-21
| | | | | | | In order to discourage new uses (there shouldn't be any, but just in case), make it illegal to set wrapperOption=true for non deprecated options. RELNOTES: None. PiperOrigin-RevId: 169477990
* Automatic code cleanup.Gravatar cushon2017-09-15
| | | | PiperOrigin-RevId: 168802886
* Remove deepExecRoot from BlazeDirectories. It is now the default. Also ↵Gravatar janakr2017-09-14
| | | | | | inline constructor that did ServerDirectories creation inline: don't pollute production code for tests' convenience. PiperOrigin-RevId: 168652349
* bes: cancel stream on bazel shutdownGravatar buchgr2017-09-12
| | | | | | | | | | | | | | | | | Introduce a new method BuildEventTransport#closeNow() which forcefully closes the BES upload even if there are still events to upload. This is executed if the bazel server is being shutdown. The implementation in the BuildEventServiceTransport works by shutting down the ExecutorService handling the upload, which in turn will interrupt the upload thread, which in turn will cancel the streaming RPC. The BuildEventServiceTransport will wait up to 100ms for that cancellation to happen. Also fix a bug where the BES transport would not wait for ACKS. PiperOrigin-RevId: 168359947
* More BUILD file refactorings.Gravatar philwo2017-09-06
| | | | | | | | | Split collect, concurrent, vfs, windows into package-level BUILD files. Move clock classes out of "util", into their own Java package. Move CompactHashSet into its own Java package to break a dependency cycle. Give nestedset and inmemoryfs their own package-level BUILD files. PiperOrigin-RevId: 167702127
* Replace TransitionApplier interface with a dedicated class.Gravatar gregce2017-08-21
| | | | | | | | | | | | | | | This accomplishes a few goals: 1. Removes the outdated BuildConfiguration.StaticConfigurationApplier code. 2. Removes the TransitionApplier abstraction completely. This was an awkward bridge meant to support both static and dynamic implementations. 3. Moves transition logic to its own dedicated class: ConfigurationResolver. This no longer belongs in BuildConfiguration, which we ultimately want to become a simple <key, value> map. Part of the static config cleanup effort. PiperOrigin-RevId: 165736955
* Add Skylark flags as common command optionsGravatar brandjon2017-08-18
| | | | | | | | | This makes these flags accessible to all commands, including some that don't do anything with Skylark (e.g. canonicalize-flags). This leads to spammier help messages. But the benefit is that it allows them to appear on a "common" line in the .bazelrc. Fixes #3538. RELNOTES: Skylark-related options may now appear as "common" command options in the .bazelrc PiperOrigin-RevId: 165620829
* Remove ConfigurationCollectionFactory as part of the static config cleanup.Gravatar gregce2017-08-18
| | | | PiperOrigin-RevId: 165578449
* Move core test classes to lib.analysis.testGravatar ulfjack2017-08-11
| | | | | | | | These are depended upon by analysis code, so need to live in the same library as lib.analysis. Moving them here makes it possible to split the build-base library into separate libraries for analysis, execution, and rules. PiperOrigin-RevId: 164847161
* Make the @Option annotation depend on the java version of the tagging enums.Gravatar ccalvarin2017-07-18
| | | | | | | The option filters proto dependency can be removed from the OptionsParser. This is in response to option parser users that want to avoid the bazel-internal proto file in their dependencies. RELNOTES: None. PiperOrigin-RevId: 162249778
* BEP: Report configuration for all actionsGravatar Klaus Aehlig2017-07-14
| | | | | | | | | | Whenever we report an action in the build event protocol that has a configuration associated with it, report the configuration as well in the protocol (and not only the checksum, if the configuration is not that of one of the top-level configured targets). Change-Id: I9b085d5381b3c3509b4f3b99c8a77bc8fba6abfe PiperOrigin-RevId: 161789745
* BuildEventWithConfiguration: represent configurations as BuildEventGravatar Klaus Aehlig2017-07-11
| | | | | | | | | | | From a BEP point of view, the only interface of a configuration we care about is its BuildEvent structure. Represent it as such, so that we can move this class to the rest of the buildeventstream module. This is a prerequisite for ActionOwners refering to configurations in the BEP. Change-Id: I6d1c1bf2951aac91607e83cad664553cd6620df8 PiperOrigin-RevId: 161510049
* BEP: Positively identify the last messageGravatar Klaus Aehlig2017-07-10
| | | | | | | | | | | Set an additional flag in the last BEP message to indicate the end of the proto sequence. While the last event is implicit from the child-relation (the sequence of protos is finished, if at least one event is seen and all announced events have occurred in the stream), an explicit marking of the last event simplifies processing. Change-Id: I6554476f975dc9e52fdb27fb3221bd27f6097ed9 PiperOrigin-RevId: 161377092
* Factor out BuildConfigurationCollection.Transitions.getDynamicTransition.Gravatar gregce2017-07-07
| | | | | | | | | | | | This is a legacy dependency on the configuration transition table, which is only needed for static configurations. Dynamic configurations didn't actually use anything in that table: this was just a convenience interface that could have equally been defined somewhere else. So this cl defines it somewhere else. There's still one last dependency: Transitions.configurationHook. We'll tackle that in a followup cl. PiperOrigin-RevId: 161141650
* Explicitly state the default options categories for options used in blaze ↵Gravatar ccalvarin2017-06-29
| | | | | | | | testing. Unlike in the production flags, these flags are only used for internal testing. Tagged them as NO_OP instead of the default UNKNOWN to make it clear that we do not need these to become properly tagged. PiperOrigin-RevId: 160526472
* Allow expansion flags to have values.Gravatar Googler2017-06-28
| | | | | | | This lets us change what it expands based on the argument passed to the flag. RELNOTES: Allows flags that expand to take values. PiperOrigin-RevId: 160298412
* Update the --incompatible_ flag requirements to use the metadata tag.Gravatar ccalvarin2017-06-28
| | | | | | Leave the category for now as the generated docs still do not use the new categorization. PiperOrigin-RevId: 160290297
* Make BuildEventStreamer flushable before first eventGravatar Klaus Aehlig2017-06-26
| | | | | | | | | | | | | | | Make the BuildEventStreamer capable of handling the flush() method, even before the initial build event is seen. This happens, if a lot (more than 10240 bytes) of output is produced before the BuildStartingEvent is generated- This is not unlikely, as the whole option processing happens before the build starts. As the promise of flush() is that the OutErrProvider is cleared, and that all output between two flush() get into separate events, numbered in order, we have to temporarily store a list of stdout/stderr String pairs. RELNOTES: None. PiperOrigin-RevId: 160137478
* Safe-guard against broken client environmentsGravatar Ulf Adams2017-06-19
| | | | | | | | | | | | | | The code previously threw StringIndexOutOfBoundsException if the client env contained just a variable name with no '=' or value. Fixed #3196. Change-Id: I5afcaa398ab2e8bacc709445f50ba363659cadbb Closes #3197. Change-Id: I5afcaa398ab2e8bacc709445f50ba363659cadbb PiperOrigin-RevId: 159222809
* BES: Consolidate BuildEventServiceModule and BuildEventStreamerModule.Gravatar buchgr2017-06-14
| | | | | | | | | | | | | | After having open sourced the Build Event Service code, there is no more need to have two separate bazel modules that both create a BuildEventStreamer. This change merges the BuildEventStreamerModule logic into the BuildEventServiceModule. DELTA=677 (330 added, 316 deleted, 31 changed) DELTA_BY_EXTENSION=java=293,oss=32 RELNOTES: None. PiperOrigin-RevId: 158960687
* Move InvocationPolicy to the options parser package.Gravatar ccalvarin2017-06-09
| | | | | | | | | | | | | | It was originally included in runtime due to external dependencies, and a desire to keep the options parser a general options library. These dependencies have been or will be removed, and there are plenty of other general flag libraries. InvocationPolicy is fundamentally acting on the properties of this specific OptionsParser and needs proper access to it for the proper solution to a number of existing bugs, which means having access to things that should be package private. PiperOrigin-RevId: 158523111
* Use ImmutableList for Option expansion functionsGravatar Googler2017-06-08
| | | | | RELNOTES: None PiperOrigin-RevId: 158279811
* Remove the CommandEnvironment from BlazeCommand.editOptionsGravatar ulfjack2017-06-06
| | | | | | | | | | | BlazeCommand.editOptions is currently called fairly late in the startup process, so it must be restrictive in what it does, as any change to the options can potentially introduce inconsistencies between different parts of Bazel. Removing the CommandEnvironment reduces the amount of damage it can do, and may allow us to move the call earlier in the startup process (maybe even to a point where the CommandEnvironment does not exist yet). PiperOrigin-RevId: 158133862
* Migrate Java tests to Truth.Gravatar lberki2017-05-30
| | | | | | RELNOTES: None. PiperOrigin-RevId: 157446717
* Experimental UI: Support multiple BuildEventTransportAnnouncedEventsGravatar buchgr2017-05-29
| | | | | | | | We initially didn't consider that multiple BuildEventStreamer instances might exist during a build. RELNOTES: None. PiperOrigin-RevId: 157385069
* Automated g4 rollback of commit c4134802dd15d6ef5cca6521f6bf6aac395ee2ad.Gravatar kchodorow2017-05-23
| | | | | | | | | | | | | | | | | | | | | | | | | | *** Reason for rollback *** Roll forward of directory name change *** Original change description *** Automated g4 rollback of commit 1d9e1ac90197b1d3d7b137ba3c1ada67bb9ba31b. *** Reason for rollback *** Breaks //src/test/shell/integration:force_delete_output_test *** Original change description *** Symlink output directories to the correct directory name If the workspace directory is /path/to/my/proj and the name in the WORKSPACE file is "floop", this will symlink the output directories to output_base/execroot/floop instead of output_base/execroot/proj. More prep for #1262, fixes #1681. PiperOrigin-RevId: 156892980
* BEP: Provide infrastructure to report configurationsGravatar Klaus Aehlig2017-05-08
| | | | | | | | | | In preparation to support multi-configuration builds, provide infrastructure allowing build events to reference BuildConfigurations. The streaming mechanism will ensure that build configurations are introduced in the stream before being referenced for the first time. Change-Id: I6b96fbebc76a05eff4f75a07e8a9cfbcd57f9c22 PiperOrigin-RevId: 155368666
* Rollforward of commit aa7f9307636d38cbb93a03acac8f4c59adfa0ee8.Gravatar ccalvarin2017-05-04
| | | | | | | | | | | | | | | | | | | | | | | | *** Reason for rollback *** Filter out conflicting flag policies for applicable commands. Only enforce the filtered, applicable policy. *** Original change description *** Automated g4 rollback of commit aa7f9307636d38cbb93a03acac8f4c59adfa0ee8. *** Reason for rollback *** Broke --experimental_inmemory_dotd_files *** Original change description *** Filter out conflicting flag policies before invocation policy enforcement. This is to minimize the likelihood of obscure policy conflict. Now, the last policy on a flag (after policy expansion) will be the only one in the "canonical" invocation policy. There should be no reason for explicitly setting multiple policies on a single flag, but if an expansion flag is policy'd and one of its children has a more specific policy on it, make sure that the policy on the child flag is after the policy on the expansion flag. Note that this restriction (only the last policy gets applied) also applies for repeatable flags. Make sure all values being set to a repeatable flag are set in a single SetValue operation, with multiple flagValues set. PiperOrigin-RevId: 154969189
* BuildEventStreamer: support also providing stdout/stderrGravatar Klaus Aehlig2017-05-02
| | | | | | | | | | | | | Extend the build-event streamer to also report about stdout/stderr, if provided. This information is reported in the progress events. At the moment, we only report stdout/stderr in progress events we send anyway, but the interface is generic enough that we could add time-based reporting later, if needed. Also note, that at the end of the build, we report the final progress event, so that all stdout/stderr generated before the build-complete event get also reported in the build-event protocol. Change-Id: If5dbd59c151edbce02d0a9b2e5938b63c0a5dc58 PiperOrigin-RevId: 154811110
* Automated g4 rollback of commit aa7f9307636d38cbb93a03acac8f4c59adfa0ee8.Gravatar ajmichael2017-04-28
| | | | | | | | | *** Reason for rollback *** Broke --experimental_inmemory_dotd_files RELNOTES: None PiperOrigin-RevId: 154477949
* Automated g4 rollback of commit d9fea57268ff01c001fbcbdc2bd057c86c362e6f.Gravatar buchgr2017-04-25
| | | | | | | | | | | | | | | | | | | | *** Reason for rollback *** Rollforward with fix for test flakiness. BEP transport closed events are delivered via their own threadpool and thus might not have been sent immediately. BuildEventStreamerTest#testSimpleStream now waits for a bit until the event has been delivered. I ran the test with --runs_per_test=1000 several times and had no further failures. *** Original change description *** Automated g4 rollback of commit 3d596d63f883fff56001ed7b2e5cf51dba45f082. *** Reason for rollback *** Made BuildEventStreamerTest#testSimpleStream 3% flaky based on --runs_per_test=1000. RELNOTES: None PiperOrigin-RevId: 154170833
* Automated g4 rollback of commit 3d596d63f883fff56001ed7b2e5cf51dba45f082.Gravatar ajmichael2017-04-25
| | | | | | | | *** Reason for rollback *** Made BuildEventStreamerTest#testSimpleStream 3% flaky based on --runs_per_test=1000. RELNOTES: None PiperOrigin-RevId: 154090559
generated by cgit v1.2.3 (git 2.39.1) at 2024-07-04 04:48:15 +0000