aboutsummaryrefslogtreecommitdiffhomepage
path: root/src/main/java/com/google/devtools/build/lib/runtime/BlazeCommandDispatcher.java
Commit message (Collapse)AuthorAge
* Delete the now-useless interfaces ServerCommand and CommandExecutor.Gravatar lberki2018-02-06
| | | | | | | | | We apparently don't have any other implementations of these interfaces than BlazeCommandDispatcher, so let's not have them at all; we can always put back an interface with the exec() method if need be. RELNOTES: None. PiperOrigin-RevId: 184698573
* Remove ShutdownBlazeServerException in favor of indicating that the server ↵Gravatar lberki2018-02-06
| | | | | | | should be shut down in BlazeCommandResult. RELNOTES: None. PiperOrigin-RevId: 184678994
* Add a "direct" mode to "blaze run" that makes the process being run a directGravatar lberki2018-02-05
| | | | | | | | | | | | | | child of the process where the Blaze client itself was run. Limitations: - Untested on Windows; it should work because ExecuteProgram() is implemented there, too, but since Windows doesn't support exec(), there is at least one process in between Progress towards #2815. RELNOTES[NEW]: The new "--direct_run" flag on "blaze run" lets one run interactive binaries. PiperOrigin-RevId: 184528845
* Accept build IDs by flag. (Rollforward of ↵Gravatar ccalvarin2017-12-14
| | | | | | | | | | | | | https://github.com/bazelbuild/bazel/commit/c6b6dbadd0a93936c51154b25abc5fbba8f2d1af) We accepted these by environment variable largely because setting it via invocation policy would require changing invocation policy for each command, which had caused the Bazel server to restart, loosing incremental state. This is fixed: changing invocation policy no longer causes Bazel to restart its servers, so accept these as normal options. We will soon no longer accept these flags by environment variable, but will accept both for a transition period, so that nobody relying on these values is broken by a single release. To inform users of this environment variable, anyone setting the environment variable without the flag will receive a warning but the value will be kept. The following release will no longer accept an environment variable. Note on format: invocation_id we accept only clean UUIDs, but for build_request_id, to help differentiate otherwise undifferentiable id types, we accept arbitrary prefixes before the UUID. The user is responsible for picking prefixes that are sane. RELNOTES: None. PiperOrigin-RevId: 179063120
* Print 'waiting for other blaze command' on its own line.Gravatar Googler2017-11-28
| | | | | | | | | | It now prints a newline char immediately, rather than waiting until the lock is released, and printing 'done!' at the end of the line. This allows per-line parsers (like IntelliJ) to display this progress notification without special-case hacks. PiperOrigin-RevId: 177180918
* Change config expansion application order, gated by startup flag ↵Gravatar ccalvarin2017-11-20
| | | | | | | | | | | | | --expand_configs_in_place. --config options were expanded in a fix-point expansion, where in practice, the flags that --config values expanded to ended up between the normal bazelrc options and the command line's explicit options. Since the options parser has an order-based priority scheme and it accepts multiple mentions of a single-valued option, this conflicts with users' expectations of being able to override these config expansions by using the order in which they are mentioned. This change makes it possible to expand the config values defined in your bazelrc (or blazerc) files to occur in-place: --stuff --config=something --laterstuff will interpret the options that --config=something expands to as if they had been mentioned explicitly between --stuff and --laterstuff. In order to not break users relying on complex flag combinations to configure their builds, this behavior will not yet be turned on by default. Instead, use --expand_configs_in_place as a startup flag to test this feature. --announce_rc may be helpful for debugging any differences between the fixed point and in-place expansions. Once you've debugged your problems, add "startup --expand_configs_in_place" to your blazerc to stick to the new behavior. RELNOTES: Use --expand_configs_in_place as a startup argument to change the order in which --config expansions are interpreted. PiperOrigin-RevId: 176371289
* Make the config expansion behavior modular.Gravatar ccalvarin2017-11-10
| | | | | | | | | | | Reorganize the BlazeOptionHandler to make it easy to exchange out how the config options are expanded using the definitions in the blazerc. Also change the getOptionsMap call to actually structure the rc options in the order that we parse them: we expand them in command order (for the test command, first add all "common" options, then "build," then "test") and then within each command, we expand the options in the rc order. This somewhat simplifies the rc expansion code, and avoids the two-phase regrouping that used to happen. This change should not change the semantics of rc option ordering. A followup change will add an alternative implementation for --config's expansion. RELNOTES: None. PiperOrigin-RevId: 175208971
* Replace all usages of Blaze's Preconditions class with guava.Gravatar tomlu2017-11-09
| | | | | | | | Blaze had its own class to avoid GC from varargs array creation for the precondition happy path. Guava now (mostly) implements these, making it unnecessary to maintain our own. This change was almost entirely automated by search-and-replace. A few BUILD files needed fixing up since I removed an export of preconditions from lib:util, which was all done by add_deps. There was one incorrect usage of Preconditions that was caught by error prone (which checks Guava's version of Preconditions) that I had to change manually. PiperOrigin-RevId: 175033526
* Automated rollback of commit c6b6dbadd0a93936c51154b25abc5fbba8f2d1af.Gravatar janakr2017-10-31
| | | | | | | | Not a totally clean rollback because of the intervening https://github.com/bazelbuild/bazel/commit/b5158a9a677b149e04e844c40a01e9a9dde40783. *** Reason for rollback *** PiperOrigin-RevId: 173957187
* Accept build IDs by flag.Gravatar ccalvarin2017-10-26
| | | | | | | | | | | We accepted these by environment variable largely because setting it via invocation policy would require changing invocation policy for each command, which had caused the Bazel server to restart, loosing incremental state. This is fixed: changing invocation policy no longer causes Bazel to restart its servers, so accept these as normal options. We will soon no longer accept these flags by environment variable, but will accept both for a transition period, so that nobody relying on these values is broken by a single release. To inform users of this environment variable, anyone setting the environment variable without the flag will receive a warning but the value will be kept. The following release will no longer accept an environment variable. Note on format: invocation_id we accept only clean UUIDs, but for build_request_id, to help differentiate otherwise undifferentiable id types, we accept arbitrary prefixes before the UUID. The user is responsible for picking prefixes that are sane. RELNOTES: None. PiperOrigin-RevId: 173432904
* Split out command line handling into its own file.Gravatar ccalvarin2017-10-25
| | | | | | | | | This code is pretty self-contained: the command dispatcher needs to do options processing all at once, before initializing the command's environment. Splitting it out into its own file makes this more obviously a single step, which is less error prone. For changing --config, we will need to gate the behavior on the value of a startup option. This change will make it easier to isolate the new behavior from the old, since everything does not have to be kept in the same class. RELNOTES: None. PiperOrigin-RevId: 173288367
* Improve --config expansion logging under --announce_rcGravatar ccalvarin2017-10-24
| | | | | | | | To make the source of options more clear before migration, improve the output provided with --announce_rc. This means separating the log messages we have for unconditional rc-options and --config options. The unconditional log statement has not changed. When expanding --config options, log the following in the order that the options are parsed: INFO: Found applicable config definition <command>:<configName> in file <rcfile>: <the options config expands to> RELNOTES: Improve --config logging when --announce_rc is present. PiperOrigin-RevId: 173185451
* Track Option placement within a priority category.Gravatar ccalvarin2017-10-18
| | | | | | | | | An option has precedence over previous options at the same enum-valued priority. Track its placement in this ordering explicitly. This will allow after-the-fact expansion of expansion options such that they correctly take precedence or not compared to other mentions of the same flag. This is needed to fix --config's expansion. RELNOTES: None. PiperOrigin-RevId: 172367996
* Accept command lines from tools invoking Bazel.Gravatar ccalvarin2017-10-18
| | | | | | | | | | | For tools that wrap Bazel in some way, the original way that the tool was invoked can be a useful piece of information to track when logging what Bazel did and why. In order to output this information in the same way that Bazel outputs its command lines, we accept --tool_command_line in the structure command line format that Bazel uses in the BEP. These structured command lines are protos that we expect as a base64 encoded byte array. For simple scripts that wish to use this feature without compiling the proto, we will also accept any old string (that cannot be interpreted as a base64 encoding) as a single "chunk" in a structured command line. This is experimental for now and users should not get attached to the format. We will remove the experimental_ prefix when it is stable. RELNOTES: None. PiperOrigin-RevId: 172341216
* Report the structured Bazel command line via the BEP.Gravatar ccalvarin2017-10-10
| | | | | | | | | This is part of the effort outlined in https://bazel.build/designs/2017/07/13/improved-command-line-reporting.html. The refactoring of the options parser is not yet complete, so we still do not have complete & correct information about the canonical command line. Where the information is blatantly incorrect, a best approximation was made, with comments and tests documenting the deficiencies. Change the names of the initial CommandLine fields in the BEP to be explicitly identified as unstructured. RELNOTES: None. PiperOrigin-RevId: 171625377
* Downgrade the default invocation policy log levels to fine.Gravatar ccalvarin2017-10-06
| | | | | | | | Unfortunately, since the invocation policy gets enforced on a user's command line early in the command environment setup, so the effective log level is not yet set. For this run, keep the logs at INFO, since any other level will disappear into the ether. InvocationPolicy gets enforced not only at invocation startup but also elsewhere to recompute the default values. The log statements from these different runs overwhelm the log output from our tests, making it hard to find other events without filtering the log. Make all extra invocation policy enforcements log at FINE so that they only appear when extra detail is requested. PiperOrigin-RevId: 171151573
* ExperimentalEventHandler: clean up in afterCommandGravatar Klaus Aehlig2017-09-27
| | | | | | | | | | | The experimental event handler honors the AfterCommandEvent which tells to clean up resources. Do so properly, by also flushing output and resetting the terminal (and then flushing it). As a nice side effect, this event handler does not need a separate release handling from the command dispatcher. Change-Id: If93cfa1b0f8d12e94fa4f57c9e2ee67362bd7d11 PiperOrigin-RevId: 170175617
* Replace referrals to options by their name to option definitions.Gravatar ccalvarin2017-09-12
| | | | | | | Now that we have a standard way of referring to an option, remove all of the places that we were referring to them by their name. Since options can have multiple names, this is more clear and provides the additional information needed to understand the option. It also stops the habit of requesting unqualified strings, which was hard to read. RELNOTES: None. PiperOrigin-RevId: 168254584
* More BUILD file refactorings.Gravatar philwo2017-09-06
| | | | | | | | | Split collect, concurrent, vfs, windows into package-level BUILD files. Move clock classes out of "util", into their own Java package. Move CompactHashSet into its own Java package to break a dependency cycle. Give nestedset and inmemoryfs their own package-level BUILD files. PiperOrigin-RevId: 167702127
* Fix assorted ErrorProne warnings.Gravatar lberki2017-09-06
| | | | | RELNOTES: None. PiperOrigin-RevId: 167574104
* Ensure our "Another command is running" messages hold realistic PIDs.Gravatar jmmv2017-08-28
| | | | | | | | | | | | | | | | | | | | Instead of blocking indefinitely for the server lock to become available when other commands are running, busy-wait for the lock to be released. This allows us to detect changes in the PID of the client that is holding the lock, and thus lets us make our wait messages more accurate. There have been multiple bug reports over time (especially from macOS users) where they complain that Bazel is stuck waiting for a non-existent PID. The code doesn't look obviously bogus, so this might just be a case of confusion based on the printed PID. By improving the diagnostic messages we output, we'll either make this confusion go away or have a chance of gathering more data when/if this happens again. This change has the side-effect of homogenizing the wait messages printed by both the Blaze client and the Blaze server and also adds details to know which component is printing what. PiperOrigin-RevId: 166508406
* Add the command option --experimental_oom_more_eagerly_threshold. This will ↵Gravatar janakr2017-08-16
| | | | | | replace the startup option after a deprecation period. PiperOrigin-RevId: 165340514
* Send Bazel startup options to server.Gravatar ccalvarin2017-08-16
| | | | | | | | | | | Send the startup options tagged with their origin so that the server has correct information about the command line as the client received it. Removes the unconditional stderr printing of all bazelrc startup options in the bazel client. Instead, the startup options are sent to the server and the same informational printing is gated on the --announce_rc option. This avoids unconditional log spam to stderr early in startup. If the server is unreachable or there are errors parsing startup options, the message is still printed to stderr. Fixes https://github.com/bazelbuild/bazel/issues/2530. RELNOTES: --announce_rc now controls whether bazelrc startup options are printed to stderr. PiperOrigin-RevId: 165211007
* Merge handleOptions into beforeCommandGravatar ulfjack2017-07-17
| | | | | | | Now that we have the options before calling beforeCommand, there's no need for a separate handleOptions method in the BlazeModule API. Remove it. PiperOrigin-RevId: 162002300
* experimentalUI: agressively bufferGravatar Klaus Aehlig2017-07-14
| | | | | | | | | | | | | | | | | To update the progress bar, we first have to remove it and then write the new one. For this to look smooth, the control sequence removing the old progress bar and the characters of the new progress bar have to arrive "in one go" at the (actual) terminal. As AnsiTerminal sends each control sequence as a separate write to the underlying stream, we have to buffer the underlying stream. Therefore, if the experimental UI is used, buffer that stream unconditionally until flushed, and not by line. For the experimental UI this is save, as it flushes the stream appropriately. For the old UI, we keep the line buffering, as the old UI relies on an implicit flush whenever a new-line character is written. Change-Id: I3a914e4b93ce17c3de05df0d860cf98849c3b4a1 PiperOrigin-RevId: 161935218
* Move CommandEnvironment creation and beforeCommand call after options parsingGravatar ulfjack2017-07-07
| | | | | | | | Instead, introduce a new commandInit method in the BlazeModule API. The primary semantic change here is that beforeCommand is no longer called if options parsing fails or if we're outside a workspace for commands that require one. PiperOrigin-RevId: 161188760
* Automated conversion to Java 8Gravatar laurentlb2017-06-30
| | | | | | | With a few manual fixes for readability. RELNOTES: None. PiperOrigin-RevId: 160582556
* BlazeCommandDispatcher: make parseArgsAndConfigs not take a CommandEnvironmentGravatar ulfjack2017-06-27
| | | | | | | | | | | | | Accept an ExtendedEventHandler in ProjectFileSupport, and use an ExtendedEventHandler in BlazeCommandDispatcher that captures the posts, and delay posting the GotProjectFileEvent until the initialization is complete. This is a small step towards simplifying the BlazeModule API - the plan is to merge beforeCommand, checkEnvironment, and handleOptions into a single method, but introduce a new method (commandInit) to participate in the early command initialization. PiperOrigin-RevId: 160259628
* BlazeCommandDispatcher: all options parsing and editing in one placeGravatar ulfjack2017-06-26
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The invocation policy must be the last step in determining the active options for the current command. Take precautions to prevent accidental options editing of options after the policy is applied by declaring the options as an OptionsProvider (interface has no mutating methods); technically, this could be circumvented by casting to OptionsParser, but it's at least safer than before. As part of this, I'm moving the BlazeCommand.editOptions call to before the invocation policy, and also making minor changes to the CommandEnvironment. Commands that expect to have the last word on options could therefore see options changes after this commit. There are three commands that override editOptions: 1. TestCommand Current behavior: if test_output=streamed is set, also sets --test_sharding_strategy=disabled and --test_strategy=exclusive. Overriding --test_sharding_strategy is not a concern; in fact, maybe we shouldn't be setting that in the first place, since it can cause tests to timeout (timeout is usually applied per shard, so a 10-way sharded test will very likely timeout). If you override the test_strategy to local or standalone, then you may get interleaved stdout / stderr output from tests that run in parallel - a bit surprising, but no showstopper. If you override the test_strategy to remote, you won't get streamed output, because no remote strategy currently supports that. You may get interleaved output, if multiple tests finish very close to each other. There are no correctness concerns, it's just a slightly worse user experience. The code is safe in all cases, AFAICT. We could consider changing streamed to not require serialized execution, and instead use something like a lock - the first test to produce output gets streamed, and all others get delayed until the lock is released, but could still execute concurrently. However, it's unlikely that that's the desired behavior for most users. 2. CoverageCommand Current behavior: sets --collect_code_coverage, increases default --test_timeout. bazel coverage --nocollect_code_coverage is effectively bazel test. You can actually now run bazel test --collect_code_coverage --test_timeout and it will behave exactly the same way as bazel coverage. It's mostly a convenience thing. Note that CoverageCommand inherits TestCommand, so the case above also applies. 3. MobileInstallCommand Current behavior: sets --aspects, and --output_groups. If you override the aspect or output_groups, then the command will fail or run an old binary from a previous build. Not what you'd expect, but no correctness concerns. Summary: IMO, the impact on the existing commands is minor. The advantage of this change is that it's more secure, and gives us an escape hatch if a command ever overrides options in a way that turns out to be wrong or broken in some way. RELNOTES: None. PiperOrigin-RevId: 160114902
* Move command.log writing to a moduleGravatar ulfjack2017-06-26
| | | | | | | This is part of refactoring / simplifying the BlazeCommandDispatcher. RELNOTES: None. PiperOrigin-RevId: 160110866
* Refactor ProjectFileSupport to work without a CommandEnvironmentGravatar ulfjack2017-06-26
| | | | | | | | | | | | Reading project files happens very early in the command initialization, and my plan is to change it so CommandEnvironment is created after we did that; the reason for that is that it makes the options final upon CommandEnvironment creation, which in turn allows simplifying the BlazeModule API. Since we no longer need a CommandEnvironment, or a BlazeWorkspace, or a BlazeRuntime, make the ProjectFileSupportTest a unit test. PiperOrigin-RevId: 159830404
* Use EventHandler even before we have a Reporter instanceGravatar ulfjack2017-06-14
| | | | | | | ...instead of relying on all the methods to call printErrLn with exactly the right format string. PiperOrigin-RevId: 158951236
* Simplify BlazeModule.beforeCommandGravatar ulfjack2017-06-14
| | | | | | | | | | | Don't pass the Command annotation explicitly, but add it to CommandEnvironment instead; most modules don't need it in the first place, so it was a lot of boilerplate for not much. Also change it so that the command is passed to the constructor. Add some documentation to the beforeCommand method. PiperOrigin-RevId: 158847128
* Move InvocationPolicy to the options parser package.Gravatar ccalvarin2017-06-09
| | | | | | | | | | | | | | It was originally included in runtime due to external dependencies, and a desire to keep the options parser a general options library. These dependencies have been or will be removed, and there are plenty of other general flag libraries. InvocationPolicy is fundamentally acting on the properties of this specific OptionsParser and needs proper access to it for the proper solution to a number of existing bugs, which means having access to things that should be package private. PiperOrigin-RevId: 158523111
* Remove the CommandEnvironment from BlazeCommand.editOptionsGravatar ulfjack2017-06-06
| | | | | | | | | | | BlazeCommand.editOptions is currently called fairly late in the startup process, so it must be restrictive in what it does, as any change to the options can potentially introduce inconsistencies between different parts of Bazel. Removing the CommandEnvironment reduces the amount of damage it can do, and may allow us to move the call earlier in the startup process (maybe even to a point where the CommandEnvironment does not exist yet). PiperOrigin-RevId: 158133862
* Add a flag --allow_undefined_configs, with default value true.Gravatar Googler2017-06-01
| | | | | | | | | Setting this flag to false makes Bazel exit with an error on undefined configs. The default value is true, so this change has no impact unless users explicitly pass --noallow_undefined_configs. This change supersedes an unmerged one, I'm taking the problem from fkp@. Credit to him for the original change. PiperOrigin-RevId: 157593338
* Inform the GotOptionsEvent of invocation policy, which is no longer passed ↵Gravatar ccalvarin2017-05-15
| | | | | | as part of the startup arguments. PiperOrigin-RevId: 156090009
* Extend BlazeModule to support listening to the console outputGravatar Klaus Aehlig2017-04-28
| | | | | | | | | | Besides writing console output to the console and the command log, allow modules to register other places where the output should be recorded to. This will allow the build-event protocol to also report on the console output. Change-Id: Ie700243120b0db7c3c68d192abeb0ab7033dc175 PiperOrigin-RevId: 154528369
* When using experimental UI, only to a reset terminal if curses were usedGravatar aehlig2017-04-25
| | | | | | | | | | | | | When unregistering an event handler, that uses colors, we need to be sure to set the terminal in a normal state after unregistering that event handler. The ExperimentalEventHandler does use curses, however only if options allow to do so (otherwise, curses will be filtered out by the underlying terminal). So, we cannot conclude the use of color from the fact that the ExperimentalEventHandler was in use. Hence when releasing it, only reset the the terminal, if we did use color and avoid the unnecessary ^[[0m; otherwise. RELNOTES: None. PiperOrigin-RevId: 154177220
* Add repository override optionGravatar kchodorow2017-04-20
| | | | | | | | | | | RELNOTES: Adds a --override_repository option that takes a repository name and path. This forces Bazel to use the directory at that path for the repository. Example usage: `--override_repository=foo=/home/user/gitroot/foo`. Fixes #1266 PiperOrigin-RevId: 153599291
* CommandDispatcher: s/blaze/bazel/ in error messageGravatar laszlocsomor2017-04-11
| | | | | RELNOTES: none PiperOrigin-RevId: 152783295
* Move InvocationPolicy from a startup argument to part of the RunRequest proto.Gravatar ccalvarin2017-04-07
| | | | | | | | | The user interface is not changing. The policy will still be accepted as a flag passed to the client, as a startup flag (before the command), it will just no longer trigger server restarts and will not be passed on to a bazel server as part of the startup arguments. In batch mode, however, it will still be a startup argument, because the RunRequest proto is not sent, and all invocations restart bazel in batch mode anyway. Since invocation policy only affects command arguments, and changes in command arguments do not cause server restarts, this is consistent with other server restart behavior. RELNOTES: Changing --invocation_policy will no longer force a server restart. PiperOrigin-RevId: 152426207
* Add new exception for wrapping parser construction failuresGravatar brandjon2017-04-03
| | | | | | | | | | The exception is unchecked. The reasoning is that errors during parser construction should not occur, and when they do occur it is an internal error like a failed assertion. This allows casual uses of the options parser to stay oblivious to the possibility of failures, consistent with how DuplicateOptionDeclarationException is currently [not] handled. At the same time, the dispatcher can catch the exception to fail gracefully (by printing to stdout instead of a log file) when parser construction fails for any reason. RELNOTES: None PiperOrigin-RevId: 151839620
* Expand Invocation FlagPolicies on expansion flags.Gravatar ccalvarin2017-03-31
| | | | | | | | For SetValue and UseDefault policies on expansion flags or flags with implicitRequirements, expand the policy into policy for each of its sub-flags. For SetValue, this addresses the issue with policies on expansion flags with overridable=true not actually letting user flags overrride the expansion. For UseDefault, this formalizes the behavior where UseDefault will wipe all user-provided flags that expand from a banned expansion flag, and will allow later work to guarantee that a later policy can override the expansion policy's subflags. Since expansion flags do not have value, break if the invocation policy uses AllowValue or DisallowValue on an expansion flag. PiperOrigin-RevId: 151718539
* Split out the InvocationPolicy parser from the enforcer.Gravatar ccalvarin2017-03-29
| | | | | | The encapsulation was off, since the format in which we accept policy has nothing to do with how we then enforce the policy. PiperOrigin-RevId: 151471747
* Clarify InvocationPolicy origins.Gravatar Chloe Calvarin2017-03-28
| | | | | | | | | Some policies can come from the module API, and some come from the user via startup argument. Clarify which one is which. -- PiperOrigin-RevId: 151324618 MOS_MIGRATED_REVID=151324618
* BEP: Also report the original command line Gravatar Klaus Aehlig2017-03-23
| | | | | | | | | | | In the protocol, also report the command line as received by the server, before and option handlers modify it. -- Change-Id: If5bab172944679752477836a5f499d2837201888 Reviewed-on: https://cr.bazel.build/9453 PiperOrigin-RevId: 150894322 MOS_MIGRATED_REVID=150894322
* Global cleanup change.Gravatar Googler2017-03-14
| | | | | | -- PiperOrigin-RevId: 150052200 MOS_MIGRATED_REVID=150052200
* Make the server commit suicide if its PID file vanishes.Gravatar Lukacs Berki2017-01-13
| | | | | | -- PiperOrigin-RevId: 144434688 MOS_MIGRATED_REVID=144434688
* Use #equals() to check equality between ExitCode instances.Gravatar Lukacs Berki2016-11-15
| | | | | -- MOS_MIGRATED_REVID=139180153
generated by cgit v1.2.3 (git 2.39.1) at 2024-07-05 23:58:22 +0000