aboutsummaryrefslogtreecommitdiffhomepage
path: root/src/main/java/com/google/devtools/build/lib/server
Commit message (Collapse)AuthorAge
* Remove unused workspace variable.Gravatar twerth2018-08-09
| | | | | RELNOTES: None PiperOrigin-RevId: 208010973
* Add a option to disable idle gc.Gravatar Benjamin Peterson2018-08-08
| | | | | | | | | | | | | | | If a Bazel server is idle for 10 seconds, it unconditionally triggers a full-scale Java GC via System.gc(). This behavior doesn't have clear benefits and causes Bazel to steal resources from whatever the user does after invoking Bazel. This CL adds a startup option, --idle_server_tasks, to toggle the idle GC behavior. Also, add some logging for when idle GC is enabled, so it's easier to evaluate its effects. Example of logging: ``` 180718 17:43:04.609:I 247 [com.google.devtools.build.lib.server.IdleServerTasks.lambda$idle$0] [Idle GC] used: 157MB -> 15MB, committed: 421MB -> 422MB ``` Fixes https://github.com/bazelbuild/bazel/issues/5589. Closes #5628. PiperOrigin-RevId: 207869996
* Replace instances of Blaze with Bazel.Gravatar Dan Fabulich2018-07-05
| | | | | | | PiperOrigin-RevId: 203300374 Change-Id: Iaa47f870ab2e0cad40a202aad2c7f9430f73c856 PiperOrigin-RevId: 203407392
* Remove the unused method IdleServerTasks#continueProcessing() .Gravatar lberki2018-04-03
| | | | | | | IdleServerTasks could probably be implemented in a much simpler way, but let's keep this change a simple deletion. One step at a time. RELNOTES: None. PiperOrigin-RevId: 191418738
* Delete the now-useless interfaces ServerCommand and CommandExecutor.Gravatar lberki2018-02-06
| | | | | | | | | We apparently don't have any other implementations of these interfaces than BlazeCommandDispatcher, so let's not have them at all; we can always put back an interface with the exec() method if need be. RELNOTES: None. PiperOrigin-RevId: 184698573
* Move BlazeRuntime/BlazeCommandDispatcher shutdown to where they are created.Gravatar lberki2018-02-06
| | | | | RELNOTES: None. PiperOrigin-RevId: 184692000
* Remove ShutdownBlazeServerException in favor of indicating that the server ↵Gravatar lberki2018-02-06
| | | | | | | should be shut down in BlazeCommandResult. RELNOTES: None. PiperOrigin-RevId: 184678994
* Add a "direct" mode to "blaze run" that makes the process being run a directGravatar lberki2018-02-05
| | | | | | | | | | | | | | child of the process where the Blaze client itself was run. Limitations: - Untested on Windows; it should work because ExecuteProgram() is implemented there, too, but since Windows doesn't support exec(), there is at least one process in between Progress towards #2815. RELNOTES[NEW]: The new "--direct_run" flag on "blaze run" lets one run interactive binaries. PiperOrigin-RevId: 184528845
* Automated rollback of commit 4869c4e17d5b1410070a1570f3244148d8f97b5d.Gravatar pcloudy2017-11-15
| | | | | | | | | | | | | | | | | | | | | | *** Reason for rollback *** Causing Bazel server to crash when running bazel clean --expunge https://github.com/bazelbuild/bazel/issues/3956 *** Original change description *** Delayed rollforward of commit 8fb311b4dced234b2f799c16c7d08148619f4087. This was rolled back due to Tensorflow breakage but the patch I exported to gerrit (https://bazel-review.googlesource.com/c/bazel/+/18590) passed Tensorflow (https://ci.bazel.io/job/bazel/job/presubmit/52/Downstream_projects/). Confirmed with jcater@ that the "newly failing" projects in the Global Tests are known issues. I think we can check this in now. Additionally I had attempted to reproduce any tensorflow issues with this by building and testing Tensor... *** ROLLBACK_OF=172361085 RELNOTES:None PiperOrigin-RevId: 175821671
* Replace all usages of Blaze's Preconditions class with guava.Gravatar tomlu2017-11-09
| | | | | | | | Blaze had its own class to avoid GC from varargs array creation for the precondition happy path. Guava now (mostly) implements these, making it unnecessary to maintain our own. This change was almost entirely automated by search-and-replace. A few BUILD files needed fixing up since I removed an export of preconditions from lib:util, which was all done by add_deps. There was one incorrect usage of Preconditions that was caught by error prone (which checks Guava's version of Preconditions) that I had to change manually. PiperOrigin-RevId: 175033526
* Some minor quality of life improvements related to the fact that the default ↵Gravatar nharmata2017-11-07
| | | | | | | | | | value of --max_idle_secs is 15s when the TEST_TMPDIR environment variable is set. (i) Add a log line to blaze.INFO when the server shuts itself down due to idleness. (ii) Mention the --max_idle_secs default in the existing stderr spam when TEST_TMPDIR is set. RELNOTES: None PiperOrigin-RevId: 174745511
* Delayed rollforward of commit 8fb311b4dced234b2f799c16c7d08148619f4087.Gravatar kush2017-10-18
| | | | | | | | | | | | | | | This was rolled back due to Tensorflow breakage but the patch I exported to gerrit (https://bazel-review.googlesource.com/c/bazel/+/18590) passed Tensorflow (https://ci.bazel.io/job/bazel/job/presubmit/52/Downstream_projects/). Confirmed with jcater@ that the "newly failing" projects in the Global Tests are known issues. I think we can check this in now. Additionally I had attempted to reproduce any tensorflow issues with this by building and testing TensorFlow locally with this patch, and all tests which passed with the released bazel had also passed with this patch. ================= Original change description ========================== Reinstate idleness checks where the server self-terminates when it's idle and there is either too much memory pressure or the workspace directory is gone. Arguably, it should kill itself when the workspace directory is gone regardless of whether it's idle or not, but let's first get us back to a known good state, then we can think about improvements. RELNOTES: None. PiperOrigin-RevId: 172361085
* Throw a checked exception if there's an error reading /proc/meminfo, and ↵Gravatar janakr2017-10-12
| | | | | | handle it properly. PiperOrigin-RevId: 171906091
* More BUILD file refactorings.Gravatar philwo2017-09-06
| | | | | | | | | Split collect, concurrent, vfs, windows into package-level BUILD files. Move clock classes out of "util", into their own Java package. Move CompactHashSet into its own Java package to break a dependency cycle. Give nestedset and inmemoryfs their own package-level BUILD files. PiperOrigin-RevId: 167702127
* Rename all logger instances to "logger" (instead "LOG" or "log").Gravatar lberki2017-09-05
| | | | | RELNOTES: None. PiperOrigin-RevId: 167505493
* Send Bazel startup options to server.Gravatar ccalvarin2017-08-16
| | | | | | | | | | | Send the startup options tagged with their origin so that the server has correct information about the command line as the client received it. Removes the unconditional stderr printing of all bazelrc startup options in the bazel client. Instead, the startup options are sent to the server and the same informational printing is gated on the --announce_rc option. This avoids unconditional log spam to stderr early in startup. If the server is unreachable or there are errors parsing startup options, the message is still printed to stderr. Fixes https://github.com/bazelbuild/bazel/issues/2530. RELNOTES: --announce_rc now controls whether bazelrc startup options are printed to stderr. PiperOrigin-RevId: 165211007
* Ensure that shutdown commands end the server process before completionGravatar mschaller2017-07-12
| | | | | | | | | | | | | | | | | | | | This change ensures that the server process is terminated before the client process terminates, when evaluating a command that shuts down the server. When completing such a command, the server communicates to the client that the server will terminate itself by setting a termination_expected bit in the final RunResponse message. The client then waits up to 60s for the server process to actually terminate. If it does not, then the client SIGKILLs the server. Also makes the gRPC server stop accepting new commands before the shutdown command completes. Drive-by fix to comments on Search{Un,Null}aryOption. RELNOTES: Commands that shut down the server (like "shutdown") now ensure that the server process has terminated before the client process terminates. PiperOrigin-RevId: 161537480
* Automated conversion to Java 8Gravatar laurentlb2017-06-30
| | | | | | | With a few manual fixes for readability. RELNOTES: None. PiperOrigin-RevId: 160582556
* Fix a few style issues.Gravatar lberki2017-06-29
| | | | | RELNOTES: None. PiperOrigin-RevId: 160510979
* Migrate the lib.server package to Java 8.Gravatar lberki2017-06-29
| | | | | RELNOTES: None. PiperOrigin-RevId: 160504704
* Move InvocationPolicy to the options parser package.Gravatar ccalvarin2017-06-09
| | | | | | | | | | | | | | It was originally included in runtime due to external dependencies, and a desire to keep the options parser a general options library. These dependencies have been or will be removed, and there are plenty of other general flag libraries. InvocationPolicy is fundamentally acting on the properties of this specific OptionsParser and needs proper access to it for the proper solution to a number of existing bugs, which means having access to things that should be package private. PiperOrigin-RevId: 158523111
* Fix a bunch of issues related to server startup/shutdown:Gravatar lberki2017-06-06
| | | | | | | | | - Only try to communicate with the server if it passes verification (i.e. the indicated PID exists and its start time is correct) - Use only one thread to delete files on exit - Don't set restart reason to NEW_VERSION unless there is an existing server RELNOTES: None. PiperOrigin-RevId: 158131470
* Move InvocationPolicy from a startup argument to part of the RunRequest proto.Gravatar ccalvarin2017-04-07
| | | | | | | | | The user interface is not changing. The policy will still be accepted as a flag passed to the client, as a startup flag (before the command), it will just no longer trigger server restarts and will not be passed on to a bazel server as part of the startup arguments. In batch mode, however, it will still be a startup argument, because the RunRequest proto is not sent, and all invocations restart bazel in batch mode anyway. Since invocation policy only affects command arguments, and changes in command arguments do not cause server restarts, this is consistent with other server restart behavior. RELNOTES: Changing --invocation_policy will no longer force a server restart. PiperOrigin-RevId: 152426207
* Clarify the log lines used when a command is interrupted to more obviously ↵Gravatar Nathan Harmata2017-03-14
| | | | | | | | disambiguate the case where the blaze grpc client tells the blaze grpc server to cancel a running command (e.g. the blaze user ctrl+c's the blaze client) between the case a streaming rpc call gets cancelled (e.g. when the grpc client hangs up). -- PiperOrigin-RevId: 149960615 MOS_MIGRATED_REVID=149960615
* Rollback of commit 8fb311b4dced234b2f799c16c7d08148619f4087.Gravatar Dmitry Lomov2017-02-16
| | | | | | | | | | | | | | | | *** Reason for rollback *** Bisected by dmarting as a root cause of TF breakage: http://ci.bazel.io/job/TensorFlow/672/BAZEL_VERSION=HEAD,PLATFORM_NAME=linux-x86_64/console *** Original change description *** Reinstate idleness checks where the server self-terminates when it's idle and there is either too much memory pressure or the workspace directory is gone. Arguably, it should kill itself when the workspace directory is gone regardless of whether it's idle or not, but let's first get us back to a known good state, then we can think about improvements. -- PiperOrigin-RevId: 147726386 MOS_MIGRATED_REVID=147726386
* Reinstate idleness checks where the server self-terminates when it's idle ↵Gravatar Lukacs Berki2017-02-14
| | | | | | | | | | and there is either too much memory pressure or the workspace directory is gone. Arguably, it should kill itself when the workspace directory is gone regardless of whether it's idle or not, but let's first get us back to a known good state, then we can think about improvements. -- PiperOrigin-RevId: 147454329 MOS_MIGRATED_REVID=147454329
* Reinstate IdleServerTasks that accidentally got disabled with the switch ↵Gravatar Lukacs Berki2017-02-07
| | | | | | | | | | from AF_UNIX to gRPC. Do not reinstate the checks whether the server should terminate itself, though, because we apparently get along pretty well without those. -- PiperOrigin-RevId: 146760481 MOS_MIGRATED_REVID=146760481
* Add a little more logging so that we have more data about gRPC communication ↵Gravatar Lukacs Berki2017-02-01
| | | | | | | | issues. -- PiperOrigin-RevId: 146127950 MOS_MIGRATED_REVID=146127950
* Global cleanup change.Gravatar Googler2017-01-25
| | | | | | -- PiperOrigin-RevId: 145473478 MOS_MIGRATED_REVID=145473478
* Make the server commit suicide if its PID file vanishes.Gravatar Lukacs Berki2017-01-13
| | | | | | -- PiperOrigin-RevId: 144434688 MOS_MIGRATED_REVID=144434688
* Remove the unused class RPCService and its test.Gravatar Lukacs Berki2017-01-10
| | | | | | -- PiperOrigin-RevId: 144054816 MOS_MIGRATED_REVID=144054816
* Move ProcMeminfoParser to lib.unix, where it belongsGravatar Ulf Adams2017-01-09
| | | | | | -- PiperOrigin-RevId: 143939410 MOS_MIGRATED_REVID=143939410
* Rollback of commit 70c5790e4fb01db382d61d457596a46b68ba8d13.Gravatar Janak Ramakrishnan2017-01-05
| | | | | | | | | | | | | | | | (Tests kept.) *** Reason for rollback *** [] *** Original change description *** When --experimental_oom_more_eagerly is enabled, tell Bazel to exit with an OutOfMemoryError and have the JVM send Bazel a SIGUSR2 when it detects an OOM. This should help in certain pathological cases when Bazel GC thrashes for some time after an OOM has been detected. -- PiperOrigin-RevId: 143694970 MOS_MIGRATED_REVID=143694970
* Bazel client: delete and don't use the server.pidGravatar Laszlo Csomor2016-11-24
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Upon startup the Bazel client checks if there's already a running server process and if so then connects to it. We achieve this by checking if there's a symlink in the server directory called served.pid, pointing to /proc/<server_pid>. If so, we read the symlink's target and extract the PID; otherwise we check if there's a file in the server's directory (server.pid.txt) that contains the PID and read it from there. Since the PID file is always there, we don't need the symlink, plus on Windows we don't support symlinks anyway, which is the real motivation for this change. Just ignoring the PID symlink is not enough, we need to actively delete it so that switching between Bazel versions (one that writes a PID symlink and one that doesn't) won't result in having a symlink and PID file with different PIDs and clients trying to kill the wrong server process / not killing one that they should. See https://github.com/bazelbuild/bazel/issues/2107 -- MOS_MIGRATED_REVID=140117287
* Store and use commandId and cookie as ByteStringsGravatar Michajlo Matijkiw2016-11-03
| | | | | | | | | | | | | We wind up doing String -> UTF8 bytes conversion for every message serialized (this happens in protocol buffer land). Do the conversion once and reuse the immutable value instead of doing it for every chunk of output written. Keep this optimization local to RpcOutputStream where we see a lot of repitition - using ByteStrings in place of Strings can get confusing when it comes to logging, so only apply this optimization where it could count. -- MOS_MIGRATED_REVID=137964305
* Avoid deadlock when a client disconnects with the gRPC queue full.Gravatar Lukacs Berki2016-10-11
| | | | | -- MOS_MIGRATED_REVID=135768834
* Remove support for using AF_UNIX to communicate between the Bazel cient and ↵Gravatar Lukacs Berki2016-10-07
| | | | | | | | | the Bazel server. RELNOTES[INC]: --command_port=-1 to use AF_UNIX for client/server communications is not supported anymore. -- MOS_MIGRATED_REVID=135355673
* Line buffer stderr.Gravatar Lukacs Berki2016-10-05
| | | | | | | Ideally, stdout would be line buffered, too, except that we sometimes pump large amounts of binary data through it, so line buffering would result in performance loss. -- MOS_MIGRATED_REVID=135237450
* Chunk large stdout/err writes in RpcOutputStreamGravatar Michajlo Matijkiw2016-09-26
| | | | | | | Prevents overly large responses from overwhelming grpc. -- MOS_MIGRATED_REVID=134083479
* Fix a typo.Gravatar Lukacs Berki2016-09-21
| | | | | -- MOS_MIGRATED_REVID=133697898
* Catch StatusRuntimeException in the Cancel() RPC.Gravatar Lukacs Berki2016-09-19
| | | | | | | This avoids a server crash if the client gets killed before the server responds to the cancel message. -- MOS_MIGRATED_REVID=133574767
* Send the client the command id as soon as possibleGravatar Michajlo Matijkiw2016-09-16
| | | | | | | | | Previously the client was dependent on stdout/stderr or other interactions to receive the command id, which is necessary to cancel the command. Send the command id right away so we can cancel sooner. -- MOS_MIGRATED_REVID=133282720
* Move request cancellation off the dispatching threadGravatar Michajlo Matijkiw2016-09-14
| | | | | | | | | | | | | | | | | Because we're running our server in a directExecutor the request handling thread must not block. However in the case of interrupting the command thread we can block in at least two ways, one not so obvious. The obvious way is synchronizing on runningCommands. The not obvious way is that interrupting a thread has some tricky locking involved which can get tangled up with operations happening in other threads, which may wind up being blocked on the dispatcher thread being active. More concretely, a thread producing output for RpcOutputStream may block as a result of the backpressure mechanisms we have in place. If this winds up blocking in an opportune way, such that it holds some lock the cancellation thread needs to make progress, we had a deadlock, since the cancellation thread was preventing output from being consumed. -- MOS_MIGRATED_REVID=133134743
* Implement flow control for gRPC and cleanup the interface to it a bit.Gravatar Lukacs Berki2016-09-06
| | | | | -- MOS_MIGRATED_REVID=132039389
* Increase the deadline for cancellation request and add some extra logging in ↵Gravatar Lukacs Berki2016-08-31
| | | | | | | case the request fails. -- MOS_MIGRATED_REVID=131804959
* GRPC server waits for termination after clean commandGravatar Michajlo Matijkiw2016-08-26
| | | | | | | | | | | There's a race between sending the finished message and shutdownNow. Use shutdown instead to let the finished message send. My IDE also auto-deleted some unnecessary casts, hope that drive by change is ok. -- MOS_MIGRATED_REVID=131328436
* Log gRPC interrupt events.Gravatar Lukacs Berki2016-08-18
| | | | | -- MOS_MIGRATED_REVID=130609319
* Only call StreamObserver.onNext() from a single thread in order to avoid a ↵Gravatar Lukacs Berki2016-08-16
| | | | | | | memory leak in gRPC (or Netty). -- MOS_MIGRATED_REVID=130369785
* Rollback of commit f107debac45ddf5859b1eb963379769b5815b18f. Also includes ↵Gravatar Janak Ramakrishnan2016-08-12
| | | | | | | | | the logical rollback of commit 67ad82a319ff8959e69e774e7c15d3af904ec23d. RELNOTES[INC]: Bazel supports Unix domain sockets for communication between its client and server again, temporarily, while we diagnose a memory leak. -- MOS_MIGRATED_REVID=130027009
* Remove the AF_UNIX client/server communication protocol.Gravatar Lukacs Berki2016-08-05
| | | | | | | | | It has been superseded by gRPC. RELNOTES[INC]: Blaze doesn't support Unix domain sockets for communication between its client and server anymore. Therefore, the --command_port command line argument doesn't accept -1 as a valid value anymore. -- MOS_MIGRATED_REVID=129424092
generated by cgit v1.2.3 (git 2.39.1) at 2024-06-13 04:01:03 +0000