| Commit message (Collapse) | Author | Age |
|
|
|
|
|
|
|
|
|
|
|
|
| |
We wind up doing String -> UTF8 bytes conversion for every message serialized
(this happens in protocol buffer land). Do the conversion once and reuse the
immutable value instead of doing it for every chunk of output written.
Keep this optimization local to RpcOutputStream where we see a lot of
repitition - using ByteStrings in place of Strings can get confusing when it
comes to logging, so only apply this optimization where it could count.
--
MOS_MIGRATED_REVID=137964305
|
|
|
|
|
| |
--
MOS_MIGRATED_REVID=135768834
|
|
|
|
|
|
|
|
|
| |
the Bazel server.
RELNOTES[INC]: --command_port=-1 to use AF_UNIX for client/server communications is not supported anymore.
--
MOS_MIGRATED_REVID=135355673
|
|
|
|
|
|
|
| |
Ideally, stdout would be line buffered, too, except that we sometimes pump large amounts of binary data through it, so line buffering would result in performance loss.
--
MOS_MIGRATED_REVID=135237450
|
|
|
|
|
|
|
| |
Prevents overly large responses from overwhelming grpc.
--
MOS_MIGRATED_REVID=134083479
|
|
|
|
|
| |
--
MOS_MIGRATED_REVID=133697898
|
|
|
|
|
|
|
| |
This avoids a server crash if the client gets killed before the server responds to the cancel message.
--
MOS_MIGRATED_REVID=133574767
|
|
|
|
|
|
|
|
|
| |
Previously the client was dependent on stdout/stderr or other interactions to
receive the command id, which is necessary to cancel the command. Send the
command id right away so we can cancel sooner.
--
MOS_MIGRATED_REVID=133282720
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Because we're running our server in a directExecutor the request handling
thread must not block. However in the case of interrupting the command thread
we can block in at least two ways, one not so obvious. The obvious way is
synchronizing on runningCommands. The not obvious way is that interrupting a
thread has some tricky locking involved which can get tangled up with
operations happening in other threads, which may wind up being blocked on the
dispatcher thread being active. More concretely, a thread producing output for
RpcOutputStream may block as a result of the backpressure mechanisms we have in
place. If this winds up blocking in an opportune way, such that it holds some
lock the cancellation thread needs to make progress, we had a deadlock, since
the cancellation thread was preventing output from being consumed.
--
MOS_MIGRATED_REVID=133134743
|
|
|
|
|
| |
--
MOS_MIGRATED_REVID=132039389
|
|
|
|
|
|
|
| |
case the request fails.
--
MOS_MIGRATED_REVID=131804959
|
|
|
|
|
|
|
|
|
|
|
| |
There's a race between sending the finished message and shutdownNow. Use
shutdown instead to let the finished message send.
My IDE also auto-deleted some unnecessary casts, hope that drive by change is
ok.
--
MOS_MIGRATED_REVID=131328436
|
|
|
|
|
| |
--
MOS_MIGRATED_REVID=130609319
|
|
|
|
|
|
|
| |
memory leak in gRPC (or Netty).
--
MOS_MIGRATED_REVID=130369785
|
|
|
|
|
|
|
|
|
| |
the logical rollback of commit 67ad82a319ff8959e69e774e7c15d3af904ec23d.
RELNOTES[INC]: Bazel supports Unix domain sockets for communication between its client and server again, temporarily, while we diagnose a memory leak.
--
MOS_MIGRATED_REVID=130027009
|
|
|
|
|
|
|
|
|
| |
It has been superseded by gRPC.
RELNOTES[INC]: Blaze doesn't support Unix domain sockets for communication between its client and server anymore. Therefore, the --command_port command line argument doesn't accept -1 as a valid value anymore.
--
MOS_MIGRATED_REVID=129424092
|
|
|
|
|
| |
--
MOS_MIGRATED_REVID=129320850
|
|
|
|
|
|
|
| |
sandbox will not allow us to bind to the IPv4 one. Automatically falls back to IPv4 if binding to [::1] fails.
--
MOS_MIGRATED_REVID=129206917
|
|
|
|
|
|
|
| |
--
Change-Id: I2da9049019b3965975fab9b7f606d099d6eab2ff
Reviewed-on: https://bazel-review.googlesource.com/#/c/4040/
MOS_MIGRATED_REVID=128208129
|
|
|
|
|
|
|
| |
In theory, now we can cancel Ping() and Cancel() RPCs, too, but since we don't tell their UUID anyone, we are fine.
--
MOS_MIGRATED_REVID=127703598
|
|
|
|
|
|
|
| |
unlike System#currentTimeMillis().
--
MOS_MIGRATED_REVID=127697254
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
*** Reason for rollback ***
Underlying issue fixed in commit 5b12f6e759c1f3137a7149d3026ff96686d07696
*** Original change description ***
Automated [] rollback of commit a3381b6ac136a0cab3ba86020c739fe16b42cee9.
*** Reason for rollback ***
Broke bazel_rules_test
See https://github.com/bazelbuild/bazel/issues/1501.
*** Original change description ***
Fix default for temporary directories to honor TMPDIR
...and only use the hard-coded "/tmp" as default for the default.
Note that is unchanged that blaze.rpcserver.tmpdir still overrides.
--
MOS_MIGRATED_REVID=127201018
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
*** Reason for rollback ***
Broke bazel_rules_test
See https://github.com/bazelbuild/bazel/issues/1501.
*** Original change description ***
Fix default for temporary directories to honor TMPDIR
...and only use the hard-coded "/tmp" as default for the default.
Note that is unchanged that blaze.rpcserver.tmpdir still overrides.
--
MOS_MIGRATED_REVID=127179462
|
|
|
|
|
|
|
|
|
|
| |
...and only use the hard-coded "/tmp" as default for the default.
Note that is unchanged that blaze.rpcserver.tmpdir still overrides.
--
Change-Id: I2ad6b9904b7cde3917968090e85cf2d6c8ad88ab
Reviewed-on: https://bazel-review.googlesource.com/#/c/3962
MOS_MIGRATED_REVID=127076270
|
|
|
|
|
|
|
|
|
|
|
| |
passes between the Ping() and Run() calls and make Ping() and Cancel() calls restart the server timeout interval.
This "fixes" a race condition where the client would call Ping(), the server would time out and then the Run() call would fail. Of course, this is not an principled fix because in theory, the timeout can still happen between the two calls, but now there are only simple file system operations and a tiny bit of CPU use between the two so it should be vanishingly unlikely. We use ->Ping() to verify server liveness after we start it up, so in theory, the timeout could strike between those two calls, too...
TESTED=By running "bazel --max_timeout_secs=2 info install_base" 100 times in a test and running the test 200 times. This procedure triggered the bug pretty reliably.
--
MOS_MIGRATED_REVID=126902519
|
|
|
|
|
| |
--
MOS_MIGRATED_REVID=126685659
|
|
|
|
|
|
|
| |
way that actually works.
--
MOS_MIGRATED_REVID=120997894
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
2009!).
If a "blaze clean --expunge" was run concurrently with another command (that was waiting for the lock), it's possible that the clean command deletes the lock file, the new server starts up, then the JVM shutdown hooks delete the PID files from the *new* server.
There is still a slight possibility of a race condition if the lock is deleted then IOException occurs which prevents the BlazeShutdownException from being raised, but I'd rather not introduce another channel from command implementations to RPCServer to close that loophole.
This issue was triggered by commit 5a78166ee4edbd295f5d5fdb94785025285e764b, after which the PID files for the new server are written a bit more early, thus increasing the time window in which the race condition can happen.
--
MOS_MIGRATED_REVID=120805163
|
|
|
|
|
|
|
|
|
| |
new ones.
Add server.pid.txt that contains the same information in text form. ExecuteDaemon() on Windows will simply not write server.pid .
--
MOS_MIGRATED_REVID=120802055
|
|
|
|
|
| |
--
MOS_MIGRATED_REVID=120699557
|
|
|
|
|
|
|
| |
This is because OsUtils.getpid() cannot work under msys2 since java.exe is not an msys2 binary. We might make it work by including JNI code, but the current plan is to go without JNI on Windows.
--
MOS_MIGRATED_REVID=120694746
|
|
|
|
|
|
|
| |
This is necessary because on Windows/msys2, the Java gRPC server listens only on IPv6 but the C++ client only tries to connect over IPv4, resulting in breakage.
--
MOS_MIGRATED_REVID=120689437
|
|
|
|
|
|
|
| |
I wonder why it was implemented like this in the first place. Unsurprisingly, it doesn't work on Windows.
--
MOS_MIGRATED_REVID=120682316
|
|
|
|
|
|
|
|
|
|
|
| |
In particular:
- Make a SIGINT to the server interrupt every command
- Parse negative numbers on the command line correctly (std::stoi throws an exception, and I'd rather not start using C++ exceptions)
- Use "bytes" for command line arguments instead of "string" in the client/server proto . This is more principled, although we pretend all arguments are strings all over the place and it works for "blaze run" mostly by accident.
--
MOS_MIGRATED_REVID=120434432
|
|
|
|
|
|
|
|
|
| |
- Actually make it work again (commit 00cfb7df61b1f3d9fac8ee29d92b315cbfe6d28f broke it, maybe I shouldn't send out changes in a hurry next time)
- Rename --grpc_port to --command_port (it's a bit better name)
- Do not send a kill signal to the server that can't be connected if we only connect to it to verify its presence
--
MOS_MIGRATED_REVID=120418784
|
|
|
|
|
|
|
| |
when in gRPC mode.
--
MOS_MIGRATED_REVID=120233121
|
|
|
|
|
|
|
| |
Work towards #930.
--
MOS_MIGRATED_REVID=120205147
|
|
|
|
|
|
|
|
|
|
|
| |
Drive-by fix: eliminate the GRPC messages from the console by passing a null logging function.
This also prepares us for the time when multiple commands will be running, because then we'll need to tell which command exactly we want to interrupt.
Work towards #930.
--
MOS_MIGRATED_REVID=120203008
|
|
|
|
|
|
|
| |
Work towards #930. With this, it's conceivable that server mode works on Windows to some degree (I haven't tried, though, because there are many issues that need to be fixed)
--
MOS_MIGRATED_REVID=120202037
|
|
|
|
|
|
|
|
|
| |
C symbols.
Fixed #1155.
--
MOS_MIGRATED_REVID=120107746
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This still has a number of issues, including, but not limited to:
- When switching between gRPC and AF_UNIX mode, you need to do a manual shutdown
- The console is spammed with "connection refused" messages on server startup
- When in gRPC mode, server also starts up an AF_UNIX server even though it's not necessary and concurrent requests probably make Bazel crash and burn
- I have no idea how concurrent gRPC requests are handled and now many threads gRPC creates
- Not tested except under Linux
- The request/response cookies are written in an odd format (negative bytes are not handled correctly). This is only a cosmetic issue, since the data content of the string is the same either way.
Can be tested with the --grpc_port=0 (or a valid port number) startup option.
--
MOS_MIGRATED_REVID=119948959
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Note that the presence of server/grpc_port does not guarantee that the server actually listens to it and we can't guarantee it, either, because it can always be kill -9'd.
I haven't decided yet how the transition between AF_UNIX and gRPC will work. For now, I'm happy that we can start up a Java server.
The way to get the kernel-chosen port is truly awful, but it is apparently impossible to do so in a different way:
https://github.com/grpc/grpc-java/issues/72
--
MOS_MIGRATED_REVID=119828354
|
|
|
|
|
|
|
| |
The code doesn't do anything yet and it's unused code for now. This change only serves to add all the necessary dependencies to BUILD files that gRPC needs.
--
MOS_MIGRATED_REVID=119628697
|
|
|
|
|
| |
--
MOS_MIGRATED_REVID=118940625
|
|
|
|
|
|
|
| |
This check is awful and should be done somewhat differently, but since we are migrating to gRPC, there isn't much point in investing much in this.
--
MOS_MIGRATED_REVID=118656266
|
|
|
|
|
|
|
|
|
| |
Windows.
Progress towards #930.
--
MOS_MIGRATED_REVID=117799006
|
|
|
|
|
|
|
| |
OutOfMemoryError and have the JVM send Bazel a SIGUSR2 when it detects an OOM. This should help in certain pathological cases when Bazel GC thrashes for some time after an OOM has been detected.
--
MOS_MIGRATED_REVID=116819359
|
|
|
|
|
|
|
| |
Fixed bazel github issue #978
--
MOS_MIGRATED_REVID=116164610
|
|
|
|
|
|
|
| |
invocation could be lost. Note that the client sends an empty request to the Blaze server to check its existence. Previously, we were treating this as a full command invocation in the server and resetting the interrupt bit. Now we detect this and do not alter the interrupt bit.
--
MOS_MIGRATED_REVID=115337972
|
|
|
|
|
|
|
| |
This helps avoid confusion with File*S*ystemUtils, which differs in only the case of a character but is a completely different class.
--
MOS_MIGRATED_REVID=113054116
|