| Commit message (Collapse) | Author | Age |
|---|
| |
|
|
|
|
|
|
|
| |
- Replace the existing Retrier with Retrier2.
- Rename Retrier2 to Retrier and remove the old Retrier + RetryException
class.
RELNOTES: None.
PiperOrigin-RevId: 177835070
|
| |
|
|
|
|
|
|
| |
Blaze had its own class to avoid GC from varargs array creation for the precondition happy path. Guava now (mostly) implements these, making it unnecessary to maintain our own.
This change was almost entirely automated by search-and-replace. A few BUILD files needed fixing up since I removed an export of preconditions from lib:util, which was all done by add_deps. There was one incorrect usage of Preconditions that was caught by error prone (which checks Guava's version of Preconditions) that I had to change manually.
PiperOrigin-RevId: 175033526
|
| |
|
|
|
|
|
| |
The old field is the error on Operation proto. The new field is the ExecuteResponse status field.
Note that the new field will also allow us to fetch logs for timing out tests, resolving a TODO, but this is not yet done is this change.
PiperOrigin-RevId: 170370676
|
| |
|
|
|
|
| |
TESTED=unit tests
RELNOTES: none
PiperOrigin-RevId: 169395919
|
| |
|
|
| |
PiperOrigin-RevId: 166981977
|
| |
|
|
|
| |
RELNOTES: None.
PiperOrigin-RevId: 161937673
|
| |
|
|
|
|
| |
TESTED=remote worker, triggered some errors
RELNOTES: fixes #3305, helps #3356
PiperOrigin-RevId: 161722997
|
| |
|
|
|
|
|
|
|
|
| |
- Only pass to the GrpcRemoteExecutor constructor the objects it really
needs.
- Share a Retrier between GrpcRemoteCache and GrpcRemoteExecutor.
RELNOTES: None
PiperOrigin-RevId: 161660054
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- Make the RemoteSpawnRunner match RemoteSpawnStrategy functionality, including
local fallback, remote caching, and execution. This is done so we can
actually finish the migration to the SpawnRunner API, for which I've had a
pending change for several months now.
- Never throw StatusRuntimeException from the GrpcRemoteCache or the
GrpcExecutor. We almost never do, since the Retrier catches them implcitly,
so a number of catch blocks were already unreachable. Carefully document
the cases where we still need to handle it.
- RemoteSpawnStrategy / RemoteSpawnRunner no longer catch gRPC-specific
exceptions; they should be able to handle any reasonable remote caching /
execution implementation (except we don't have a common interface for
GrpcRemoteExecutor yet), with no dependency on gRPC as such. Note that the
RemoteSpawnStrategy class will actually go away after the SpawnRunner
migration (eventually).
- However, ensure that we _are_ actually throwing CacheNotFoundException;
the retrier implicitly catches that also, so we need to manually unwrap
from RetryException.
- Don't call into the EventHandler from RemoteSpawnStrategy; instead, throw
an exception with the message, and let the higher levels handle the
reporting (we only allow this for exception + local fallback, for which
there's no good reporting API right now).
PiperOrigin-RevId: 161195666
|
| |
|
|
|
|
| |
than per-call.
PiperOrigin-RevId: 160892006
|
| |
|
|
|
|
|
|
| |
retry strategy may need tuning.
Other behavior changes: swallowing gRPC CANCELLED errors when the thread is interrupted, as these are expected and just make debugging difficult. Also, distinguishing between the gRPC DEADLINE_EXCEEDED caused by the actual command timing out on the server vs. other causes (the former should not be retriable, while the latter should retry).
TESTED=unit tests, remote worker on Bazel
PiperOrigin-RevId: 160605830
|
| |
|
|
|
|
|
|
|
| |
deprecating the wait_for_completion field.
Note on errors: in the RemoteWorker, I currently handle all errors as onError of the watch call. Other options are: pass them as the operation error field, and pass some of them as the onError of the execute call. For now, I'm just using the simplest option; the Bazel client is ready to handle all possible options.
RELNOTES: none
PiperOrigin-RevId: 158974207
|
| |
|
|
|
|
|
|
|
| |
https://docs.google.com/document/d/1AaGk7fOPByEvpAbqeXIyE8HX_A3_axxNnvroblTZ_6s/edit
Also refactored away the various *Interface* files, no need since unit testing can be done with mocking the appropriate gRPC Impl classes directly (see tests). This also fixes the RemoteSpawnRunner, which should use different objects for remote caching and remote execution, the same way RemoteSpawnStrategy does.
RELNOTES: n/a
PiperOrigin-RevId: 158473700
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Update the command line flags used by remote execution/caching as well as the
build event service (BES).
Major changes:
- Remote execution/caching and BES share flags for authentication and TLS.
- Removed API Key authentication from BES, as it's not being used.
- Add TLS support to BES upload.
- Add --bes_project_id flag. If set, the value is propagated as part of BES
lifecycle events.
For reviewers:
Start your review at CommonRemoteAndBesOptions, BuildEventServiceOptions and
RemoteOptions. The other changes are mostly automatic IDE renames of fields and
flag updates in shell script tests.
RELNOTES: None.
PiperOrigin-RevId: 156553857
|
| |
|
|
|
|
| |
TESTED: local server
RELNOTES: n/a
PiperOrigin-RevId: 153599636
|
|
|
The new RemoteExecutionClient class only performs remote execution, and
nothing else; all higher-level functions, local rety, etc. will live outside
of the client.
In order to add unit tests, I had to add another layer of indirection between
the Grpc{RemoteExecutor,ActionCache} and GRPC, since GRPC generates final,
non-mockable classes. While a testing approach that uses a fake server can
also get some test coverage (as in GrpcActionCacheTest), it doesn't allow us
to test the full range of bad things that can happen at the GRPC layer.
The cloned implementation uses a single GRPC channel, as was recommended to
me by Jakob, who worked on GRPC. A single channel should be sufficiently
scalable, it's thread-safe, and it performs chunking internally. On the
server-side, the requests from a single channel can be dispatched to a thread
pool, so this should not be a blocker for server-side parallelism.
I also changed it to throw an exception whenever anything bad happens - this
makes it much more obvious if there's still bug in this code; the old code
silently swallows many errors, falling back to local execution, which papers
over many issues.
Furthermore, we now return a RemoteExecutionResult to indicate whether the
action ran at all (regardless of exit code), as well as the exit code.
All in all, this implementation is closer to the production code we're using
internally, although quite a few things are still missing.
The cloned implementation is not hooked up to RemoteSpawnStrategy yet. It
also does not support combining remote caching with local execution, but note
that RemoteSpawnStrategy regressed in that respect and currently also does
not support that mode.
PiperOrigin-RevId: 151578409
|
buchgr
tomlu
olaola
ulfjack