| Commit message (Collapse) | Author | Age |
|---|
| |
|
|
|
|
|
|
|
|
|
| |
This adds support for Unix sockets to Bazel for the
remote http cache. See corresponding issue #5098
for discussion.
RELNOTES: Introduce the --remote_cache_proxy flag,
which allows for remote http caching to connect
via a unix domain socket.
PiperOrigin-RevId: 204111667
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This change introduces concurrent downloads of action outputs
for remote caching/execution. So far, for an action we would
download one output after the other which isn't as bad as it
sounds as we would typically run dozens or hundreds of actions
in parallel. However, for actions with a lot of outputs or graphs
that allow limited parallelism we expect this change to positively
impact performance.
Note, that with this change the AbstractRemoteActionCache will
attempt to always download all outputs concurrently. The actual
parallelism is controlled by the underlying network transport.
The gRPC transport currently enforces no limits on the concurrent
calls, which should be fine given that all calls are multiplexed
on a single network connection. The HTTP/1.1 transport also
enforces no parallelism by default, but I have added the
--remote_max_connections=INT flag which allows to specify an upper
bound on the number of network connections to be open concurrently.
I have introduced this flag as a defensive mechanism for users
who's environment might enforce an upper bound on the number of open
connections, as with this change its possible for the number of
concurrently open connections to dramatically increase (from
NumParallelActions to NumParallelActions * SumParallelActionOutputs).
A side effect of this change is that it puts the infrastructure
for retries and circuit breaking for the HttpBlobStore in place.
RELNOTES: None
PiperOrigin-RevId: 199005510
|
| |
|
|
|
|
|
|
| |
Netty's HttpClientCodec always emits a LastHttpContent event and so we don't need to track the received bytes manually to know when we are done reading. The HttpBlobStore compares the hashes of the received bytes to give us confidence that what we received is correct.
Closes #5244.
PiperOrigin-RevId: 197887877
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When the http response has a status other than 200 and
also has a Content-Length header set, then wait until
all content has been received before completing the user
promise.
In case of any errors, close the channel in order to make
sure it's not reused as we don't know what data is left
on the wire.
Closes #5101.
PiperOrigin-RevId: 194787393
|
| |
|
|
|
|
|
|
|
|
| |
Also, remove unused SO_TIMEOUT. Fixes #4890
cc @benjaminp
Closes #4895.
PiperOrigin-RevId: 190051030
|
| |
|
|
|
|
|
|
| |
Increase the connect timeout to 30 seconds Windows sometimes
seems to need more time.
RELNOTES: None
PiperOrigin-RevId: 188702864
|
| |
|
|
|
|
| |
Closes #4622.
PiperOrigin-RevId: 188595430
|
| |
|
|
|
|
| |
Closes #4609.
PiperOrigin-RevId: 185032751
|
|
|
* This puts in the foundation of HTTP/2 support for remote caching.
* Allows us to remove the Apache HTTP library as a dependency, reducing
the Bazel binary size by 1MiB.
On fast networks (i.e. GCE to GCS) we can see a >2x speed improvement for TLS
throughput. Even from my workstation to GCS I get significant build time
improvements when using Netty's TLS 18s vs 12s.
Closes #4481.
PiperOrigin-RevId: 183411787
|
buchgr