aboutsummaryrefslogtreecommitdiffhomepage
path: root/doc/load-balancing.md
diff options
context:
space:
mode:
authorGravatar Mark D. Roth <roth@google.com>2016-12-14 07:52:47 -0800
committerGravatar Mark D. Roth <roth@google.com>2016-12-14 07:52:47 -0800
commit3cadfb152fee3160be689ee4545e15dfcf8ebd37 (patch)
tree5b5d722c86f3fa28d1957ee3fadb81ae71d9d8c1 /doc/load-balancing.md
parent4603a1c7a43347fd7139c619112de82c03f7b18a (diff)
Started updating docs.
Diffstat (limited to 'doc/load-balancing.md')
-rw-r--r--doc/load-balancing.md128
1 files changed, 73 insertions, 55 deletions
diff --git a/doc/load-balancing.md b/doc/load-balancing.md
index dfaa7a7f33..03c9bd2cba 100644
--- a/doc/load-balancing.md
+++ b/doc/load-balancing.md
@@ -44,30 +44,31 @@ list of servers to the client.
### External Load Balancing Service
The client load balancing code is kept simple and portable, implementing
-well-known algorithms (ie, Round Robin) for server selection.
-Complex load balancing algorithms are instead provided by the load balancer. The
-client relies on the load balancer to provide _load balancing configuration_ and
-_the list of servers_ to which the client should send requests. The balancer
-updates the server list as needed to balance the load as well as handle server
+well-known algorithms (e.g., Round Robin) for server selection.
+Complex load balancing algorithms are instead provided by the load
+balancer. The client relies on the load balancer to provide _load
+balancing configuration_ and _the list of servers_ to which the client
+should send requests. The balancer updates the server list as needed
+to balance the load as well as handle server unavailability or health
+issues. The load balancer will make any necessary complex decisions and
+inform the client. The load balancer may communicate with the backend
+servers to collect load and health information.
+
+# Requirements
+
+### Simple API and client
+
+The gRPC client load balancing code must be simple and portable. The
+client should only contain simple algorithms (e.g., Round Robin) for
+server selection. For complex algorithms, the client should rely on
+a load balancer to provide load balancing configuration and the list of
+servers to which the client should send requests. The balancer will update
+the server list as needed to balance the load as well as handle server
unavailability or health issues. The load balancer will make any necessary
-complex decisions and inform the client. The load balancer may communicate with
-the backend servers to collect load and health information.
+complex decisions and inform the client. The load balancer may communicate
+with the backend servers to collect load and health information.
-
-## Requirements
-
-#### Simple API and client
-
-The gRPC client load balancing code must be simple and portable. The client
-should only contain simple algorithms (ie Round Robin) for server selection. For
-complex algorithms, the client should rely on a load balancer to provide load
-balancing configuration and the list of servers to which the client should send
-requests. The balancer will update the server list as needed to balance the load
-as well as handle server unavailability or health issues. The load balancer will
-make any necessary complex decisions and inform the client. The load balancer
-may communicate with the backend servers to collect load and health information.
-
-#### Security
+### Security
The load balancer may be separate from the actual server backends and a
compromise of the load balancer should only lead to a compromise of the
@@ -75,43 +76,60 @@ loadbalancing functionality. In other words, a compromised load balancer should
not be able to cause a client to trust a (potentially malicious) backend server
any more than in a comparable situation without loadbalancing.
-# Proposed Architecture
+# Architecture
+
+## Overview
+
+The primary mechanism for load-balancing in gRPC is external
+load-balancing, where an external load balancer provides simple clients
+with an up-to-date list of servers.
+
+The gRPC client does support an API for built-in load balancing policies.
+However, there are only a small number of these (one of which is the
+`grpclb` policy, which implements external load balancing), and users
+are discouraged from trying to extend gRPC by adding more. Instead, new
+load balancing policies should be implemented in external load balancers.
+
+## Load Balancing Policies
-The gRPC load balancing implements the external load balancing server approach:
-an external load balancer provides simple clients with an up-to-date list of
-servers.
+Load-balancing policies fit into the gRPC client workflow in between
+name resolution and the connection to the server. Here's how it all
+works:
![image](images/load_balancing_design.png)
-1. On startup, the gRPC client issues a name resolution request for the service.
- The name will resolve to one or more IP addresses to gRPC servers, a hint on
- whether the IP address(es) point to a load balancer or not, and also return a
- client config.
-2. The gRPC client connects to a gRPC Server.
- 1. If the name resolution has hinted that the endpoint is a load balancer,
- the client's gRPC LB policy will attempt to open a stream to the load
- balancer service. The server may respond in only one of the following
- ways.
- 1. `status::UNIMPLEMENTED`. There is no loadbalancing in use. The client
- call will fail.
- 2. "I am a Load Balancer and here is the server list." (Goto Step 4.)
- 3. "Please contact Load Balancer X" (See Step 3.) The client will close
- this connection and cancel the stream.
- 4. If the server fails to respond, the client will wait for some timeout
- and then re-resolve the name (process to Step 1 above).
- 2. If the name resolution has not hinted that the endpoint is a load
- balancer, the client connects directly to the service it wants to talk to.
-3. The gRPC client's gRPC LB policy opens a separate connection to the Load
- Balancer. If this fails, it will go back to step 1 and try another address.
- 1. During channel initialization to the Load Balancer, the client will
- attempt to open a stream to the Load Balancer service.
- 2. The Load Balancer will return a server list to the gRPC client. If the
- server list is empty, the call will wait until a non-empty one is
- received. Optional: The Load Balancer will also open channels to the gRPC
- servers if load reporting is needed.
-4. The gRPC client will send RPCs to the gRPC servers contained in the server
- list from the Load Balancer.
-5. Optional: The gRPC servers may periodically report load to the Load Balancer.
+1. On startup, the gRPC client issues a [name resolution](naming.md) request
+ for the server name. The name will resolve to one or more IP addresses,
+ each of which will indicate whether it is a server address or
+ a load balancer address, and a [service config](service_config.md)
+ that indicates which client-side load-balancing policy to use (e.g.,
+ `round_robin` or `grpclb`).
+2. The client instantiates the load balancing policy, which is then
+ responsible for deciding which requests will be sent to which
+ addresses.
+ - Note: If all addresses returned by the resolver are balancer
+ addresses, then the client will use the `grpclb` policy, regardless
+ of what load-balancing policy was requested by the service config.
+ Otherwise, the client will use the load-balancing policy requested
+ by the service config. If no load-balancing policy is requested
+ by the service config, then the client will default to a policy
+ that picks the first available server address.
+3. In the case of the `grpclb` policy, it opens a stream to one of the
+ balancer addresses returned by the resolver. It asks the balancer for
+ the server addresses to use for the server name originally requested by
+ the client (i.e., the same one originally passed to the name resolver).
+ - Note: Currently, the `grpclb` policy ignores any non-balancer
+ addresses returned by the resolver. However, in the future, it may
+ be changed to use these addresses as a fallback in case no balancers
+ can be contacted.
+4. The gRPC servers to which the load balancer is directing the client
+ may report load to the load balancers, if that information is needed
+ by the load balancer's configuration.
+5. The load balancer returns a server list to the gRPC client. If the
+ server list is empty, the call will block until a non-empty one is
+ received.
+6. The gRPC client will send RPCs to the gRPC servers contained in
+ the server list from the Load Balancer.
## Client