From 3cadfb152fee3160be689ee4545e15dfcf8ebd37 Mon Sep 17 00:00:00 2001 From: "Mark D. Roth" Date: Wed, 14 Dec 2016 07:52:47 -0800 Subject: Started updating docs. --- doc/load-balancing.md | 128 ++++++++++++++++++++++++++++---------------------- 1 file changed, 73 insertions(+), 55 deletions(-) (limited to 'doc/load-balancing.md') diff --git a/doc/load-balancing.md b/doc/load-balancing.md index dfaa7a7f33..03c9bd2cba 100644 --- a/doc/load-balancing.md +++ b/doc/load-balancing.md @@ -44,30 +44,31 @@ list of servers to the client. ### External Load Balancing Service The client load balancing code is kept simple and portable, implementing -well-known algorithms (ie, Round Robin) for server selection. -Complex load balancing algorithms are instead provided by the load balancer. The -client relies on the load balancer to provide _load balancing configuration_ and -_the list of servers_ to which the client should send requests. The balancer -updates the server list as needed to balance the load as well as handle server +well-known algorithms (e.g., Round Robin) for server selection. +Complex load balancing algorithms are instead provided by the load +balancer. The client relies on the load balancer to provide _load +balancing configuration_ and _the list of servers_ to which the client +should send requests. The balancer updates the server list as needed +to balance the load as well as handle server unavailability or health +issues. The load balancer will make any necessary complex decisions and +inform the client. The load balancer may communicate with the backend +servers to collect load and health information. + +# Requirements + +### Simple API and client + +The gRPC client load balancing code must be simple and portable. The +client should only contain simple algorithms (e.g., Round Robin) for +server selection. For complex algorithms, the client should rely on +a load balancer to provide load balancing configuration and the list of +servers to which the client should send requests. The balancer will update +the server list as needed to balance the load as well as handle server unavailability or health issues. The load balancer will make any necessary -complex decisions and inform the client. The load balancer may communicate with -the backend servers to collect load and health information. +complex decisions and inform the client. The load balancer may communicate +with the backend servers to collect load and health information. - -## Requirements - -#### Simple API and client - -The gRPC client load balancing code must be simple and portable. The client -should only contain simple algorithms (ie Round Robin) for server selection. For -complex algorithms, the client should rely on a load balancer to provide load -balancing configuration and the list of servers to which the client should send -requests. The balancer will update the server list as needed to balance the load -as well as handle server unavailability or health issues. The load balancer will -make any necessary complex decisions and inform the client. The load balancer -may communicate with the backend servers to collect load and health information. - -#### Security +### Security The load balancer may be separate from the actual server backends and a compromise of the load balancer should only lead to a compromise of the @@ -75,43 +76,60 @@ loadbalancing functionality. In other words, a compromised load balancer should not be able to cause a client to trust a (potentially malicious) backend server any more than in a comparable situation without loadbalancing. -# Proposed Architecture +# Architecture + +## Overview + +The primary mechanism for load-balancing in gRPC is external +load-balancing, where an external load balancer provides simple clients +with an up-to-date list of servers. + +The gRPC client does support an API for built-in load balancing policies. +However, there are only a small number of these (one of which is the +`grpclb` policy, which implements external load balancing), and users +are discouraged from trying to extend gRPC by adding more. Instead, new +load balancing policies should be implemented in external load balancers. + +## Load Balancing Policies -The gRPC load balancing implements the external load balancing server approach: -an external load balancer provides simple clients with an up-to-date list of -servers. +Load-balancing policies fit into the gRPC client workflow in between +name resolution and the connection to the server. Here's how it all +works: ![image](images/load_balancing_design.png) -1. On startup, the gRPC client issues a name resolution request for the service. - The name will resolve to one or more IP addresses to gRPC servers, a hint on - whether the IP address(es) point to a load balancer or not, and also return a - client config. -2. The gRPC client connects to a gRPC Server. - 1. If the name resolution has hinted that the endpoint is a load balancer, - the client's gRPC LB policy will attempt to open a stream to the load - balancer service. The server may respond in only one of the following - ways. - 1. `status::UNIMPLEMENTED`. There is no loadbalancing in use. The client - call will fail. - 2. "I am a Load Balancer and here is the server list." (Goto Step 4.) - 3. "Please contact Load Balancer X" (See Step 3.) The client will close - this connection and cancel the stream. - 4. If the server fails to respond, the client will wait for some timeout - and then re-resolve the name (process to Step 1 above). - 2. If the name resolution has not hinted that the endpoint is a load - balancer, the client connects directly to the service it wants to talk to. -3. The gRPC client's gRPC LB policy opens a separate connection to the Load - Balancer. If this fails, it will go back to step 1 and try another address. - 1. During channel initialization to the Load Balancer, the client will - attempt to open a stream to the Load Balancer service. - 2. The Load Balancer will return a server list to the gRPC client. If the - server list is empty, the call will wait until a non-empty one is - received. Optional: The Load Balancer will also open channels to the gRPC - servers if load reporting is needed. -4. The gRPC client will send RPCs to the gRPC servers contained in the server - list from the Load Balancer. -5. Optional: The gRPC servers may periodically report load to the Load Balancer. +1. On startup, the gRPC client issues a [name resolution](naming.md) request + for the server name. The name will resolve to one or more IP addresses, + each of which will indicate whether it is a server address or + a load balancer address, and a [service config](service_config.md) + that indicates which client-side load-balancing policy to use (e.g., + `round_robin` or `grpclb`). +2. The client instantiates the load balancing policy, which is then + responsible for deciding which requests will be sent to which + addresses. + - Note: If all addresses returned by the resolver are balancer + addresses, then the client will use the `grpclb` policy, regardless + of what load-balancing policy was requested by the service config. + Otherwise, the client will use the load-balancing policy requested + by the service config. If no load-balancing policy is requested + by the service config, then the client will default to a policy + that picks the first available server address. +3. In the case of the `grpclb` policy, it opens a stream to one of the + balancer addresses returned by the resolver. It asks the balancer for + the server addresses to use for the server name originally requested by + the client (i.e., the same one originally passed to the name resolver). + - Note: Currently, the `grpclb` policy ignores any non-balancer + addresses returned by the resolver. However, in the future, it may + be changed to use these addresses as a fallback in case no balancers + can be contacted. +4. The gRPC servers to which the load balancer is directing the client + may report load to the load balancers, if that information is needed + by the load balancer's configuration. +5. The load balancer returns a server list to the gRPC client. If the + server list is empty, the call will block until a non-empty one is + received. +6. The gRPC client will send RPCs to the gRPC servers contained in + the server list from the Load Balancer. ## Client -- cgit v1.2.3