diff options
Diffstat (limited to 'doc')
| -rw-r--r-- | doc/health-checking.md | 70 | ||||
| -rw-r--r-- | doc/interop-test-descriptions.md | 131 | ||||
| -rw-r--r-- | doc/naming.md | 52 | ||||
| -rw-r--r-- | doc/server-reflection.md | 183 |
4 files changed, 398 insertions, 38 deletions
diff --git a/doc/health-checking.md b/doc/health-checking.md new file mode 100644 index 0000000000..0b3f9c6a03 --- /dev/null +++ b/doc/health-checking.md @@ -0,0 +1,70 @@ +GRPC Health Checking Protocol +================================ + +Health checks are used to probe whether the server is able to handle rpcs. The +client-to-server health checking can happen from point to point or via some +control system. A server may choose to reply “unhealthy” because it +is not ready to take requests, it is shutting down or some other reason. +The client can act accordingly if the response is not received within some time +window or the response says unhealthy in it. + + +A GRPC service is used as the health checking mechanism for both simple +client-to-server scenario and other control systems such as load-balancing. +Being a high +level service provides some benefits. Firstly, since it is a GRPC service +itself, doing a health check is in the same format as a normal rpc. Secondly, +it has rich semantics such as per-service health status. Thirdly, as a GRPC +service, it is able reuse all the existing billing, quota infrastructure, etc, +and thus the server has full control over the access of the health checking +service. + +## Service Definition + +The server should export a service defined in the following proto: + +``` +syntax = "proto3"; + +package grpc.health.v1alpha; + +message HealthCheckRequest { + string service = 1; +} + +message HealthCheckResponse { + enum ServingStatus { + UNKNOWN = 0; + SERVING = 1; + NOT_SERVING = 2; + } + ServingStatus status = 1; +} + +service Health { + rpc Check(HealthCheckRequest) returns (HealthCheckResponse); +} +``` + +A client can query the server’s health status by calling the `Check` method, and +a deadline should be set on the rpc. The client can optionally set the service +name it wants to query for health status. The suggested format of service name +is `package_names.ServiceName`, such as `grpc.health.v1alpha.Health`. + +The server should register all the services manually and set +the individual status, including an empty service name and its status. For each +request received, if the service name can be found in the registry, +a response must be sent back with an `OK` status and the status field should be +set to `SERVING` or `NOT_SERVING` accordingly. If the service name is not +registered, the server returns a `NOT_FOUND` GRPC status. + +The server should use an empty string as the key for server’s +overall health status, so that a client not interested in a specific service can +query the server's status with an empty request. The server can just do exact +matching of the service name without support of any kind of wildcard matching. +However, the service owner has the freedom to implement more complicated +matching semantics that both the client and server agree upon. + +A client can declare the server as unhealthy if the rpc is not finished after +some amount of time. The client should be able to handle the case where server +does not have the Health service. diff --git a/doc/interop-test-descriptions.md b/doc/interop-test-descriptions.md index 065e107c24..84ceaa3081 100644 --- a/doc/interop-test-descriptions.md +++ b/doc/interop-test-descriptions.md @@ -55,7 +55,7 @@ Server features: Procedure: 1. Client calls EmptyCall with the default Empty message -Asserts: +Client asserts: * call was successful * response is non-null @@ -84,7 +84,7 @@ Procedure: } ``` -Asserts: +Client asserts: * call was successful * response payload type is COMPRESSABLE * response payload body is 314159 bytes in size @@ -110,6 +110,7 @@ Procedure: } } ``` + 3. Client then sends: ``` @@ -119,6 +120,7 @@ Procedure: } } ``` + 4. Client then sends: ``` @@ -128,6 +130,7 @@ Procedure: } } ``` + 5. Client then sends: ``` @@ -137,9 +140,10 @@ Procedure: } } ``` - 6. Client halfCloses -Asserts: + 6. Client half-closes + +Client asserts: * call was successful * response aggregated_payload_size is 74922 @@ -172,7 +176,7 @@ Procedure: } ``` -Asserts: +Client asserts: * call was successful * exactly four responses * response payloads are COMPRESSABLE @@ -202,6 +206,7 @@ Procedure: } } ``` + 2. After getting a reply, it sends: ``` @@ -215,6 +220,7 @@ Procedure: } } ``` + 3. After getting a reply, it sends: ``` @@ -228,6 +234,7 @@ Procedure: } } ``` + 4. After getting a reply, it sends: ``` @@ -242,7 +249,9 @@ Procedure: } ``` -Asserts: + 5. After getting a reply, client half-closes + +Client asserts: * call was successful * exactly four responses * response payloads are COMPRESSABLE @@ -261,7 +270,7 @@ Server features: Procedure: 1. Client calls FullDuplexCall and then half-closes -Asserts: +Client asserts: * call was successful * exactly zero responses @@ -300,7 +309,7 @@ Procedure: } ``` -Asserts: +Client asserts: * call was successful * received SimpleResponse.username equals the value of `--default_service_account` flag * received SimpleResponse.oauth_scope is in `--oauth_scope` @@ -328,7 +337,7 @@ Server features: * [Echo OAuth Scope][] Procedure: - 1. Client configures the channel to use ServiceAccountCredentials. + 1. Client configures the channel to use ServiceAccountCredentials 2. Client calls UnaryCall with: ``` @@ -343,7 +352,7 @@ Procedure: } ``` -Asserts: +Client asserts: * call was successful * received SimpleResponse.username is in the json key file read from `--service_account_key_file` @@ -370,7 +379,7 @@ Server features: * [Echo OAuth Scope][] Procedure: - 1. Client configures the channel to use JWTTokenCredentials. + 1. Client configures the channel to use JWTTokenCredentials 2. Client calls UnaryCall with: ``` @@ -384,7 +393,7 @@ Procedure: } ``` -Asserts: +Client asserts: * call was successful * received SimpleResponse.username is in the json key file read from `--service_account_key_file` @@ -422,7 +431,7 @@ Server features: Procedure: 1. Client uses the auth library to obtain an authorization token - 2. Client configures the channel to use AccessTokenCredentials with the access token obtained in step 1. + 2. Client configures the channel to use AccessTokenCredentials with the access token obtained in step 1 3. Client calls UnaryCall with the following message ``` @@ -431,8 +440,8 @@ Procedure: fill_oauth_scope: true } ``` - -Asserts: + +Client asserts: * call was successful * received SimpleResponse.username is in the json key file used by the auth library to obtain the authorization token @@ -464,10 +473,10 @@ Server features: Procedure: 1. Client uses the auth library to obtain an authorization token - 2. Client configures the channel with just SSL credentials. + 2. Client configures the channel with just SSL credentials 3. Client calls UnaryCall, setting per-call credentials to - AccessTokenCredentials with the access token obtained in step 1. The request is - the following message + AccessTokenCredentials with the access token obtained in step 1. The request + is the following message ``` { @@ -475,8 +484,8 @@ Procedure: fill_oauth_scope: true } ``` - -Asserts: + +Client asserts: * call was successful * received SimpleResponse.username is in the json key file used by the auth library to obtain the authorization token @@ -496,8 +505,14 @@ Server features: * [Echo Metadata][] Procedure: - 1. While sending custom metadata (ascii + binary) in the header, client calls - UnaryCall with: + 1. The client attaches custom metadata with the following keys and values: + + ``` + key: "x-grpc-test-echo-initial", value: "test_initial_metadata_value" + key: "x-grpc-test-echo-trailing-bin", value: 0xababab + ``` + + to a UnaryCall with request: ``` { @@ -508,23 +523,41 @@ Procedure: } } ``` -The client attaches custom metadata with the following keys and values: + + 2. The client attaches custom metadata with the following keys and values: + ``` key: "x-grpc-test-echo-initial", value: "test_initial_metadata_value" key: "x-grpc-test-echo-trailing-bin", value: 0xababab ``` - 2. Client repeats step 1. with FullDuplexCall instead of UnaryCall. -Asserts: + to a FullDuplexCall with request: + + ``` + { + response_type: COMPRESSABLE + response_size: 314159 + payload:{ + body: 271828 bytes of zeros + } + } + ``` + + and then half-closes + +Client asserts: * call was successful -* metadata with key `"x-grpc-test-echo-initial"` and value `"test_initial_metadata_value"`is received in the initial metadata. -* metadata with key `"x-grpc-test-echo-trailing-bin"` and value `0xababab` is received in the trailing metadata. +* metadata with key `"x-grpc-test-echo-initial"` and value + `"test_initial_metadata_value"`is received in the initial metadata for calls + in Procedure steps 1 and 2. +* metadata with key `"x-grpc-test-echo-trailing-bin"` and value `0xababab` is + received in the trailing metadata for calls in Procedure steps 1 and 2. ### status_code_and_message -This test verifies unary calls succeed in sending messages, and propagates back +This test verifies unary calls succeed in sending messages, and propagate back status code and message sent along with the messages. Server features: @@ -543,12 +576,26 @@ Procedure: } } ``` -2. Client repeats step 1. with FullDuplexCall instead of UnaryCall. + 2. Client calls FullDuplexCall with: + + ``` + { + response_status:{ + code: 2 + message: "test status message" + } + } + ``` + + and then half-closes -Asserts: -* received status code is the same with sent code -* received status message is the same with sent message + +Client asserts: +* received status code is the same as the sent code for both Procedure steps 1 + and 2 +* received status message is the same as the sent message for both Procedure + steps 1 and 2 ### unimplemented_method @@ -556,15 +603,19 @@ Status: Ready for implementation. Blocking beta. This test verifies calling unimplemented RPC method returns the UNIMPLEMENTED status code. +Server features: +N/A + Procedure: -* Client calls `grpc.testing.UnimplementedService/UnimplementedCall` with an empty request (defined as `grpc.testing.Empty`): +* Client calls `grpc.testing.UnimplementedService/UnimplementedCall` with an + empty request (defined as `grpc.testing.Empty`): ``` { } ``` -Asserts: +Client asserts: * received status code is 12 (UNIMPLEMENTED) * received status message is empty or null/unset @@ -580,7 +631,7 @@ Procedure: 1. Client starts StreamingInputCall 2. Client immediately cancels request -Asserts: +Client asserts: * Call completed with status CANCELLED ### cancel_after_first_response @@ -606,9 +657,10 @@ Procedure: } } ``` + 2. After receiving a response, client cancels request -Asserts: +Client asserts: * Call completed with status CANCELLED ### timeout_on_sleeping_server @@ -620,7 +672,8 @@ Server features: * [FullDuplexCall][] Procedure: - 1. Client calls FullDuplexCall with the following request and sets its timeout to 1ms. + 1. Client calls FullDuplexCall with the following request and sets its timeout + to 1ms ``` { @@ -630,7 +683,9 @@ Procedure: } ``` -Asserts: + 2. Client waits + +Client asserts: * Call completed with status DEADLINE_EXCEEDED. ### concurrent_large_unary diff --git a/doc/naming.md b/doc/naming.md new file mode 100644 index 0000000000..5ad7e6622e --- /dev/null +++ b/doc/naming.md @@ -0,0 +1,52 @@ +#gRPC Naming and Discovery Support + +## Overview + +gRPC supports DNS as the default name-system. A number of alternative name-systems are used in various deployments. We propose an API that is general enough to support a range of name-systems and the corresponding syntax for names. The gRPC client library in various languages will provide a plugin mechanism so resolvers for different name-systems can be plugged in. + +## Detailed Proposal + + A fully qualified, self contained name used for gRPC channel construction uses the syntax: + +``` +scheme://authority/endpoint_name +``` + +Here, scheme indicates the name-system to be used. Example schemes to be supported include: + +* `dns` + +* `zookeeper` + +* `etcd` + +Authority indicates some scheme-specific bootstrap information, e.g., for DNS, the authority may include the IP[:port] of the DNS server to use. Often, a DNS name may used as the authority, since the ability to resolve DNS names is already built into all gRPC client libraries. + +Finally, the endpoint_name indicates a concrete name to be looked up in a given name-system identified by the scheme and the authority. The syntax of endpoint name is dictated by the scheme in use. + +### Plugins + +The gRPC client library will switch on the scheme to pick the right resolver plugin and pass it the fully qualified name string. + +Resolvers should be able to contact the authority and get a resolution that they return back to the gRPC client library. The returned contents include a list of IP:port, an optional config and optional auth config data to be used for channel authentication. The plugin API allows the resolvers to continuously watch an endpoint_name and return updated resolutions as needed. + +## Zookeeper + +Apache [ZooKeeper](https://zookeeper.apache.org/) is a popular solution for building name-systems. Curator is a service discovery system built on to of ZooKeeper. We propose to organize names hierarchically as `/path/service/instance` similar to Apache Curator. + +A fully-qualified ZooKeeper name used to construct a gRPC channel will look as follows: + +``` +zookeeper://host:port/path/service/instance +``` +Here `zookeeper` is the scheme identifying the name-system. `host:port` identifies an authoritative name-server for this scheme (i.e., a Zookeeper server). The host can be an IP address or a DNS name. +Finally `/path/service/instance` is the Zookeeper name to be resolved. + +## Service Registration + + +Service providers can register their services in Zookeeper by using a Zookeeper client. + +Each service is a zookeeper node, and each instance is a child node of the corresponding service. For example, a MySQL service may have multiple instances, `/mysql/1`, `/mysql/2`, `/mysql/3`. The name of the service or instance, as well as an optional path is specified by the service provider. + +The data in service nodes is empty. Each instance node stores its address in the format of `host:port`, where host can be either hostname or IP address. diff --git a/doc/server-reflection.md b/doc/server-reflection.md new file mode 100644 index 0000000000..cceee1647f --- /dev/null +++ b/doc/server-reflection.md @@ -0,0 +1,183 @@ +GRPC Server Reflection Protocol +=============================== + +This document describes server reflection as an optional extension for servers +to assist clients in runtime construction of requests without having stub +information precompiled into the client. + +The primary usecase for server reflection is to write (typically) command line +debugging tools for talking to a grpc server. In particular, such a tool will +take in a method and a payload (in human readable text format) send it to the +server (typically in binary proto wire format), and then take the response and +decode it to text to present to the user. + +This broadly involves two problems: determining what formats (which protobuf +messages) a server’s method uses, and determining how to convert messages +between human readable format and the (likely binary) wire format. + +## Method reflection + +We want to be able to answer the following queries: + 1. What methods does a server export? + 2. For a particular method, how do we call it? +Specifically, what are the names of the methods, are those methods unary or +streaming, and what are the types of the argument and result? + +``` +#TODO(dklempner): link to an actual .proto later. +package grpc.reflection.v1alpha; + +message ListApisRequest { +} + +message ListApisResponse { + repeated google.protobuf.Api apis = 1; +} + +message GetMethodRequest { + string method = 1; +} +message GetMethodResponse { + google.protobuf.Method method = 1; +} + +service ServerReflection { + rpc ListApis (ListApisRequest) returns (ListApisResponse); + rpc GetMethod (GetMethodRequest) returns (GetMethodResponse); +} +``` + +Note that a server is under no obligation to return a complete list of all +methods it supports. For example, a reverse proxy may support server reflection +for methods implemented directly on the proxy but not enumerate all methods +supported by its backends. + + +### Open questions on method reflection + * Consider how to extend this protocol to support non-protobuf methods. + +## Argument reflection +The second half of the problem is converting between the human readable +input/output of a debugging tool and the binary format understood by the +method. + +This is obviously dependent on protocol type. At one extreme, if both the +server and the debugging tool accept JSON, there may be no need for such a +conversion in the first place. At the opposite extreme, a server using a custom +binary format has no hope of being supported by a generic system. The +intermediate interesting common case is a server which speaks binary-proto and +a debugging client which speaks either ascii-proto or json-proto. + +One approach would be to require servers directly support human readable input. +In the future method reflection may be extended to document such support, +should it become widespread or standardized. + +## Protobuf descriptors + +A second would be for the server to export its +google::protobuf::DescriptorDatabase over the wire. This is very easy to +implement in C++, and Google implementations of a similar protocol already +exist in C++, Go, and Java. + +This protocol mostly returns FileDescriptorProtos, which are a proto encoding +of a parsed .proto file. It supports four queries: + 1. The FileDescriptorProto for a given file name + 2. The FileDescriptorProto for the file with a given symbol + 3. The FileDescriptorProto for the file with a given extension + 4. The list of known extension tag numbers of a given type + +These directly correspond to the methods of +google::protobuf::DescriptorDatabase. Note that this protocol includes support +for extensions, which have been removed from proto3 but are still in widespread +use in Google’s codebase. + +Because most usecases will require also requesting the transitive dependencies +of requested files, the queries will also return all transitive dependencies of +the returned file. Should interesting usecases for non-transitive queries turn +up later, we can easily extend the protocol to support them. + +### Reverse proxy traversal + +One potential issue with naive reverse proxies is that, while any individual +server will have a consistent and valid picture of the proto DB which is +sufficient to handle incoming requests, incompatibilities will arise if the +backend servers have a mix of builds. For example, if a given message is moved +from foo.proto to bar.proto, and the client requests foo.proto from an old +server and bar.proto from a new server, the resulting database will have a +double definition. + +To solve this problem, the protocol is structured as a bidirectional stream, +ensuring all related requests go to a single server. This has the additional +benefit that overlapping recursive requests don’t require sending a lot of +redundant information, because there is a single stream to maintain context +between queries. + +``` +package grpc.reflection.v1alpha; +message DescriptorDatabaseRequest { + string host = 1; + oneof message_request { + string files_for_file_name = 3; + string files_for_symbol_name = 4; + FileContainingExtensionRequest file_containing_extension = 5; + string list_all_extensions_of_type = 6; + } +} + +message FileContainingExtensionRequest { + string base_message = 1; + int64 extension_id = 2; +} + +message DescriptorDatabaseResponse { + string valid_host = 1; + DescriptorDatabaseRequest original_request = 2; + oneof message_response { + // These are proto2 type google.protobuf.FileDescriptorProto, but + // we avoid taking a dependency on descriptor.proto, which uses + // proto2 only features, by making them opaque + // bytes instead + repeated bytes fd_proto = 4; + ListAllExtensionsResponse extensions_response = 5; + // Notably includes error code 5, NOT FOUND + int32 error_code = 6; + } +} + +message ListAllExtensionsResponse { + string base_type_name; + repeated int64 extension_number; +} + +service ProtoDescriptorDatabase { + rpc DescriptorDatabaseInfo(stream DescriptorDatabaseRequest) returns (stream DescriptorDatabaseResponse); +} +``` + +Any given request must either result in an error code or an answer, usually in +the form of a series of FileDescriptorProtos with the requested file itself +and all previously unsent transitive imports of that file. Servers may track +which FileDescriptorProtos have been sent on a given stream, for a given value +of valid_host, and avoid sending them repeatedly for overlapping requests. + +| message_request message | Result | +| files_for_file_name | transitive closure of file name | +| files_for_symbol_name | transitive closure file containing symbol | +| file_containing_extension | transitive closure of file containing a given extension number of a given symbol | +| list_all_extensions_of_type | ListAllExtensionsResponse containing all known extension numbers of a given type | + +At some point it would make sense to additionally also support any.proto’s +format. Note that known any.proto messages can be queried by symbol using this +protocol even without any such support, by parsing the url and extracting the +symbol name from it. + +## Language specific implementation thoughts +All of the information needed to implement Proto reflection is available to the +code generator, but I’m not certain we actually generate this in every +language. If the proto implementation in the language doesn’t have something +like google::protobuf::DescriptorPool the grpc implementation for that language +will need to index those FileDescriptorProtos by file and symbol and imports. + +One issue is that some grpc implementations are very loosely coupled with +protobufs; in such implementations it probably makes sense to split apart these +reflection APIs so as not to take an additional proto dependency. |