Merge remote-tracking branch 'upstream/master' into remove_initial_connect_string

6 files changed, 227 insertions, 88 deletions

diff --git a/doc/PROTOCOL-WEB.md b/doc/PROTOCOL-WEB.md
index 35448d683f..5f01af3627 100644
--- a/doc/PROTOCOL-WEB.md
+++ b/doc/PROTOCOL-WEB.md
@@ -39,6 +39,7 @@ Content-Type
   * e.g. application/grpc-web+[proto, json, thrift]
 2. application/grpc-web-text
   * text-encoded streams of “application/grpc-web”
+  * e.g. application/grpc-web-text+[proto, thrift]
 
 ---
 
@@ -60,10 +61,18 @@ HTTP/2 related behavior (specified in [gRPC over HTTP2](http://www.grpc.io/docs/
 Message framing (vs. [http2-transport-mapping](http://www.grpc.io/docs/guides/wire.html#http2-transport-mapping))
 
 1. Response status encoded as part of the response body
-  * Key-value pairs encoded as a HTTP/1 headers block (without the terminating newline).
+  * Key-value pairs encoded as a HTTP/1 headers block (without the terminating newline), per https://tools.ietf.org/html/rfc7230#section-3.2
+  ```
+    key1: foo\r\n
+    key2: bar\r\n
+  ```
 2. 8th (MSB) bit of the 1st gRPC frame byte
   * 0: data
   * 1: trailers
+  ```
+    10000000b: an uncompressed trailer (as part of the body)
+    10000001b: a compressed trailer
+  ```
 3. Trailers must be the last message of the response, as enforced
 by the implementation
 4. Trailers-only responses: no change to the gRPC protocol spec.
@@ -72,10 +81,9 @@ in the body.
 
 ---
 
-User Agent and Server headers
+User Agent
 
-* U-A: grpc-web-javascript/0.1
-* Server: grpc-web-gateway/0.1
+* U-A: grpc-web-javascript
 
 ---
 
@@ -86,14 +94,12 @@ the response stream needs to be text encoded e.g. when XHR is used or due
 to security policies with XHR
   * Accept: application/grpc-web-text
 2. The default text encoding is base64
-  * Text encoding may be specified with Content-Type or Accept headers as
-      * application/grpc-web-text-base64
   * Note that “Content-Transfer-Encoding: base64” should not be used.
   Due to in-stream base64 padding when delimiting messages, the entire
   response body is not necessarily a valid base64-encoded entity
   * While the server runtime will always base64-encode and flush gRPC messages
   atomically the client library should not assume base64 padding always
-  happens at the boundary of message frames.
+  happens at the boundary of message frames. That is, the implementation may send base64-encoded "chunks" with potential padding whenever the runtime needs to flush a byte buffer.
 3. For binary trailers, when the content-type is set to
 application/grpc-web-text, the extra base64 encoding specified
 in [gRPC over HTTP2](http://www.grpc.io/docs/guides/wire.html)
@@ -131,6 +137,10 @@ Security
 
 CORS preflight
 
+* Should follow the [CORS spec](https://developer.mozilla.org/en-US/docs/Web/HTTP/Server-Side_Access_Control)
+  * Access-Control-Allow-Credentials to allow Authorization headers
+  * Access-Control-Allow-Methods to allow POST and (preflight) OPTIONS only
+  * Access-Control-Allow-Headers to whatever the preflight request carries
 * The client library may support header overwrites to avoid preflight
   * https://github.com/whatwg/fetch/issues/210
 * CSP support to be specified
@@ -149,3 +159,10 @@ Bidi-streaming, with flow-control
 * Pending on [whatwg fetch/streams](https://github.com/whatwg/fetch) to be
 finalized and implemented in modern browsers
 * gRPC-Web client will support the native gRPC protocol with modern browsers
+
+---
+
+Versioning
+
+* Special headers may be introduced to support features that may break compatiblity.
+
diff --git a/doc/g_stands_for.md b/doc/g_stands_for.md
index 53a1fdf193..d2fc7a50f9 100644
--- a/doc/g_stands_for.md
+++ b/doc/g_stands_for.md
@@ -7,3 +7,4 @@ future), and the corresponding version numbers that used them:
 - 1.0 'g' stands for 'gRPC'
 - 1.1 'g' stands for 'good'
 - 1.2 'g' stands for 'green'
+- 1.3 'g' stands for 'gentle'
diff --git a/doc/internationalization.md b/doc/internationalization.md
new file mode 100644
index 0000000000..1b614cbd26
--- /dev/null
+++ b/doc/internationalization.md
@@ -0,0 +1,45 @@
+gRPC Internationalization
+=========================
+
+As a universal RPC framework, gRPC needs to be fully usable within/across different international environments. 
+This document describes gRPC API and behavior specifics when used in a non-english environment.
+
+## API Concepts
+
+While some API elements need to be able to represent non-english content, some are intentionally left as ASCII-only
+for simplicity & performance reasons.
+
+### Method name (in RPC Invocation)
+Method names are ASCII-only and may only contain characters allowed by HTTP/2 text header values. That should not
+be very limiting as most gRPC services will use protobuf which only allows method names from an even more restricted ASCII subset.
+Also, handling method names is a very hot code path so any additional encoding/decoding step is to be avoided.
+
+Recommended representation in language-specific APIs: string type.
+
+### Host name (in RPC Invocation)
+Host names are punycode encoded, but the user is responsible for providing the punycode-encoded string if she wishes to use an internationalized host name.
+
+Recommended representation in language-specific APIs: string/unicode string.
+
+NOTE: overriding host name when invoking RPCs is only supported by C-core based gRPC implementations.
+
+### Status detail/message (accompanies RPC status code)
+
+Status messages are expected to contain national-alphabet characters.
+Allowed values are unicode strings (content will be percent-encoded on the wire).
+
+Recommended representation in language-specific APIs: unicode string.
+
+### Metadata key
+Allowed values are defined by HTTP/2 standard (metadata keys are represented as HTTP/2 header/trailer names).
+
+Recommended representation in language-specific APIs: string.
+
+### Metadata value (text-valued metadata)
+Allowed values are defined by HTTP/2 standard (metadata values are represented as HTTP/2 header/trailer text values).
+
+Recommended representation in language-specific APIs: string.
+
+### Channel target (in channel creation)
+
+TBD
diff --git a/doc/server_side_auth.md b/doc/server_side_auth.md
new file mode 100644
index 0000000000..288c6e9cb3
--- /dev/null
+++ b/doc/server_side_auth.md
@@ -0,0 +1,60 @@
+Server-side API for Authenticating Clients
+==========================================
+
+NOTE: This document describes how server-side authentication works in C-core based gRPC implementations only. In gRPC Java and Go, server side authentication is handled differently.
+
+## AuthContext
+
+To perform server-side authentication, gRPC exposes the *authentication context* for each call. The context exposes important authentication-related information about the RPC such as the type of security/authentication type being used and the peer identity.
+
+The authentication context is structured as a multi-map of key-value pairs - the *auth properties*. In addition to that, for authenticated RPCs, the set of properties corresponding to a selected key will represent the verified identity of the caller - the *peer identity*.
+
+The contents of the *auth properties* are populated by an *auth interceptor*. The interceptor also chooses which property key will act as the peer identity (e.g. for client certificate authentication this property will be `"x509_common_name"` or `"x509_subject_alternative_name"`).
+
+WARNING: AuthContext is the only reliable source of truth when it comes to authenticating RPCs. Using any other call/context properties for authentication purposes is wrong and inherently unsafe.
+
+####Example AuthContext contents
+
+For secure channel using mutual TLS authentication with both client and server certificates (test certificates from this repository are used).
+
+Populated auth properties:
+```
+"transport_security_type": "ssl"  # connection is secured using TLS/SSL
+"x509_common_name": "*.test.google.com"  # from client's certificate
+"x509_pem_cert": "-----BEGIN CERTIFICATE-----\n..."  # client's PEM encoded certificate
+"x509_subject_alternative_name": "*.test.google.fr"
+"x509_subject_alternative_name": "waterzooi.test.google.be"
+"x509_subject_alternative_name": "*.test.youtube.com"
+"x509_subject_alternative_name": "192.168.1.3"
+```
+
+The peer identity is set of all properties named `"x509_subject_alternative_name"`:
+```
+peer_identity_property_name = "x509_subject_alternative_name"
+```
+
+## AuthProperty
+
+Auth properties are elements of the AuthContext. They have a name (a key of type string) and a value which can be a string or binary data.
+
+## Auth Interceptors
+
+Auth interceptors are gRPC components that populate contents of the auth context based on gRPC's internal state and/or call metadata.
+gRPC comes with some basic "interceptors" already built-in.
+
+WARNING: While there is a public API that allows anyone to write their own custom interceptor, please think twice before using it.
+There are legitimate uses for custom interceptors but you should keep in mind that as auth interceptors essentially decide which RPCs are authenticated and which are not, their code is very sensitive from the security perspective and getting things wrong might have serious consequences. If unsure, we strongly recommend to rely on official & proven interceptors that come with gRPC.
+
+####Available auth interceptors
+- TLS/SSL certificate authentication (built into gRPC's security layer, automatically used whenever you use a secure connection)
+- (coming soon) JWT auth token authentication
+- more will be added over time
+
+## Status (by language)
+C-core exposes low level API to access auth context contents and to implement an auth interceptor.
+In C++, the auth interceptor API is exposed as `AuthMetadataProcessor`.
+
+A high level API to access AuthContext contents is available in these languages:
+- C++
+- C# (implementation in-progress)
+- other languages coming soon
diff --git a/doc/service_config.md b/doc/service_config.md
index 2dabb83a37..ecc23817d1 100644
--- a/doc/service_config.md
+++ b/doc/service_config.md
@@ -12,105 +12,105 @@ The service config is a JSON string of the following form:
 
 ```
 {
-  # Load balancing policy name.
-  # Supported values are 'round_robin' and 'grpclb'.
-  # Optional; if unset, the default behavior is pick the first available
-  # backend.
-  # Note that if the resolver returns only balancer addresses and no
-  # backend addresses, gRPC will always use the 'grpclb' policy,
-  # regardless of what this field is set to.
+  // Load balancing policy name.
+  // Supported values are 'round_robin' and 'grpclb'.
+  // Optional; if unset, the default behavior is pick the first available
+  // backend.
+  // Note that if the resolver returns only balancer addresses and no
+  // backend addresses, gRPC will always use the 'grpclb' policy,
+  // regardless of what this field is set to.
   'loadBalancingPolicy': string,
 
-  # Per-method configuration.  Optional.
+  // Per-method configuration.  Optional.
   'methodConfig': [
     {
-      # The names of the methods to which this method config applies. There
-      # must be at least one name. Each name entry must be unique across the
-      # entire service config. If the 'method' field is empty, then this
-      # method config specifies the defaults for all methods for the specified
-      # service.
-      #
-      # For example, let's say that the service config contains the following
-      # method config entries:
-      #
-      # 'methodConfig': [
-      #   { 'name': [ { 'service': 'MyService' } ] ... },
-      #   { 'name': [ { 'service': 'MyService', 'method': 'Foo' } ] ... }
-      # ]
-      #
-      # For a request for MyService/Foo, we will use the second entry, because
-      # it exactly matches the service and method name.
-      # For a request for MyService/Bar, we will use the first entry, because
-      # it provides the default for all methods of MyService.
+      // The names of the methods to which this method config applies. There
+      // must be at least one name. Each name entry must be unique across the
+      // entire service config. If the 'method' field is empty, then this
+      // method config specifies the defaults for all methods for the specified
+      // service.
+      //
+      // For example, let's say that the service config contains the following
+      // method config entries:
+      //
+      // 'methodConfig': [
+      //   { 'name': [ { 'service': 'MyService' } ] ... },
+      //   { 'name': [ { 'service': 'MyService', 'method': 'Foo' } ] ... }
+      // ]
+      //
+      // For a request for MyService/Foo, we will use the second entry, because
+      // it exactly matches the service and method name.
+      // For a request for MyService/Bar, we will use the first entry, because
+      // it provides the default for all methods of MyService.
       'name': [
         {
-          # RPC service name.  Required.
-          # If using gRPC with protobuf as the IDL, then this will be of
-          # the form "pkg.service_name", where "pkg" is the package name
-          # defined in the proto file.
+          // RPC service name.  Required.
+          // If using gRPC with protobuf as the IDL, then this will be of
+          // the form "pkg.service_name", where "pkg" is the package name
+          // defined in the proto file.
           'service': string,
 
-          # RPC method name.  Optional (see above).
+          // RPC method name.  Optional (see above).
           'method': string,
         }
       ],
 
-      # Whether RPCs sent to this method should wait until the connection is
-      # ready by default. If false, the RPC will abort immediately if there
-      # is a transient failure connecting to the server. Otherwise, gRPC will
-      # attempt to connect until the deadline is exceeded.
-      #
-      # The value specified via the gRPC client API will override the value
-      # set here. However, note that setting the value in the client API will
-      # also affect transient errors encountered during name resolution,
-      # which cannot be caught by the value here, since the service config
-      # is obtained by the gRPC client via name resolution.
+      // Whether RPCs sent to this method should wait until the connection is
+      // ready by default. If false, the RPC will abort immediately if there
+      // is a transient failure connecting to the server. Otherwise, gRPC will
+      // attempt to connect until the deadline is exceeded.
+      //
+      // The value specified via the gRPC client API will override the value
+      // set here. However, note that setting the value in the client API will
+      // also affect transient errors encountered during name resolution,
+      // which cannot be caught by the value here, since the service config
+      // is obtained by the gRPC client via name resolution.
       'waitForReady': bool,
 
-      # The default timeout in seconds for RPCs sent to this method. This can
-      # be overridden in code. If no reply is received in the specified amount
-      # of time, the request is aborted and a deadline-exceeded error status
-      # is returned to the caller.
-      #
-      # The actual deadline used will be the minimum of the value specified
-      # here and the value set by the application via the gRPC client API.
-      # If either one is not set, then the other will be used.
-      # If neither is set, then the request has no deadline.
-      #
-      # The format of the value is that of the 'Duration' type defined here:
-      # https://developers.google.com/protocol-buffers/docs/proto3#json
+      // The default timeout in seconds for RPCs sent to this method. This can
+      // be overridden in code. If no reply is received in the specified amount
+      // of time, the request is aborted and a deadline-exceeded error status
+      // is returned to the caller.
+      //
+      // The actual deadline used will be the minimum of the value specified
+      // here and the value set by the application via the gRPC client API.
+      // If either one is not set, then the other will be used.
+      // If neither is set, then the request has no deadline.
+      //
+      // The format of the value is that of the 'Duration' type defined here:
+      // https://developers.google.com/protocol-buffers/docs/proto3#json
       'timeout': string,
 
-      # The maximum allowed payload size for an individual request or object
-      # in a stream (client->server) in bytes. The size which is measured is
-      # the serialized, uncompressed payload in bytes. This applies both
-      # to streaming and non-streaming requests.
-      #
-      # The actual value used is the minimum of the value specified here and
-      # the value set by the application via the gRPC client API.
-      # If either one is not set, then the other will be used.
-      # If neither is set, then the built-in default is used.
-      #
-      # If a client attempts to send an object larger than this value, it
-      # will not be sent and the client will see an error.
-      # Note that 0 is a valid value, meaning that the request message must
-      # be empty.
+      // The maximum allowed payload size for an individual request or object
+      // in a stream (client->server) in bytes. The size which is measured is
+      // the serialized, uncompressed payload in bytes. This applies both
+      // to streaming and non-streaming requests.
+      //
+      // The actual value used is the minimum of the value specified here and
+      // the value set by the application via the gRPC client API.
+      // If either one is not set, then the other will be used.
+      // If neither is set, then the built-in default is used.
+      //
+      // If a client attempts to send an object larger than this value, it
+      // will not be sent and the client will see an error.
+      // Note that 0 is a valid value, meaning that the request message must
+      // be empty.
       'maxRequestMessageBytes': number,
 
-      # The maximum allowed payload size for an individual response or object
-      # in a stream (server->client) in bytes. The size which is measured is
-      # the serialized, uncompressed payload in bytes. This applies both
-      # to streaming and non-streaming requests.
-      #
-      # The actual value used is the minimum of the value specified here and
-      # the value set by the application via the gRPC client API.
-      # If either one is not set, then the other will be used.
-      # If neither is set, then the built-in default is used.
-      #
-      # If a server attempts to send an object larger than this value, it
-      # will not be sent, and the client will see an error.
-      # Note that 0 is a valid value, meaning that the response message must
-      # be empty.
+      // The maximum allowed payload size for an individual response or object
+      // in a stream (server->client) in bytes. The size which is measured is
+      // the serialized, uncompressed payload in bytes. This applies both
+      // to streaming and non-streaming requests.
+      //
+      // The actual value used is the minimum of the value specified here and
+      // the value set by the application via the gRPC client API.
+      // If either one is not set, then the other will be used.
+      // If neither is set, then the built-in default is used.
+      //
+      // If a server attempts to send an object larger than this value, it
+      // will not be sent, and the client will see an error.
+      // Note that 0 is a valid value, meaning that the response message must
+      // be empty.
       'maxResponseMessageBytes': number
     }
   ]
diff --git a/doc/status_ordering.md b/doc/status_ordering.md
new file mode 100644
index 0000000000..fccfa863a3
--- /dev/null
+++ b/doc/status_ordering.md
@@ -0,0 +1,16 @@
+Ordering Status and Reads in the gRPC API
+-----------------------------------------
+
+Rules for implementors:
+1. Reads and Writes Must not succeed after Status has been delivered.
+2. OK Status is only delivered after all buffered messages are read.
+3. Reads May continue to succeed after a failing write.
+   However, once a write fails, all subsequent writes Must fail,
+   and similarly, once a read fails, all subsequent reads Must fail.
+4. When an error status is known to the library, if the user asks for status,
+   the library Should discard messages received in the library but not delivered
+   to the user and then deliver the status. If the user does not ask for status
+   but continues reading, the library Should deliver buffered messages before
+   delivering status. The library MAY choose to implement the stricter version
+   where errors cause all buffered messages to be dropped, but this is not a
+   requirement.