From ec5e730d47561d14045747031f824216a694f3dc Mon Sep 17 00:00:00 2001 From: Jan Tattermusch Date: Wed, 22 Feb 2017 05:41:46 +0100 Subject: Create internationalization.md --- doc/internationalization.md | 44 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 44 insertions(+) create mode 100644 doc/internationalization.md (limited to 'doc') diff --git a/doc/internationalization.md b/doc/internationalization.md new file mode 100644 index 0000000000..a39a5eef89 --- /dev/null +++ b/doc/internationalization.md @@ -0,0 +1,44 @@ +gRPC Internationalization +========================= + +As a universal RPC framework, gRPC needs to be fully usable within/across different international environments. +This document describes gRPC API and behavior specifics when used in a non-english environment. + +## API Concepts + +While some API elements need to be able to represent non-english content, some are intentionally left as ASCII-only +for simplicity & performance reasons. + +### Method name (in RPC Invocation) +Method names are ASCII-only. Most gRPC services will use protobuf which only allows ASCII based method names anyway. +Also, handling method names is a very hot code path. + +Recommended representation in language-specific APIs: string type. + +### Host name (in RPC Invocation) +Host names are punycode encoded. Currently, the punycode needs to be provided by the user. + +Recommended representation in language-specific APIs: string/unicode string. + +NOTE: overriding host name when invoking RPCs is only supported by C-core based gRPC implementations. + +### Status detail/message (accompanies RPC status code) + +Status messages are expected to contain national-alphabet characters. +Allowed values are unicode strings (content will be percent-encoded on the wire). + +Recommended representation in language-specific APIs: unicode string. + +### Metadata key +Allowed values are defined by HTTP/2 standard (metadata keys are represented as HTTP/2 header/trailer names). + +Recommended representation in language-specific APIs: string. + +### Metadata value (text-valued metadata) +Allowed values are defined by HTTP/2 standard (metadata values are represented as HTTP/2 header/trailer text values). + +Recommended representation in language-specific APIs: string. + +### Channel name + +TBD -- cgit v1.2.3 From 012fc00ada6504b75fff5da2cc5bde53344236ab Mon Sep 17 00:00:00 2001 From: Jan Tattermusch Date: Wed, 22 Feb 2017 15:18:34 +0100 Subject: Address comments. --- doc/internationalization.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) (limited to 'doc') diff --git a/doc/internationalization.md b/doc/internationalization.md index a39a5eef89..b82785207b 100644 --- a/doc/internationalization.md +++ b/doc/internationalization.md @@ -10,8 +10,9 @@ While some API elements need to be able to represent non-english content, some a for simplicity & performance reasons. ### Method name (in RPC Invocation) -Method names are ASCII-only. Most gRPC services will use protobuf which only allows ASCII based method names anyway. -Also, handling method names is a very hot code path. +Method names are ASCII-only and may only contain characters allowed by HTTP/2 text header values. That should not +be very limiting as most gRPC services will use protobuf which only allows method names from an even more restricted ASCII subset. +Also, handling method names is a very hot code path so any additional encoding/decoding step is to be avoided. Recommended representation in language-specific APIs: string type. @@ -39,6 +40,6 @@ Allowed values are defined by HTTP/2 standard (metadata values are represented a Recommended representation in language-specific APIs: string. -### Channel name +### Channel target (in channel creation) TBD -- cgit v1.2.3 From 31c64171fda313d4432c57cc9c48627c18f0e576 Mon Sep 17 00:00:00 2001 From: Jan Tattermusch Date: Mon, 27 Feb 2017 11:12:50 +0100 Subject: Update host name i18n docs. --- doc/internationalization.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc') diff --git a/doc/internationalization.md b/doc/internationalization.md index b82785207b..1b614cbd26 100644 --- a/doc/internationalization.md +++ b/doc/internationalization.md @@ -17,7 +17,7 @@ Also, handling method names is a very hot code path so any additional encoding/d Recommended representation in language-specific APIs: string type. ### Host name (in RPC Invocation) -Host names are punycode encoded. Currently, the punycode needs to be provided by the user. +Host names are punycode encoded, but the user is responsible for providing the punycode-encoded string if she wishes to use an internationalized host name. Recommended representation in language-specific APIs: string/unicode string. -- cgit v1.2.3 From 0258444a81347d16f47cdb19445d12308184ffb9 Mon Sep 17 00:00:00 2001 From: Wenbo Zhu Date: Mon, 27 Feb 2017 14:21:42 -0800 Subject: Update PROTOCOL-WEB.md Addressed @mwitkow comments. https://github.com/grpc/grpc-web/issues/57 --- doc/PROTOCOL-WEB.md | 27 +++++++++++++++++++++++---- 1 file changed, 23 insertions(+), 4 deletions(-) (limited to 'doc') diff --git a/doc/PROTOCOL-WEB.md b/doc/PROTOCOL-WEB.md index 35448d683f..b7a930a7d4 100644 --- a/doc/PROTOCOL-WEB.md +++ b/doc/PROTOCOL-WEB.md @@ -39,6 +39,7 @@ Content-Type * e.g. application/grpc-web+[proto, json, thrift] 2. application/grpc-web-text * text-encoded streams of “application/grpc-web” + * e.g. application/grpc-web+[proto, thrift] --- @@ -61,9 +62,17 @@ Message framing (vs. [http2-transport-mapping](http://www.grpc.io/docs/guides/wi 1. Response status encoded as part of the response body * Key-value pairs encoded as a HTTP/1 headers block (without the terminating newline). + ``` + key1: foo\r\n + key2: bar\r\n + ``` 2. 8th (MSB) bit of the 1st gRPC frame byte * 0: data * 1: trailers + ``` + 10000000b: an uncompressed trailer (as part of the body) + 10000001b: a compressed trailer + ``` 3. Trailers must be the last message of the response, as enforced by the implementation 4. Trailers-only responses: no change to the gRPC protocol spec. @@ -72,10 +81,9 @@ in the body. --- -User Agent and Server headers +User Agent -* U-A: grpc-web-javascript/0.1 -* Server: grpc-web-gateway/0.1 +* U-A: grpc-web-javascript --- @@ -93,7 +101,7 @@ to security policies with XHR response body is not necessarily a valid base64-encoded entity * While the server runtime will always base64-encode and flush gRPC messages atomically the client library should not assume base64 padding always - happens at the boundary of message frames. + happens at the boundary of message frames. That is, the implementation may send base64-encoded "chunks" with potential padding whenever the runtime needs to flush a byte buffer. 3. For binary trailers, when the content-type is set to application/grpc-web-text, the extra base64 encoding specified in [gRPC over HTTP2](http://www.grpc.io/docs/guides/wire.html) @@ -131,6 +139,10 @@ Security CORS preflight +* Should follow the [CORS spec](https://developer.mozilla.org/en-US/docs/Web/HTTP/Server-Side_Access_Control) + * Access-Control-Allow-Credentials to allow Authorization headers + * Access-Control-Allow-Methods to allow POST and (preflight) OPTIONS only + * Access-Control-Allow-Headers to whatever the preflight request carries * The client library may support header overwrites to avoid preflight * https://github.com/whatwg/fetch/issues/210 * CSP support to be specified @@ -149,3 +161,10 @@ Bidi-streaming, with flow-control * Pending on [whatwg fetch/streams](https://github.com/whatwg/fetch) to be finalized and implemented in modern browsers * gRPC-Web client will support the native gRPC protocol with modern browsers + +--- + +Versioning + +* Special headers may be introduced to support features that may break compatiblity. + -- cgit v1.2.3 From ea16fa1598efcd29308cf5d2745a032bce0947f0 Mon Sep 17 00:00:00 2001 From: Wenbo Zhu Date: Mon, 27 Feb 2017 16:20:43 -0800 Subject: Update PROTOCOL-WEB.md --- doc/PROTOCOL-WEB.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc') diff --git a/doc/PROTOCOL-WEB.md b/doc/PROTOCOL-WEB.md index b7a930a7d4..cdc0531dd3 100644 --- a/doc/PROTOCOL-WEB.md +++ b/doc/PROTOCOL-WEB.md @@ -39,7 +39,7 @@ Content-Type * e.g. application/grpc-web+[proto, json, thrift] 2. application/grpc-web-text * text-encoded streams of “application/grpc-web” - * e.g. application/grpc-web+[proto, thrift] + * e.g. application/grpc-web-text+[proto, thrift] --- -- cgit v1.2.3 From a56edc43f404e2fddbe0d38d28c0189444e8a616 Mon Sep 17 00:00:00 2001 From: Wenbo Zhu Date: Mon, 27 Feb 2017 16:21:45 -0800 Subject: Update PROTOCOL-WEB.md --- doc/PROTOCOL-WEB.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) (limited to 'doc') diff --git a/doc/PROTOCOL-WEB.md b/doc/PROTOCOL-WEB.md index cdc0531dd3..0b54e0f34f 100644 --- a/doc/PROTOCOL-WEB.md +++ b/doc/PROTOCOL-WEB.md @@ -61,7 +61,7 @@ HTTP/2 related behavior (specified in [gRPC over HTTP2](http://www.grpc.io/docs/ Message framing (vs. [http2-transport-mapping](http://www.grpc.io/docs/guides/wire.html#http2-transport-mapping)) 1. Response status encoded as part of the response body - * Key-value pairs encoded as a HTTP/1 headers block (without the terminating newline). + * Key-value pairs encoded as a HTTP/1 headers block (without the terminating newline), per https://tools.ietf.org/html/rfc7230#section-3.2 ``` key1: foo\r\n key2: bar\r\n -- cgit v1.2.3 From 4ead63723618d34a5d6f2c9f20f2c4cc0c7e8c1d Mon Sep 17 00:00:00 2001 From: Jan Tattermusch Date: Fri, 10 Feb 2017 20:06:30 -0800 Subject: Add high-level server-side auth docs --- doc/server_side_auth.md | 60 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 60 insertions(+) create mode 100644 doc/server_side_auth.md (limited to 'doc') diff --git a/doc/server_side_auth.md b/doc/server_side_auth.md new file mode 100644 index 0000000000..288c6e9cb3 --- /dev/null +++ b/doc/server_side_auth.md @@ -0,0 +1,60 @@ +Server-side API for Authenticating Clients +========================================== + +NOTE: This document describes how server-side authentication works in C-core based gRPC implementations only. In gRPC Java and Go, server side authentication is handled differently. + +## AuthContext + +To perform server-side authentication, gRPC exposes the *authentication context* for each call. The context exposes important authentication-related information about the RPC such as the type of security/authentication type being used and the peer identity. + +The authentication context is structured as a multi-map of key-value pairs - the *auth properties*. In addition to that, for authenticated RPCs, the set of properties corresponding to a selected key will represent the verified identity of the caller - the *peer identity*. + +The contents of the *auth properties* are populated by an *auth interceptor*. The interceptor also chooses which property key will act as the peer identity (e.g. for client certificate authentication this property will be `"x509_common_name"` or `"x509_subject_alternative_name"`). + +WARNING: AuthContext is the only reliable source of truth when it comes to authenticating RPCs. Using any other call/context properties for authentication purposes is wrong and inherently unsafe. + +####Example AuthContext contents + +For secure channel using mutual TLS authentication with both client and server certificates (test certificates from this repository are used). + +Populated auth properties: +``` +"transport_security_type": "ssl" # connection is secured using TLS/SSL +"x509_common_name": "*.test.google.com" # from client's certificate +"x509_pem_cert": "-----BEGIN CERTIFICATE-----\n..." # client's PEM encoded certificate +"x509_subject_alternative_name": "*.test.google.fr" +"x509_subject_alternative_name": "waterzooi.test.google.be" +"x509_subject_alternative_name": "*.test.youtube.com" +"x509_subject_alternative_name": "192.168.1.3" +``` + +The peer identity is set of all properties named `"x509_subject_alternative_name"`: +``` +peer_identity_property_name = "x509_subject_alternative_name" +``` + +## AuthProperty + +Auth properties are elements of the AuthContext. They have a name (a key of type string) and a value which can be a string or binary data. + +## Auth Interceptors + +Auth interceptors are gRPC components that populate contents of the auth context based on gRPC's internal state and/or call metadata. +gRPC comes with some basic "interceptors" already built-in. + +WARNING: While there is a public API that allows anyone to write their own custom interceptor, please think twice before using it. +There are legitimate uses for custom interceptors but you should keep in mind that as auth interceptors essentially decide which RPCs are authenticated and which are not, their code is very sensitive from the security perspective and getting things wrong might have serious consequences. If unsure, we strongly recommend to rely on official & proven interceptors that come with gRPC. + +####Available auth interceptors +- TLS/SSL certificate authentication (built into gRPC's security layer, automatically used whenever you use a secure connection) +- (coming soon) JWT auth token authentication +- more will be added over time + +## Status (by language) +C-core exposes low level API to access auth context contents and to implement an auth interceptor. +In C++, the auth interceptor API is exposed as `AuthMetadataProcessor`. + +A high level API to access AuthContext contents is available in these languages: +- C++ +- C# (implementation in-progress) +- other languages coming soon -- cgit v1.2.3 From 880b7981d85ef134ebb6fb5831669eb1e4f46821 Mon Sep 17 00:00:00 2001 From: Wenbo Zhu Date: Thu, 2 Mar 2017 17:41:36 -0800 Subject: Update PROTOCOL-WEB.md Remove text-base64 C-T. Will revisit if we ever need support non-default (base64) encoding. --- doc/PROTOCOL-WEB.md | 2 -- 1 file changed, 2 deletions(-) (limited to 'doc') diff --git a/doc/PROTOCOL-WEB.md b/doc/PROTOCOL-WEB.md index 0b54e0f34f..5f01af3627 100644 --- a/doc/PROTOCOL-WEB.md +++ b/doc/PROTOCOL-WEB.md @@ -94,8 +94,6 @@ the response stream needs to be text encoded e.g. when XHR is used or due to security policies with XHR * Accept: application/grpc-web-text 2. The default text encoding is base64 - * Text encoding may be specified with Content-Type or Accept headers as - * application/grpc-web-text-base64 * Note that “Content-Transfer-Encoding: base64” should not be used. Due to in-stream base64 padding when delimiting messages, the entire response body is not necessarily a valid base64-encoded entity -- cgit v1.2.3 From 8918aaeccd4cd22cc6e549dcb3ddc0d661993e97 Mon Sep 17 00:00:00 2001 From: Craig Tiller Date: Mon, 6 Mar 2017 15:59:08 -0800 Subject: Document status ordering rules This documents a rule that's existed in a hard to find internal document that's existed since Feb 2016 by abhikumar@google.com. Since that rule is critical to untangling some gRPC C core behavior, we should document it publically. --- doc/status_ordering.md | 16 ++++++++++++++++ test/core/end2end/tests/streaming_error_response.c | 3 +++ 2 files changed, 19 insertions(+) create mode 100644 doc/status_ordering.md (limited to 'doc') diff --git a/doc/status_ordering.md b/doc/status_ordering.md new file mode 100644 index 0000000000..fccfa863a3 --- /dev/null +++ b/doc/status_ordering.md @@ -0,0 +1,16 @@ +Ordering Status and Reads in the gRPC API +----------------------------------------- + +Rules for implementors: +1. Reads and Writes Must not succeed after Status has been delivered. +2. OK Status is only delivered after all buffered messages are read. +3. Reads May continue to succeed after a failing write. + However, once a write fails, all subsequent writes Must fail, + and similarly, once a read fails, all subsequent reads Must fail. +4. When an error status is known to the library, if the user asks for status, + the library Should discard messages received in the library but not delivered + to the user and then deliver the status. If the user does not ask for status + but continues reading, the library Should deliver buffered messages before + delivering status. The library MAY choose to implement the stricter version + where errors cause all buffered messages to be dropped, but this is not a + requirement. diff --git a/test/core/end2end/tests/streaming_error_response.c b/test/core/end2end/tests/streaming_error_response.c index 42055907c8..2b9c404b15 100644 --- a/test/core/end2end/tests/streaming_error_response.c +++ b/test/core/end2end/tests/streaming_error_response.c @@ -31,6 +31,9 @@ * */ +/** \file Verify that status ordering rules are obeyed. + \ref doc/status_ordering.md */ + #include "test/core/end2end/end2end_tests.h" #include -- cgit v1.2.3 From 008c1f09115b23196d192d9ddd92b431a824edd7 Mon Sep 17 00:00:00 2001 From: "Nicolas \"Pixel\" Noble" Date: Tue, 14 Mar 2017 21:12:44 +0100 Subject: Updating g_stands_for.md --- doc/g_stands_for.md | 1 + 1 file changed, 1 insertion(+) (limited to 'doc') diff --git a/doc/g_stands_for.md b/doc/g_stands_for.md index 53a1fdf193..d2fc7a50f9 100644 --- a/doc/g_stands_for.md +++ b/doc/g_stands_for.md @@ -7,3 +7,4 @@ future), and the corresponding version numbers that used them: - 1.0 'g' stands for 'gRPC' - 1.1 'g' stands for 'good' - 1.2 'g' stands for 'green' +- 1.3 'g' stands for 'gentle' -- cgit v1.2.3