diff options
41 files changed, 474 insertions, 138 deletions
diff --git a/doc/connection-backoff-interop-test-description.md b/doc/connection-backoff-interop-test-description.md new file mode 100644 index 0000000000..0f00c86dca --- /dev/null +++ b/doc/connection-backoff-interop-test-description.md @@ -0,0 +1,77 @@ +Connection Backoff Interop Test Descriptions +=============================================== + +This test is to verify the client is reconnecting the server with correct +backoffs as specified in +[the spec](http://github.com/grpc/grpc/blob/master/doc/connection-backoff.md). +The test server has a port (control_port) running a rpc service for controlling +the server and another port (retry_port) to close any incoming tcp connections. +The test has the following flow: + +1. The server starts listening on control_port. +2. The client calls Start rpc on server control_port. +3. The server starts listening on retry_port. +4. The client connects to server retry_port and retries with backoff for 540s, +which translates to about 13 retries. +5. The client calls Stop rpc on server control port. +6. The client checks the response to see whether the server thinks the backoffs +are conforming the spec or do its own check on the backoffs in the response. + +Client and server use +[test.proto](https://github.com/grpc/grpc/blob/master/test/proto/test.proto). +Each language should implement its own client. The C++ server is shared among +languages. + +Client +------ + +Clients should accept these arguments: +* --server_control_port=PORT + * The server port to connect to for rpc. For example, "8080" +* --server_retry_port=PORT + * The server port to connect to for testing backoffs. For example, "8081" + +The client must connect to the control port without TLS. The client should +either assert on the server returned backoff status or check the returned +backoffs on its own. + +Procedure of client: + +1. Calls Start on server control port with a large deadline or no deadline, +waits for its finish and checks it succeeded. +2. Initiates a channel connection to server retry port, which should perform +reconnections with proper backoffs. A convienent way to achieve this is to +call Start with a deadline of 540s. The rpc should fail with deadline exceeded. +3. Calls Stop on server control port and checks it succeeded. +4. Checks the response to see whether the server thinks the backoffs passed the + test. +5. Optionally, the client can do its own check on the returned backoffs. + + +Server +------ + +A C++ server can be used for the test. Other languages do NOT need to implement +a server. To minimize the network delay, the server binary should run on the +same machine or on a nearby machine (in terms of network distance) with the +client binary. + +A server implements the ReconnectService to its state. It also opens a +tcp server on the retry_port, which just shuts down all incoming tcp +connections to simulate connection failures. The server will keep a record of +all the reconnection timestamps and return the connection backoffs in the +response in milliseconds. The server also checks the backoffs to see whether +they conform the spec and returns whether the client passes the test. + +If the server receives a Start call when another client is being tested, it +finishes the call when the other client is done. If some other host connects +to the server retry_port when a client is being tested, the server will log an +error but likely would think the client fails the test. + +The server accepts these arguments: + +* --control_port=PORT + * The port to listen on for control rpcs. For example, "8080" +* --retry_port=PORT + * The tcp server port. For example, "8081" + diff --git a/doc/interop-test-descriptions.md b/doc/interop-test-descriptions.md index 3ee5d0f032..065e107c24 100644 --- a/doc/interop-test-descriptions.md +++ b/doc/interop-test-descriptions.md @@ -483,18 +483,17 @@ library to obtain the authorization token * received SimpleResponse.oauth_scope is in `--oauth_scope` -### Metadata (TODO: fix name) - -Status: Not yet implementable +### custom_metadata This test verifies that custom metadata in either binary or ascii format can be -sent in header and trailer. +sent as initial-metadata by the client and as both initial- and trailing-metadata +by the server. Server features: * [UnaryCall][] +* [FullDuplexCall][] * [Compressable Payload][] -* Ability to receive custom metadata from client in header and send custom data - back to client in both header and trailer. (TODO: this is not defined) +* [Echo Metadata][] Procedure: 1. While sending custom metadata (ascii + binary) in the header, client calls @@ -509,21 +508,29 @@ Procedure: } } ``` +The client attaches custom metadata with the following keys and values: + ``` + key: "x-grpc-test-echo-initial", value: "test_initial_metadata_value" + key: "x-grpc-test-echo-trailing-bin", value: 0xababab + ``` + 2. Client repeats step 1. with FullDuplexCall instead of UnaryCall. Asserts: * call was successful -* custom metadata is echoed back in the response header. -* custom metadata is echoed back in the response trailer. +* metadata with key `"x-grpc-test-echo-initial"` and value `"test_initial_metadata_value"`is received in the initial metadata. +* metadata with key `"x-grpc-test-echo-trailing-bin"` and value `0xababab` is received in the trailing metadata. + -### status_code_and_message -Status: Not yet implementable +### status_code_and_message This test verifies unary calls succeed in sending messages, and propagates back status code and message sent along with the messages. Server features: * [UnaryCall][] +* [FullDuplexCall][] +* [Echo Status][] Procedure: 1. Client calls UnaryCall with: @@ -536,6 +543,8 @@ Procedure: } } ``` +2. Client repeats step 1. with FullDuplexCall instead of UnaryCall. + Asserts: * received status code is the same with sent code @@ -543,21 +552,15 @@ Asserts: ### unimplemented_method -Status: Not yet implementable +Status: Ready for implementation. Blocking beta. -This test verifies calling unimplemented RPC method returns unimplemented -status. +This test verifies calling unimplemented RPC method returns the UNIMPLEMENTED status code. Procedure: -* Client calls UnimplementedCall with: +* Client calls `grpc.testing.UnimplementedService/UnimplementedCall` with an empty request (defined as `grpc.testing.Empty`): ``` { - response_type: COMPRESSABLE - response_size: 314159 - payload:{ - body: 271828 bytes of zeros - } } ``` @@ -767,6 +770,22 @@ When the client requests COMPRESSABLE payload, the response includes a payload of the size requested containing all zeros and the payload type is COMPRESSABLE. +### Echo Status +[Echo Status]: #echo-status +When the client sends a response_status in the request payload, the server closes +the stream with the status code and messsage contained within said response_status. +The server will not process any further messages on the stream sent by the client. +This can be used by clients to verify correct handling of different status codes and +associated status messages end-to-end. + +### Echo Metadata +[Echo Metadata]: #echo-metadata +When the client sends metadata with the key `"x-grpc-test-echo-initial"` with its +request, the server sends back exactly this key and the corresponding value back to +the client as part of initial metadata. When the client sends metadata with the key +`"x-grpc-test-echo-trailing-bin"` with its request, the server sends back exactly this +key and the corresponding value back to the client as trailing metadata. + ### Observe ResponseParameters.interval_us [Observe ResponseParameters.interval_us]: #observe-responseparametersinterval_us diff --git a/include/grpc++/client_context.h b/include/grpc++/client_context.h index 9df76699d2..5cf4d3328a 100644 --- a/include/grpc++/client_context.h +++ b/include/grpc++/client_context.h @@ -110,7 +110,7 @@ class ClientContext { creds_ = creds; } - grpc_compression_algorithm get_compression_algorithm() const { + grpc_compression_algorithm compression_algorithm() const { return compression_algorithm_; } @@ -119,8 +119,8 @@ class ClientContext { std::shared_ptr<const AuthContext> auth_context() const; // Get and set census context - void set_census_context(census_context* ccp) { census_context_ = ccp; } - census_context* get_census_context() const { return census_context_; } + void set_census_context(struct census_context* ccp) { census_context_ = ccp; } + struct census_context* census_context() const { return census_context_; } void TryCancel(); @@ -170,7 +170,7 @@ class ClientContext { grpc::string authority_; std::shared_ptr<Credentials> creds_; mutable std::shared_ptr<const AuthContext> auth_context_; - census_context* census_context_; + struct census_context* census_context_; std::multimap<grpc::string, grpc::string> send_initial_metadata_; std::multimap<grpc::string, grpc::string> recv_initial_metadata_; std::multimap<grpc::string, grpc::string> trailing_metadata_; diff --git a/include/grpc++/dynamic_thread_pool.h b/include/grpc++/dynamic_thread_pool.h index bc4e2d4d74..f0cd35940f 100644 --- a/include/grpc++/dynamic_thread_pool.h +++ b/include/grpc++/dynamic_thread_pool.h @@ -41,6 +41,7 @@ #include <grpc++/thread_pool_interface.h> #include <list> +#include <memory> #include <queue> namespace grpc { diff --git a/include/grpc++/impl/sync_no_cxx11.h b/include/grpc++/impl/sync_no_cxx11.h index fda668957e..5869b04c76 100644 --- a/include/grpc++/impl/sync_no_cxx11.h +++ b/include/grpc++/impl/sync_no_cxx11.h @@ -87,7 +87,7 @@ class condition_variable { ~condition_variable() { gpr_cv_destroy(&cv_); } void wait(lock_guard<mutex> &mu) { mu.locked = false; - gpr_cv_wait(&cv_, &mu.mu_.mu_, gpr_inf_future(GPR_CLOCK_REALTIME); + gpr_cv_wait(&cv_, &mu.mu_.mu_, gpr_inf_future(GPR_CLOCK_REALTIME)); mu.locked = true; } void notify_one() { gpr_cv_signal(&cv_); } diff --git a/include/grpc++/server_context.h b/include/grpc++/server_context.h index 3bfa48fbb6..e204d66456 100644 --- a/include/grpc++/server_context.h +++ b/include/grpc++/server_context.h @@ -46,6 +46,7 @@ struct gpr_timespec; struct grpc_metadata; struct grpc_call; +struct census_context; namespace grpc { @@ -104,18 +105,20 @@ class ServerContext { return client_metadata_; } - grpc_compression_level get_compression_level() const { + grpc_compression_level compression_level() const { return compression_level_; } void set_compression_level(grpc_compression_level level); - grpc_compression_algorithm get_compression_algorithm() const { + grpc_compression_algorithm compression_algorithm() const { return compression_algorithm_; } void set_compression_algorithm(grpc_compression_algorithm algorithm); std::shared_ptr<const AuthContext> auth_context() const; + const struct census_context* census_context() const; + private: friend class ::grpc::testing::InteropContextInspector; friend class ::grpc::Server; diff --git a/include/grpc/census.h b/include/grpc/census.h index 379783905a..7603dfdce1 100644 --- a/include/grpc/census.h +++ b/include/grpc/census.h @@ -44,26 +44,30 @@ extern "C" { #endif -/* Identify census functionality that can be enabled via census_initialize(). */ -enum census_functions { - CENSUS_NONE = 0, /* Do not enable census. */ - CENSUS_TRACING = 1, /* Enable census tracing. */ - CENSUS_STATS = 2, /* Enable Census stats collection. */ - CENSUS_CPU = 4, /* Enable Census CPU usage collection. */ - CENSUS_ALL = CENSUS_TRACING | CENSUS_STATS | CENSUS_CPU +/* Identify census features that can be enabled via census_initialize(). */ +enum census_features { + CENSUS_FEATURE_NONE = 0, /* Do not enable census. */ + CENSUS_FEATURE_TRACING = 1, /* Enable census tracing. */ + CENSUS_FEATURE_STATS = 2, /* Enable Census stats collection. */ + CENSUS_FEATURE_CPU = 4, /* Enable Census CPU usage collection. */ + CENSUS_FEATURE_ALL = + CENSUS_FEATURE_TRACING | CENSUS_FEATURE_STATS | CENSUS_FEATURE_CPU }; -/* Shutdown and startup census subsystem. The 'functions' argument should be - * the OR (|) of census_functions values. If census fails to initialize, then +/** Shutdown and startup census subsystem. The 'features' argument should be + * the OR (|) of census_features values. If census fails to initialize, then * census_initialize() will return a non-zero value. It is an error to call * census_initialize() more than once (without an intervening * census_shutdown()). */ -int census_initialize(int functions); -void census_shutdown(); +int census_initialize(int features); +void census_shutdown(void); -/* If any census feature has been initialized, this funtion will return a - * non-zero value. */ -int census_available(); +/** Return the features supported by the current census implementation (not all + * features will be available on all platforms). */ +int census_supported(void); + +/** Return the census features currently enabled. */ +int census_enabled(void); /* Internally, Census relies on a context, which should be propagated across * RPC's. From the RPC subsystems viewpoint, this is an opaque data structure. diff --git a/src/core/census/grpc_context.c b/src/core/census/grpc_context.c index 0ed63469b6..d4243cb246 100644 --- a/src/core/census/grpc_context.c +++ b/src/core/census/grpc_context.c @@ -39,7 +39,7 @@ static void grpc_census_context_destroy(void *context) { } void grpc_census_call_set_context(grpc_call *call, census_context *context) { - if (!census_available()) { + if (census_enabled() == CENSUS_FEATURE_NONE) { return; } if (context == NULL) { diff --git a/src/core/census/initialize.c b/src/core/census/initialize.c index 8016520641..8d60f790eb 100644 --- a/src/core/census/initialize.c +++ b/src/core/census/initialize.c @@ -33,20 +33,25 @@ #include <grpc/census.h> -static int census_fns_enabled = CENSUS_NONE; +static int features_enabled = CENSUS_FEATURE_NONE; -int census_initialize(int functions) { - if (census_fns_enabled != CENSUS_NONE) { +int census_initialize(int features) { + if (features_enabled != CENSUS_FEATURE_NONE) { return 1; } - if (functions != CENSUS_NONE) { + if (features != CENSUS_FEATURE_NONE) { return 1; } else { - census_fns_enabled = functions; + features_enabled = features; return 0; } } -void census_shutdown() { census_fns_enabled = CENSUS_NONE; } +void census_shutdown(void) { features_enabled = CENSUS_FEATURE_NONE; } -int census_available() { return (census_fns_enabled != CENSUS_NONE); } +int census_supported(void) { + /* TODO(aveitch): improve this as we implement features... */ + return CENSUS_FEATURE_NONE; +} + +int census_enabled(void) { return features_enabled; } diff --git a/src/core/iomgr/tcp_server_windows.c b/src/core/iomgr/tcp_server_windows.c index cc680507ff..bcd2aa8536 100644 --- a/src/core/iomgr/tcp_server_windows.c +++ b/src/core/iomgr/tcp_server_windows.c @@ -79,6 +79,8 @@ struct grpc_tcp_server { /* active port count: how many ports are actually still listening */ int active_ports; + /* number of iomgr callbacks that have been explicitly scheduled during shutdown */ + int iomgr_callbacks_pending; /* all listening ports */ server_port *ports; @@ -93,6 +95,7 @@ grpc_tcp_server *grpc_tcp_server_create(void) { gpr_mu_init(&s->mu); gpr_cv_init(&s->cv); s->active_ports = 0; + s->iomgr_callbacks_pending = 0; s->cb = NULL; s->cb_arg = NULL; s->ports = gpr_malloc(sizeof(server_port) * INIT_PORT_CAP); @@ -112,10 +115,10 @@ void grpc_tcp_server_destroy(grpc_tcp_server *s, for (i = 0; i < s->nports; i++) { server_port *sp = &s->ports[i]; sp->shutting_down = 1; - grpc_winsocket_shutdown(sp->socket); + s->iomgr_callbacks_pending += grpc_winsocket_shutdown(sp->socket); } /* This happens asynchronously. Wait while that happens. */ - while (s->active_ports) { + while (s->active_ports || s->iomgr_callbacks_pending) { gpr_cv_wait(&s->cv, &s->mu, gpr_inf_future(GPR_CLOCK_REALTIME)); } gpr_mu_unlock(&s->mu); @@ -254,8 +257,16 @@ static void on_accept(void *arg, int from_iocp) { /* The general mechanism for shutting down is to queue abortion calls. While this is necessary in the read/write case, it's useless for the accept - case. Let's do nothing. */ - if (!from_iocp) return; + case. We only need to adjust the pending callback count */ + if (!from_iocp) { + gpr_mu_lock(&sp->server->mu); + GPR_ASSERT(sp->server->iomgr_callbacks_pending > 0); + if (0 == --sp->server->iomgr_callbacks_pending) { + gpr_cv_broadcast(&sp->server->cv); + } + gpr_mu_unlock(&sp->server->mu); + return; + } /* The IOCP notified us of a completed operation. Let's grab the results, and act accordingly. */ @@ -264,11 +275,12 @@ static void on_accept(void *arg, int from_iocp) { &transfered_bytes, FALSE, &flags); if (!wsa_success) { if (sp->shutting_down) { - /* During the shutdown case, we ARE expecting an error. So that's swell, + /* During the shutdown case, we ARE expecting an error. So that's well, and we can wake up the shutdown thread. */ sp->shutting_down = 0; sp->socket->read_info.outstanding = 0; gpr_mu_lock(&sp->server->mu); + GPR_ASSERT(sp->server->active_ports > 0); if (0 == --sp->server->active_ports) { gpr_cv_broadcast(&sp->server->cv); } diff --git a/src/core/iomgr/tcp_windows.c b/src/core/iomgr/tcp_windows.c index 38ae74abec..5e0457a37b 100644 --- a/src/core/iomgr/tcp_windows.c +++ b/src/core/iomgr/tcp_windows.c @@ -368,7 +368,14 @@ static grpc_endpoint_write_status win_write(grpc_endpoint *ep, return GRPC_ENDPOINT_WRITE_PENDING; } -static void win_add_to_pollset(grpc_endpoint *ep, grpc_pollset *pollset) { +static void win_add_to_pollset(grpc_endpoint *ep, grpc_pollset *ps) { + (void) ps; + grpc_tcp *tcp = (grpc_tcp *) ep; + grpc_iocp_add_socket(tcp->socket); +} + +static void win_add_to_pollset_set(grpc_endpoint *ep, grpc_pollset_set *pss) { + (void) pss; grpc_tcp *tcp = (grpc_tcp *) ep; grpc_iocp_add_socket(tcp->socket); } @@ -402,8 +409,9 @@ static char *win_get_peer(grpc_endpoint *ep) { } static grpc_endpoint_vtable vtable = {win_notify_on_read, win_write, - win_add_to_pollset, win_shutdown, - win_destroy, win_get_peer}; + win_add_to_pollset, win_add_to_pollset_set, + win_shutdown, win_destroy, + win_get_peer}; grpc_endpoint *grpc_tcp_create(grpc_winsocket *socket, char *peer_string) { grpc_tcp *tcp = (grpc_tcp *) gpr_malloc(sizeof(grpc_tcp)); diff --git a/src/core/surface/init.c b/src/core/surface/init.c index a015262612..442bc72f21 100644 --- a/src/core/surface/init.c +++ b/src/core/surface/init.c @@ -80,8 +80,11 @@ void grpc_init(void) { grpc_security_pre_init(); grpc_iomgr_init(); grpc_tracer_init("GRPC_TRACE"); - if (census_initialize(CENSUS_NONE)) { - gpr_log(GPR_ERROR, "Could not initialize census."); + /* Only initialize census if noone else has. */ + if (census_enabled() == CENSUS_FEATURE_NONE) { + if (census_initialize(census_supported())) { /* enable all features. */ + gpr_log(GPR_ERROR, "Could not initialize census."); + } } grpc_timers_global_init(); } diff --git a/src/cpp/client/channel.cc b/src/cpp/client/channel.cc index da31d000b3..9c2b0b1dbc 100644 --- a/src/cpp/client/channel.cc +++ b/src/cpp/client/channel.cc @@ -69,7 +69,7 @@ Call Channel::CreateCall(const RpcMethod& method, ClientContext* context, ? target_.c_str() : context->authority().c_str(), context->raw_deadline()); - grpc_census_call_set_context(c_call, context->get_census_context()); + grpc_census_call_set_context(c_call, context->census_context()); GRPC_TIMER_MARK(GRPC_PTAG_CPP_CALL_CREATED, c_call); context->set_call(c_call, shared_from_this()); return Call(c_call, this, cq); diff --git a/src/cpp/server/server_context.cc b/src/cpp/server/server_context.cc index bf7a4ba5ec..f54ed8aa29 100644 --- a/src/cpp/server/server_context.cc +++ b/src/cpp/server/server_context.cc @@ -39,6 +39,7 @@ #include <grpc++/impl/sync.h> #include <grpc++/time.h> +#include "src/core/census/grpc_context.h" #include "src/core/channel/compress_filter.h" #include "src/cpp/common/create_auth_context.h" @@ -179,4 +180,8 @@ std::shared_ptr<const AuthContext> ServerContext::auth_context() const { return auth_context_; } +const struct census_context* ServerContext::census_context() const { + return grpc_census_call_get_context(call_); +} + } // namespace grpc diff --git a/src/node/examples/perf_test.js b/src/node/examples/perf_test.js index da919eced5..0f38725f72 100644 --- a/src/node/examples/perf_test.js +++ b/src/node/examples/perf_test.js @@ -41,7 +41,8 @@ var interop_server = require('../interop/interop_server.js'); function runTest(iterations, callback) { var testServer = interop_server.getServer(0, false); testServer.server.listen(); - var client = new testProto.TestService('localhost:' + testServer.port); + var client = new testProto.TestService('localhost:' + testServer.port, + grpc.Credentials.createInsecure()); function runIterations(finish) { var start = process.hrtime(); diff --git a/src/node/examples/qps_test.js b/src/node/examples/qps_test.js index 00293b464a..1ce4dbe070 100644 --- a/src/node/examples/qps_test.js +++ b/src/node/examples/qps_test.js @@ -61,7 +61,8 @@ var interop_server = require('../interop/interop_server.js'); function runTest(concurrent_calls, seconds, callback) { var testServer = interop_server.getServer(0, false); testServer.server.listen(); - var client = new testProto.TestService('localhost:' + testServer.port); + var client = new testProto.TestService('localhost:' + testServer.port, + grpc.Credentials.createInsecure()); var warmup_num = 100; diff --git a/src/node/examples/route_guide_client.js b/src/node/examples/route_guide_client.js index 8cd532fef1..647f3ffaba 100644 --- a/src/node/examples/route_guide_client.js +++ b/src/node/examples/route_guide_client.js @@ -40,7 +40,8 @@ var path = require('path'); var _ = require('lodash'); var grpc = require('..'); var examples = grpc.load(__dirname + '/route_guide.proto').examples; -var client = new examples.RouteGuide('localhost:50051'); +var client = new examples.RouteGuide('localhost:50051', + grpc.Credentials.createInsecure()); var COORD_FACTOR = 1e7; diff --git a/src/node/examples/stock_client.js b/src/node/examples/stock_client.js index b37e66df07..ab9b050e9b 100644 --- a/src/node/examples/stock_client.js +++ b/src/node/examples/stock_client.js @@ -38,7 +38,8 @@ var examples = grpc.load(__dirname + '/stock.proto').examples; * This exports a client constructor for the Stock service. The usage looks like * * var StockClient = require('stock_client.js'); - * var stockClient = new StockClient(server_address); + * var stockClient = new StockClient(server_address, + * grpc.Credentials.createInsecure()); * stockClient.getLastTradePrice({symbol: 'GOOG'}, function(error, response) { * console.log(error || response); * }); diff --git a/src/node/ext/channel.cc b/src/node/ext/channel.cc index c43b55f115..d02ed95672 100644 --- a/src/node/ext/channel.cc +++ b/src/node/ext/channel.cc @@ -98,31 +98,30 @@ NAN_METHOD(Channel::New) { if (args.IsConstructCall()) { if (!args[0]->IsString()) { - return NanThrowTypeError("Channel expects a string and an object"); + return NanThrowTypeError( + "Channel expects a string, a credential and an object"); } grpc_channel *wrapped_channel; // Owned by the Channel object NanUtf8String *host = new NanUtf8String(args[0]); NanUtf8String *host_override = NULL; - if (args[1]->IsUndefined()) { + grpc_credentials *creds; + if (!Credentials::HasInstance(args[1])) { + return NanThrowTypeError( + "Channel's second argument must be a credential"); + } + Credentials *creds_object = ObjectWrap::Unwrap<Credentials>( + args[1]->ToObject()); + creds = creds_object->GetWrappedCredentials(); + grpc_channel_args *channel_args_ptr; + if (args[2]->IsUndefined()) { + channel_args_ptr = NULL; wrapped_channel = grpc_insecure_channel_create(**host, NULL); - } else if (args[1]->IsObject()) { - grpc_credentials *creds = NULL; - Handle<Object> args_hash(args[1]->ToObject()->Clone()); + } else if (args[2]->IsObject()) { + Handle<Object> args_hash(args[2]->ToObject()->Clone()); if (args_hash->HasOwnProperty(NanNew(GRPC_SSL_TARGET_NAME_OVERRIDE_ARG))) { host_override = new NanUtf8String(args_hash->Get(NanNew(GRPC_SSL_TARGET_NAME_OVERRIDE_ARG))); } - if (args_hash->HasOwnProperty(NanNew("credentials"))) { - Handle<Value> creds_value = args_hash->Get(NanNew("credentials")); - if (!Credentials::HasInstance(creds_value)) { - return NanThrowTypeError( - "credentials arg must be a Credentials object"); - } - Credentials *creds_object = - ObjectWrap::Unwrap<Credentials>(creds_value->ToObject()); - creds = creds_object->GetWrappedCredentials(); - args_hash->Delete(NanNew("credentials")); - } Handle<Array> keys(args_hash->GetOwnPropertyNames()); grpc_channel_args channel_args; channel_args.num_args = keys->Length(); @@ -149,16 +148,19 @@ NAN_METHOD(Channel::New) { return NanThrowTypeError("Arg values must be strings"); } } - if (creds == NULL) { - wrapped_channel = grpc_insecure_channel_create(**host, &channel_args); - } else { - wrapped_channel = - grpc_secure_channel_create(creds, **host, &channel_args); - } - free(channel_args.args); + channel_args_ptr = &channel_args; } else { return NanThrowTypeError("Channel expects a string and an object"); } + if (creds == NULL) { + wrapped_channel = grpc_insecure_channel_create(**host, channel_args_ptr); + } else { + wrapped_channel = + grpc_secure_channel_create(creds, **host, channel_args_ptr); + } + if (channel_args_ptr != NULL) { + free(channel_args_ptr->args); + } Channel *channel; if (host_override == NULL) { channel = new Channel(wrapped_channel, host); @@ -168,8 +170,8 @@ NAN_METHOD(Channel::New) { channel->Wrap(args.This()); NanReturnValue(args.This()); } else { - const int argc = 2; - Local<Value> argv[argc] = {args[0], args[1]}; + const int argc = 3; + Local<Value> argv[argc] = {args[0], args[1], args[2]}; NanReturnValue(constructor->GetFunction()->NewInstance(argc, argv)); } } diff --git a/src/node/ext/credentials.cc b/src/node/ext/credentials.cc index d6cff0631d..21d61f1a7f 100644 --- a/src/node/ext/credentials.cc +++ b/src/node/ext/credentials.cc @@ -81,6 +81,8 @@ void Credentials::Init(Handle<Object> exports) { NanNew<FunctionTemplate>(CreateGce)->GetFunction()); ctr->Set(NanNew("createIam"), NanNew<FunctionTemplate>(CreateIam)->GetFunction()); + ctr->Set(NanNew("createInsecure"), + NanNew<FunctionTemplate>(CreateInsecure)->GetFunction()); constructor = new NanCallback(ctr); exports->Set(NanNew("Credentials"), ctr); } @@ -92,9 +94,6 @@ bool Credentials::HasInstance(Handle<Value> val) { Handle<Value> Credentials::WrapStruct(grpc_credentials *credentials) { NanEscapableScope(); - if (credentials == NULL) { - return NanEscapeScope(NanNull()); - } const int argc = 1; Handle<Value> argv[argc] = { NanNew<External>(reinterpret_cast<void *>(credentials))}; @@ -128,7 +127,11 @@ NAN_METHOD(Credentials::New) { NAN_METHOD(Credentials::CreateDefault) { NanScope(); - NanReturnValue(WrapStruct(grpc_google_default_credentials_create())); + grpc_credentials *creds = grpc_google_default_credentials_create(); + if (creds == NULL) { + NanReturnNull(); + } + NanReturnValue(WrapStruct(creds)); } NAN_METHOD(Credentials::CreateSsl) { @@ -152,9 +155,12 @@ NAN_METHOD(Credentials::CreateSsl) { return NanThrowTypeError( "createSSl's third argument must be a Buffer if provided"); } - - NanReturnValue(WrapStruct(grpc_ssl_credentials_create( - root_certs, key_cert_pair.private_key == NULL ? NULL : &key_cert_pair))); + grpc_credentials *creds = grpc_ssl_credentials_create( + root_certs, key_cert_pair.private_key == NULL ? NULL : &key_cert_pair); + if (creds == NULL) { + NanReturnNull(); + } + NanReturnValue(WrapStruct(creds)); } NAN_METHOD(Credentials::CreateComposite) { @@ -169,13 +175,21 @@ NAN_METHOD(Credentials::CreateComposite) { } Credentials *creds1 = ObjectWrap::Unwrap<Credentials>(args[0]->ToObject()); Credentials *creds2 = ObjectWrap::Unwrap<Credentials>(args[1]->ToObject()); - NanReturnValue(WrapStruct(grpc_composite_credentials_create( - creds1->wrapped_credentials, creds2->wrapped_credentials))); + grpc_credentials *creds = grpc_composite_credentials_create( + creds1->wrapped_credentials, creds2->wrapped_credentials); + if (creds == NULL) { + NanReturnNull(); + } + NanReturnValue(WrapStruct(creds)); } NAN_METHOD(Credentials::CreateGce) { NanScope(); - NanReturnValue(WrapStruct(grpc_compute_engine_credentials_create())); + grpc_credentials *creds = grpc_compute_engine_credentials_create(); + if (creds == NULL) { + NanReturnNull(); + } + NanReturnValue(WrapStruct(creds)); } NAN_METHOD(Credentials::CreateIam) { @@ -188,8 +202,17 @@ NAN_METHOD(Credentials::CreateIam) { } NanUtf8String auth_token(args[0]); NanUtf8String auth_selector(args[1]); - NanReturnValue( - WrapStruct(grpc_iam_credentials_create(*auth_token, *auth_selector))); + grpc_credentials *creds = grpc_iam_credentials_create(*auth_token, + *auth_selector); + if (creds == NULL) { + NanReturnNull(); + } + NanReturnValue(WrapStruct(creds)); +} + +NAN_METHOD(Credentials::CreateInsecure) { + NanScope(); + NanReturnValue(WrapStruct(NULL)); } } // namespace node diff --git a/src/node/ext/credentials.h b/src/node/ext/credentials.h index 794736fe1a..62957e61c3 100644 --- a/src/node/ext/credentials.h +++ b/src/node/ext/credentials.h @@ -68,6 +68,7 @@ class Credentials : public ::node::ObjectWrap { static NAN_METHOD(CreateGce); static NAN_METHOD(CreateFake); static NAN_METHOD(CreateIam); + static NAN_METHOD(CreateInsecure); static NanCallback *constructor; // Used for typechecking instances of this javascript class static v8::Persistent<v8::FunctionTemplate> fun_tpl; diff --git a/src/node/interop/interop_client.js b/src/node/interop/interop_client.js index e810e68e45..236b36616c 100644 --- a/src/node/interop/interop_client.js +++ b/src/node/interop/interop_client.js @@ -397,6 +397,7 @@ var test_cases = { function runTest(address, host_override, test_case, tls, test_ca, done) { // TODO(mlumish): enable TLS functionality var options = {}; + var creds; if (tls) { var ca_path; if (test_ca) { @@ -405,13 +406,14 @@ function runTest(address, host_override, test_case, tls, test_ca, done) { ca_path = process.env.SSL_CERT_FILE; } var ca_data = fs.readFileSync(ca_path); - var creds = grpc.Credentials.createSsl(ca_data); - options.credentials = creds; + creds = grpc.Credentials.createSsl(ca_data); if (host_override) { options['grpc.ssl_target_name_override'] = host_override; } + } else { + creds = grpc.Credentials.createInsecure(); } - var client = new testProto.TestService(address, options); + var client = new testProto.TestService(address, creds, options); test_cases[test_case](client, done); } diff --git a/src/node/src/client.js b/src/node/src/client.js index f843669bd0..b2b4423707 100644 --- a/src/node/src/client.js +++ b/src/node/src/client.js @@ -531,11 +531,13 @@ exports.makeClientConstructor = function(methods, serviceName) { * Create a client with the given methods * @constructor * @param {string} address The address of the server to connect to + * @param {grpc.Credentials} credentials Credentials to use to connect + * to the server * @param {Object} options Options to pass to the underlying channel * @param {function(string, Object, function)=} updateMetadata function to * update the metadata for each request */ - function Client(address, options, updateMetadata) { + function Client(address, credentials, options, updateMetadata) { if (!updateMetadata) { updateMetadata = function(uri, metadata, callback) { callback(null, metadata); @@ -545,7 +547,7 @@ exports.makeClientConstructor = function(methods, serviceName) { options = {}; } options['grpc.primary_user_agent'] = 'grpc-node/' + version; - this.channel = new grpc.Channel(address, options); + this.channel = new grpc.Channel(address, credentials, options); this.server_address = address.replace(/\/$/, ''); this.auth_uri = this.server_address + '/' + serviceName; this.updateMetadata = updateMetadata; diff --git a/src/node/test/call_test.js b/src/node/test/call_test.js index 942c31ac68..c5edab8bcd 100644 --- a/src/node/test/call_test.js +++ b/src/node/test/call_test.js @@ -48,6 +48,8 @@ function getDeadline(timeout_secs) { return deadline; } +var insecureCreds = grpc.Credentials.createInsecure(); + describe('call', function() { var channel; var server; @@ -55,7 +57,7 @@ describe('call', function() { server = new grpc.Server(); var port = server.addHttp2Port('localhost:0'); server.start(); - channel = new grpc.Channel('localhost:' + port); + channel = new grpc.Channel('localhost:' + port, insecureCreds); }); after(function() { server.shutdown(); @@ -82,7 +84,7 @@ describe('call', function() { }); }); it('should fail with a closed channel', function() { - var local_channel = new grpc.Channel('hostname'); + var local_channel = new grpc.Channel('hostname', insecureCreds); local_channel.close(); assert.throws(function() { new grpc.Call(channel, 'method'); diff --git a/src/node/test/channel_test.js b/src/node/test/channel_test.js index 3e61d3bbc6..c991d7b25b 100644 --- a/src/node/test/channel_test.js +++ b/src/node/test/channel_test.js @@ -36,11 +36,13 @@ var assert = require('assert'); var grpc = require('bindings')('grpc.node'); +var insecureCreds = grpc.Credentials.createInsecure(); + describe('channel', function() { describe('constructor', function() { it('should require a string for the first argument', function() { assert.doesNotThrow(function() { - new grpc.Channel('hostname'); + new grpc.Channel('hostname', insecureCreds); }); assert.throws(function() { new grpc.Channel(); @@ -49,38 +51,49 @@ describe('channel', function() { new grpc.Channel(5); }); }); - it('should accept an object for the second parameter', function() { + it('should require a credential for the second argument', function() { assert.doesNotThrow(function() { - new grpc.Channel('hostname', {}); + new grpc.Channel('hostname', insecureCreds); }); assert.throws(function() { new grpc.Channel('hostname', 5); }); + assert.throws(function() { + new grpc.Channel('hostname'); + }); + }); + it('should accept an object for the third argument', function() { + assert.doesNotThrow(function() { + new grpc.Channel('hostname', insecureCreds, {}); + }); + assert.throws(function() { + new grpc.Channel('hostname', insecureCreds, 'abc'); + }); }); it('should only accept objects with string or int values', function() { assert.doesNotThrow(function() { - new grpc.Channel('hostname', {'key' : 'value'}); + new grpc.Channel('hostname', insecureCreds,{'key' : 'value'}); }); assert.doesNotThrow(function() { - new grpc.Channel('hostname', {'key' : 5}); + new grpc.Channel('hostname', insecureCreds, {'key' : 5}); }); assert.throws(function() { - new grpc.Channel('hostname', {'key' : null}); + new grpc.Channel('hostname', insecureCreds, {'key' : null}); }); assert.throws(function() { - new grpc.Channel('hostname', {'key' : new Date()}); + new grpc.Channel('hostname', insecureCreds, {'key' : new Date()}); }); }); }); describe('close', function() { it('should succeed silently', function() { - var channel = new grpc.Channel('hostname', {}); + var channel = new grpc.Channel('hostname', insecureCreds, {}); assert.doesNotThrow(function() { channel.close(); }); }); it('should be idempotent', function() { - var channel = new grpc.Channel('hostname', {}); + var channel = new grpc.Channel('hostname', insecureCreds, {}); assert.doesNotThrow(function() { channel.close(); channel.close(); @@ -89,7 +102,7 @@ describe('channel', function() { }); describe('getTarget', function() { it('should return a string', function() { - var channel = new grpc.Channel('localhost', {}); + var channel = new grpc.Channel('localhost', insecureCreds, {}); assert.strictEqual(typeof channel.getTarget(), 'string'); }); }); diff --git a/src/node/test/end_to_end_test.js b/src/node/test/end_to_end_test.js index 5d3baf823d..9133ceca92 100644 --- a/src/node/test/end_to_end_test.js +++ b/src/node/test/end_to_end_test.js @@ -57,6 +57,8 @@ function multiDone(done, count) { }; } +var insecureCreds = grpc.Credentials.createInsecure(); + describe('end-to-end', function() { var server; var channel; @@ -64,7 +66,7 @@ describe('end-to-end', function() { server = new grpc.Server(); var port_num = server.addHttp2Port('0.0.0.0:0'); server.start(); - channel = new grpc.Channel('localhost:' + port_num); + channel = new grpc.Channel('localhost:' + port_num, insecureCreds); }); after(function() { server.shutdown(); diff --git a/src/node/test/health_test.js b/src/node/test/health_test.js index bb700cc46c..93b068be31 100644 --- a/src/node/test/health_test.js +++ b/src/node/test/health_test.js @@ -56,7 +56,8 @@ describe('Health Checking', function() { before(function() { var port_num = healthServer.bind('0.0.0.0:0'); healthServer.start(); - healthClient = new health.Client('localhost:' + port_num); + healthClient = new health.Client('localhost:' + port_num, + grpc.Credentials.createInsecure()); }); after(function() { healthServer.shutdown(); diff --git a/src/node/test/math_client_test.js b/src/node/test/math_client_test.js index f2751857ff..1bd85ca495 100644 --- a/src/node/test/math_client_test.js +++ b/src/node/test/math_client_test.js @@ -53,7 +53,8 @@ describe('Math client', function() { before(function(done) { var port_num = server.bind('0.0.0.0:0'); server.start(); - math_client = new math.Math('localhost:' + port_num); + math_client = new math.Math('localhost:' + port_num, + grpc.Credentials.createInsecure()); done(); }); after(function() { diff --git a/src/node/test/surface_test.js b/src/node/test/surface_test.js index 98f9b15bfc..18c99a5994 100644 --- a/src/node/test/surface_test.js +++ b/src/node/test/surface_test.js @@ -124,7 +124,7 @@ describe('Echo service', function() { }); var port = server.bind('localhost:0'); var Client = surface_client.makeProtobufClientConstructor(echo_service); - client = new Client('localhost:' + port); + client = new Client('localhost:' + port, grpc.Credentials.createInsecure()); server.start(); }); after(function() { @@ -169,7 +169,8 @@ describe('Generic client and server', function() { var port = server.bind('localhost:0'); server.start(); var Client = grpc.makeGenericClientConstructor(string_service_attrs); - client = new Client('localhost:' + port); + client = new Client('localhost:' + port, + grpc.Credentials.createInsecure()); }); after(function() { server.shutdown(); @@ -216,7 +217,7 @@ describe('Echo metadata', function() { }); var port = server.bind('localhost:0'); var Client = surface_client.makeProtobufClientConstructor(test_service); - client = new Client('localhost:' + port); + client = new Client('localhost:' + port, grpc.Credentials.createInsecure()); server.start(); }); after(function() { @@ -337,7 +338,7 @@ describe('Other conditions', function() { }); port = server.bind('localhost:0'); var Client = surface_client.makeProtobufClientConstructor(test_service); - client = new Client('localhost:' + port); + client = new Client('localhost:' + port, grpc.Credentials.createInsecure()); server.start(); }); after(function() { @@ -382,7 +383,8 @@ describe('Other conditions', function() { }; var Client = surface_client.makeClientConstructor(test_service_attrs, 'TestService'); - misbehavingClient = new Client('localhost:' + port); + misbehavingClient = new Client('localhost:' + port, + grpc.Credentials.createInsecure()); }); it('should respond correctly to a unary call', function(done) { misbehavingClient.unary(badArg, function(err, data) { @@ -602,7 +604,7 @@ describe('Cancelling surface client', function() { }); var port = server.bind('localhost:0'); var Client = surface_client.makeProtobufClientConstructor(mathService); - client = new Client('localhost:' + port); + client = new Client('localhost:' + port, grpc.Credentials.createInsecure()); server.start(); }); after(function() { diff --git a/test/core/security/oauth2_utils.c b/test/core/security/oauth2_utils.c index ecd04fd9e1..2df2f99269 100644 --- a/test/core/security/oauth2_utils.c +++ b/test/core/security/oauth2_utils.c @@ -84,7 +84,7 @@ char *grpc_test_fetch_oauth2_token_with_credentials(grpc_credentials *creds) { gpr_mu_lock(GRPC_POLLSET_MU(&request.pollset)); while (!request.is_done) - grpc_pollset_work(&request.pollset, gpr_inf_future(GPR_CLOCK_REALTIME)); + grpc_pollset_work(&request.pollset, gpr_inf_future(GPR_CLOCK_MONOTONIC)); gpr_mu_unlock(GRPC_POLLSET_MU(&request.pollset)); grpc_pollset_shutdown(&request.pollset, do_nothing, NULL); diff --git a/test/cpp/qps/qps_test.cc b/test/cpp/qps/qps_test.cc index 7b93443f7c..ba980a6664 100644 --- a/test/cpp/qps/qps_test.cc +++ b/test/cpp/qps/qps_test.cc @@ -53,8 +53,8 @@ static void RunQPS() { ClientConfig client_config; client_config.set_client_type(ASYNC_CLIENT); client_config.set_enable_ssl(false); - client_config.set_outstanding_rpcs_per_channel(10); - client_config.set_client_channels(800); + client_config.set_outstanding_rpcs_per_channel(1000); + client_config.set_client_channels(8); client_config.set_payload_size(1); client_config.set_async_client_threads(8); client_config.set_rpc_type(UNARY); diff --git a/tools/gce_setup/cloud_prod_runner.sh b/tools/gce_setup/cloud_prod_runner.sh index 4206a66db7..7343702883 100755 --- a/tools/gce_setup/cloud_prod_runner.sh +++ b/tools/gce_setup/cloud_prod_runner.sh @@ -34,8 +34,8 @@ log_link=https://pantheon.corp.google.com/m/cloudstorage/b/stoked-keyword-656-ou main() { source grpc_docker.sh - test_cases=(large_unary empty_unary ping_pong client_streaming server_streaming cancel_after_begin cancel_after_first_response) - auth_test_cases=(service_account_creds compute_engine_creds jwt_token_creds) + test_cases=(large_unary empty_unary ping_pong client_streaming server_streaming cancel_after_begin cancel_after_first_response empty_stream timeout_on_sleeping_server) + auth_test_cases=(service_account_creds compute_engine_creds jwt_token_creds oauth2_auth_token per_rpc_creds) clients=(cxx java go ruby node csharp_mono csharp_dotnet python php) for test_case in "${test_cases[@]}" do diff --git a/tools/gce_setup/grpc_docker.sh b/tools/gce_setup/grpc_docker.sh index b53aa98aab..c06780a699 100755 --- a/tools/gce_setup/grpc_docker.sh +++ b/tools/gce_setup/grpc_docker.sh @@ -508,7 +508,12 @@ grpc_cloud_prod_auth_test_args() { grpc_gen_test_cmd="grpc_cloud_prod_auth_" [[ -n $1 ]] && { # test_case test_case=$1 - grpc_gen_test_cmd+="$1" + test_command="service_account_creds" + if [ "$test_case" == "compute_engine_creds" ] + then + test_command="compute_engine_creds" + fi + grpc_gen_test_cmd+=$test_command shift } || { echo "$FUNCNAME: missing arg: test_case" 1>&2 diff --git a/tools/gce_setup/interop_test_runner.sh b/tools/gce_setup/interop_test_runner.sh index 6ff3731985..506471411b 100755 --- a/tools/gce_setup/interop_test_runner.sh +++ b/tools/gce_setup/interop_test_runner.sh @@ -37,7 +37,7 @@ fail_log_link=https://pantheon.corp.google.com/m/cloudstorage/b/stoked-keyword-6 main() { source grpc_docker.sh - test_cases=(large_unary empty_unary ping_pong client_streaming server_streaming cancel_after_begin cancel_after_first_response) + test_cases=(large_unary empty_unary ping_pong client_streaming server_streaming cancel_after_begin cancel_after_first_response empty_stream timeout_on_sleeping_server) clients=(cxx java go ruby node python csharp_mono php) servers=(cxx java go ruby node python csharp_mono) for test_case in "${test_cases[@]}" diff --git a/tools/jenkins/run_jenkins.sh b/tools/jenkins/run_jenkins.sh index 56f9e82ca5..93cf82d260 100755 --- a/tools/jenkins/run_jenkins.sh +++ b/tools/jenkins/run_jenkins.sh @@ -1,4 +1,4 @@ -#!/bin/bash +#!/bin/sh # Copyright 2015, Google Inc. # All rights reserved. # @@ -31,6 +31,8 @@ # This script is invoked by Jenkins and triggers a test run based on # env variable settings. # +# Bootstrap into bash +[ -z $1 ] && exec bash $0 bootstrapped # Setting up rvm environment BEFORE we set -ex. [[ -s /etc/profile.d/rvm.sh ]] && . /etc/profile.d/rvm.sh # To prevent cygwin bash complaining about empty lines ending with \r @@ -90,7 +92,9 @@ then docker cp $DOCKER_CID:/var/local/git/grpc/report.xml $git_root sleep 4 docker rm $DOCKER_CID || true - +elif [ "$platform" == "interop" ] +then + python tools/run_tests/run_interops.py --language=$language elif [ "$platform" == "windows" ] then echo "building $language on Windows" @@ -103,11 +107,18 @@ then /cygdrive/c/nuget/nuget.exe restore src/csharp/Grpc.sln python tools/run_tests/run_tests.py -t -l $language -x report.xml || true + elif [ "$platform" == "macos" ] then echo "building $language on MacOS" ./tools/run_tests/run_tests.py -t -l $language -c $config -x report.xml || true + +elif [ "$platform" == "freebsd" ] +then + echo "building $language on FreeBSD" + + MAKE=gmake ./tools/run_tests/run_tests.py -t -l $language -c $config -x report.xml || true else echo "Unknown platform $platform" exit 1 diff --git a/tools/run_tests/build_ruby.sh b/tools/run_tests/build_ruby.sh index b6c4e32b7e..259f336ef2 100755 --- a/tools/run_tests/build_ruby.sh +++ b/tools/run_tests/build_ruby.sh @@ -36,5 +36,7 @@ export GRPC_CONFIG=${CONFIG:-opt} # change to grpc's ruby directory cd $(dirname $0)/../../src/ruby +rm -rf ./tmp + bundle install rake compile:grpc diff --git a/tools/run_tests/jobset.py b/tools/run_tests/jobset.py index 46137f01f4..ec25b47610 100755 --- a/tools/run_tests/jobset.py +++ b/tools/run_tests/jobset.py @@ -206,7 +206,7 @@ class Job(object): do_newline=self._newline_on_success or self._travis) if self._bin_hash: update_cache.finished(self._spec.identity(), self._bin_hash) - elif self._state == _RUNNING and time.time() - self._start > 600: + elif self._state == _RUNNING and time.time() - self._start > 900: self._tempfile.seek(0) stdout = self._tempfile.read() filtered_stdout = filter(lambda x: x in string.printable, stdout.decode(errors='ignore')) diff --git a/tools/run_tests/run_interops.py b/tools/run_tests/run_interops.py new file mode 100755 index 0000000000..1cf268526d --- /dev/null +++ b/tools/run_tests/run_interops.py @@ -0,0 +1,36 @@ +import argparse +import xml.etree.cElementTree as ET +import jobset + +argp = argparse.ArgumentParser(description='Run interop tests.') +argp.add_argument('-l', '--language', + choices=['build_only', 'c++'], + nargs='+', + default=['build_only']) +args = argp.parse_args() + +# build job +build_steps = 'tools/run_tests/run_interops_build.sh' +build_job = jobset.JobSpec(cmdline=build_steps, shortname='build') + +# test jobs +_TESTS = ['large_unary', 'empty_unary', 'ping_pong', 'client_streaming', 'server_streaming'] +jobs = [] +jobNumber = 0 +for lang in args.language: + for test in _TESTS: + test_job = jobset.JobSpec(cmdline=['tools/run_tests/run_interops_test.sh', '%s' % lang, '%s' % test], shortname=test) + jobs.append(test_job) + jobNumber+=1 + +root = ET.Element('testsuites') +testsuite = ET.SubElement(root, 'testsuite', id='1', package='grpc', name='tests') + +# always do the build of docker first, and then all the tests can run in parallel +jobset.run([build_job], maxjobs=1, xml_report=testsuite) +jobset.run(jobs, maxjobs=jobNumber, xml_report=testsuite) + +tree = ET.ElementTree(root) +tree.write('report.xml', encoding='UTF-8') + + diff --git a/tools/run_tests/run_interops_build.sh b/tools/run_tests/run_interops_build.sh new file mode 100755 index 0000000000..23441a5300 --- /dev/null +++ b/tools/run_tests/run_interops_build.sh @@ -0,0 +1,47 @@ +#!/bin/sh + +# Copyright 2015, Google Inc. +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions are +# met: +# +# * Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# * Redistributions in binary form must reproduce the above +# copyright notice, this list of conditions and the following disclaimer +# in the documentation and/or other materials provided with the +# distribution. +# * Neither the name of Google Inc. nor the names of its +# contributors may be used to endorse or promote products derived from +# this software without specific prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +set -e + +#clean up any old docker files and start mirroring repository if not started already +sudo docker rmi -f grpc/cxx || true +sudo docker rmi -f grpc/base || true +sudo docker rmi -f 0.0.0.0:5000/grpc/base || true +sudo docker run -d -e GCS_BUCKET=docker-interop-images -e STORAGE_PATH=/admin/docker_images -p 5000:5000 google/docker-registry || true + +#prepare building by pulling down base images and necessary files +sudo docker pull 0.0.0.0:5000/grpc/base +sudo docker tag -f 0.0.0.0:5000/grpc/base grpc/base +gsutil cp -R gs://docker-interop-images/admin/service_account tools/dockerfile/grpc_cxx +gsutil cp -R gs://docker-interop-images/admin/cacerts tools/dockerfile/grpc_cxx + +#build docker file, add more languages later +sudo docker build --no-cache -t grpc/cxx tools/dockerfile/grpc_cxx diff --git a/tools/run_tests/run_interops_test.sh b/tools/run_tests/run_interops_test.sh new file mode 100755 index 0000000000..1d0eedad85 --- /dev/null +++ b/tools/run_tests/run_interops_test.sh @@ -0,0 +1,43 @@ +#!/bin/sh + +# Copyright 2015, Google Inc. +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions are +# met: +# +# * Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# * Redistributions in binary form must reproduce the above +# copyright notice, this list of conditions and the following disclaimer +# in the documentation and/or other materials provided with the +# distribution. +# * Neither the name of Google Inc. nor the names of its +# contributors may be used to endorse or promote products derived from +# this software without specific prior written permission. +# +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS +# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT +# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR +# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT +# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, +# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT +# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, +# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY +# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT +# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE +# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + +language=$1 +test_case=$2 + +set -e +if [ "$language" = "c++" ] +then + sudo docker run grpc/cxx /var/local/git/grpc/bins/opt/interop_client --enable_ssl --use_prod_roots --server_host_override=grpc-test.sandbox.google.com --server_host=grpc-test.sandbox.google.com --server_port=443 --test_case=$test_case +else + echo "interop testss not added for $language" + exit 1 +fi + diff --git a/tools/run_tests/run_tests.py b/tools/run_tests/run_tests.py index 77cc02a087..38da1f043a 100755 --- a/tools/run_tests/run_tests.py +++ b/tools/run_tests/run_tests.py @@ -253,7 +253,7 @@ class RubyLanguage(object): environ=_FORCE_ENVIRON_FOR_WRAPPERS)] def make_targets(self): - return ['run_dep_checks'] + return ['static_c'] def build_steps(self): return [['tools/run_tests/build_ruby.sh']] @@ -458,7 +458,7 @@ if platform.system() == 'Windows': cwd='vsprojects', shell=True) else: def make_jobspec(cfg, targets): - return jobset.JobSpec(['make', + return jobset.JobSpec([os.getenv('MAKE', 'make'), '-j', '%d' % (multiprocessing.cpu_count() + 1), 'EXTRA_DEFINES=GRPC_TEST_SLOWDOWN_MACHINE_FACTOR=%f' % args.slowdown, |