aboutsummaryrefslogtreecommitdiffhomepage
path: root/tools/run_tests/performance
diff options
context:
space:
mode:
authorGravatar apolcyn <apolcyn@google.com>2017-03-31 12:13:49 -0700
committerGravatar GitHub <noreply@github.com>2017-03-31 12:13:49 -0700
commit09ed43f1d1670064cc98863479498676bb8c2502 (patch)
tree5f969f110090164ddecb0b20fc5110b961c97f5c /tools/run_tests/performance
parent3885578bb5008f9bd2aaeedd9d0e97e6c4bfd6c9 (diff)
parent679981069d4085235bddf08022995c0b34dad7aa (diff)
Merge pull request #10311 from apolcyn/doc_perf_tests
add doc for perf qps tests with steps for running
Diffstat (limited to 'tools/run_tests/performance')
-rw-r--r--tools/run_tests/performance/README.md106
1 files changed, 106 insertions, 0 deletions
diff --git a/tools/run_tests/performance/README.md b/tools/run_tests/performance/README.md
new file mode 100644
index 0000000000..5fd64f6ee9
--- /dev/null
+++ b/tools/run_tests/performance/README.md
@@ -0,0 +1,106 @@
+# Overview of performance test suite, with steps for manual runs:
+
+For design of the tests, see
+http://www.grpc.io/docs/guides/benchmarking.html.
+
+## Pre-reqs for running these manually:
+In general the benchmark workers and driver build scripts expect
+[linux_performance_worker_init.sh](../../gce/linux_performance_worker_init.sh) to have been ran already.
+
+### To run benchmarks locally:
+* From the grpc repo root, start the
+[run_performance_tests.py](../run_performance_tests.py) runner script.
+
+### On remote machines, to start the driver and workers manually:
+The [run_performance_test.py](../run_performance_tests.py) top-level runner script can also
+be used with remote machines, but for e.g., profiling the server,
+it might be useful to run workers manually.
+
+1. You'll need a "driver" and separate "worker" machines.
+For example, you might use one GCE "driver" machine and 3 other
+GCE "worker" machines that are in the same zone.
+
+2. Connect to each worker machine and start up a benchmark worker with a "driver_port".
+ * For example, to start the grpc-go benchmark worker:
+ [grpc-go worker main.go](https://github.com/grpc/grpc-go/blob/master/benchmark/worker/main.go) --driver_port <driver_port>
+
+#### Comands to start workers in different languages:
+ * Note that these commands are what the top-level
+ [run_performance_test.py](../run_performance_tests.py) script uses to
+ build and run different workers through the
+ [build_performance.sh](./build_performance.sh) script and "run worker"
+ scripts (such as the [run_worker_java.sh](./run_worker_java.sh)).
+
+##### Running benchmark workers for C-core wrapped languages (C++, Python, C#, Node, Ruby):
+ * These are more simple since they all live in the main grpc repo.
+
+```
+$ cd <grpc_repo_root>
+$ tools/run_tests/performance/build_performance.sh
+$ tools/run_tests/performance/run_worker_<language>.sh
+```
+
+ * Note that there is one "run_worker" script per language, e.g.,
+ [run_worker_csharp.sh](./run_worker_csharp.sh) for c#.
+
+##### Running benchmark workers for gRPC-Java:
+ * You'll need the [grpc-java](https://github.com/grpc/grpc-java) repo.
+
+```
+$ cd <grpc-java-repo>
+$ ./gradlew -PskipCodegen=true :grpc-benchmarks:installDist
+$ benchmarks/build/install/grpc-benchmarks/bin/benchmark_worker --driver_port <driver_port>
+```
+
+##### Running benchmark workers for gRPC-Go:
+ * You'll need the [grpc-go repo](https://github.com/grpc/grpc-go)
+
+```
+$ cd <grpc-go-repo>/benchmark/worker && go install
+$ # if profiling, it might be helpful to turn off inlining by building with "-gcflags=-l"
+$ $GOPATH/bin/worker --driver_port <driver_port>
+```
+
+#### Build the driver:
+* Connect to the driver machine (if using a remote driver) and from the grpc repo root:
+```
+$ tools/run_tests/performance/build_performance.sh
+```
+
+#### Run the driver:
+1. Get the 'scenario_json' relevant for the scenario to run. Note that "scenario
+ json" configs are generated from [scenario_config.py](./scenario_config.py).
+ The [driver](../../../test/cpp/qps/qps_json_driver.cc) takes a list of these configs as a json string of the form: `{scenario: <json_list_of_scenarios> }`
+ in its `--scenarios_json` command argument.
+ One quick way to get a valid json string to pass to the driver is by running
+ the [run_performance_tests.py](./run_performance_tests.py) locally and copying the logged scenario json command arg.
+
+2. From the grpc repo root:
+
+* Set `QPS_WORKERS` environment variable to a comma separated list of worker
+machines. Note that the driver will start the "benchmark server" on the first
+entry in the list, and the rest will be told to run as clients against the
+benchmark server.
+
+Example running and profiling of go benchmark server:
+```
+$ export QPS_WORKERS=<host1>:<10000>,<host2>,10000,<host3>:10000
+$ bins/opt/qps_json_driver --scenario_json='<scenario_json_scenario_config_string>'
+```
+
+### Example profiling commands
+
+While running the benchmark, a profiler can be attached to the server.
+
+Example to count syscalls in grpc-go server during a benchmark:
+* Connect to server machine and run:
+```
+$ netstat -tulpn | grep <driver_port> # to get pid of worker
+$ perf stat -p <worker_pid> -e syscalls:sys_enter_write # stop after test complete
+```
+
+Example memory profile of grpc-go server, with `go tools pprof`:
+* After a run is done on the server, see its alloc profile with:
+```
+$ go tool pprof --text --alloc_space http://localhost:<pprof_port>/debug/heap
+```