aboutsummaryrefslogtreecommitdiffhomepage
path: root/benchmarks/README.md
diff options
context:
space:
mode:
authorGravatar Yilun Chong <chongyilun250@sina.com>2018-01-03 09:42:26 -0800
committerGravatar GitHub <noreply@github.com>2018-01-03 09:42:26 -0800
commit43caa38d6ed68129d28bf4528488e4f389a33b34 (patch)
treea0f12d536972bde077d1be2b3b48c02ea5ff9782 /benchmarks/README.md
parent156161dfcde38b72c55ead02cacff7087c93a4d8 (diff)
parentaca6c155981859e99460835ccc791fac9d350b47 (diff)
Merge pull request #4014 from BSBandme/JavaCaliper
Add caliper supported to java benchmark
Diffstat (limited to 'benchmarks/README.md')
-rw-r--r--benchmarks/README.md42
1 files changed, 38 insertions, 4 deletions
diff --git a/benchmarks/README.md b/benchmarks/README.md
index 0273e389..09c06907 100644
--- a/benchmarks/README.md
+++ b/benchmarks/README.md
@@ -5,7 +5,36 @@ This directory contains benchmarking schemas and data sets that you
can use to test a variety of performance scenarios against your
protobuf language runtime.
-The schema for the datasets is described in `benchmarks.proto`.
+## Prerequisite
+
+First, you need to follow the instruction in the root directory's README to
+build your language's protobuf, then:
+
+### CPP
+You need to install [cmake](https://cmake.org/) before building the benchmark.
+
+We are using [google/benchmark](https://github.com/google/benchmark) as the
+benchmark tool for testing cpp. This will be automaticly made during build the
+cpp benchmark.
+
+### JAVA
+We're using maven to build the java benchmarks, which is the same as to build
+the Java protobuf. There're no other tools need to install. We're using
+[google/caliper](https://github.com/google/caliper) as benchmark tool, which
+can be automaticly included by maven.
+
+### Big data
+
+There's some optional big testing data which is not included in the directory initially, you need to
+run the following command to download the testing data:
+
+```
+$ ./download_data.sh
+```
+
+After doing this the big data file will automaticly generated in the benchmark directory.
+
+## Run instructions
To run all the benchmark dataset:
@@ -26,22 +55,27 @@ To run a specific dataset:
For java:
```
-$ make java
-$ ./java-benchmark $(specific generated dataset file name)
+$ make java-benchmark
+$ ./java-benchmark $(specific generated dataset file name) [-- $(caliper option)]
```
For cpp:
```
-$ make cpp
+$ make cpp-benchmark
$ ./cpp-benchmark $(specific generated dataset file name)
```
+## Benchmark datasets
+
Each data set is in the format of benchmarks.proto:
+
1. name is the benchmark dataset's name.
2. message_name is the benchmark's message type full name (including package and message name)
3. payload is the list of raw data.
+The schema for the datasets is described in `benchmarks.proto`.
+
Benchmark likely want to run several benchmarks against each data set (parse,
serialize, possibly JSON, possibly using different APIs, etc).