aboutsummaryrefslogtreecommitdiffhomepage
path: root/benchmarks/README.md
diff options
context:
space:
mode:
authorGravatar Yilun Chong <yilunchong@google.com>2017-12-13 14:34:52 -0800
committerGravatar Yilun Chong <yilunchong@google.com>2017-12-13 14:34:52 -0800
commit34843eddfe65686a86d12780d1ca709a997d83ad (patch)
tree670a2ed6cd727a214a4f59147c10ce2ee9c375a6 /benchmarks/README.md
parent5e732e35c1aea3a00903aae50f63cf1cf9166833 (diff)
Fix bugs
Diffstat (limited to 'benchmarks/README.md')
-rw-r--r--benchmarks/README.md32
1 files changed, 24 insertions, 8 deletions
diff --git a/benchmarks/README.md b/benchmarks/README.md
index 00a63704..f266d41e 100644
--- a/benchmarks/README.md
+++ b/benchmarks/README.md
@@ -5,20 +5,31 @@ This directory contains benchmarking schemas and data sets that you
can use to test a variety of performance scenarios against your
protobuf language runtime.
-The schema for the datasets is described in `benchmarks.proto`.
+## Benchmark tools and build instructions
-The benchmark is based on some submodules. To initialize the submodues:
+First, you need to follow the instruction in the root directory's README to
+build your language's protobuf, then:
-For java:
-```
-$ ./initialize_submodule.sh java
-```
+### CPP
+We are using [google/benchmark](https://github.com/google/benchmark) as the
+benchmark tool for testing cpp. This is included as submodule under third_party
+directory. To init and build this tools, you need to do this under root dirctory:
-For java:
```
-$ ./initialize_submodule.sh cpp
+$ cd third_party
+$ git submodule update --init -r
+$ cd benchmark && cmake -DCMAKE_BUILD_TYPE=Release && make && cd ../..
```
+### JAVA
+We're using maven to build the java benchmarks, which is the same as to build
+the Java protobuf. There're no other tools need to install. We're using
+[google/caliper](https://github.com/google/caliper) as benchmark tool, which
+can be automaticly included by maven.
+
+
+## Run instructions
+
To run all the benchmark dataset:
For java:
@@ -49,6 +60,8 @@ $ make cpp-benchmark
$ ./cpp-benchmark $(specific generated dataset file name)
```
+## Benchmark datasets
+
There's some big testing data not included in the directory initially, you need to
run the following command to download the testing data:
@@ -57,10 +70,13 @@ $ ./download_data.sh
```
Each data set is in the format of benchmarks.proto:
+
1. name is the benchmark dataset's name.
2. message_name is the benchmark's message type full name (including package and message name)
3. payload is the list of raw data.
+The schema for the datasets is described in `benchmarks.proto`.
+
Benchmark likely want to run several benchmarks against each data set (parse,
serialize, possibly JSON, possibly using different APIs, etc).