Add fuzzing support for proxygen (#2856)

This adds support for compiling and running the fuzzers present in the proxygen repository.
Right now there's only one fuzzer committed there, but this build script is generic
and will pull all of them in as we add more (if oss-fuzz integration proves fruitful).

Test plan is below - following https://google.github.io/oss-fuzz/getting-started/new-project-guide/#testing-locally

I verified the base image builds:

    python infra/helper.py build_image proxygen

I built and verified the ASAN fuzzer works:

    python infra/helper.py build_fuzzers --sanitizer address proxygen
    python infra/helper.py check_build proxygen
    python infra/helper.py run_fuzzer proxygen ProxygenHTTP1xFuzzer

Similar thing for UBSAN:

    python infra/helper.py build_fuzzers --sanitizer undefined proxygen
    python infra/helper.py check_build proxygen
    python infra/helper.py run_fuzzer proxygen ProxygenHTTP1xFuzzer

Note the last one seemed to run ASAN build by default so I pulled out the command it runs and ran it manually:

    docker run --rm -i --privileged -e FUZZING_ENGINE=libfuzzer -e SANITIZER=undefined -e ARCHITECTURE=x86_64 -v /home/mhl/oss-fuzz/build/out/proxygen:/out -t gcr.io/oss-fuzz-base/base-runner test_all

I tested the coverage build:

    python infra/helper.py build_fuzzers --sanitizer coverage proxygen
    python infra/helper.py coverage proxygen ProxygenHTTP1xFuzzer

Note that this "runs" but threw some warnings which I will file a separate issue for.
It does generate the files though.

NOTE: I didn't run the MSAN build as I would have to figure out instrumenting all dependencies.
We can investigate that in a follow up.

Similarly, I haven't yet tried the dataflow build.

Note that I haven't tried testing this with the AFL build yet either. There were no instructions on the page (https://google.github.io/oss-fuzz/getting-started/new-project-guide/#testing-locally) on how to do so -- if someone can mention them here I am happy to test that too before committing.

3 files changed, 177 insertions, 0 deletions

diff --git a/projects/proxygen/Dockerfile b/projects/proxygen/Dockerfile
new file mode 100644
index 00000000..82e3521d
--- /dev/null
+++ b/projects/proxygen/Dockerfile
@@ -0,0 +1,140 @@
+# Copyright 2019 Google Inc.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+################################################################################
+
+FROM gcr.io/oss-fuzz-base/base-builder
+MAINTAINER mhl@fb.com
+
+# Install packages we need to build dependencies
+RUN apt-get update && \
+    apt-get install -y \
+    make \
+    autoconf \
+    automake \
+    libtool \
+    sudo \
+    wget \
+    gcc \
+    g++ \
+    python \
+    python-dev
+
+# We need a newer version of CMake than is available in the base builder image
+RUN wget https://github.com/Kitware/CMake/releases/download/v3.15.2/cmake-3.15.2.tar.gz && \
+    tar xzf cmake-3.15.2.tar.gz && \
+    cd cmake-3.15.2 && \
+    export CC=clang && \
+    export CXX=clang++ && \
+    CXXFLAGS="" && \
+    CFLAGS="" && \
+    ./bootstrap && \
+    make -j$(nproc) && \
+    make install && \
+    cd .. && \
+    rm -rf cmake-3.15.2
+
+# Install and build boost from source so we can have it use libc++
+RUN wget https://dl.bintray.com/boostorg/release/1.70.0/source/boost_1_70_0.tar.gz && \
+    tar xzf boost_1_70_0.tar.gz && \
+    cd boost_1_70_0 && \
+    ./bootstrap.sh --with-toolset=clang && \
+    ./b2 clean && \
+    ./b2 toolset=clang cxxflags="-stdlib=libc++" linkflags="-stdlib=libc++" -j$(nproc) install && \
+    cd .. && \
+    rm -rf boost_1_70_0
+
+# Build gflags/glog/gtest from source so we use libc++ and avoid incompatibilities with the std::string ABI breaking changes
+RUN sudo apt-get purge libgflags-dev
+
+RUN wget https://github.com/gflags/gflags/archive/v2.2.2.tar.gz && \
+    tar xzf v2.2.2.tar.gz && \
+    cd gflags-2.2.2 && \
+    mkdir build && \
+    cd build && \
+    export CC=clang && \
+    export CXX=clang++ && \
+    export CXXFLAGS="-stdlib=libc++" && \
+    cmake -DCMAKE_POSITION_INDEPENDENT_CODE=ON .. && \
+    make -j$(nproc) && \
+    sudo make install && \
+    cd ../../ && \
+    rm -rf gflags-2.2.2
+
+RUN wget https://github.com/google/glog/archive/v0.4.0.tar.gz && \
+    tar xzf v0.4.0.tar.gz && \
+    cd glog-0.4.0 && \
+    export CC=clang && \
+    export CXX=clang++ && \
+    export CXXFLAGS="-stdlib=libc++" && \
+    mkdir build && \
+    cd build && \
+    cmake -DCMAKE_POSITION_INDEPENDENT_CODE=ON -DCMAKE_VERBOSE_MAKEFILE=ON .. && \
+    make -j$(nproc) && \
+    sudo make install && \
+    cd ../.. && \
+    rm -rf glog-0.4.0
+
+RUN wget https://github.com/google/googletest/archive/release-1.8.1.tar.gz && \
+    tar xzf release-1.8.1.tar.gz && \
+    cd googletest-release-1.8.1 && \
+    export CC=clang && \
+    export CXX=clang++ && \
+    export CXXFLAGS="-stdlib=libc++" && \
+    mkdir build && \
+    cd build && \
+    cmake -DCMAKE_POSITION_INDEPENDENT_CODE=ON -DCMAKE_VERBOSE_MAKEFILE=ON .. && \
+    make -j$(nproc) && \
+    sudo make install && \
+    cd ../.. && \
+    rm -rf googletest-release-1.8.1
+
+# Build and install zstd from source so we have it available for proxygen
+RUN wget https://github.com/facebook/zstd/releases/download/v1.4.2/zstd-1.4.2.tar.gz && \
+    tar xzf zstd-1.4.2.tar.gz && \
+    cd zstd-1.4.2 && \
+    export CC=clang && \
+    export CXX=clang++ && \
+    export CXXFLAGS="-stdlib=libc++" && \
+    sudo make -j$(nproc) install && \
+    cd .. && \
+    rm -rf zstd-1.4.2
+
+# Replicate `install-dependencies` from the proxygen `build.sh` script
+RUN apt-get install -y \
+    git \
+    flex \
+    bison \
+    libkrb5-dev \
+    libsasl2-dev \
+    libnuma-dev \
+    pkg-config \
+    libssl-dev \
+    libcap-dev \
+    gperf \
+    libevent-dev \
+    libtool \
+    libjemalloc-dev \
+    unzip \
+    libiberty-dev \
+    liblzma-dev \
+    zlib1g-dev \
+    binutils-dev \
+    libsodium-dev \
+    libdouble-conversion-dev
+
+# Fetch source and copy over files
+RUN git clone --depth 1 https://github.com/facebook/proxygen.git proxygen
+WORKDIR proxygen
+COPY build.sh $SRC/
diff --git a/projects/proxygen/build.sh b/projects/proxygen/build.sh
new file mode 100755
index 00000000..49175512
--- /dev/null
+++ b/projects/proxygen/build.sh
@@ -0,0 +1,23 @@
+#!/bin/bash -eu
+# Copyright 2019 Google Inc.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+################################################################################
+
+cd proxygen
+# We need to link to libunwind to ensure exception handling works
+# See https://clang.llvm.org/docs/Toolchain.html#unwind-library
+export LDFLAGS="-lunwind"
+./build.sh -m --no-install-dependencies --build-for-fuzzing
+find ./_build/proxygen/fuzzers -type f -executable -exec cp {} $OUT/ \;
diff --git a/projects/proxygen/project.yaml b/projects/proxygen/project.yaml
new file mode 100644
index 00000000..71500980
--- /dev/null
+++ b/projects/proxygen/project.yaml
@@ -0,0 +1,14 @@
+homepage: "https://github.com/facebook/proxygen"
+primary_contact: "mhl@fb.com"
+auto_ccs:
+  - "gulshan@fb.com"
+  - "reed@fb.com"
+  - "lniccolini@fb.com"
+vendor_ccs:
+  - "oss-fuzz@fb.com"
+fuzzing_engines:
+  - libfuzzer
+  - afl
+sanitizers:
+  - address
+  - undefined