diff options
author | 2021-08-05 12:01:07 -0700 | |
---|---|---|
committer | 2021-08-05 12:01:07 -0700 | |
commit | af2617d7d0bb96113ac616829cbc3b9b57fd1ac5 (patch) | |
tree | 1e71439123edc0f0860da513cb02e00121a59368 /docs | |
parent | be08bbd5249e31345dc42140558a3a3f4720e71d (diff) |
[ClusterFuzzLite][docs] Start writing docs (#6165)
I started writing some important docs, such as overview, build_integration, and running_clusterfuzzlite. However, most important docs are still TODO.
This PR at least adds a skeleton for those docs.
Diffstat (limited to 'docs')
-rw-r--r-- | docs/clusterfuzzlite/advanced/adding_ci_support.md | 12 | ||||
-rw-r--r-- | docs/clusterfuzzlite/advanced/architecture.md | 12 | ||||
-rw-r--r-- | docs/clusterfuzzlite/advanced_topics.md | 10 | ||||
-rw-r--r-- | docs/clusterfuzzlite/build_integration.md | 286 | ||||
-rw-r--r-- | docs/clusterfuzzlite/clusterfuzzlite.md | 27 | ||||
-rw-r--r-- | docs/clusterfuzzlite/overview.md | 29 | ||||
-rw-r--r-- | docs/clusterfuzzlite/running-clusterfuzzlite/generic.md | 17 | ||||
-rw-r--r-- | docs/clusterfuzzlite/running-clusterfuzzlite/github_actions.md | 12 | ||||
-rw-r--r-- | docs/clusterfuzzlite/running_clusterfuzzlite.md | 101 |
9 files changed, 506 insertions, 0 deletions
diff --git a/docs/clusterfuzzlite/advanced/adding_ci_support.md b/docs/clusterfuzzlite/advanced/adding_ci_support.md new file mode 100644 index 00000000..2e855edd --- /dev/null +++ b/docs/clusterfuzzlite/advanced/adding_ci_support.md @@ -0,0 +1,12 @@ +--- +layout: default +parent: Advanced Topics +grand_parent: ClusterFuzzLite +title: Adding Support for a New CI +nav_order: 1 +permalink: /clusterfuzzlite/advanced-topics/adding-ci-support/ +--- + +# Adding Support for a New CI + +TODO diff --git a/docs/clusterfuzzlite/advanced/architecture.md b/docs/clusterfuzzlite/advanced/architecture.md new file mode 100644 index 00000000..a564f378 --- /dev/null +++ b/docs/clusterfuzzlite/advanced/architecture.md @@ -0,0 +1,12 @@ +--- +layout: default +parent: Advanced Topics +grand_parent: ClusterFuzzLite +title: Architecture +nav_order: 1 +permalink: /clusterfuzzlite/advanced-topics/architecture/ +--- + +# Architecture + +TODO diff --git a/docs/clusterfuzzlite/advanced_topics.md b/docs/clusterfuzzlite/advanced_topics.md new file mode 100644 index 00000000..c5b49e12 --- /dev/null +++ b/docs/clusterfuzzlite/advanced_topics.md @@ -0,0 +1,10 @@ +--- +layout: default +parent: ClusterFuzzLite +title: Advanced Topics +has_children: true +nav_order: 4 +permalink: /clusterfuzzlite/advanced-topics/ +--- + +# Advanced topics diff --git a/docs/clusterfuzzlite/build_integration.md b/docs/clusterfuzzlite/build_integration.md new file mode 100644 index 00000000..e52bee1c --- /dev/null +++ b/docs/clusterfuzzlite/build_integration.md @@ -0,0 +1,286 @@ +--- +layout: default +parent: ClusterFuzzLite +title: Build Integration +has_children: true +nav_order: 2 +permalink: /clusterfuzzlite/build-integration/ +--- +# Build integration +{: .no_toc} + +- TOC +{:toc} +--- + +## Prerequisites +ClusterFuzzLite supports statically linked [libFuzzer targets]({{ site.baseurl }}/reference/glossary/#fuzz-target) on Linux. + +Before you can start setting up your new project for fuzzing, you must do the following: +- [Integrate]({{ site.baseurl }}/advanced-topics/ideal-integration/) one or more [fuzz targets]({{ site.baseurl }}/reference/glossary/#fuzz-target) + with the project you want to fuzz. For examples, see TODO. +- [Install Docker](https://docs.docker.com/engine/installation) + (Googlers can visit [go/installdocker](https://goto.google.com/installdocker)). + [Why Docker?]({{ site.baseurl }}/faq/#why-do-you-use-docker) + + If you want to run `docker` without `sudo`, you can + [create a docker group](https://docs.docker.com/engine/installation/linux/ubuntulinux/#/create-a-docker-group). + + **Note:** Docker images can consume significant disk space. Run + [docker-cleanup](https://gist.github.com/mikea/d23a839cba68778d94e0302e8a2c200f) + periodically to garbage-collect unused images. + +- Clone the OSS-Fuzz repo: `git clone https://github.com/google/oss-fuzz.git` + +## Generating an empty build integration +Build integrations consist of two files, a `Dockerfile` that can install +dependencies for your project, and a `build.sh` file that actually builds your +project. These must live in the `.clusterfuzzlite` directory in the root of your +project's source code checkout. +You can generate empty versions of these files with the following command: + +```bash +$ cd /path/to/oss-fuzz +$ export PATH_TO_PROJECT=<path_to_your_project> +$ python infra/helper.py generate $PATH_TO_PROJECT --external +``` + +Once the configuration files are generated, you should modify them to fit your +project. + +## Dockerfile {#dockerfile} + +This integration file defines the Docker image for your project. +Your [build.sh](#buildsh) script will be executed in inside the container you +define. +For most projects, the image is simple: +```docker +FROM gcr.io/oss-fuzz-base/base-builder # base image with clang toolchain +RUN apt-get update && apt-get install -y ... # install required packages to build your project +COPY . $SRC/<project_name> # checkout all sources needed to build your project +WORKDIR $SRC/<project_name> # current directory for the build script +COPY ./clusterfuzzlite/build.sh fuzzer.cc $SRC/ # copy build script into src dir +``` +TODO: Provide examples. + +## build.sh {#buildsh} + +This file defines how to build binaries for [fuzz targets]({{ site.baseurl }}/reference/glossary/#fuzz-target) in your project. +The script is executed within the image built from your [Dockerfile](#Dockerfile). +If you are familiar with `build.sh` files from OSS-Fuzz, they work exactly the +same in ClusterFuzzLite. + +In general, this script should do the following: + +- Build the project using your build system with the ClusterFuzzLite's compiler. +- Provide ClusterFuzzLite's compiler flags (defined as [environment variables](#Requirements)) to the build system. +- Build your [fuzz targets]({{ site.baseurl }}/reference/glossary/#fuzz-target) + and link your project's build with `$LIB_FUZZING_ENGINE` (libFuzzer). + +Resulting binaries should be placed in `$OUT`. + +Here's an example from Expat +([source](https://github.com/google/oss-fuzz/blob/master/projects/expat/build.sh)): + +```bash +#!/bin/bash -eu + +./buildconf.sh +# configure scripts usually use correct environment variables. +./configure + +make clean +make -j$(nproc) all + +$CXX $CXXFLAGS -std=c++11 -Ilib/ \ + $SRC/parse_fuzzer.cc -o $OUT/parse_fuzzer \ + $LIB_FUZZING_ENGINE .libs/libexpat.a + +cp $SRC/*.dict $SRC/*.options $OUT/ +``` + +If your project is written in Go, check out the [Integrating a Go project]({{ site.baseurl }}//getting-started/new-project-guide/go-lang/) page. + +**Note:** +1. Make sure that the binary names for your [fuzz targets]({{ site.baseurl }}/reference/glossary/#fuzz-target) contain only +alphanumeric characters, underscore(_) or dash(-). Otherwise, they won't run. +1. Don't remove source code files. They are needed for code coverage. + + +### Temporarily disabling code instrumentation during builds + +In some cases, it's not necessary to instrument every 3rd party library or tool that supports the build target. Use the following snippet to build tools or libraries without instrumentation: + +``` +CFLAGS_SAVE="$CFLAGS" +CXXFLAGS_SAVE="$CXXFLAGS" +unset CFLAGS +unset CXXFLAGS +export AFL_NOOPT=1 + +# +# build commands here that should not result in instrumented code. +# + +export CFLAGS="${CFLAGS_SAVE}" +export CXXFLAGS="${CXXFLAGS_SAVE}" +unset AFL_NOOPT +``` +TODO: Figure out if we should include this AFL code. + +### build.sh script environment + +When your build.sh script is executed, the following locations are available within the image: + +| Location| Env Variable | Description | +|---------| ------------ | ---------- | +| `/out/` | `$OUT` | Directory to store build artifacts (fuzz targets, dictionaries, options files, seed corpus archives). | +| `/src/` | `$SRC` | Directory to checkout source files. | +| `/work/`| `$WORK` | Directory to store intermediate files. | + +Although the files layout is fixed within a container, environment variables are +provided so you can write retargetable scripts. + +In case your fuzz target uses the [FuzzedDataProvider] class, make sure it is +included via `#include <fuzzer/FuzzedDataProvider.h>` directive. + +[FuzzedDataProvider]: https://github.com/google/fuzzing/blob/master/docs/split-inputs.md#fuzzed-data-provider + +### build.sh requirements {#Requirements} + +Only binaries without an extension are accepted as targets. Extensions are reserved for other artifacts, like .dict. + +You *must* use the special compiler flags needed to build your project and fuzz targets. +These flags are provided in the following environment variables: + +| Env Variable | Description +| ------------- | -------- +| `$CC`, `$CXX`, `$CCC` | The C and C++ compiler binaries. +| `$CFLAGS`, `$CXXFLAGS` | C and C++ compiler flags. +| `$LIB_FUZZING_ENGINE` | C++ compiler argument to link fuzz target against the prebuilt engine library (e.g. libFuzzer). + +You *must* use `$CXX` as a linker, even if your project is written in pure C. + +Most well-crafted build scripts will automatically use these variables. If not, +pass them manually to the build tool. + +See the [Provided Environment Variables](https://github.com/google/oss-fuzz/blob/master/infra/base-images/base-builder/README.md#provided-environment-variables) section in +`base-builder` image documentation for more details. + +## Fuzzer execution environment + +For more on the environment that +your [fuzz targets]({{ site.baseurl }}/reference/glossary/#fuzz-target) run in, and the assumptions you can make, see the [fuzzer environment]({{ site.baseurl }}/further-reading/fuzzer-environment/) page. + +## Testing locally + +You can build your docker image and fuzz targets locally, so you can test them +before running ClusterFuzzLite. +1. Run the same helper script you used to create your directory structure, this time using it to build your docker image and [fuzz targets]({{ site.baseurl }}/reference/glossary/#fuzz-target): + + ```bash + $ cd /path/to/oss-fuzz + $ python infra/helper.py build_image $PATH_TO_PROJECT --external + $ python infra/helper.py build_fuzzers $PATH_TO_PROJECT --sanitizer <address/undefined/coverage> --external + ``` + + The built binaries appear in the `/path/to/oss-fuzz/build/out/$PROJECT_NAME` + directory on your machine (and `$OUT` in the container). Note that + `$PROJECT_NAME` is the name of the directory of your project (e.g. if + `$PATH_TO_PROJECT` is `/path/to/systemd`, `$PROJECT_NAME` is systemd. + + **Note:** You *must* run your fuzz target binaries inside the base-runner docker + container to make sure that they work properly. + +2. Find failures to fix by running the `check_build` command: + + ```bash + $ python infra/helper.py check_build $PATH_TO_PROJECT --external + ``` + +3. If you want to test changes against a particular fuzz target, run the following command: + + ```bash + $ python infra/helper.py run_fuzzer --external --corpus-dir=<path-to-temp-corpus-dir> $PATH_TO_PROJECT <fuzz_target> + ``` + +4. We recommend taking a look at your code coverage as a test to ensure that +your fuzz targets get to the code you expect. This would use the corpus +generated from the previous `run_fuzzer` step in your local corpus directory. + + ```bash + $ python infra/helper.py build_fuzzers --sanitizer coverage $PATH_TO_PROJECT + $ python infra/helper.py coverage $PATH_TO_PROJECT --fuzz-target=<fuzz_target> --corpus-dir=<path-to-temp-corpus-dir> --external + ``` + +You may need to run `python infra/helper.py pull_images` to use the latest +coverage tools. Please refer to +[code coverage]({{ site.baseurl }}/advanced-topics/code-coverage/) for detailed +information on code coverage generation. + +**Note:** Currently, ClusterFuzzLite only supports AddressSanitizer (address) +and UndefinedBehaviorSanitizer (undefined) configurations. +<b>Make sure to test each +of the supported build configurations with the above commands (build_fuzzers -> run_fuzzer -> coverage).</b> + +If everything works locally, it should also work on ClusterFuzzLite. If you +check in your files and experience failures, review your [dependencies]({{site.baseurl }}/further-reading/fuzzer-environment/#dependencies). + +## Debugging Problems + +If you run into problems, the [Debugging page]({{ site.baseurl }}/advanced-topics/debugging/) lists ways to debug your build scripts and +[fuzz targets]({{ site.baseurl }}/reference/glossary/#fuzz-target). + +## Efficient fuzzing + +To improve your fuzz target ability to find bugs faster, you should consider the +following ways: + +### Seed Corpus + +OSS-Fuzz uses evolutionary fuzzing algorithms. Supplying seed corpus consisting +of good sample inputs is one of the best ways to improve [fuzz target]({{ site.baseurl }}/reference/glossary/#fuzz-target)'s coverage. + +To provide a corpus for `my_fuzzer`, put `my_fuzzer_seed_corpus.zip` file next +to the [fuzz target]({{ site.baseurl }}/reference/glossary/#fuzz-target)'s binary in `$OUT` during the build. Individual files in this +archive will be used as starting inputs for mutations. The name of each file in the corpus is the sha1 checksum (which you can get using the `sha1sum` or `shasum` command) of its contents. You can store the corpus +next to source files, generate during build or fetch it using curl or any other +tool of your choice. +(example: [boringssl](https://github.com/google/oss-fuzz/blob/master/projects/boringssl/build.sh#L41)). + +Seed corpus files will be used for cross-mutations and portions of them might appear +in bug reports or be used for further security research. + +### Dictionaries + +Dictionaries hugely improve fuzzing efficiency for inputs with lots of similar +sequences of bytes. [libFuzzer documentation](http://libfuzzer.info#dictionaries) + +Put your dict file in `$OUT`. If the dict filename is the same as your target +binary name (i.e. `%fuzz_target%.dict`), it will be automatically used. If the +name is different (e.g. because it is shared by several targets), specify this +in .options file: + +``` +[libfuzzer] +dict = dictionary_name.dict +``` + +It is common for several [fuzz targets]({{ site.baseurl }}/reference/glossary/#fuzz-target) +to reuse the same dictionary if they are fuzzing very similar inputs. +(example: [expat](https://github.com/google/oss-fuzz/blob/master/projects/expat/parse_fuzzer.options)). + +### Input Size + +By default, the fuzzing engine will generate input of any arbitrary length. +This might be useful to try corner cases that could lead to a +security vulnerability. However, if large inputs are not necessary to +increase the coverage of your target API, it is important to add a limit +here to significantly improve performance. + +```cpp +if (size < kMinInputLength || size > kMaxInputLength) + return 0; +``` + +TODO(metzman): We probably want a TOC for lang-specific guides (which we still need to add). diff --git a/docs/clusterfuzzlite/clusterfuzzlite.md b/docs/clusterfuzzlite/clusterfuzzlite.md new file mode 100644 index 00000000..9dfd30a2 --- /dev/null +++ b/docs/clusterfuzzlite/clusterfuzzlite.md @@ -0,0 +1,27 @@ +--- +layout: default +title: ClusterFuzzLite +has_children: true +nav_order: 8 +permalink: /clusterfuzzlite/ +<!-- Hide for now by setting "published: false" --> +published: false +--- + +# ClusterFuzzLite +ClusterFuzzLite is a lightweight, easy-to-use, fuzzing infrastructure that is +based off [ClusterFuzz]. ClusterFuzzLite is designed to run on [continuous integration] (CI) +systems, which means it is easy to set up and provides a familiar interface for +users. +Currently CIFuzz fully supports [GitHub Actions]. However ClusterFuzzLite is +designed so that supporting new CI systems is trivial and core features can be +used on any CI system without any additional effort. + +See [Overview] for a more detailed description of how ClusterFuzzLite works and +how you can use it. + +[continous integration]: https://en.wikipedia.org/wiki/Continuous_integration +[fuzzing]: https://en.wikipedia.org/wiki/Fuzzing +[ClusterFuzz]: https://google.github.io/clusterfuzz/ +[GitHub Actions]: https://docs.github.com/en/actions +[Overview]: {{ site.baseurl }}/clusterfuzzlite/overview/ diff --git a/docs/clusterfuzzlite/overview.md b/docs/clusterfuzzlite/overview.md new file mode 100644 index 00000000..5e1e2497 --- /dev/null +++ b/docs/clusterfuzzlite/overview.md @@ -0,0 +1,29 @@ +--- +layout: default +parent: ClusterFuzzLite +title: Overview +nav_order: 1 +permalink: /clusterfuzzlite/overview/ +--- + +# Overview + +ClusterFuzzLite makes fuzzing more valuable by: +* Fuzzing continuously. +* Catching bugs before they land in your codebase by fuzzing pull + requests/commits. +* Providing coverage reports on which code is fuzzed. +* Managing your corpus, pruning it daily or a specified-interval. + +ClusterFuzzLite supports [libFuzzer], [AddressSanitizer], and +[UndefinedBehavior]. +ClusterFuzzLite is modular, so you can decide which features you want to use. +Using ClusterFuzzLite entails two major steps: +1. [Integrating with ClusterFuzzLite's build system] so ClusterFuzzLite can + build your project's fuzzers. +2. [Running ClusterFuzzLite]. +[libFuzzer]: https://libfuzzer.info +[AddressSanitizer]: https://clang.llvm.org/docs/AddressSanitizer.html +[UndefinedBehaviorSanitizer]: https://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html +[Integrating with ClusterFuzzLite's build system]: {{ site.baseurl }}/clusterfuzzlite/build-integration/ +[Running ClusterFuzzLite]: {{ site.baseurl }}/clusterfuzzlite/running-clusterfuzzlite/ diff --git a/docs/clusterfuzzlite/running-clusterfuzzlite/generic.md b/docs/clusterfuzzlite/running-clusterfuzzlite/generic.md new file mode 100644 index 00000000..045cd243 --- /dev/null +++ b/docs/clusterfuzzlite/running-clusterfuzzlite/generic.md @@ -0,0 +1,17 @@ +--- +layout: default +parent: Running ClusterFuzzLite +grand_parent: ClusterFuzzLite +title: Running ClusterFuzzLite in a Generic Environment +nav_order: 2 +permalink: /clusterfuzzlite/running-clusterfuzzlite/generic/ +--- + +# Running ClusterFuzzLite in a Generic Environment +TODO: Rename run_cifuzz.py to run_clusterfuzzlite.py + +This page describes how to run clusterfuzzlite using the run_clusterfuzzlite.py +script, a low level script that is used by CI systems to run ClusterFuzzLite. +This guide is useful if you want to run ClusterFuzzLite outside of CI or if you +want to run it on a CI system that isn't supported yet. +TODO diff --git a/docs/clusterfuzzlite/running-clusterfuzzlite/github_actions.md b/docs/clusterfuzzlite/running-clusterfuzzlite/github_actions.md new file mode 100644 index 00000000..430f6752 --- /dev/null +++ b/docs/clusterfuzzlite/running-clusterfuzzlite/github_actions.md @@ -0,0 +1,12 @@ +--- +layout: default +parent: Running ClusterFuzzLite +grand_parent: ClusterFuzzLite +title: GitHub Actions +nav_order: 1 +permalink: /clusterfuzzlite/running-clusterfuzzlite/github-actions/ +--- + +# GitHub Actions + +TODO diff --git a/docs/clusterfuzzlite/running_clusterfuzzlite.md b/docs/clusterfuzzlite/running_clusterfuzzlite.md new file mode 100644 index 00000000..068fc03e --- /dev/null +++ b/docs/clusterfuzzlite/running_clusterfuzzlite.md @@ -0,0 +1,101 @@ +--- +layout: default +parent: ClusterFuzzLite +title: Running ClusterFuzzLite +has_children: true +nav_order: 3 +permalink: /clusterfuzzlite/running-clusterfuzzlite/ +--- +# Running ClusterFuzzLite +{: .no_toc} + +- TOC +{:toc} +--- + +## Overview +TODO: add a diagram. + +Once your project's fuzzers can be built and run by the helper script, it is +ready to be fuzzed by ClusterFuzzLite. +The exact method for doing this will depend on the how you are running +ClusterFuzzLite. For guides on how to run ClusterFuzzLite in your particular +environment (e.g. GitHub Actions) see the subguides. +The rest of this page will explain concepts configuration options and that are +agnostic to how ClusterFuzzLite is being run. + +## ClusterFuzzLite Tasks + +ClusterFuzzLite has the concept of tasks which instruct ClusterFuzzLite what to +do when running. + +### Code Review Fuzzing + +TODO(metzman): Work on a generic name for CIFuzz/PR fuzzing. + +One of the core ways for ClusterFuzzLite to be used is for fuzzing code that is +in review that was just commited. +This use-case is important because it allows ClusterFuzzLite to find bugs before +they are commited into your code and while they are easiest to fix. +To use Code Review Fuzzing, set the configuration option `clusterfuzzlite-task` +to `code-review`. +If you are familiar with OSS-Fuzz's CIFuzz, this task is similar to CIFuzz. +Running other ClusterFuzzLite tasks enhances ClusterFuzzLite's ability to do +Code Review Fuzzing. + +If [Batch Fuzzing] is enabled, Code Review Fuzzing will report only newly +introduced bugs and use the corpus developed during batch fuzzing. +If [Code Coverage Reporting] is enabled, Code Review Fuzzing will try to only +run the fuzzers affected by the code change. + +### Batch Fuzzing + +ClusterFuzzLite can also run in a batch fuzzing mode where all fuzzers are run +for a long amount of time. Unlike Code Review Fuzzing, this task is not meant to +be interactive, it is meant to be long-lasting and generally is more similar to +fuzzing in ClusterFuzz than Code Review Fuzzing. Batch Fuzzing allows +ClusterFuzzLite to build up a corpus for each of your fuzz targets. This corpus +will be used in Code Coverage Reporting as well as Code Review Fuzzing. + +### Corpus Prune + +If multiple Batch Fuzzing tasks are run concurrently then we strongly recommend +running a pruning task as well. This task is run according to some set schedule +(once a day is probably sufficient) to prune the corpus of redundant testcases, +which can happen if multiple Batch Fuzzing jobs are done concurrently. + +### Code Coverage Report + +The last task ClusterFuzzLite offers is Code Coverage Reports. This task will +run your fuzzers on the corpus developed during Batch Fuzzing and will generate +an HTML report that shows you which part of your code is covered by batch +fuzzing. + +## Configuration Options + +Below are some configuration options that you can set when running +ClusterFuzzLite. +We will explain how to set these in each of the subguides. + +`language`: (optional) The language your target program is written in. Defaults +to `c++`. This should be the same as the value you set in `project.yaml`. See +[this explanation]({{ site.baseurl }}//getting-started/new-project-guide/#language) +for more details. + +`fuzz-time`: Determines how long ClusterFuzzLite spends fuzzing your project in +seconds. The default is 600 seconds. + +`sanitizer`: Determines a sanitizer to build and run fuzz targets with. The +choices are `'address'`, and `'undefined'`. The default is `'address'`. + +`task`: The task for ClusterFuzzLite to execute. `code-review` +by default. See [ClusterFuzzLite Tasks] for more details on how to run different +tasks. +TODO(metzman): change run_fuzzers_mode to this. + +`dry-run`: Determines if ClusterFuzzLite surfaces bugs/crashes. The default +value is `false`. When set to `true`, ClusterFuzzLite will never report a +failure even if it finds a crash in your project. This requires the user to +manually check the logs for detected bugs. + +TODO(metzman): We probably want a TOC on this page for subguides. |