aboutsummaryrefslogtreecommitdiffhomepage
path: root/infra/cifuzz/continuous_integration.py
Commit message (Collapse)AuthorAge
* infra: allow passing architecture=i386 to CIFuzz (#7779)Gravatar Evgeny Vereshchagin2022-06-08
| | | | | | | | | | | | | | to mostly make sure that fuzz targets are buildable with architecture=i386. Ideally CIFuzz should also download the latest corpora using the "clusterfuzz-builds-i386" links but it kind of works even without that. It was tested in https://github.com/evverx/oss-fuzz/pull/13 by pointing https://github.com/evverx/systemd/pull/110 to that fork of the oss-fuzz repository. To judge from https://github.com/evverx/systemd/actions/runs/2406321298 it seems to be working more or less. The "i386" job failed there because https://github.com/systemd/systemd/commit/89b6a3f13e5f3b8a375dc82cb2a1c2c204a5067e to test "i386" as much as possible.
* [cifuzz] Copy repo from image before checking out. (#6941)Gravatar jonathanmetzman2021-12-01
| | | | | | | | | | | | * [cifuzz] Copy repo from image before checking out. Do this instead of cloning repo anew. Fixes: https://github.com/google/oss-fuzz/issues/6755 * fix tests * Add test code for cifuzz-example * fix
* Abstract away duplicate code in prepare_for_fuzzer_build (#6706)Gravatar jonathanmetzman2021-11-01
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Share more code between the 4 implementations of `prepare_for_fuzzer_build`. This simplifies the code of these implementations, reduces repetition and makes them easier to understand. Create helper functions/methods for: 1. Creating a failed `BuildPreparationResult`: `get_build_preparation_failure` 2. Building an external project docker image: `_build_external_project_docker_image` 3. Cloning a repo and checking out the specified commit/pr: `_clone_repo_and_checkout` 4. Detecting the main repo `_detect_main_repo` 5. Creating a repo manager from an existing checkout: `_create_repo_manager_for_project_src_path` Change `ExternalGeneric` implementation of `prepare_for_fuzzer_build` to: 1. Call `_create_repo_manager_for_project_src_path` 2. Call `_build_external_project_docker_image` Change `InternalGeneric` implementation of `prepare_for_fuzzer_build` to: 1. Call `_detect_main_repo` 2. Call `_create_repo_manager_for_project_src_path` Change `ExternalGithub` implementation of `prepare_for_fuzzer_build` to: 1. Call `_clone_repo_and_checkout` 2. Call `_build_external_project_docker_image` Change `InternalGithub` implementation of `prepare_for_fuzzer_build` to: 1. Call `_detect_main_repo` 2. Call `_clone_repo_and_checkout`
* Fix diffing on non-GitHub and improve config (#6707)Gravatar jonathanmetzman2021-10-31
| | | | | | | | | | | | | | | | | | | Fix diffing on non-GitHub and improve config 1. Remove obsolete comments. 2. Remove unused function get_pr_ref 3. Correct comment on git_sha 4. Rename commit_sha to git_sha 5. Make base_commit, pr_ref, and base_ref non-github specific and move to CiEnvironment. 6. Hoist get_diff_base to base class to allow diffing 7. Fix downloading coverage on non-Github. 8. Add TODO so that we don't assume github actions is run on github.com (enterprise users) 9. Rename repo_url to git_url. 10. Add missing git_url method base class. 11. Clarify what git_url, git_sha and pr_ref are for and leave TODOs about how we can eliminate them. 12. Fix typos.
* Fix download_latest_build for pull requests. (#6688)Gravatar Oliver Chang2021-10-29
| | | | | | | | | | | | Getting the LATEST_BUILD_WINDOW latest commits from HEAD^ is wrong. We should use the diff base instead. Also modify the Ci.get_diff_base methods to not include "..." in the return value. This will be appended in get_changed_code_under_test instead. This also potentially fixes a behaviour mismatch with GitHub PRs and diffing against the latest base branch rather than git merge-base base-branch HEAD.
* [ClusterFuzzLite] Support GCB and gsutil/gcs as filestore. (#6629)Gravatar jonathanmetzman2021-10-27
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * add gsutil filestore * lint * Fix * Add build image script * get gcb fuzzing working * fmt and fix config_utils_test * Check that crashes are uploaded * Add no_filestore * fix test * fix tests * fix * Print crash URL * Fix * fix * fmt * lnt * fix * fmt
* cflite: Support private repos (#6592)Gravatar Oliver Chang2021-10-20
| | | | | Pass github token during cloning. Fixes #6584
* Don't upload builds in run_fuzzers. (#6151)Gravatar Oliver Chang2021-08-17
| | | | | | | | | | The current way adds a lot of ordering assumptions, and doesn't fit too well with parallel batch fuzzing either. Add a "upload-build" boolean action input that can be added to "build_fuzzers" to upload latest builds instead. Builds are now uploaded by commit hash, rather than a fixed "latest" name. ClusterFuzzLite's download_latest_build will check the last 3 commits and download the first available build by git hash.
* [CIFuzz] Don't overwrite the repo in external builds. (#6131)Gravatar jonathanmetzman2021-07-29
| | | | | Fixes: #6119 Also add more tests for continuous_integration module.
* Make oss-fuzz-project-name optional. (#6071)Gravatar Oliver Chang2021-07-21
| | | | | | | | | Also rename code uses from `project_name` to `oss_fuzz_project_name`, to be more explicit, and use it to determine whether or not we're running an OSS-Fuzz project or not. For external fuzzing #6051.
* [CIFuzz] Support non-github action CI for external projects (#5824)Gravatar jonathanmetzman2021-05-24
| | | | | This is incomplete for a few reasons. 1. It probably can't diff properly. 2. It assumes the project source should be put in /src/$PROJECT_NAME.
* [cifuzz] Abstract-away OSS-Fuzz specific bits (#5088)Gravatar jonathanmetzman2021-02-03
| | | | | Abstract away OSS-Fuzz specific bits into the OSSFuzz implementation of the ClusterFuzzDeployment class. This will make it easier to implement support for other deployments of ClusterFuzz (including ClusterFuzzLite).
* [cifuzz] Support a batch fuzzing mode (#5073)Gravatar jonathanmetzman2021-02-01
| | | In this mode, CIFuzz will keep fuzzing until the time limit is reached, even if a crash was found.
* [CIFuzz] Fix diffing + Refactor (#5032)Gravatar jonathanmetzman2021-01-28
* [CIFuzz] Fix diffing + Refactor Make diffing work in two scenarios where it previously failed: 1. Commit fuzzing. In this case, we diff $COMMIT against $COMMIT^1 because the intent here is to fuzz the commit. 2. Fuzzing PRs that aren't to master. In this case, we previously were diffing against origin/master. Instead, diff against the local version of the base repo. This also has the nice effect of handling PRs that havent pulled from master recently enough. Also do refactoring. 1. Move code that is different for differenct CI systems to continuous_integration.py 2. Change how configuration in build_fuzzers is handled. Previously configuration was gotten in build_fuzzers_entrypoint and passed as individual params. This made code ugly and hard to read. Instead, move code dealing with config to it's own module config_utils. This module implements a config class which can be used to create objects that are passed around to code that needs it. Making the code much easier to read. TODO: Move run_fuzzers code to new config system.