diff options
author | 2017-01-17 07:26:41 -0500 | |
---|---|---|
committer | 2017-01-17 14:54:12 +0000 | |
commit | 6d4d6cce5e6e73d14c5d39c20321b189d737dfd5 (patch) | |
tree | d7f273f5eaad4571143410c1e15ae6db2f71fd9e | |
parent | 7cae170c1dd0af567c3eff3d838768ef30ce4665 (diff) |
Update Buildbot doc for Task Scheduler
BUG=skia:6118
NOTRY=true
DOCS_PREVIEW= https://skia.org/?cl=7023
Change-Id: I398f4fd98959bb37bcb5499d38e517542e0df90f
Reviewed-on: https://skia-review.googlesource.com/7023
Commit-Queue: Ben Wagner <benjaminwagner@google.com>
Reviewed-by: Ben Wagner <benjaminwagner@google.com>
-rw-r--r-- | site/dev/testing/automated_testing.md | 90 | ||||
-rw-r--r-- | site/dev/testing/buildbot.md | 102 |
2 files changed, 90 insertions, 102 deletions
diff --git a/site/dev/testing/automated_testing.md b/site/dev/testing/automated_testing.md new file mode 100644 index 0000000000..d7a817a925 --- /dev/null +++ b/site/dev/testing/automated_testing.md @@ -0,0 +1,90 @@ +Skia Automated Testing +====================== + +Overview +-------- + +Skia uses [Swarming](https://github.com/luci/luci-py/blob/master/appengine/swarming/doc/Design.md) +to do the heavy lifting for our automated testing. It farms out tasks, which may +consist of compiling code, running tests, or any number of other things, to our +bots, which are virtual or real machines living in our local lab, Chrome Infra's +lab, or in GCE. + +The [Skia Task Scheduler](http://go/skia-task-scheduler) determines what tasks +should run on what bots at what time. See the link for a detailed explanation of +how relative task priorities are derived. A *task* corresponds to a single +Swarming task. A *job* is composed of a directed acyclic graph of one or more +*tasks*. The job is complete when all of its component tasks have succeeded +or is considered a failure when any of its component tasks fails. The scheduler +may automatically retry tasks within its set limits. Jobs are not retried. +Multiple jobs may share the same task, for example, tests on two different +Android devices which use the same compiled code. + +Each Skia repository has an infra/bots/tasks.json file which defines the jobs +and tasks for the repo. Most jobs will run at every commit, but it is possible +to specify nightly and weekly jobs as well. For convenience, most repos also +have a gen_tasks.go which will generate tasks.json. You will need to +[install Go](https://golang.org/doc/install). From the repository root: + + $ go get -u go.skia.org/infra/... + $ go run infra/bots/gen_tasks.go + +It is necessary to run gen_tasks.go every time it is changed or every time an +[asset](https://skia.googlesource.com/skia/+/master/infra/bots/assets/README.md) +has changed. There is also a test mode which simply verifies that the tasks.json +file is up to date: + + $ go run infra/bots/gen_tasks.go --test + + + +Try Jobs +-------- + +It is useful to know how your change will perform before it is submitted. After +uploading your CL to [Gerrit](https://skia-review.googlesource.com/), you may +trigger a try job for any job listed in tasks.json: + + $ git cl try -B <bucket name> -b <job name> + +The bucket name refers to the [Buildbucket](https://chromium.googlesource.com/infra/infra/+/master/appengine/cr-buildbucket/README.md) +bucket to which the request will be submitted. Most public Skia repos use the +"skia.primary" bucket, and most private Skia repos use the "skia.internal" +bucket. + + +Status View +------------ + +The status view shows a table with tasks, grouped by test type and platform, +on the X-axis and commits on the Y-axis. The cells are colored according to +the status of the task for each commit: + +* green: success +* orange: failure +* purple: exception (infrastructure issue) +* black border, no fill: task in progress +* blank: no task has started yet for a given revision + +Commits are listed by author, and the branch on which the commit was made is +shown on the very left. A purple result will override an orange result. + +For more detail, you can click on an individual cell to get a summary of the +task. You can also click one of the white bars at the top of each column to see +a summary of recent tasks with the same name. + +The status page has several filters which can be used to show only a subset of +task specs: + +* Interesting: Task specs which have both successes and failures within the + visible commit window. +* Failures: Task specs which have failures within the visible commit window. +* Comments: Task specs which have comments. +* Failing w/o comment: task specs which have failures within the visible commit + window but have no comments. +* All: Display all tasks. +* Search: Enter a search string. Substrings and regular expressions may be + used, per the Javascript String Match() rules: + http://www.w3schools.com/jsref/jsref_match.asp + + diff --git a/site/dev/testing/buildbot.md b/site/dev/testing/buildbot.md deleted file mode 100644 index 0bee36cab3..0000000000 --- a/site/dev/testing/buildbot.md +++ /dev/null @@ -1,102 +0,0 @@ -Skia Buildbots -============== - -Overview --------- - -Like the Chromium team, the Skia team uses [buildbot](http://trac.buildbot.net/) -to run continuous builds and tests. - -Here is a link to our main status page: https://status.skia.org/ - -There are also buildbot console pages for a detailed view of those results: - - Externally-facing: - -* http://build.chromium.org/p/client.skia/console -* http://build.chromium.org/p/client.skia.android/console -* http://build.chromium.org/p/client.skia.compile/console -* http://build.chromium.org/p/client.skia.fyi/console - - Internally-facing: - -* http://uberchromegw.corp.google.com/i/client.skia/console -* http://uberchromegw.corp.google.com/i/client.skia.android/console -* http://uberchromegw.corp.google.com/i/client.skia.compile/console -* http://uberchromegw.corp.google.com/i/client.skia.fyi/console -* http://uberchromegw.corp.google.com/i/client.skia.internal/console - - -Architecture ------------- - -The buildbot system consists of these elements: \(see -http://buildbot.net/buildbot/docs/current/manual/introduction.html#system-architecture -for more detail\) - -* Buildbot Master - - * Watches for new commits to land in the Skia repository - \(https://skia.googlesource.com/skia\) - * Whenever a new commit lands, it triggers a **Build** on each **Builder** - to test the new revision. - * Serves up status pages whenever anybody requests them - -* Build - - * One run of a particular **Builder**, at a particular code revision. - * "Build" is sort of a misnomer; it's just a list of steps (typically shell - commands) which are run by the **Buildslave** process on the host - machine, and those may include compiling and running code as well as - arbitrary other commands. - -* Builder - - * One repeatable build and/or test configuration on a given platform. The - Builder is basically a blueprint which provides logic to determine which - steps to run within a Build. - -* Buildslave \(or "buildbot slave"\) - - * A process running on a host machine that builds and runs code as directed - by the Buildbot Master. - * One or more Builders may run on a given Buildslave, but only one runs at - a time. - * One or more Buildslaves may run on a given host machine. - - -Status View ------------- - -The status view shows a table with builders, grouped by test type and platform, -on the X-axis and commits on the Y-axis. The cells are colored according to -the status of the build for each commit: - -* green: success -* orange: failure -* purple: exception (infrastructure issue) -* black border, no fill: build in progress -* blank: no build has started yet for a given revision - -Commits are listed by author, and the branch on which the commit was made is -shown on the very left. A purple result will override an orange result. - -For more detail, you can click on an individual cell to get a summary of the -steps which ran for that build. You can also click one of the white bars at -the top of each column to see a summary of recent builds for a given builder. - -The status page has several filters which can be used to show only a subset of -bots: - -* Interesting: Bots which have both successes and failures within the visible - commit window. -* Failures: Bots which have failures within the visible commit window. -* Comments: Bots which have comments. -* Failing w/o comment: Bots which have failures within the visible commit window - but have no comments. -* All: Display all bots. -* Search: Enter a search string. Substrings and regular expressions may be - used, per the Javascript String Match() rules: - http://www.w3schools.com/jsref/jsref_match.asp - - |