From 767276e4bd97ebea690884d156862d5e6e8aa7fd Mon Sep 17 00:00:00 2001 From: hcm Date: Fri, 7 Aug 2015 10:09:03 -0700 Subject: Gold Workflows Documentation BUG=skia: NOTRY=true DOCS_PREVIEW= https://skia.org/?cl=1276663003 Review URL: https://codereview.chromium.org/1276663003 --- site/dev/testing/BlameView.png | Bin 0 -> 192389 bytes site/dev/testing/ByTest.png | Bin 0 -> 204022 bytes site/dev/testing/ClusterConfig.png | Bin 0 -> 218419 bytes site/dev/testing/Digests.png | Bin 0 -> 417670 bytes site/dev/testing/DotDiagram.png | Bin 0 -> 319870 bytes site/dev/testing/Grid.png | Bin 0 -> 328877 bytes site/dev/testing/Ignores.png | Bin 0 -> 259943 bytes site/dev/testing/IssueHighlight.png | Bin 0 -> 193256 bytes site/dev/testing/Search.png | Bin 0 -> 210544 bytes site/dev/testing/skiagold.md | 150 ++++++++++++++++++++++++++++++++++++ 10 files changed, 150 insertions(+) create mode 100644 site/dev/testing/BlameView.png create mode 100644 site/dev/testing/ByTest.png create mode 100644 site/dev/testing/ClusterConfig.png create mode 100644 site/dev/testing/Digests.png create mode 100644 site/dev/testing/DotDiagram.png create mode 100644 site/dev/testing/Grid.png create mode 100644 site/dev/testing/Ignores.png create mode 100644 site/dev/testing/IssueHighlight.png create mode 100644 site/dev/testing/Search.png create mode 100644 site/dev/testing/skiagold.md (limited to 'site/dev') diff --git a/site/dev/testing/BlameView.png b/site/dev/testing/BlameView.png new file mode 100644 index 0000000000..8884dbcf25 Binary files /dev/null and b/site/dev/testing/BlameView.png differ diff --git a/site/dev/testing/ByTest.png b/site/dev/testing/ByTest.png new file mode 100644 index 0000000000..75857f1674 Binary files /dev/null and b/site/dev/testing/ByTest.png differ diff --git a/site/dev/testing/ClusterConfig.png b/site/dev/testing/ClusterConfig.png new file mode 100644 index 0000000000..6854d9527d Binary files /dev/null and b/site/dev/testing/ClusterConfig.png differ diff --git a/site/dev/testing/Digests.png b/site/dev/testing/Digests.png new file mode 100644 index 0000000000..efd811fb49 Binary files /dev/null and b/site/dev/testing/Digests.png differ diff --git a/site/dev/testing/DotDiagram.png b/site/dev/testing/DotDiagram.png new file mode 100644 index 0000000000..7bf8b990b8 Binary files /dev/null and b/site/dev/testing/DotDiagram.png differ diff --git a/site/dev/testing/Grid.png b/site/dev/testing/Grid.png new file mode 100644 index 0000000000..192d340a0b Binary files /dev/null and b/site/dev/testing/Grid.png differ diff --git a/site/dev/testing/Ignores.png b/site/dev/testing/Ignores.png new file mode 100644 index 0000000000..a4200ca36d Binary files /dev/null and b/site/dev/testing/Ignores.png differ diff --git a/site/dev/testing/IssueHighlight.png b/site/dev/testing/IssueHighlight.png new file mode 100644 index 0000000000..c21fda9b99 Binary files /dev/null and b/site/dev/testing/IssueHighlight.png differ diff --git a/site/dev/testing/Search.png b/site/dev/testing/Search.png new file mode 100644 index 0000000000..4e8491e3c9 Binary files /dev/null and b/site/dev/testing/Search.png differ diff --git a/site/dev/testing/skiagold.md b/site/dev/testing/skiagold.md new file mode 100644 index 0000000000..9fc9bbafbe --- /dev/null +++ b/site/dev/testing/skiagold.md @@ -0,0 +1,150 @@ +Skia Gold +========= + +Recommended Workflows +--------------------- +### How to best use Gold for commonly faced problems ### + +These instructions will refer to various views which are accessible via the left +navigation on [gold.skia.org](https://gold.skia.org/). +View access is public, triage access is granted to +Skia contributors. You must be logged in to triage. + +Problem #1: As sheriff, I need to triage and “assign” many incoming new images. +------------------------------------------------------------------------------- +Solution today: + +* Access the By Blame view to see digests needing triage and associated + owners/CLs + + Only untriaged digests will be shown by default + + Blame is not sorted in any particular order + + Digests are clustered by runs and the most minimal set of blame + +
+ +* Select digests for triage + + Digests will be listed in order with largest difference first + + Click to open the digest view with detailed information + +
+ +* Open bugs for identified owner(s) + + The digest detail view has a link to open a bug from the UI + + Via the Gold UI or when manually entering a bug, copy the full URL of + single digest into a bug report + + The URL reference to the digest in Issue Tracker will link the bug to + the digest in Gold + +
+ +
+ +Future improvements: + +* Smarter, more granular blamelist + +
+ +Problem #2: As a developer, I need to land a CL that may change many images. +---------------------------------------------------------------------------- +To find your results: + +* Immediately following commit, access the By Blame view to find untriaged + digest groupings associated with your ID +* Click on one of the clusters including your CL to triage +* Return to the By Blame view to walk through all untriaged digests involving + your change +* Note: It is not yet implemented in the UI but possible to filter the view + by CL. Delete hashes in the URL to only include the hash for your CL. + +
+ +To rebaseline images: + +* Access the Ignores view and create a new, short-interval (hours) ignore for + the most affected configuration(s) + +
+ + +* Click on the Ignore to bring up a search view filtered by the affected + configuration(s) +* Mark untriaged images as positive (or negative if appropriate) +* Follow one of two options for handling former positives: + + Leave former positives as-is and let them fall off with time if there is + low risk of recurrence + + Mark former positives as negative if needed to verify the change moving + forward + +Future improvements: + +* Trybot support prior to commit, with view limited to your CL +* Pre-triage prior to commit that will persist when the CL lands + +
+ +Problem #3: As a developer or infrastructure engineer, I need to add a new or updated config. +--------------------------------------------------------------------------------------------- +(ie: new bot, test mode, environment change) + +Solution today: + +* Follow the process for rebaselining images: + + Wait for the bot/test/config to be committed and show up in the Gold UI + + Access the Ignores view and create a short-interval ignore for the + configuration(s) + + Triage the ignores for that config to identify positive images + + Delete the ignore + +Future improvements: + +* Introduction of a new or updated test can make use of try jobs and pre-triage. +* New configs may be able to use these features as well. + +
+ +Problem #4: As a developer, I need to analyze the details of a particular image digest. +--------------------------------------------------------------------------------------- +Solution: + +* Access the By Test view + +
+ +* Click the magnifier to filter by configuration +* Access the Cluster view to see the distribution of digest results + + Use control-click to select and do a direct compare between data points + + Click on configurations under “parameters” to highlight data points and + compare + +
+ +* Access the Grid view to see NxN diffs + +
+ +* Access the Dot diagram to see history of commits for the trace + + Each dot represents a commit + + Each line represents a configuration + + Dot colors distinguish between digests + +
+ +
+ +Future improvements: + +* Large diff display of image vs image + +
+ +Problem #5: As a developer, I need to find results for a particular configuration. +---------------------------------------------------------------------------------- +Solution: + +* Access the Search view +* Select any parameters desired to search across tests + +
+ + -- cgit v1.2.3