aboutsummaryrefslogtreecommitdiffhomepage
path: root/site
diff options
context:
space:
mode:
authorGravatar hcm <hcm@google.com>2015-03-02 11:25:25 -0800
committerGravatar Commit bot <commit-bot@chromium.org>2015-03-02 11:25:25 -0800
commit281bf5249c2930ff4e8a500c8c26deb8e3253b11 (patch)
tree07deaaed87b4e4136a89c8124e49858160cbf454 /site
parentcc1ac862752031fa3efd7429800a0f791c24e5cf (diff)
Add testing section of docs. Consolidated trybot/buildbot section
Diffstat (limited to 'site')
-rw-r--r--site/dev/testing/buildbot.md70
-rw-r--r--site/dev/testing/fonts.md118
-rw-r--r--site/dev/testing/index.md16
-rw-r--r--site/dev/testing/testing.md (renamed from site/dev/contrib/testing.md)0
-rw-r--r--site/dev/testing/tests.md (renamed from site/dev/contrib/tests.md)0
5 files changed, 204 insertions, 0 deletions
diff --git a/site/dev/testing/buildbot.md b/site/dev/testing/buildbot.md
new file mode 100644
index 0000000000..7d9035db14
--- /dev/null
+++ b/site/dev/testing/buildbot.md
@@ -0,0 +1,70 @@
+Skia Buildbots
+==============
+
+Overview
+--------
+
+Like the Chromium team, the Skia team uses [buildbot](http://trac.buildbot.net/)
+to run continuous builds and tests.
+
+Here is a link to our main status page: https://status.skia.org/
+
+There are also Skia client, compile, Android, and FYI console pages for a detailed
+view of those results:
+
+ Externally-facing: http://build.chromium.org/p/client.skia/console
+
+ Internally-facing: http://chromegw.corp.google.com/i/client.skia/console
+ http://chromegw.corp.google.com/i/client.skia.internal/console
+ \(only visible internally\)
+
+Architecture
+------------
+
+The buildbot system consists of these elements: \(see
+http://buildbot.net/buildbot/docs/current/manual/introduction.html#system-architecture
+for more detail\)
+
+* builder
+
+ * one repeatable build and/or test configuration on a given platform.
+ * each builder maintains its own local checkout of the Skia repo
+ * only one builder is running at any given time on any single buildslave; otherwise,
+ different builders could interfere with each other's performance numbers
+
+* buildbot master
+
+ * watches for new commits to land in the Skia repository
+ \(https://skia.googlesource.com/skia\)
+ * whenever a new commit lands, it tells buildbot slaves to start building and
+ testing the latest revision
+ * serves up status pages whenever anybody requests them
+
+* buildslave \(or "buildbot slave"\)
+
+ * a process on a machine that builds and runs code as directed by the buildbot
+ master
+ * one or more builders run on each buildslave
+
+* build
+
+ * one run of a particular builder, at a particular code revision
+
+
+Status View
+------------
+
+The status view shows a table with builders, grouped by test type and platform,
+on the X-axis and commits on the Y-axis. The cells are colored according to
+the status of the build for each commit: green indicates success, red indicates
+failure, light orange indicates an in-progress build, and white indicates that
+no build has started yet for a given revision. Commits are listed by author, and
+the branch on which the commit was made is shown on the very left.
+
+For more detail, you can click on an individual cell to get a summary of the
+steps which ran for that build. You can also click one of the white bars at
+the top of each column to see a summary of recent builds for a given builder.
+
+
+
+
diff --git a/site/dev/testing/fonts.md b/site/dev/testing/fonts.md
new file mode 100644
index 0000000000..317ee9f1fa
--- /dev/null
+++ b/site/dev/testing/fonts.md
@@ -0,0 +1,118 @@
+Fonts and GM Tests
+==================
+
+Overview
+--------
+
+Each test in the gm directory draws a reference image. Their primary purpose is
+to detect when images change unexpectedly, indicating that a rendering bug has
+been introduced.
+
+The gm tests have a secondary purpose: they detect when rendering is different
+across platforms and configurations.
+
+The dm \(Diamond Master\) tool supports flags that minimize or eliminate the
+differences introduced by the font scaler native to each platform.
+
+
+Portable fonts
+--------------
+
+The most portable font format uses Skia to draw characters directly from paths,
+and contains a idealized set of font metrics. This does not exercise platform
+specific fonts at all, but does support specifying the font name, font size,
+font style, and attributes like fakeBold. The paths are generated on a reference
+platform \(currently a Mac\) and are stored as data in
+'tools/test_font_data.cpp' .
+
+To use portable fonts, pass '\-\-portableFonts' to dm.
+
+
+Resource fonts
+--------------
+
+The '\-\-resourceFonts' flag directs dm to use font files present in the resources
+directory. By using the same font set on all buildbots, the generated gm images
+become more uniform across platforms.
+
+Today, the set of fonts used by gm, and present in my resources directory,
+include:
+
+ * Courier New Bold Italic.ttf
+ * Courier New Bold.ttf
+ * Courier New Italic.ttf
+ * Courier New.ttf
+ * LiberationSans-Bold.ttf
+ * LiberationSans-BoldItalic.ttf
+ * LiberationSans-Italic.ttf
+ * LiberationSans-Regular.ttf
+ * Papyrus.ttc
+ * Pro W4.otf
+ * Times New Roman Bold Italic.ttf
+ * Times New Roman Bold.ttf
+ * Times New Roman Italic.ttf
+ * Times New Roman.ttf
+
+
+System fonts
+------------
+
+If neither '\-\-portableFonts' nor '\-\-resourceFonts' is specified, dm uses the fonts
+present on the system. Also, if '\-\-portableFonts' or '\-\-resourceFonts' is specified
+and the desired font is not available, the native font lookup algorithm is
+invoked.
+
+
+GM font selection
+-----------------
+
+Each gm specifies the typeface to use when drawing text. For now, to set the
+portable typeface on the paint, call:
+
+~~~~
+sk_tool_utils::set_portable_typeface(SkPaint* , const char* name = NULL,
+SkTypeface::Style style = SkTypeface::kNormal );
+~~~~
+
+To create a portable typeface, use:
+
+~~~~
+SkTypeface* typeface = sk_tool_utils::create_portable_typeface(const char* name,
+SkTypeface::Style style);
+~~~~
+
+Eventually, both 'set_portable_typeface()' and 'create_portable_typeface()' will be
+removed. Instead, a test-wide 'SkFontMgr' will be selected to choose portable
+fonts or resource fonts.
+
+
+Adding new fonts and glyphs to a GM
+-----------------------------------
+
+If a font is missing from the portable data or the resource directory, the
+system font is used instead. If a glyph is missing from the portable data, the
+first character, usually a space, is drawn instead.
+
+Running dm with '\-\-portableFonts' and '\-\-reportUsedChars' generates
+'tools/test_font_data_chars.cpp', which describes the fonts and characters used by
+all gm tests. Subsequently running the 'create_test_font' tool generates new paths
+and writes them into 'tools/test_font_data.cpp' .
+
+
+Future work
+-----------
+
+The font set used by gm tests today is arbitrary and not intended to be
+cross-platform. By choosing fonts without licensing issues, all bots can freely
+contain the same fonts. By narrowing the font selection, the size of the test
+font data will be more manageable.
+
+Adding support for selecting from multiple font managers at runtime permits
+removing manual typeface selection in the gm tests. Today, options to dm like
+'\-\-pipe' fail with '\-\-portableFonts' because we're hard-coded to using the default
+font manage when pictures are serialized.
+
+Some gm tests explicitly always want to use system fonts and system metrics;
+other gm tests use text only to label the drawing; yet other gm tests use text
+to generate paths for testing. Additional discrimination is needed to
+distinguish these cases.
diff --git a/site/dev/testing/index.md b/site/dev/testing/index.md
new file mode 100644
index 0000000000..fb90ec106f
--- /dev/null
+++ b/site/dev/testing/index.md
@@ -0,0 +1,16 @@
+Testing
+=======
+
+Skia relies heavily on our suite of unit and Golden Master \(GM\) tests, which
+are served by our Diamond Master \(DM\) test tool, for correctness testing.
+Tests are executed by our trybots, for every commit, across most of our
+supported platforms and configurations.
+Skia [Gold](https://gold.skia.org) is a web interface for triaging these results.
+
+We also have a robust set of performance tests, served by the nanobench tool and
+accessible via the Skia [Perf](https://perf.skia.org) web interface.
+
+Cluster Telemetry is a powerful framework that helps us capture and benchmark
+SKP files, a binary format for draw commands, across up to one million websites.
+
+See the individual subpages for more details on our various test tools.
diff --git a/site/dev/contrib/testing.md b/site/dev/testing/testing.md
index 1ed56235bf..1ed56235bf 100644
--- a/site/dev/contrib/testing.md
+++ b/site/dev/testing/testing.md
diff --git a/site/dev/contrib/tests.md b/site/dev/testing/tests.md
index fbe84dfeff..fbe84dfeff 100644
--- a/site/dev/contrib/tests.md
+++ b/site/dev/testing/tests.md