aboutsummaryrefslogtreecommitdiffhomepage
path: root/tools/tests
Commit message (Collapse)AuthorAge
* rebaseline.py: add --keep-going-on-failure option, off by defaultGravatar epoger@google.com2013-07-16
| | | | | | | | R=borenet@google.com Review URL: https://codereview.chromium.org/18092004 git-svn-id: http://skia.googlecode.com/svn/trunk@10109 2bbb7eff-a529-9590-31e7-b0007b416f81
* Fix Housekeeper bot after r9961Gravatar epoger@google.com2013-07-10
| | | | git-svn-id: http://skia.googlecode.com/svn/trunk@9965 2bbb7eff-a529-9590-31e7-b0007b416f81
* rebaseline.py: error out early if --expectations-root not pointing at ↵Gravatar epoger@google.com2013-07-10
| | | | | | | | | | | gm-expected dir BUG=https://code.google.com/p/skia/issues/detail?id=1403 R=senorblanco@chromium.org Review URL: https://codereview.chromium.org/18416005 git-svn-id: http://skia.googlecode.com/svn/trunk@9961 2bbb7eff-a529-9590-31e7-b0007b416f81
* rebaseline.py: use specified configs/tests as a FILTER over actually-run testsGravatar epoger@google.com2013-07-10
| | | | | | | | | | | | | Now that we have the actual-results.json file for every bot type, we know the full set of tests that actual results are available for. If configs and/or tests are specified, just use those as FILTERS over the actually available results. R=senorblanco@chromium.org Review URL: https://codereview.chromium.org/18324018 git-svn-id: http://skia.googlecode.com/svn/trunk@9956 2bbb7eff-a529-9590-31e7-b0007b416f81
* rebaseline.py: if expectations dir contains JSON format results, update ↵Gravatar epoger@google.com2013-07-08
| | | | | | | | | | | | those instead of image files Part of Step 3 in https://goto.google.com/ChecksumTransitionDetail R=senorblanco@chromium.org Review URL: https://codereview.chromium.org/18348018 git-svn-id: http://skia.googlecode.com/svn/trunk@9910 2bbb7eff-a529-9590-31e7-b0007b416f81
* rebaseline.py self-tests: prepare to add tests of JSON-format rebaseliningGravatar epoger@google.com2013-07-03
| | | | | | | | | | | | For now, rebaseline.py still does not support JSON-format rebaselining, so the one self-test reports failure. But this change will allow us to test the JSON-format rebaselining once rebaseline.py supports it. R=scroggo@google.com Review URL: https://codereview.chromium.org/18348015 git-svn-id: http://skia.googlecode.com/svn/trunk@9896 2bbb7eff-a529-9590-31e7-b0007b416f81
* Create jsondiff tool to capture diffs between two JSON expectations files.Gravatar epoger@google.com2013-07-02
| | | | | | | | | | | | | This will be used to view diffs before committing rebaselines, or to review exactly what changed between two baseline revisions in the past. Part of Step 3 in https://goto.google.com/ChecksumTransitionDetail R=scroggo@google.com Review URL: https://codereview.chromium.org/18103005 git-svn-id: http://skia.googlecode.com/svn/trunk@9878 2bbb7eff-a529-9590-31e7-b0007b416f81
* Fix rebaseline.py self-test to account for r9859Gravatar epoger@google.com2013-07-02
| | | | | | | | R=robertphillips@google.com Review URL: https://codereview.chromium.org/18559002 git-svn-id: http://skia.googlecode.com/svn/trunk@9872 2bbb7eff-a529-9590-31e7-b0007b416f81
* Treat empty directory properly in skimage.Gravatar scroggo@google.com2013-06-21
| | | | | | | | | | | | | | | | Instead of treating an empty directory as a file that failed to decode, treat it as an empty directory. Add self tests to check for this. Builds upon from https://codereview.chromium.org/16866010, which was reverted. R=epoger@google.com Review URL: https://codereview.chromium.org/17101029 git-svn-id: http://skia.googlecode.com/svn/trunk@9730 2bbb7eff-a529-9590-31e7-b0007b416f81
* rebaseline.py: split image-based rebaselining, which will go away soon, into ↵Gravatar epoger@google.com2013-06-19
| | | | | | | | | | | | | | its own script (part of step 3 in https://goto.google.com/ChecksumTransitionDetail ) also adds new --expectations-root option R=scroggo@google.com, senorblanco@chromium.org Review URL: https://codereview.chromium.org/17379004 git-svn-id: http://skia.googlecode.com/svn/trunk@9689 2bbb7eff-a529-9590-31e7-b0007b416f81
* skimage: revert r9642Gravatar epoger@google.com2013-06-18
| | | | git-svn-id: http://skia.googlecode.com/svn/trunk@9647 2bbb7eff-a529-9590-31e7-b0007b416f81
* skimage: if --readPath points at an empty dir, create an empty expectations fileGravatar epoger@google.com2013-06-17
| | | | | | | | TBR=scroggo Review URL: https://codereview.chromium.org/17294006 git-svn-id: http://skia.googlecode.com/svn/trunk@9642 2bbb7eff-a529-9590-31e7-b0007b416f81
* skimage self-tests: confirm correctness of bitmap hash valuesGravatar epoger@google.com2013-06-13
| | | | | | | | R=scroggo@google.com Review URL: https://codereview.chromium.org/16866010 git-svn-id: http://skia.googlecode.com/svn/trunk@9591 2bbb7eff-a529-9590-31e7-b0007b416f81
* rebaseline.py: try to download images from Google Storage before skia-autogenGravatar epoger@google.com2013-06-12
| | | | | | | | | | This will allow us to complete Step 1 of https://goto.google.com/ChecksumTransitionDetail R=scroggo@google.com Review URL: https://codereview.chromium.org/16311011 git-svn-id: http://skia.googlecode.com/svn/trunk@9534 2bbb7eff-a529-9590-31e7-b0007b416f81
* rebaseline.py: add --add-new option, and only add new expectations if it is setGravatar epoger@google.com2013-06-12
| | | | | | | | R=senorblanco@chromium.org Review URL: https://codereview.chromium.org/16363006 git-svn-id: http://skia.googlecode.com/svn/trunk@9527 2bbb7eff-a529-9590-31e7-b0007b416f81
* rebaseline.py: improve readability of dry-run outputGravatar epoger@google.com2013-06-11
| | | | | | | | Split off of already-reviewed https://codereview.chromium.org/15789010/ ('rebaseline.py: --tests and --configs are now FILTERS within json results') Review URL: https://codereview.chromium.org/16509006 git-svn-id: http://skia.googlecode.com/svn/trunk@9507 2bbb7eff-a529-9590-31e7-b0007b416f81
* rebaseline.py: in option names, replace underscores with hyphensGravatar epoger@google.com2013-06-11
| | | | | | | | Split off of already-reviewed https://codereview.chromium.org/15789010/ ('rebaseline.py: --tests and --configs are now FILTERS within json results') Review URL: https://codereview.chromium.org/15904018 git-svn-id: http://skia.googlecode.com/svn/trunk@9504 2bbb7eff-a529-9590-31e7-b0007b416f81
* Preparations for running skimage tool on bots.Gravatar scroggo@google.com2013-06-07
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | skimage_main: Specify the full name of the expectations file to write on the command line, rather than the folder to write 'results.json'. This is consistent with writeJsonSummaryPath in GM, as well as the format for reading the file. Further, it allows more flexibility to name the file. Do not force opaque when writing bitmaps. skimage does its comparison on the raw bitmaps. The images are written for visual comparison, and mostly transparent bitmaps (like resources/plane.png) aren't meaningful when forced opaque. Provide an option to to write only mismatches to a directory. Handled similar to the writePath. Report a failure if there is no expectations file. This allows writing to the mismatchPath so we can create an initial baseline. Pass SkBitmap by const&. skimage_self_test: Include the name of the json file in the parameter passed to createExpectationsPath. The name has been changed to be more specific. .gitignore: Ignore the new name of the file created by skimage_self_test. run.sh: Run skimage_self_tests.py BUG=http://code.google.com/p/skia/issues/detail?id=1241 R=epoger@google.com Review URL: https://codereview.chromium.org/15813016 git-svn-id: http://skia.googlecode.com/svn/trunk@9474 2bbb7eff-a529-9590-31e7-b0007b416f81
* Run skimage as part of run.shGravatar scroggo@google.com2013-06-07
| | | | | | | | | | | | | Test skimage to ensure that it does not break. Split off from https://codereview.chromium.org/15813016/ BUG=http://code.google.com/p/skia/issues/detail?id=1241 R=epoger@google.com Review URL: https://codereview.chromium.org/15774017 git-svn-id: http://skia.googlecode.com/svn/trunk@9473 2bbb7eff-a529-9590-31e7-b0007b416f81
* rebaseline.py self-tests: split test inputs and outputs into completely ↵Gravatar epoger@google.com2013-06-06
| | | | | | | | separate directories Review URL: https://codereview.chromium.org/16557002 git-svn-id: http://skia.googlecode.com/svn/trunk@9459 2bbb7eff-a529-9590-31e7-b0007b416f81
* add more rebaseline test resources (no behavioral changes)Gravatar epoger@google.com2013-06-05
| | | | | | | | (SkipBuildbotRuns) Review URL: https://codereview.chromium.org/15774019 git-svn-id: http://skia.googlecode.com/svn/trunk@9446 2bbb7eff-a529-9590-31e7-b0007b416f81
* rebaseline.py: if --tests is not specified, get test list from ↵Gravatar epoger@google.com2013-06-05
| | | | | | | | | | actual-results.json R=scroggo@google.com Review URL: https://codereview.chromium.org/16093025 git-svn-id: http://skia.googlecode.com/svn/trunk@9443 2bbb7eff-a529-9590-31e7-b0007b416f81
* rebaseline.py : add self-testsGravatar epoger@google.com2013-06-04
| | | | | | | | R=senorblanco@chromium.org Review URL: https://codereview.chromium.org/15660014 git-svn-id: http://skia.googlecode.com/svn/trunk@9426 2bbb7eff-a529-9590-31e7-b0007b416f81
* Create a self test for skimage.Gravatar scroggo@google.com2013-05-14
| | | | | | | | | | | | Runs skimage twice: once to create an expectations file, and a second time comparing against the file. Uses the files in resources as test files. R=epoger@google.com Review URL: https://codereview.chromium.org/14969007 git-svn-id: http://skia.googlecode.com/svn/trunk@9123 2bbb7eff-a529-9590-31e7-b0007b416f81
* bench_graph_svg: add r7618 results (NOT per-tile) to self-test source dataGravatar epoger@google.com2013-02-17
| | | | | | Review URL: https://codereview.appspot.com/7305098 git-svn-id: http://skia.googlecode.com/svn/trunk@7760 2bbb7eff-a529-9590-31e7-b0007b416f81
* bench_graph_svg self-test: add non-tile bench dataGravatar epoger@google.com2013-02-15
| | | | | | | TBR=benchen Review URL: https://codereview.appspot.com/7317048 git-svn-id: http://skia.googlecode.com/svn/trunk@7750 2bbb7eff-a529-9590-31e7-b0007b416f81
* tools/tests/run.sh: download test bench data from Google Storage rather than SVNGravatar epoger@google.com2013-02-15
| | | | | | Review URL: https://codereview.appspot.com/7343045 git-svn-id: http://skia.googlecode.com/svn/trunk@7749 2bbb7eff-a529-9590-31e7-b0007b416f81
* bench_graph_svg: generate output in consistently sorted order, to make diffs ↵Gravatar epoger@google.com2013-02-14
| | | | | | | | | more useful TBR=benchen Review URL: https://codereview.appspot.com/7330044 git-svn-id: http://skia.googlecode.com/svn/trunk@7744 2bbb7eff-a529-9590-31e7-b0007b416f81
* disable new bench_graph_svg test in tools/tests/run.sh that fails on housekeeperGravatar epoger@google.com2013-02-14
| | | | | | | TBR=bensong Review URL: https://codereview.appspot.com/7303092 git-svn-id: http://skia.googlecode.com/svn/trunk@7739 2bbb7eff-a529-9590-31e7-b0007b416f81
* tools/tests/run.sh: add self-test for bench_graph_svg.pyGravatar epoger@google.com2013-02-14
| | | | | | Review URL: https://codereview.appspot.com/7319044 git-svn-id: http://skia.googlecode.com/svn/trunk@7735 2bbb7eff-a529-9590-31e7-b0007b416f81
* Enable per-tile logging in bench_picturesGravatar borenet@google.com2013-01-11
| | | | | | | | | - Modify bench_pictures buildstep to translate "key: True" --> "--key" rather than requiring "key: value" --> "--key value" - Add whacky TileArgs helper which includes "timeIndividualTiles: True" Review URL: https://codereview.appspot.com/7092046 git-svn-id: http://skia.googlecode.com/svn/trunk@7141 2bbb7eff-a529-9590-31e7-b0007b416f81
* * Extended skdiff to report alpha channel differences.Gravatar rmistry@google.com2013-01-03
| | | | | | | | * Created a tools/tests/rebaseline.sh to copy output-actual into output-expected. * Sample results are available here: http://www.corp.google.com/~rmistry/skia/gm-playback-windows/images-skdiff/ Review URL: https://codereview.appspot.com/7038048 git-svn-id: http://skia.googlecode.com/svn/trunk@7003 2bbb7eff-a529-9590-31e7-b0007b416f81
* Add more warnings about https://code.google.com/p/skia/issues/detail?id=677 ↵Gravatar epoger@google.com2013-01-02
| | | | | | ('make tools/tests/run.sh work cross-platform') git-svn-id: http://skia.googlecode.com/svn/trunk@6965 2bbb7eff-a529-9590-31e7-b0007b416f81
* Address comments for r6822: https://codereview.appspot.com/6946052/Gravatar borenet@google.com2012-12-17
| | | | | | Review URL: https://codereview.appspot.com/6943059 git-svn-id: http://skia.googlecode.com/svn/trunk@6847 2bbb7eff-a529-9590-31e7-b0007b416f81
* Fix bench_pictures.cfg and add sanity checkGravatar borenet@google.com2012-12-14
| | | | | | Review URL: https://codereview.appspot.com/6946052 git-svn-id: http://skia.googlecode.com/svn/trunk@6822 2bbb7eff-a529-9590-31e7-b0007b416f81
* Update skdiff.Gravatar bungeman@google.com2012-12-05
| | | | | | | https://codereview.appspot.com/6850115/ git-svn-id: http://skia.googlecode.com/svn/trunk@6681 2bbb7eff-a529-9590-31e7-b0007b416f81
* Add a self-test for gm, similar to our self-test for skdiff.Gravatar epoger@google.com2012-11-12
| | | | | | Review URL: https://codereview.appspot.com/6827068 git-svn-id: http://skia.googlecode.com/svn/trunk@6380 2bbb7eff-a529-9590-31e7-b0007b416f81
* Adding checkboxes to skdiff output so a user can select a subset of images ↵Gravatar jkleinert@google.com2012-08-30
| | | | | | | | | to rebaseline. BUG=745 Review URL: https://codereview.appspot.com/6497054 git-svn-id: http://skia.googlecode.com/svn/trunk@5360 2bbb7eff-a529-9590-31e7-b0007b416f81
* skdiff: recurse over subdirectories, unless --norecurse option is givenGravatar epoger@google.com2012-08-16
| | | | | | | in anticipation of https://code.google.com/p/skia/issues/detail?id=743 ('move gm baselines outside of trunk, and modify naming convention') Review URL: https://codereview.appspot.com/6465053 git-svn-id: http://skia.googlecode.com/svn/trunk@5121 2bbb7eff-a529-9590-31e7-b0007b416f81
* skdiff output: indicate which result types would cause failureGravatar epoger@google.com2012-07-19
| | | | | | Review URL: https://codereview.appspot.com/6419053 git-svn-id: http://skia.googlecode.com/svn/trunk@4676 2bbb7eff-a529-9590-31e7-b0007b416f81
* skdiff: replace --failonmismatches with repeatable --failonresult optionGravatar epoger@google.com2012-07-13
| | | | | | Review URL: https://codereview.appspot.com/6392064 git-svn-id: http://skia.googlecode.com/svn/trunk@4611 2bbb7eff-a529-9590-31e7-b0007b416f81
* skdiff: add --failonmismatches and --listfilename options, plus cleanupGravatar epoger@google.com2012-07-12
| | | | | | | These changes are needed in order to switch the buildbots from using "gm -r" to "skdiff" to compare gm results, and should be generally good for humans too. Review URL: https://codereview.appspot.com/6392054 git-svn-id: http://skia.googlecode.com/svn/trunk@4579 2bbb7eff-a529-9590-31e7-b0007b416f81
* Improve sorting of skdiff output, and make it consistent across platformsGravatar epoger@google.com2012-06-28
| | | | | | | BUG=https://code.google.com/p/skia/issues/detail?id=677 Review URL: https://codereview.appspot.com/6351045 git-svn-id: http://skia.googlecode.com/svn/trunk@4388 2bbb7eff-a529-9590-31e7-b0007b416f81
* skdiff: return nonzero exit value if there are any mismatchesGravatar epoger@google.com2012-05-31
| | | | | | | This will allow us to replace "gm -r" in our buildbots with "skdiff". This will speed things up, and work around http://code.google.com/p/skia/issues/detail?id=473 ('PDF gradtext gm image results are nondeterministic') Review URL: https://codereview.appspot.com/6242071 git-svn-id: http://skia.googlecode.com/svn/trunk@4097 2bbb7eff-a529-9590-31e7-b0007b416f81
* test-only: add test coverage of skdiff over input with identical bits or pixelsGravatar epoger@google.com2012-05-30
| | | | | | Review URL: https://codereview.appspot.com/6260055 git-svn-id: http://skia.googlecode.com/svn/trunk@4084 2bbb7eff-a529-9590-31e7-b0007b416f81
* Add to skdiff: test images for bitwise equality before testing for ↵Gravatar epoger@google.com2012-05-22
| | | | | | | | | | | | | | | | | | | pixel-wise equality - we now check for bitwise equality of files before bothering with pixel comparisons (this will help with formats for which we don't have good decoders, like PDF) - unparsable files are now reported as differing, unless they are bitwise equal (before, they were always reported as matching!) - "largest area mismatch" is now based on same-size image pairs only; previously, if any image pairs had mismatching size, it was 100% - removed repetitive "image size mismatch, so no diff to display" messages - changed format of leftmost table cells to be more readable BUG=http://code.google.com/p/skia/issues/detail?id=473 Review URL: https://codereview.appspot.com/6208089 git-svn-id: http://skia.googlecode.com/svn/trunk@4027 2bbb7eff-a529-9590-31e7-b0007b416f81
* skdiff test-only change: add test of --nodiffs mode, make tests validate ↵Gravatar epoger@google.com2012-05-18
| | | | | | | | return value Review URL: https://codereview.appspot.com/6206085 git-svn-id: http://skia.googlecode.com/svn/trunk@3999 2bbb7eff-a529-9590-31e7-b0007b416f81
* test-only change: update run.sh and rearrange dirs to ease addition of more ↵Gravatar epoger@google.com2012-05-17
| | | | | | | | tests Review URL: https://codereview.appspot.com/6220044 git-svn-id: http://skia.googlecode.com/svn/trunk@3995 2bbb7eff-a529-9590-31e7-b0007b416f81
* test-only changes: add more image files to test more skdiff behaviorGravatar epoger@google.com2012-05-16
| | | | | | | | | Tests output order for very vs. slightly different pixels, very vs. slightly different image sizes, etc. TBR=tomhudson@google.com Review URL: https://codereview.appspot.com/6213049 git-svn-id: http://skia.googlecode.com/svn/trunk@3977 2bbb7eff-a529-9590-31e7-b0007b416f81
* skdiff: only write out diff images that are included in index.htmlGravatar epoger@google.com2012-05-16
| | | | | | | Not only are those files unnecessary, in some different-image-size cases, their contents are nondeterministic! Review URL: https://codereview.appspot.com/6208073 git-svn-id: http://skia.googlecode.com/svn/trunk@3975 2bbb7eff-a529-9590-31e7-b0007b416f81
generated by cgit v1.2.3 (git 2.39.1) at 2024-07-02 00:00:28 +0000