6 visual regression testing tools for UI design

Responsive layouts are very difficult to test. It is more tedious to design and test all of a project’s screen elements at every breakpoint than it is to code them in the first place. Visual regression allows you to make visual comparisons between the baseline versions of the site and the version in development. The complete process is nothing but taking the screenshot of the original design and comparing it with the new, looking for pixel differences. Visual regression comes in various flavors, using a variety of technologies and workflows as mentioned below. Pick any one to suit your purpose.

BackstopJS

BackstopJS automates visual regression testing of your responsive web UI by comparing DOM screenshots over time. BackstopJS is a config-driven automated screenshot test application for responsive websites and web-applications. Specify a set of DOM elements and viewport sizes and BackstopJS will work like an extra set of eyes on your workflow. The workflow is as simple as below:

1. BackstopJS Creates reference screenshots of your webpage/web-app at multiple screen sizes.
2. After editing your CSS, use BackstopJS to find any unwanted CSS regressions.
3. BackstopJS then uses Resemble.js, CasperJS, PhantomJS and SlimerJS to report on visual changes.

Wraith

Wraith is a screenshot comparison tool, created by developers at BBC News. Wraith uses a headless browser to create screenshots of webpages on different environments (or at different moments in time) and then creates a diff of the two images; the affected areas are highlighted in blue. The workflow process is as below:

1. Takes screenshots of your webpages.
2. Run a comparison task across them.
3. Output a diff PNG file comparing the two images, and a data.txt file which contains the percentage of pixels that have changed.
4. Package all of this up into a gallery.html, ready for you to view.
5. If any screenshot’s diff is above the threshold you specified in your configuration file, the task exits with a system error code (useful for CI).
6. The failed screenshot will also be highlighted in the gallery.

PhantomCSS

PhantomCSS is a CasperJS module for automating visual regression testing with PhantomJS 2 or SlimerJS and Resemble.js. For testing Web apps, live style guides and responsive layouts. PhantomCSS takes screenshots captured by CasperJS and compares them to baseline images using Resemble.js to test for rgb pixel differences. PhantomCSS then generates image diffs to help you find the cause.

CasperJS

CasperJS CasperJS is a navigation scripting & testing utility for PhantomJS and SlimerJS. It eases the process of defining a full navigation scenario and provides useful high-level functions, methods & syntactic sugar for doing common tasks such as:

– defining & ordering navigation steps
– filling forms
– clicking links
– capturing screenshots of a page (or an area)
– making assertions on remote DOM
– logging & events
– downloading resources, even binary ones
– catching errors and react accordingly
– writing functional test suites, exporting results as JUnit XML (xUnit)ML (xUnit)

Spectre

Spectre is a free web application to diff screenshots. A simple UI for browsing and inspecting diffs, and an API for runner scripts to submit screenshots to and receive a pass or fail in real time. (For use with Wraith, Backstop, Selenium etc). It’s heavily influenced by VisualReview, BackstopJS and Wraith.You can either run the app using docker, or you can run it natively on your machine.

VisualReview

VisualReview, a human-friendly tool for testing and reviewing visual regressions. VisualReview’s goal is to provide a productive and human-friendly workflow for testing and reviewing your web application’s layout for any regressions.

VisualReview functions as a server which accepts screenshots of your web application, sent for example, from your selenium or protractor scripts. These shots are then compared to screenshots you uploaded and reviewed earlier. VisualReview will display and highlight any differences between the two and allows you to either accept or reject these changes.

2 thoughts on “6 visual regression testing tools for UI design

  1. You should consider trying Screenster (http://screenster.io). It is a visual testing tool which does not require coding skills to build tests. On my recent projects we have PMs and developers recording tests and we find it to be a much better alternative to Selenium. We even automated CSS testing of web pages because there’s a lot of smart logic targeted at test maintenance.

  2. Whoa, a big list of great resources. All these tools are really helpful for our testers. I have heard about wraith earlier but not much aware of the others. I would definitely going to try one of them. Thanks for sharing!

Comments are closed.