Browse Source

Add screenshot testing

master
Timmy Huang 2 years ago
committed by james hadfield
parent
commit
7a851a0c73
  1. 4
      .github/workflows/ci.yaml
  2. 5
      .gitignore
  3. 77
      DEV_DOCS.md
  4. 1097
      package-lock.json
  5. 2
      package.json
  6. 1
      puppeteer.config.js
  7. 5
      puppeteer.setup.js
  8. 5
      src/components/controls/color-by.js
  9. BIN
      test/integration/__image_snapshots__/Color-by:author-snap.png
  10. BIN
      test/integration/__image_snapshots__/Color-by:country-snap.png
  11. BIN
      test/integration/__image_snapshots__/Color-by:date-snap.png
  12. BIN
      test/integration/__image_snapshots__/Color-by:region-snap.png
  13. 42
      test/integration/helpers.js
  14. 88
      test/integration/zika.test.js

4
.github/workflows/ci.yaml

@ -19,7 +19,7 @@ jobs:
- run: npm ci
- run: npm test
integration-test:
runs-on: ubuntu-latest
runs-on: macos-10.15
strategy:
matrix:
node: [10]
@ -66,4 +66,4 @@ jobs:
with:
node-version: 10
- run: npm install
- run: npx bundlesize
- run: npx bundlesize

5
.gitignore

@ -8,6 +8,7 @@ s3/
/local_narratives/
/narratives/
/datasets/
**/__diff_output__
### OSX ###
.DS_Store
@ -16,4 +17,6 @@ s3/
node_modules/
npm-debug.log
*tgz
.vscode/launch.json
### IDE ###
.vscode/*

77
DEV_DOCS.md

@ -1,5 +1,5 @@
---
title: Contributing to Auspice development
Title: Contributing to Auspice development
---
Thank you for helping us to improve Nextstrain!
@ -10,42 +10,85 @@ This project strictly adheres to the [Contributor Covenant Code of Conduct](http
Please see the [project boards](https://github.com/orgs/nextstrain/projects) for currently available issues.
## Contributing code
Code contributions are welcomed! [Please see the main auspice docs](https://nextstrain.github.io/auspice/introduction/install) for details on how to install and run auspice from source.
## Contributing code
Code contributions are welcomed! [Please see the main auspice docs](https://nextstrain.github.io/auspice/introduction/install) for details on how to install and run auspice from source.
Please comment on an open issue if you are working on it.
For changes unrelated to an open issue, please make an issue outlining what you would like to change/add.
Where possible, **please rebase** your work onto master rather than merging changes from master into your PR.
From a fork: `git pull --rebase upstream master`
### Make sure tests are passing
We use the following libraries for all kinds of testing, so it'd help to read the docs to get familiar with their APIs and features:
1. [`Jest`](https://github.com/facebook/jest)
2. [`Puppeteer`](https://github.com/puppeteer/puppeteer/)
3. [`Jest-Puppeteer`](https://github.com/smooth-code/jest-puppeteer)
4. [`Jest-Image-Snapshot`](https://github.com/americanexpress/jest-image-snapshot)
When you submit a pull request to the auspice repository, a variety of integration and unit tests will need to pass before it can be merged.
You will likely want to run these tests locally before submitting:
First, install the dependencies with `npm i`, then:
#### For unit tests:
#### For unit tests
Run `npm test`.
#### For linting:
#### For linting
Run `npm run lint`. If there are issues run `npm run lint:fix`.
#### For integration tests:
#### For integration tests
> For integration tests to work, you'll need to have `git-lfs` installed (see below) as it stores the images that the snapshot tests will use.
1. Fetch the datasets with `npm run get-data` and `npm run get-narratives`.
2. Ensure you are **not** currently running the site locally, then run `npm run integration-test:ci`.
#### For smoke tests:
#### How to update test snapshots
1. Unit tests: `npm run test -- -u`
2. Integration tests `npm run integration-test -- -u`
#### For smoke tests
1. Fetch the datasets with `npm run get-data` and `npm run get-narratives`.
2. Ensure you are **not** currently running the site locally, then run `npm run smoke-test:ci`.
#### Test Tips
1. Run a single `describe()`, `it()`, or `test()` **within a file**, add `.only()`:
E.g., `describe.only()`, `it.only()`, or `test.only()`
2. Run a single test file, append the following to your test command `-- relative_or_absolute_path/to/file`:
E.g., `npm run integration-test -- test/integration/zika.test.js`
3. Run integration tests in headful mode, prepend `HEADLESS=false` to your command:
E.g., `HEADLESS=false npm run integration-test`
4. For integration tests, please try to use [`expect-puppeteer`](https://github.com/smooth-code/jest-puppeteer/blob/master/packages/expect-puppeteer/README.md#api) as much as possible, and only resort to `puppeteer`'s native API when we can't do it with `expect-puppeteer`. The reason is because `expect-puppeteer` has better DX, as explained [here](https://github.com/smooth-code/jest-puppeteer/blob/master/packages/expect-puppeteer/README.md#why-do-i-need-it)
5. How to add a new integration image test:
1. Wrap your image test with helper function `toMatchImageSnapshot()` from `test/integration/helpers.js`, and it will take a screenshot every `100ms` until it matches the expected snapshot or timeout (default: `10s`)
2. Temporarily add `page.waitFor(__ENOUGH__MS__)` before taking the new snapshot to give the browser enough time to render a complete image
3. Example: `test/integration/zika.test.js`
## git-lfs
We use [Git Large File Storage](https://github.com/git-lfs/git-lfs) to manage certain assets.
@ -68,49 +111,49 @@ This documentation is built from files contained within the Auspice GitHub repo
Note that currently the documentation must be rebuilt & pushed to GitHub _after_ a new version is released in order for the changelog to correctly appear at [nextstrain.github.io/auspice/releases/changelog](https://nextstrain.github.io/auspice/releases/changelog).
## Contributing to Internationalization and Localization (i18n/l18n)
If you can assist in efforts to translate the Auspice interface to more languages your assistance would be very much appreciated.
The currently available languages are displayed via a drop-down at the bottom of the sidebar.
#### Adding a new language:
## Adding a new language
1) Add the language to the `getlanguageOptions` function in [this file](https://github.com/nextstrain/auspice/blob/master/src/components/controls/language.js#L24)
2) If this is a new language, copy the folder (and the JSONs within it) `src/locales/en` and name it to match the language code for the new translation -- e.g. for Spanish this would be `src/locales/es`
3) For each key-value in the JSONs, translate the english phrase to the new locale. (Do not modify the strings within `{{...}}` sections.)
For example, a spanish translation would change the English:
```json
"sampled between {{from}} and {{to}}": "sampled between {{from}} and {{to}}",
"and comprising": "and comprising",
```
to
to
```json
"sampled between {{from}} and {{to}}": "aislados entre {{from}} y {{to}}",
"and comprising": "y compuesto de",
```
#### Helper script to check what parts of a translation are out-of-date or missing:
## Helper script to check what parts of a translation are out-of-date or missing
Run `npm run diff-lang -- X`, where `X` is the language you wish to check, for instance `es`.
This will display the strings which:
* need to be added to the translation
* are present but should be removed as they are no longer used
* are present but are simply a copy of the English version & need to be translated
> Running `npm run diff-lang` will check all available languages.
#### Improving an existing translation:
## Improving an existing translation
If a translation of a particular string is not yet available, then auspice will fall-back to the english version.
1) Find the relevant key in the (EN) JSONs [in this directory](https://github.com/nextstrain/auspice/tree/master/src/locales/en)
2) Add the key to the JSON with the same name, but in the directory corresponding to the language you are translating into (see above for an example).
## Releases & versioning
New versions are released via the `./releaseNewVersion.sh` script from an up-to-date `master` branch. It will prompt you for the version number increase, push changes to the `release` branch and, as long as Travis-CI is successful then a new version will be automatically published to [npm](https://www.npmjs.com/package/auspice).

1097
package-lock.json
File diff suppressed because it is too large
View File

2
package.json

@ -138,6 +138,8 @@
"eslint-plugin-react": "^7.2.1",
"eslint-plugin-react-hooks": "^1.6.0",
"jest": "^25.1.0",
"jest-circus": "^25.3.0",
"jest-image-snapshot": "^3.0.1",
"jest-puppeteer": "^4.4.0",
"puppeteer": "^2.1.1",
"quoted-printable": "^1.0.1",

1
puppeteer.config.js

@ -1,4 +1,5 @@
module.exports = {
testRunner: "jest-circus/runner",
preset: "jest-puppeteer",
globals: {
BASE_URL:

5
puppeteer.setup.js

@ -1,4 +1,7 @@
import { setDefaultOptions } from 'expect-puppeteer';
import { toMatchImageSnapshot } from 'jest-image-snapshot';
expect.extend({ toMatchImageSnapshot });
// (tihuan): This is the max time a test can take to run.
// Since when debugging, we run slowMo and !headless, this means
@ -7,6 +10,8 @@ import { setDefaultOptions } from 'expect-puppeteer';
jest.setTimeout(30 * 1000);
setDefaultOptions({ timeout: 3 * 1000 });
jest.retryTimes(2);
beforeEach(async () => {
await jestPuppeteer.resetBrowser();
const userAgent = await browser.userAgent();

5
src/components/controls/color-by.js

@ -189,13 +189,14 @@ class ColorBy extends React.Component {
render() {
const styles = this.getStyles();
const colorOptions = Object.keys(this.props.colorings)
.map((key) => ({value: key, label: this.props.colorings[key].title}));
return (
<div style={styles.base}>
<div style={styles.base} id="selectColorBy">
<Select
name="selectColorBy"
id="selectColorBy"
value={this.state.colorBySelected}
options={colorOptions}
clearable={false}

BIN
test/integration/__image_snapshots__/Color-by:author-snap.png (Stored with Git LFS)

size 247382

BIN
test/integration/__image_snapshots__/Color-by:country-snap.png (Stored with Git LFS)

size 236376

BIN
test/integration/__image_snapshots__/Color-by:date-snap.png (Stored with Git LFS)

size 241689

BIN
test/integration/__image_snapshots__/Color-by:region-snap.png (Stored with Git LFS)

size 238657

42
test/integration/helpers.js

@ -0,0 +1,42 @@
/**
* @param {function} assert Function that calls `page.screenshot()` and `expect(image).toMatchImageSnapshot()`
* @param {object} options
* @param options.timeoutMS The max timeout before failing the test in milliseconds
* @param options.pollingMS The polling frequency in milliseconds
*/
export async function toMatchImageSnapshot(assert, options = {}) {
const DEFAULT_TIMEOUT_MS = 10 * 1000;
const DEFAULT_POLLING_MS = 100;
const {
timeoutMS = DEFAULT_TIMEOUT_MS,
pollingMS = DEFAULT_POLLING_MS
} = options;
const startTime = Date.now();
/**
* (tihuan): We want `await` to complete before the next iteration
*/
/* eslint-disable no-await-in-loop */
// eslint-disable-next-line no-constant-condition
while (true) {
try {
await assert();
break;
} catch (error) {
if (isTimeoutExceeded()) {
throw error;
}
page.waitFor(pollingMS);
continue;
}
}
/* eslint-enable no-await-in-loop */
function isTimeoutExceeded() {
return Date.now() - startTime > timeoutMS;
}
}

88
test/integration/zika.test.js

@ -1,20 +1,90 @@
import { toMatchImageSnapshot } from './helpers';
describe("Zika", () => {
it('displays the title on the page', async () => {
await goToZikaPage()
it("displays the title on the page", async () => {
await goToZikaPage();
await expect(page).toMatch("Real-time tracking of Zika virus evolution");
});
it('clicks `Play` on the `TransmissionsCard`', async () => {
await goToZikaPage()
it("clicks `Play` on the `TransmissionsCard`", async () => {
await goToZikaPage();
await expect(page).toClick("button", { text: "Play" });
await expect(page).toMatchElement("button", { text: "Pause" });
});
describe("Color by", () => {
describe("author", () => {
it("matches the screenshot", async () => {
await matchSelectOptionScreenshot("author", async () => {
await page.keyboard.press("ArrowUp");
await page.keyboard.press("Enter");
});
});
});
await expect(page).toMatchElement('#TransmissionsCard')
describe("country", () => {
it("matches the screenshot", async () => {
await matchSelectOptionScreenshot("country", () => {});
});
});
await expect(page).toClick('button', { text: 'Play' })
describe("date", () => {
it("matches the screenshot", async () => {
await matchSelectOptionScreenshot("date", async () => {
await page.keyboard.press("ArrowUp");
await page.keyboard.press("ArrowUp");
await page.keyboard.press("Enter");
});
});
});
await expect(page).toMatchElement('button', { text: 'Pause' })
})
describe("region", () => {
it("matches the screenshot", async () => {
await matchSelectOptionScreenshot("region", async () => {
await page.keyboard.press("ArrowDown");
await page.keyboard.press("Enter");
});
});
});
});
});
async function matchSelectOptionScreenshot(option, selectOptionTest) {
await goToZikaPage();
const colorBySelector = await expect(page).toMatchElement("#selectColorBy");
await colorBySelector.click();
await selectOptionTest();
await expect(colorBySelector).toMatch(option);
const treeTitle = await expect(page).toMatchElement("#Title");
await expect(treeTitle).toMatch(option);
await toMatchImageSnapshot(async () => {
const image = await page.screenshot();
/**
* (tihuan): Apply `blur` to ignore minor noises.
* Also `customSnapshotIdentifier` is needed, since we use `jest.retryTimes()`
* https://github.com/americanexpress/jest-image-snapshot/pull/122/files
* https://github.com/americanexpress/jest-image-snapshot#%EF%B8%8F-api
*/
const SNAPSHOT_CONFIG = {
blur: 2,
customSnapshotIdentifier: `Color-by:${option}`
};
expect(image).toMatchImageSnapshot(SNAPSHOT_CONFIG);
});
}
async function goToZikaPage() {
await page.goto(`${BASE_URL}/zika?p=grid`, { waitUntil: 'networkidle2' })
await page.goto(`${BASE_URL}/zika?p=grid`, { waitUntil: "networkidle2" });
await expect(page).toMatchElement("#TransmissionsCard");
}
Loading…
Cancel
Save