Tests for the Electron app are written using [Spectron](https://github.com/electron-userland/spectron#application-api) (and [spectron-keys](https://github.com/jsantell/spectron-keys) for key inputs), while tests for the CLI use [execa](https://github.com/sindresorhus/execa).
Sometimes, you might need to run tests against a _packaged_ application. A packaged application is the final artifact which bundles all of the various resources together, and is created for distribution in the form of a `.dmg` or `.exe`, etc. Packaging takes longer to do and is only required for edge cases (such as a <!-- TODO(TSCONVERSION) update this link -->[plugin installation](https://github.com/Kong/insomnia/blob/357b8f05f89fd5c07a75d8418670abe37b2882dc/packages/insomnia-smoke-test/designer/app.test.js#L36)), so we typically run tests against a build. To run packaged tests, from the root:
Each of the above commands will automatically run the Express server, so you do not need to take any extra steps.
## How to write
When writing tests, it is recommended to use the scripts in this project directly (instead of from the root, as per the section above). After building and/or packaging your application under test, it will be available under `packages/insomnia-app/{build|dist}` and you can begin writing your test.
In order to run tests for development, open two terminal tabs in `packages/insomnia-smoke-test`:
This will allow you to write and monitor the server separately from each test, speeding up the development cycle.
You may also need to run a test multiple times. You can focus a particular test or test suite using [Jest globals](https://jestjs.io/docs/en/api#testonlyname-fn-timeout) such as `it.only` and `describe.skip`, or their aliases `fit` and `xdescribe`, etc.
## General guidelines
### Data
Individual tests will automatically run against a clean Insomnia data directory to keep data isolated.
### Dependencies
A test should not depend on any external services unless absolutely necessary. If a particular endpoint is required (eg. for authentication or a specific content type), implement a new endpoint in `/server`.
### Element selection
Spectron is built heavily on top of WebdriverIO, and WebdriverIO's `browser` object is available under `app.client` ([docs](https://github.com/electron-userland/spectron#client)). This is the primary API you will need for user interactions, see examples with existing tests.
Through WebdriverIO you can use a host of CSS or React selectors. There is no clear guideline about which selector to use, but whichever approach is used it must favour stability and be understandable.
There are trade-offs with each selector approach but it's important to know how generic or specific a particular component or CSS class is, in order to ensure that the correct element is always selected as the application evolves.
#### Select by component and props
Sometimes selecting by a React component and props, directly from `app.client` is the cleanest approach, as the following two examples show:
It is important to scope an element to an appropriate ancestor. In a way the selector becomes self-documenting, but also ensures stability as the UI evolves.
In the following example, it is possible for multiple buttons which match the `button#enabled` selector to exist on the page. By chaining a React and CSS selector, we can ensure the test runner will always click the expected button within the `BasicAuth` component.
A similar approach can be achieved through a CSS selector. In the following example, after sending a successful request, we want to detect an element containing the CSS classes `tag bg-success` and ensure it contains the text `200 OK`.
These classes are fairly generic and could exist multiple times on the page, but the HTTP response code will always be in the response pane (`response-pane`) header (`pane__header`). As such, the selector is scoped to always select the expected element, wait for it to show, and ensure it has the expected text.
const tag = await app.client.$('.response-pane .pane__header .tag.bg-success');
await tag.waitForDisplayed();
await expectText(tag, '200 OK');
};
```
### Interactions
As is common with all smoke testing frameworks, before interacting with an element (click, hover, etc) it is generally good to check whether you _can_ interact with it. For instance, clicking a button will fail if the button is not yet clickable.
Sometimes you will need to add explicit pauses to allow for UI to refresh or database writes to occur (`await app.client.pause(500)`). Try to keep these to a minimum, though, exploring all other avenues first, such as WebdriverIO's `waitFor*` functions. Avoiding explicit waits ensures each test runs in the short amount of time.
When typing in the url bar for HTTP requests, we first wait for it to exist on the page before clicking on it and typing, because request activation can take some time.
In addition, sometimes we want to wait for an element to hide instead of show. To achieve this, we can use the `reverse` option available through WebdriverIO, as shown in the following example.
It is important for a smoke test to be _readable_ so the flow can be understood, and the (often complicated) implementation details hidden, like in the example below.
In most cases, it will be beneficial to create helper functions under `/modules`, regardless of how reusable they are. Some modules (such as `dropdown`, `tabs` and `settings`) are reusable, while some are specific to certain pages (eg `debug`, `home`, `onboarding`). These can be broken down into more granular modules as the test suite grows.
### Extend tests
Unlike unit tests, the application startup time for a smoke test can sometimes be longer than the test itself. As such, in cases where it is appropriate, **extend** a smoke test with additional steps instead of creating a **new** test.
Smoke tests can potentially be flaky, and one attempt to avoid flaky tests in the default branch is to run the final implementation of a test at least 20 times locally to prove its stability. If a test is unable to achieve this, it is very unlikely to be accepted into the test suite.
