electron/docs/tutorial/automated-testing.md

# Automated Testing

Test automation is an efficient way of validating that your application code works as intended.
While Electron doesn't actively maintain its own testing solution, this guide will go over a couple
ways you can run end-to-end automated tests on your Electron app.

## Using the WebDriver interface

From [ChromeDriver - WebDriver for Chrome][chrome-driver]:

> WebDriver is an open source tool for automated testing of web apps across many
> browsers. It provides capabilities for navigating to web pages, user input,
> JavaScript execution, and more. ChromeDriver is a standalone server which
> implements WebDriver's wire protocol for Chromium. It is being developed by
> members of the Chromium and WebDriver teams.

There are a few ways that you can set up testing using WebDriver.

### With WebdriverIO

[WebdriverIO](https://webdriver.io/) (WDIO) is a test automation framework that provides a
Node.js package for testing with WebDriver. Its ecosystem also includes various plugins
(e.g. reporter and services) that can help you put together your test setup.

If you already have an existing WebdriverIO setup, it is recommended to update your dependencies and validate your existing configuration with how it is [outlined in the docs](https://webdriver.io/docs/desktop-testing/electron#configuration).

#### Install the test runner

If you don't use WebdriverIO in your project yet, you can add it by running the starter toolkit in your project root directory:

```sh npm2yarn
npm init wdio@latest ./
```

This starts a configuration wizard that helps you put together the right setup, installs all necessary packages, and generates a `wdio.conf.js` configuration file. Make sure to select _"Desktop Testing - of Electron Applications"_ on one of the first questions asking _"What type of testing would you like to do?"_.

#### Connect WDIO to your Electron app

After running the configuration wizard, your `wdio.conf.js` should include roughly the following content:

```js title='wdio.conf.js' @ts-nocheck
export const config = {
  // ...
  services: ['electron'],
  capabilities: [{
    browserName: 'electron',
    'wdio:electronServiceOptions': {
      // WebdriverIO can automatically find your bundled application
      // if you use Electron Forge or electron-builder, otherwise you
      // can define it here, e.g.:
      // appBinaryPath: './path/to/bundled/application.exe',
      appArgs: ['foo', 'bar=baz']
    }
  }]
  // ...
}
```

#### Write your tests

Use the [WebdriverIO API](https://webdriver.io/docs/api) to interact with elements on the screen. The framework provides custom "matchers" that make asserting the state of your application easy, e.g.:

```js @ts-nocheck
import { browser, $, expect } from '@wdio/globals'

describe('keyboard input', () => {
  it('should detect keyboard input', async () => {
    await browser.keys(['y', 'o'])
    await expect($('keypress-count')).toHaveText('YO')
  })
})
```

Furthermore, WebdriverIO allows you to access Electron APIs to get static information about your application:

```js @ts-nocheck
import { browser, $, expect } from '@wdio/globals'

describe('when the make smaller button is clicked', () => {
  it('should decrease the window height and width by 10 pixels', async () => {
    const boundsBefore = await browser.electron.browserWindow('getBounds')
    expect(boundsBefore.width).toEqual(210)
    expect(boundsBefore.height).toEqual(310)

    await $('.make-smaller').click()
    const boundsAfter = await browser.electron.browserWindow('getBounds')
    expect(boundsAfter.width).toEqual(200)
    expect(boundsAfter.height).toEqual(300)
  })
})
```

or to retrieve other Electron process information:

```js @ts-nocheck
import fs from 'node:fs'
import path from 'node:path'
import { browser, expect } from '@wdio/globals'

const packageJson = JSON.parse(fs.readFileSync(path.join(__dirname, '..', 'package.json'), { encoding: 'utf-8' }))
const { name, version } = packageJson

describe('electron APIs', () => {
  it('should retrieve app metadata through the electron API', async () => {
    const appName = await browser.electron.app('getName')
    expect(appName).toEqual(name)
    const appVersion = await browser.electron.app('getVersion')
    expect(appVersion).toEqual(version)
  })

  it('should pass args through to the launched application', async () => {
    // custom args are set in the wdio.conf.js file as they need to be set before WDIO starts
    const argv = await browser.electron.mainProcess('argv')
    expect(argv).toContain('--foo')
    expect(argv).toContain('--bar=baz')
  })
})
```

#### Run your tests

To run your tests:

```sh
$ npx wdio run wdio.conf.js
```

WebdriverIO helps launch and shut down the application for you.

#### More documentation

Find more documentation on Mocking Electron APIs and other useful resources in the [official WebdriverIO documentation](https://webdriver.io/docs/desktop-testing/electron).

### With Selenium

[Selenium](https://www.selenium.dev/) is a web automation framework that
exposes bindings to WebDriver APIs in many languages. Their Node.js bindings
are available under the `selenium-webdriver` package on NPM.

#### Run a ChromeDriver server

In order to use Selenium with Electron, you need to download the `electron-chromedriver`
binary, and run it:

```sh npm2yarn
npm install --save-dev electron-chromedriver
./node_modules/.bin/chromedriver
Starting ChromeDriver (v2.10.291558) on port 9515
Only local connections are allowed.
```

Remember the port number `9515`, which will be used later.

#### Connect Selenium to ChromeDriver

Next, install Selenium into your project:

```sh npm2yarn
npm install --save-dev selenium-webdriver
```

Usage of `selenium-webdriver` with Electron is the same as with
normal websites, except that you have to manually specify how to connect
ChromeDriver and where to find the binary of your Electron app:

```js title='test.js' @ts-expect-error=[1]
const webdriver = require('selenium-webdriver')
const driver = new webdriver.Builder()
  // The "9515" is the port opened by ChromeDriver.
  .usingServer('http://localhost:9515')
  .withCapabilities({
    'goog:chromeOptions': {
      // Here is the path to your Electron binary.
      binary: '/Path-to-Your-App.app/Contents/MacOS/Electron'
    }
  })
  .forBrowser('chrome') // note: use .forBrowser('electron') for selenium-webdriver <= 3.6.0
  .build()
driver.get('https://www.google.com')
driver.findElement(webdriver.By.name('q')).sendKeys('webdriver')
driver.findElement(webdriver.By.name('btnG')).click()
driver.wait(() => {
  return driver.getTitle().then((title) => {
    return title === 'webdriver - Google Search'
  })
}, 1000)
driver.quit()
```

## Using Playwright

[Microsoft Playwright](https://playwright.dev) is an end-to-end testing framework built
using browser-specific remote debugging protocols, similar to the [Puppeteer][] headless
Node.js API but geared towards end-to-end testing. Playwright has experimental Electron
support via Electron's support for the [Chrome DevTools Protocol][] (CDP).

### Install dependencies

You can install Playwright through your preferred Node.js package manager. It comes with its
own [test runner][playwright-intro], which is built for end-to-end testing:

```sh npm2yarn
npm install --save-dev @playwright/test
```

:::caution Dependencies
This tutorial was written with `@playwright/test@1.41.1`. Check out
[Playwright's releases][playwright-releases] page to learn about
changes that might affect the code below.
:::

### Write your tests

Playwright launches your app in development mode through the `_electron.launch` API.
To point this API to your Electron app, you can pass the path to your main process
entry point (here, it is `main.js`).

```js {5} @ts-nocheck
const { test, _electron: electron } = require('@playwright/test')

test('launch app', async () => {
  const electronApp = await electron.launch({ args: ['main.js'] })
  // close app
  await electronApp.close()
})
```

After that, you will access to an instance of Playwright's `ElectronApp` class. This
is a powerful class that has access to main process modules for example:

```js {5-10} @ts-nocheck
const { test, _electron: electron } = require('@playwright/test')

test('get isPackaged', async () => {
  const electronApp = await electron.launch({ args: ['main.js'] })
  const isPackaged = await electronApp.evaluate(async ({ app }) => {
    // This runs in Electron's main process, parameter here is always
    // the result of the require('electron') in the main app script.
    return app.isPackaged
  })
  console.log(isPackaged) // false (because we're in development mode)
  // close app
  await electronApp.close()
})
```

It can also create individual [Page][playwright-page] objects from Electron BrowserWindow instances.
For example, to grab the first BrowserWindow and save a screenshot:

```js {6-7} @ts-nocheck
const { test, _electron: electron } = require('@playwright/test')

test('save screenshot', async () => {
  const electronApp = await electron.launch({ args: ['main.js'] })
  const window = await electronApp.firstWindow()
  await window.screenshot({ path: 'intro.png' })
  // close app
  await electronApp.close()
})
```

Putting all this together using the Playwright test-runner, let's create a `example.spec.js`
test file with a single test and assertion:

```js title='example.spec.js' @ts-nocheck
const { test, expect, _electron: electron } = require('@playwright/test')

test('example test', async () => {
  const electronApp = await electron.launch({ args: ['.'] })
  const isPackaged = await electronApp.evaluate(async ({ app }) => {
    // This runs in Electron's main process, parameter here is always
    // the result of the require('electron') in the main app script.
    return app.isPackaged
  })

  expect(isPackaged).toBe(false)

  // Wait for the first BrowserWindow to open
  // and return its Page object
  const window = await electronApp.firstWindow()
  await window.screenshot({ path: 'intro.png' })

  // close app
  await electronApp.close()
})
```

Then, run Playwright Test using `npx playwright test`. You should see the test pass in your
console, and have an `intro.png` screenshot on your filesystem.

```console
☁  $ npx playwright test

Running 1 test using 1 worker

  ✓  example.spec.js:4:1 › example test (1s)
```

:::info
Playwright Test will automatically run any files matching the `.*(test|spec)\.(js|ts|mjs)` regex.
You can customize this match in the [Playwright Test configuration options][playwright-test-config].
It also works with TypeScript out of the box.
:::

:::tip Further reading
Check out Playwright's documentation for the full [Electron][playwright-electron]
and [ElectronApplication][playwright-electronapplication] class APIs.
:::

## Using a custom test driver

It's also possible to write your own custom driver using Node.js' built-in IPC-over-STDIO.
Custom test drivers require you to write additional app code, but have lower overhead and let you
expose custom methods to your test suite.

To create a custom driver, we'll use Node.js' [`child_process`](https://nodejs.org/api/child_process.html) API.
The test suite will spawn the Electron process, then establish a simple messaging protocol:

```js title='testDriver.js' @ts-nocheck
const childProcess = require('node:child_process')
const electronPath = require('electron')

// spawn the process
const env = { /* ... */ }
const stdio = ['inherit', 'inherit', 'inherit', 'ipc']
const appProcess = childProcess.spawn(electronPath, ['./app'], { stdio, env })

// listen for IPC messages from the app
appProcess.on('message', (msg) => {
  // ...
})

// send an IPC message to the app
appProcess.send({ my: 'message' })
```

From within the Electron app, you can listen for messages and send replies using the Node.js
[`process`](https://nodejs.org/api/process.html) API:

```js title='main.js'
// listen for messages from the test suite
process.on('message', (msg) => {
  // ...
})

// send a message to the test suite
process.send({ my: 'message' })
```

We can now communicate from the test suite to the Electron app using the `appProcess` object.

For convenience, you may want to wrap `appProcess` in a driver object that provides more
high-level functions. Here is an example of how you can do this. Let's start by creating
a `TestDriver` class:

```js title='testDriver.js' @ts-nocheck
class TestDriver {
  constructor ({ path, args, env }) {
    this.rpcCalls = []

    // start child process
    env.APP_TEST_DRIVER = 1 // let the app know it should listen for messages
    this.process = childProcess.spawn(path, args, { stdio: ['inherit', 'inherit', 'inherit', 'ipc'], env })

    // handle rpc responses
    this.process.on('message', (message) => {
      // pop the handler
      const rpcCall = this.rpcCalls[message.msgId]
      if (!rpcCall) return
      this.rpcCalls[message.msgId] = null
      // reject/resolve
      if (message.reject) rpcCall.reject(message.reject)
      else rpcCall.resolve(message.resolve)
    })

    // wait for ready
    this.isReady = this.rpc('isReady').catch((err) => {
      console.error('Application failed to start', err)
      this.stop()
      process.exit(1)
    })
  }

  // simple RPC call
  // to use: driver.rpc('method', 1, 2, 3).then(...)
  async rpc (cmd, ...args) {
    // send rpc request
    const msgId = this.rpcCalls.length
    this.process.send({ msgId, cmd, args })
    return new Promise((resolve, reject) => this.rpcCalls.push({ resolve, reject }))
  }

  stop () {
    this.process.kill()
  }
}

module.exports = { TestDriver }
```

In your app code, can then write a simple handler to receive RPC calls:

```js title='main.js'
const METHODS = {
  isReady () {
    // do any setup needed
    return true
  }
  // define your RPC-able methods here
}

const onMessage = async ({ msgId, cmd, args }) => {
  let method = METHODS[cmd]
  if (!method) method = () => new Error('Invalid method: ' + cmd)
  try {
    const resolve = await method(...args)
    process.send({ msgId, resolve })
  } catch (err) {
    const reject = {
      message: err.message,
      stack: err.stack,
      name: err.name
    }
    process.send({ msgId, reject })
  }
}

if (process.env.APP_TEST_DRIVER) {
  process.on('message', onMessage)
}
```

Then, in your test suite, you can use your `TestDriver` class with the test automation
framework of your choosing. The following example uses
[`ava`](https://www.npmjs.com/package/ava), but other popular choices like Jest
or Mocha would work as well:

```js title='test.js' @ts-nocheck
const test = require('ava')
const electronPath = require('electron')
const { TestDriver } = require('./testDriver')

const app = new TestDriver({
  path: electronPath,
  args: ['./app'],
  env: {
    NODE_ENV: 'test'
  }
})
test.before(async t => {
  await app.isReady
})
test.after.always('cleanup', async t => {
  await app.stop()
})
```

[chrome-driver]: https://sites.google.com/chromium.org/driver/
[Puppeteer]: https://github.com/puppeteer/puppeteer
[playwright-intro]: https://playwright.dev/docs/intro
[playwright-electron]: https://playwright.dev/docs/api/class-electron/
[playwright-electronapplication]: https://playwright.dev/docs/api/class-electronapplication
[playwright-page]: https://playwright.dev/docs/api/class-page
[playwright-releases]: https://playwright.dev/docs/release-notes
[playwright-test-config]: https://playwright.dev/docs/api/class-testconfig#test-config-test-match
[Chrome DevTools Protocol]: https://chromedevtools.github.io/devtools-protocol/