Version: Next

Test runner

While Detox was created to test mobile applications, effectively it is not a test runner – instead, it runs on top of a test runner. There are many third-party solutions for running tests, so we're happy to not reinvent the wheel and to devote our time to the mobile domain itself.

Since we focus on React Native and yet require some test runner under the hood, the most logical choice was to provide an official integration with Jest, which is the default test runner for such projects. This is why all our guides presume you have Jest under the hood, and the structure generated via detox init is no exception.

The integration with a test runner is a matter of the configuration, not the implementation – Detox source code has no hard-coded logic for Jest save for a few minor places¹. Furthermore, we're looking forward to new third-party integrations with popular test runners as the Internals API keeps improving.

Location

You can define the testRunner config section in two ways: globally and locally (per a configuration):

.detoxrc.js
/** @type {Detox.DetoxConfig} */
module.exports = {
  testRunner: {
    /* global section */
  },
  devices: { /* … */ },
  apps: { /* … */ },
  configurations: {
    'ios.sim.debug': {
      device: 'iphone',
      app: 'ios.debug',
      testRunner: {
        /* local (per-configuration) section */
      },
    },
  },
};

Properties

`testRunner.args` [object]

This section is responsible for building the test runner command that is going to be spawned when you run:

detox test
# $0 --key1 value1 … ---keyN valueN ...positionalArguments

For example, this configuration of a test runner:

{
  "testRunner": {
    "args": {
      "$0": "nyc jest",
      "bail": true,
      "config": "e2e/jest.config.js",
      "_": ["e2e/sanity-tests"]
    }
  }
}

would eventually spawn:

nyc jest --bail --config e2e/jest.config.js e2e/sanity-tests

Now, when you have an idea of what it does, let's overview the properties one by one.

`testRunner.args.$0` [string]

Default: jest.

Defines the beginning of the test runner command, usually just the name of the executable. The path to the executable is resolved according to your PATH environment variable.

Although not recommended, you can specify composite commands like node -r ./preload.js node_modules/.bin/my-runner.

`testRunner.args[…]` [string | number | boolean]

You can define arbitrary arguments in the key-value format, e.g.:

{
  "args": {
    "$0": "jest",
    "color": false,
    "bail": true,
    "testTimeout": 60000,
    "config": "e2e/jest.ci.config.js"
  }
}

For example, the config above would generate a command like this:

jest --no-color --bail --testTimeout 60000 --config e2e/jest.ci.config.js

As you can see, false boolean values produce keys prefixed with --no-.

`testRunner.args._` [string[]]

Default: [].

This property defines an array of default positional arguments to pass to the test runner. Consider an example:

{
  "args": {
    "$0": "jest",
    "_": ["e2e/sanity-tests"]
  }
}

If you run tests without extra positional arguments, you’ll get _ contents appended:

detox test -c ios.sim.debug
# jest … e2e/sanity-tests

If you run tests with custom positional arguments, the _ contents get replaced:

detox test e2e/regression-tests
# jest … e2e/regression-tests

If you use the retry mechanism of detox test, be prepared that the failed test file paths will override _ in all the subsequent re-runs.

`testRunner.retries` [number]

Default: 0.

Tells detox test to keep re-running the test runner with failed test files until they pass, or the number of repeated attempts exceeds the specified value:

detox test
DETOX_CONFIGURATION="…" jest --config e2e/jest.config.js e2e/sanity-tests
# …
# There were failing tests in the following files:
#   1. /path/to/your/test.js
#
# Detox CLI is going to restart the test runner with those files...
DETOX_CONFIGURATION="…" jest --config e2e/jest.config.js /path/to/your/test.js
# …

`testRunner.bail` [boolean]

Default: false.

When true, tells detox test to cancel next retrying if it gets at least one report about a permanent test suite failure. Has no effect, if testRunner.retries is undefined or set to zero.

`testRunner.detached` [boolean]

Default: false.

When true, tells detox test to spawn the test runner in a detached mode.

This is useful in CI environments, where you want to intercept SIGINT and SIGTERM signals to gracefully shut down the test runner and the device.

Instead of passing the kill signal to the child process (the test runner), Detox will send an emergency shutdown request to all the workers, and then it will wait for them to finish.

`testRunner.forwardEnv` [boolean]

Default: false.

When enabled, tells detox test to pass command-line arguments as environment variables to the test runner, e.g.:

detox test -c ios.sim.debug --record-logs all
DETOX_CONFIGURATION=ios.sim.debug DETOX_RECORD_LOGS=all jest …

Nevertheless, even if it is disabled, Detox will keep printing hints how to call your test runner without Detox CLI, so that you can copy and paste the command into your IDE when you want to debug something.

`testRunner.inspectBrk` [function]

caution

This property is intended primarily for developing integrations with third-party test runners.

Default: a Jest-specific callback that sets $0, --runInBand and cleans -w, --maxWorkers.

The provided function is called when detox test is called with --inspect-brk. Your implementation should prepare testRunner.args for debugging with Node.js inspector, e.g. for Jest that would be:

.detoxrc.js
/* @type {Detox.DetoxConfig} */
module.exports = {
  testRunner: {
    /** @param {Detox.DetoxTestRunnerConfig} config */
    inspectBrk: (config) => {
      config.args.$0 = os.platform() === 'win32'
        ? `node --inspect-brk ./node_modules/jest/bin/jest.js`
        : `node --inspect-brk ./node_modules/.bin/jest`;
      config.args.runInBand = true;
      delete config.args.w;
      delete config.args.workers;
    },
  },
};

`testRunner.jest` [object]

This is an add-on section used by our Jest integration code (but not Detox core itself). In other words, if you’re implementing (or using) a custom integration with some other test runner, feel free to define a section for yourself (e.g. testRunner.mocha)

`testRunner.jest.setupTimeout` [number]

Default: 120000 (2 minutes).

As a part of the environment setup), Detox boots the device and installs the apps. If that takes longer than the specified value, the entire test suite will be considered as failed, e.g.:

 FAIL  e2e/starter.test.js
  ● Test suite failed to run

    Exceeded timeout of 120000ms while setting up Detox environment

`testRunner.jest.teardownTimeout` [number]

Default: 30000 (30 seconds).

If the environment teardown) takes longer than the specified value, Detox will throw a timeout error.

`testRunner.jest.reportSpecs` [boolean | undefined]

Default: undefined (auto).

By default, Jest prints the test names and their status (passed or failed) at the very end of the test session. This might be fine for sub-second unit tests, but it is uncomfortable to wait a couple of minutes until you actually see anything.

When enabled, it makes Detox to print messages like these each time the new test starts and ends:

03:36.258 detox[40125] i Sanity: should have welcome screen
03:37.495 detox[40125] i Sanity: should have welcome screen [OK]
03:37.496 detox[40125] i Sanity: should show hello screen after tap
03:38.928 detox[40125] i Sanity: should show hello screen after tap [OK]
03:38.929 detox[40125] i Sanity: should show world screen after tap
03:40.351 detox[40125] i Sanity: should show world screen after tap [OK]

By default, it is enabled automatically in test sessions with a single worker. And vice versa, if multiple tests are executed concurrently, Detox turns it off to avoid confusion in the log. Use boolean values, true or false, to turn off the automatic choice.

`testRunner.jest.reportWorkerAssign` [boolean]

Default: true.

Like already mentioned, in the init phase, Detox boots the device and installs the apps. This flag tells Detox to print messages like these every time the device gets assigned to a specific suite:

18:03:29.869 detox[40125] i starter.test.js is assigned to 4EC84833-C7EA-4CA3-A6E9-5C30A29EA596 (iPhone 12 Pro Max)

`testRunner.jest.retryAfterCircusRetries` [boolean]

Default: false.

Jest provides an API to re-run individual failed tests: jest.retryTimes(count). When Detox detects the use of this API, it suppresses its own CLI retry mechanism controlled via detox test … --retries <N> or testRunner.retries. The motivation is simple – activating the both mechanisms is apt to increase your test duration dramatically, if your tests are flaky.

If you wish nevertheless to use both the mechanisms simultaneously, set it to true.

Jest config

Jest config generated by detox init is helpful for understanding how Detox integrates with Jest:

e2e/jest.config.js
/** @type {import('@jest/types').Config.InitialOptions} */
module.exports = {
  rootDir: '..',
  testMatch: ['<rootDir>/e2e/**/*.test.js'],
  testTimeout: 120000,
  maxWorkers: 1,
  globalSetup: 'detox/runners/jest/globalSetup',
  globalTeardown: 'detox/runners/jest/globalTeardown',
  reporters: ['detox/runners/jest/reporter'],
  testEnvironment: 'detox/runners/jest/testEnvironment',
  verbose: true,
};

All the listed properties vary from mandatory to strongly recommended, and below we'll be explaining why (and, more importantly, how to customize them correctly). If you need to add extra properties, please consult the Configuring Jest article on its official website.

rootDir and testMatch enforce the convention that your tests have .test.js extension and reside somewhere in e2e folder together with the Jest config:

├── …
├── e2e
│   ├── feature1.test.js
│   ├── feature2
│   │   ├── subfeature1.test.js
│   │   └── subfeature2.test.js
│   ├── …
│   └── jest.config.js
├── …
├── .detoxrc.js
└── package.json

testTimeout: 120000 overrides the default value (5 seconds), which is usually too short to complete a single end-to-end test. Two minutes should be safe enough, but you’re welcome to increase or decrease depending on your needs.
maxWorkers: 1 prevents potential over-allocation of mobile devices according to the default Jest strategy. By default, Jest picks cpusCount — 1 which is too much (e.g. 6-core laptop would spawn 11 devices). Note that casually you can override it via forwarding command-line argument --maxWorkers <N>:
```
detox test … --maxWorkers 2
# … jest … --maxWorkers 2
```
Change it only if you want to change the default value. For instance, you could use different number of workers depending on the environment, e.g.:
```
/** @type {import('@jest/types').Config.InitialOptions} */
module.exports = {
  // …
  maxWorkers: process.env.CI ? 2 : 1,
};
```
globalSetup file is essential as it integrates with Detox Internals API. If you need to set up something in addition, you should wrap it like this:
```
module.exports = async () => {
   await require('detox/runners/jest').globalSetup();
   await yourGlobalSetupFunction();
};
```

globalTeardown file is essential as it integrates with Detox Internals API. If you need to tear down something in addition, you should wrap it like this:

module.exports = async () => {
  try {
    await yourGlobalTeardownFunction();
  } finally {
    await require('detox/runners/jest').globalTeardown();
  }
};

reporters array should always include a reporter from Detox. We reserve right to add anytime some integration code there. Although currently it is rather empty, not having it puts you under risk every time you upgrade Detox versions.

testEnvironment is the most important part of the integration. If you need to add something on top of it, please inherit like shown below:

e2e/testEnvironment.js
const { DetoxCircusEnvironment } = require('detox/runners/jest');

class CustomDetoxEnvironment extends DetoxCircusEnvironment {
  constructor(config, context) {
    super(config, context);
    // custom code
  }

  async setup(config, context) {
    await super.setup(config, context);
    // custom code
  }

  async handleTestEvent(event, state) {
    await super.handleTestEvent(event, state);
    // custom code
  }

  async teardown(config, context) {
    try {
      // custom code
    } finally {
      await super.teardown(config, context);
    }
  }
}

module.exports = CustomDetoxEnvironment;

verbose: true disables batching of Jest logs and ensures you see the logs in real time.

Globals

Unless behavior.init.exposeGlobals is set to false, Detox exposes its primitives (expect, device, ...) globally, and it will override Jest’s global expect object. If you need to use it nevertheless, import it explicitly:

import jestExpect from 'expect';

Mocking

Don’t use jest.mock() or any other similar mocking mechanism. Follow our Mocking guide instead.

Parallel Test Execution

Detox relies on test runners to execute tests in parallel.

If you’re using Jest under the hood, the easiest way is to specify -w, --maxWorkers, e.g.:

detox test … --maxWorkers 2

In the other cases, consult your test runner documentation.

Forwarding CLI arguments

If Detox does not recognize CLI arguments you pass, it forwards them as-is to the underlying test runner, e.g.:

detox test -c ios.sim.debug --key1 value1 --key2
# DETOX_CONFIGURATION=ios.sim.debug jest --key1 value1 --key2
#
# ● Unrecognized CLI Parameters:
#
#   Following options were not recognized:
#   ["key1", "key2"]
#
#   CLI Options Documentation:
#   https://jestjs.io/docs/cli

Therefore, if test runner rejects such arguments, it is your responsibility to fix that.

Since there might be argument clashes between Detox and a test runner, you can use -- (double dash) to forward the arguments as-is, e.g.:

detox test -c ios.sim.debug -- --help
# DETOX_CONFIGURATION=ios.sim.debug jest --help
# Usage: jest [--config=<pathToConfigFile>] [TestPathPattern]
#
# Options:
# …

Detox has a few hard-coded default values for Jest: testRunner.args.$0 and testRunner.inspectBrk hook. Also detox test CLI is aware of Jest boolean arguments (e.g. -i, --runInBand, --bail, etc.), and it can auto-fix ambiguous commands like detox test --runInBand e2e/starter.test.js --bail. We're looking forward to make the code even more agnostic, but currently these caveats are worth mentioning for the developers of third-party test runner integrations.↩

Test runner

Location​

Properties​

testRunner.args [object]​

testRunner.args.$0 [string]​

testRunner.args[…] [string | number | boolean]​

testRunner.args._ [string[]]​

testRunner.retries [number]​

testRunner.bail [boolean]​

testRunner.detached [boolean]​

testRunner.forwardEnv [boolean]​

testRunner.inspectBrk [function]​

testRunner.jest [object]​

testRunner.jest.setupTimeout [number]​

testRunner.jest.teardownTimeout [number]​

testRunner.jest.reportSpecs [boolean | undefined]​

testRunner.jest.reportWorkerAssign [boolean]​

testRunner.jest.retryAfterCircusRetries [boolean]​

Jest config​

Globals​

Mocking​

Parallel Test Execution​

Forwarding CLI arguments​

Location

Properties

`testRunner.args` [object]

`testRunner.args.$0` [string]

`testRunner.args[…]` [string | number | boolean]

`testRunner.args._` [string[]]

`testRunner.retries` [number]

`testRunner.bail` [boolean]

`testRunner.detached` [boolean]

`testRunner.forwardEnv` [boolean]

`testRunner.inspectBrk` [function]

`testRunner.jest` [object]

`testRunner.jest.setupTimeout` [number]

`testRunner.jest.teardownTimeout` [number]

`testRunner.jest.reportSpecs` [boolean | undefined]

`testRunner.jest.reportWorkerAssign` [boolean]

`testRunner.jest.retryAfterCircusRetries` [boolean]

Jest config

Globals

Mocking

Parallel Test Execution

Forwarding CLI arguments