Artifacts

info

This article is incomplete. We're looking forward to improve this article as soon as we have an opportunity to do so.

Detox can store artifacts such as transient screenshots and device logs.

Location

You can define the artifacts config section in two ways: globally and locally (per a configuration):

.detoxrc.js

/** @type {Detox.DetoxConfig} */
module.exports = {
  artifacts: {
    /* global section */
  },
  devices: { /* … */ },
  apps: { /* … */ },
  configurations: {
    'ios.sim.debug': {
      device: 'iphone',
      app: 'ios.debug',
      artifacts: {
        /* local (per-configuration) section */
      },
    },
  },
};

Example

You can control artifacts collection via Detox configuration:

{
  "artifacts": {
    "rootDir": ".artifacts",
    "pathBuilder": "./config/pathbuilder.js",
    "plugins": {
      "instruments": {"enabled": false},
      "log": {"enabled": true},
      "uiHierarchy": "enabled",
      "screenshot": {
        "shouldTakeAutomaticSnapshots": true,
        "keepOnlyFailedTestsArtifacts": true,
        "takeWhen": {
          "testStart": false,
          "testDone": true
        }
      },
      "video": {
        "android": {
          "bitRate": 4000000
        },
        "simulator": {
          "codec": "hevc"
        }
      }
    }
  },
  "configurations": {
    "ios.sim.release": {
      // ...
      "artifacts": {
        "rootDir": ".artifacts/ios",
        "plugins": {
          "instruments": "all"
        }
      }
    }
  }
}

As can be seen from the example above, in a specific configuration you may override individual properties from the default artifacts configuration. For instance, in the example above you can see that specifically in ios.sim.release we turn on instruments plugin.

CLI arguments (e.g., --artifacts-location, --record-logs) still have the highest priority and override their counterparts from JSON.

Also, that example demonstrates that you can use strings (identical to the ones from CLI) in parallel to the object configurations for plugins. Below you can see mappings between the string presets and the corresponding objects:

preset	object
none	{ "enabled": false }
all	{ "enabled": true }
failing	{ "enabled": true, "keepOnlyFailedTestsArtifacts": true }
manual	{ "enabled": true, "shouldTakeAutomaticSnapshots": false }

There is also a shortcut to disable artifacts for a specific configuration:

{
  "configurations": {
    "ios.no-artifacts": {
      // ...
      "artifacts": false
    }
  }
}

Artifacts are various recordings during tests including, but not limited to, device logs, device screenshots and screen recordings (videos).

Enabling Artifacts

Artifacts are disabled by default. To enable them, specify via launch arguments or a configuration object what artifacts you want to record.

Launch Arguments

• To record .log files, add --record-logs all (or --record-logs failing, if you want to keep logs only for failing tests).
• To record .mp4 test run videos, add --record-videos all (or --record-videos failing, if you want to keep video recordings only for failing tests).
• To record .dtxrec (Detox Instruments recordings) for each test, add --record-performance all. To open those recordings, you’ll need Detox Instruments. NOTE: only iOS is supported.
• To capture .uihierarchy snapshots (iOS only, Xcode 12.0+) on view action failures, add --capture-view-hierarchy enabled.
• To take .png screenshots before and after each test, add --take-screenshots all (or --take-screenshots failing, if you want to keep only screenshots of failing tests).\ Alternatively, you might leverage the device.takeScreenshot() API for manual control.

Artifacts root directory

• To change artifacts root directory location (by default it is ./artifacts), add --artifacts-location <path>.\ NOTE: There is a slightly obscure convention. If you want to create automatically a subdirectory with timestamp and configuration name (to avoid file overwrites upon consequent reruns), specify a path to directory that does not end with a slash. Otherwise, if you want to put artifacts straight to the specified directory (in a case where you make a single run only, e.g. on CI), add a slash (or a backslash) to the end.

detox test --artifacts-location /tmp/detox_artifacts  # will also append /android.emu.release.2018-06-14 08:54:11Z
detox test --artifacts-location /tmp/detox_artifacts/ # won’t append anything, hereby treating it as a root

Configuration Object

Detox artifacts can be configured in a more advanced way with the artifacts configuration in package.json (or .detoxrc):

{
  "artifacts": {},
  "configurations": {
    "some.device": {
      "artifacts": {},
    },
  },
}

NOTE: As you can see, there is a global and a local (per-configuration) configuration of the artifacts. Detox merges those configurations, and the per-device artifacts configuration has a higher priority over the general one.

The artifacts object has the following properties:

Property	Example values	Default value	Description
rootDir	".artifacts/"	./artifacts	A directory, where all the recorded artifacts will be placed in. Please note that there is a trailing slash convention described above.
pathBuilder	"./e2e/config/pathbuilder.js"	undefined	Path to a module that exports a custom PathBuilder ᵃ
plugins	{ ... }	... see below	... see below

Path builder

ᵃ PathBuilder should be either an object with a method buildPathForTestArtifact or a class — see the corresponding interfaces below:

interface PathBuilder {
    buildPathForTestArtifact(artifactPath: string, testSummary?: TestSummary): string;
}

interface PathBuilderClass {
    new(opts: { rootDir: string; }): PathBuilder;
}

As one can see, if a custom implementation of PathBuilder exports a class instead of an object, then the class constructor can also get and save rootDir location:

class MyPathBuilder {
  constructor({ rootDir }) {
    this._rootDir = rootDir;
  }

  buildPathForTestArtifact(artifactName, testSummary) {
    /* ... use this._rootDir ... */
  }
}

module.exports = MyPathBuilder;

Its main method, buildPathForTestArtifact should return a full path to the custom artifact location, when called with a suggested artifact name (e.g., testDone.png, device.log) and the current TestSummary, where TestSummary is:

interface TestSummary {
    /**
     * Name of the current test, e.g., for:
     * describe('that screen', () =>
     *   it('should have a menu', () =>
     * The expected string would be: "should have a menu".
     */
    title: string;
    /**
     * Full name of the current test, usually preceded by a suite name, e.g.:
     * describe('that screen', () =>
     *   it('should have a menu', () =>
     * The expected string would be: "that screen should have a menu".
     */
    fullName: string;
    /**
     * Status of the current test. Free-form strings are not allowed.
     */
    status: 'running' | 'passed' | 'failed';
    /**
     * Clarifies the reason for why the test has failed.
     * Expected to coincide only with status: 'failed'.
     */
    timedOut?: boolean;
    /**
     * If the test runner is capable of retrying failed tests, then
     * this property indicates for which time this test is running.
     * When the property is undefined, its value is considered to be 1.
     * */
    invocations?: number;
}

For more technical details, search for ArtifactPathBuilder.js in Detox source code.

The further subsections describe the plugins object structure.

Screenshot Plugin

Below is a default screenshot plugin object configuration, which is loaded implicitly and corresponds to the manual preset:

{
  "plugins": {
    "screenshot": {
      "enabled": true,
      "shouldTakeAutomaticSnapshots": false,
      "keepOnlyFailedTestsArtifacts": false,
      "takeWhen": {
        "testStart": true,
        "testDone": true,
        "appNotReady": true,
      },
    }
  }
}

The other string presets override the following properties compared to the default configuration:

• none => { enabled: false }.
• failing => { shouldTakeAutomaticSnapshots: true, keepOnlyFailedTestsArtifacts: true }.
• all => { shouldTakeAutomaticSnapshots: true, keepOnlyFailedTestsArtifacts: false }

The individual property behavior is the following:

• If enabled is false, then the screenshots will never be saved to the artifacts' folder.
• If shouldTakeAutomaticSnapshots is false, then no one of the events described in takeWhen object is going to trigger a screenshot.
• If keepOnlyFailedTestsArtifacts is true, then only screenshots from a failed test will be saved to the artifacts folder.
• If takeWhen is undefined, it is going to have the default value described above (all props are true).
• If takeWhen is set to be an empty object {}, that is equivalent to:

{
  "testStart": false,
  "testDone": false,
  "appNotReady": true,
}

Hence, for example, if you wish to enable only testDone screenshots and leave taking appNotReady screenshots as-is, you have to pass:

{
  "artifacts": {
    "plugins": {
      "screenshot": {
        "takeWhen": { "testDone": true }
      }
    }
  }
}

Video Plugin

To be done. See meanwhile the example above.

Log Plugin

To be done. See meanwhile the example above.

Instruments Plugin

To be done. See meanwhile the example above.

UI hierarchy Plugin

To be done. See meanwhile the example above.

Artifacts Structure

1. Artifacts root folder is created per detox test run. If, for instance,--artifacts-location /tmp is used with --configuration ios.sim.release configuration on 14th June 2018 at 11:02:11 GMT+02, then the folder /tmp/ios.sim.release.2018-06-14 09:02:11Z is created.

2. Test folder is created per test inside the root folder. The folder name consists of the test number, and the test’s full name provided to detox.afterEach(testSummary) as explained above and in detox object documentation. For instance, for the above example, the following folders will be created inside /tmp/ios.sim.release.2018-06-14 09:02:11Z:

   ✗ Assertions should assert an element has (accessibility)
   ✓ Network Synchronization Sync with short network requests - 100ms

3. Artifacts files are created inside the test folders. The files suffixes stand for the files types (currently there are .err.log and .out.log), and the files prefixes are the launch numbers of the application per test (if the app was executed more than once per test, you will have several artifacts of each type - one per launch). For instance, a test folder may contain the following artifacts files:

   test.log
   test.mp4
   test.dtxrec/
   beforeEach.png
   afterEach.png

Example of the structure

artifacts/android.emu.release.2018-06-12 06:36:18Z/startup.log
artifacts/android.emu.release.2018-06-12 06:36:18Z/✗ Assertions should assert an element has (accessibility) id/beforeEach.png
artifacts/android.emu.release.2018-06-12 06:36:18Z/✗ Assertions should assert an element has (accessibility) id/test.log
artifacts/android.emu.release.2018-06-12 06:36:18Z/✗ Assertions should assert an element has (accessibility) id/test.mp4
artifacts/android.emu.release.2018-06-12 06:36:18Z/✗ Assertions should assert an element has (accessibility) id/afterEach.png