One post tagged with "major-release"

Today we're proud to announce the new major release, Detox 20 (codename "Ashán"), which brings:

• official support for Genymotion SaaS
• improved integration with test runners
• configurable logging subsystem
• headless mode for iOS via configs and CLI
• reversing TCP ports via Android app configs
• and more optimizations to land in the next minor versions.

Genymotion SaaS​

Highlights: Using Genymotion SaaS.

Two years ago we added elementary support for cloud-based Android emulators provided by Genymotion SaaS and started a beta testing phase across mobile projects at Wix.

Previously our mobile infrastructure engineers had been maintaining Android virtual devices on CI build agents on their own, so switching to cloud devices cleared up their time for more productive tasks. Another improvement was particularly noticeable for teams with a vast number of tests. We could reduce the duration of their CI pipelines almost by half after they scaled up from 2 parallel devices to 6¹.

This positive impact encouraged us to adopt Genymotion SaaS for CI as quickly as possible, ignoring some unresolved issues in the initial pull request. For the most part, those were minor problems in global lifecycle management. Yet that made us feel uncertain about releasing it as-is, so we decided to take time and gain more production experience before taking any direction.

The further experience was surprisingly smooth and rarely presented issues, spare for a few minor glitches in advanced scenarios. Admittedly, revamping the Detox lifecycle took us longer than expected, which is all the more reason for us to celebrate today.

We're looking forward to providing our users with more opportunities for testing in the cloud, and this step is only the first of many to come. We hope you'll utilize this new feature to your delight.

Integration with test runners​

Highlights: Config file > Test runner, Internals API, Dropping Mocha support.

It took about a few months of work to formalize the contract between Detox and a test runner. While there's still a lot of place for improvement, the new Detox release refines their interaction and lays the groundwork for third-party integrations.

Mocha was our first supported test runner, but unfortunately, it could not keep up with our scaling requirements as the number of end-to-end tests grew. By the time it acquired the ability to run tests in parallel, we already had to place bets on another horse, and that was Jest.

We attempted to keep compatibility with both Jest and Mocha, but the farther we went, the more obvious it was that we couldn't have it both ways. As it turned out, Jest wasn't easy to get along with – our first integration with it was too simplistic. Over a couple of years of use in production, we kept discovering various issues that forced us to rewrite our "glue" code from scratch twice, and this isn't over yet. All combined didn't leave much time and energy for tinkering with Mocha anymore.

In this release, we discontinued Mocha support to focus on the attunement of Jest with the new runner-independent test runner config and Internals API. If there's enough demand, now it is up to the open-source community to build a new integration between Detox and Mocha.

Configurable logger​

Highlights: Config file > Logger, Logger API.

The rigidity of the logging subsystem has always been showing itself since its very creation in the summer of 2019. Due to time constraints and existing tech debts, it was impossible to do it right the first time, so we lived about three years with a proof-of-concept rather than a full-fledged feature.

The inconveniences weren't fatal but quite noticeable, nevertheless. Here are a few syndromes you could have spotted if you have ever used Detox timeline and log artifacts, especially when running tests in parallel:

• an uncanny file array: detox_pid_7505.log, detox_pid_7505.log.json, detox_pid_7506.log;
• a relatively shallow detox.trace.json: test suites, test functions, and some user-defined segments.

The good news is that the new Detox release condenses all those numerous logs into two files:

• the plain, human-readable detox.log;
• the raw, machine-readable detox.trace.json for chrome://trace, Perfetto and other utilities.

With the help of the new Logger API, you can add custom duration events to the timeline, too, e.g.:

await detox.log.trace.complete('Login', async () => {
  await element(by.id('email')).typeText('john@example.com');
  await element(by.id('password')).typeText('123456');

  detox.log.info('Trying to log in...');
  await element(by.id('submit')).tap();
});

Besides, it is possible now to customize the console output of Detox via the new logger config, e.g.:

/** @type {Detox.DetoxConfig} */
module.exports = {
  // ...
  logger: {
    options: {
      showDate: false,
      showLoggerName: false,
      showPid: false,
      prefixers: {
        ph: null,
      },
    },
  },
};

In the example above, we minimize all the metadata around the log messages – see the screenshot below:

Minor features​

Headless iOS​

One of Detox known issues was always booting iOS simulators in a hidden mode. You could see tests running on your local simulator only if you had manually opened the Simulator app beforehand. So, we unified the headless property for both iOS and Android, and now both the platforms visibly boot a device unless you configure it otherwise, e.g.:

/* @type {Detox.DetoxConfig} */
module.exports = {
  devices: {
    iphone: {
      type: 'ios.simulator',
      headless: process.env.CI ? true : undefined,
      device: {
        type: 'iPhone 14'
      },
      /* ... */
    }
  },
};

or, via CLI:

detox test -c ios.sim.release --headless

Reverse ports​

Your apps might try to access some localhost:* addresses (e.g., mock servers), but this is a bit more problematic in the case of Android. The Android emulators are separate virtual devices with their own loopback network interface. In such cases, you must set up reverse port forwarding via adb reverse.

Local servers are quite a common prerequisite for apps in debug mode – one could recall React Native bundler on port 8081, Storybook server on 9009, etc. That's why we decided to add an optional config property for Android apps, reversePorts:

/** @type {Detox.DetoxConfig} */
module.exports = {
  // ...
  apps: {
    'android.debug': {
      type: 'android.apk',
      binaryPath: '...',
      reversePorts: [8081, 3000],
    },
  },
};

In other words, this is a convenience API that tells Detox to run device.reverseTcpPort(portNumber) after installing the app. It should be helpful for anyone who prefers to keep such things as configs rather than as code.

Read-only emulators by default​

The -read-only flag appeared in Android emulator 28.0.16. Detox promptly adopted it since the read-only mode allowed it to run multiple instances of a single Android virtual device (AVD) concurrently. This feature helped us to implement parallel test execution support for Android back then.

Being overcautious, we implemented that partially, only for cases when the user starts multiple concurrent workers. This decision created a moderately annoying UX issue. Imagine you run tests sequentially first, using one worker only. That provides you with a regular AVD instance, i.e., not a read-only one. After that, you switch to multiple workers only to get an error from the Android emulator, complaining about mixing regular and read-only instances.

While the fix itself has always been straightforward – close the running AVD and try again – this entire overcaution brought more issues than solving them. That's why, from now on, Android emulators will always be starting in -read-only mode unless you configure readonly: false in your device config.

Reset lock file​

This release adds a small CLI tool, detox reset-lock-file, to help users with one specific use scenario.

Imagine you want to run tests for multiple Detox configurations at once, e.g.:

detox test -c iphoneSE2020.release e2e/ui.test.js
detox test -c iphone14ProMax.release e2e/ui.test.js

The problem is that Detox uses a file-locking mechanism to avoid situations when parallel test workers would take control of the same device. The detox test command, upon start, erases that file contents, creating a race condition risk.

To eliminate that risk, use a combination of detox reset-lock-file and --keepLockFile like this:

detox reset-lock-file & \
detox test --keepLockFile -c iphoneSE2020.release e2e/ui.test.js & \
detox test --keepLockFile -c iphone14ProMax.release e2e/ui.test.js & \
wait

In the future, we plan to minimize using lock files so that you don't have to think about this low-level implementation detail. So, this tool adds some convenience until we provide a next-gen solution.

Deprecations​

Detox 20 executes many pending deprecations, so make sure to check out our Migration Guide before upgrading:

• JS: minimum supported Node.js version is 14.x;
• JS: minimum supported Jest version is 27.2.5;
• JS: Mocha test runner is no longer supported;
• JS: discontinued old adapters for Jest (jest-jasmine, first generation of jest-circus adapter);
• JS: discontinued { permanent: true } option in device.appLaunchArgs.* methods (#3360);
• CLI: dropped -w, --workers and -o, --runner-config args – see a dedicated section in the migration guide;
• CLI: dropped deprecated --device-launch-args (#3665);
• Config: discontinued kebab-case properties: test-runner, runner-config (#3371)
• Config: discontinued skipLegacyWorkersInjections property ((#3286))
• Config: deprecated specs and runnerConfig properties
• Config: changed semantics of testRunner property
• Config: dropped support for all-in-one configurations (#3386);
• Android: remove deprecated native IdlePolicyConfig (#3332)
• iOS: discontinued ios.none device type – see the new way to debug native code (#3361)

Afterword​

Over the last year and a half, we have established a centralized configuration system for more than 50 projects using Detox at Wix. While it never seemed to be a cakewalk, the entire experience of troubleshooting over a hundred issues across the organization did not leave us unchanged.

We see numerous things to improve in Detox, but most of them boil down to the same thing – scaling. Surprisingly, "scaling" makes an excellent umbrella term for nearly every challenge we've been encountering lately:

• scaling up the number of users requires us to improve the onboarding and troubleshooting experience;
• scaling up the number of projects forces us to centralize scattered configs into flexible organization presets;
• scaling up the number of tests prompts us to optimize the codebase and incline it towards cloud and remote execution.

Our core team has been facing challenges of limited human resource constraints and growing scaling needs for a long time. In many ways, that has shaped a specific mindset within the team. We evaluate every discussed feature by asking a simple question: will it save other people and us time to focus on more important things? Teaching a man to fish is better than giving fish, so our success at preventing support issues matters more than our success at solving them ourselves.

That's why we'll be making subsequent efforts in these three areas, hoping to get back to you soon with even more exciting updates.

Enjoy your drive with Detox 20!

Cheers! 👋