runners running in parallel on a track

Photo by Austrian National Library on Unsplash

Sometimes it’s helpful to be able to identify which WebdriverIO worker is running a particular test. This is especially important when running tests in parallel.

For instance, where I work, we struggled with the length of time it took to run all of our end to end tests. These tests ran slowly, and it could take up to 45 minutes for a development team to see the results in the test reports.

To get faster feedback, we realized running the tests in parallel on different Chrome browsers would give us results 5x faster. To isolate the tests from each other, we created a pool of credentials for various test accounts. To run tests in parallel, it’s critical that no two tests use the same credentials at the same time as another test. Some tests need a certain setting to be enabled, while other tests need that setting to be disabled. By isolating each test, we guarantee that one test cannot hijack the test environment of another.

Automated tests, whether they be unit tests, functional tests, or end to end tests, should run in isolation of the others. When tests are independent, it’s much easier to run them in parallel and guarantee the same results as if we ran them sequentially, one by one.

For each test to know which browser it is running in, we need to be able to find a way to have the test retrieve the correct credentials from the credentials pool. My research shows that browser.options.credentials, which is set in the wdio.conf.js file, is accessible from all of the WebdriverIO hooks, as well as from within the spec files themselves. Thus, we built our credentials pool right into the WebdriverIO configuration file as follows:

wdio.conf.js

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38

capabilities: [
    {
        browserName: 'chrome',
        // Test Set 1
        specs: './tests/set1/**/*.spec.js',
        'wdio:credentials': {
            'username': 'testUser1',
            'password': 'PASSWORD'
        }
    },
    {
        browserName: 'chrome',
        // Test Set 2
        specs: './tests/set2/**/*.spec.js',
        'wdio:credentials': {
            'username': 'testUser2',
            'password': 'PASSWORD'
        }
    },
    {
        browserName: 'chrome',
        // Test Set 3
        specs: './tests/set3/**/*.spec.js',
        'wdio:credentials': {
            'username': 'testUser3',
            'password': 'PASSWORD'
        }
    },
    {
        browserName: 'chrome',
        // Test Set 4
        specs: './tests/set4/**/*.spec.js',
        'wdio:credentials': {
            'username': 'testUser4',
            'password': 'PASSWORD'
        }
    }
],

WebdriverIO allows us to use a vendor prefix to add our own properties to each capability. The ‘wdio:’ is our vendor prefix, and we use the ‘credentials’ property to store the credentials we’ll use for each browser.

We didn’t do any manual setup for these test accounts, although you could if you wanted to. Instead, we felt it would be much more scalable to write the test in a way that it uses the application’s API to perform all of the test setup prior to executing the test. Instead of a testing engineer manually flipping switches in the account settings, each test does this automatically with a quick 300ms API call. For instance, if one test needs the user’s timezone to be set in India, then we’ll change the account timezone in the spec file’s before hook. Another test, which might use Syndey’s timezone, would also use the API to set the timezone as needed.

When running the tests in parallel, the credentials are available via browser.options.credentials. To run tests, we’ll first need to login. So our test code might look like this:

sometest.spec.js

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20

describe('Tests for the Thingamajig', () => {
    
    // get the assigned credentials for whichever browser this test is running on
    const credentials = browser.options.capabilities['wdio:credentials'];

    before(() => {
        // here we use the API to change the timezone or do other setup as needed for that specific test account
    });

    it('should check that the widget behaves as expected', () => {
        browser.url('/login');

        // login using the assigned credentials
        $('#username').setValue(credentials.username);
        $('#password').setValue(credentials.password);
        $('button#login').click();

        // perform the test and validations here
    });
});

If we’re running tests with 4 capabilities, then 4 workers are running at any given time. In the WebdriverIO logs, you may have observed seeing something like this at the beginning of every logged output:

1

[0-1] ...

This is the capability ID, and WebdriverIO passes it into the onWorkerStart hook when each worker is about to start up. WebdriverIO also passes in the capabilities object into this same hook. Therefore, if you need the capability ID, for whatever reason, you can add it to the capabilities object and then access it from within another hook or within the spec files themselves.

wdio.conf.js

1
2
3
4
5

onWorkerStart: function (cid, caps, specs, args, execArgv) {
    console.log('onWorkerStart: cid = ' + cid);  // something like [1-0] or [1-1], etc...
    caps['wdio:cid'] = cid;    // assign the CID to the vendor prefixed wdio:cid property
    // ...
},

wdio.conf.js

1
2
3

beforeTest: function(test, context) {
    console.log('beforeTest: cid = ' + browser.options.capabilities['wdio:cid']);
}

The CID changes with each new test suite, so this can be helpful for things like building custom reporters to capture data about various tests.

Also, perhaps the biggest takeaway from all of this is that any data that you need to share between tests, which pertain to the environment the test is running on, can be assigned to browser.options.capabilities, and subsequently accessed by each test.