REserve — Testing UI5 — Executing tests

In this third article, the runner is improved to enable the execution of the tests (qUnit and OPA). The web server is modified to inject hooking scripts and new endpoints are provided to receive the tests results. Also, a basic execution queue is implemented so that we can control the number of instances that are executed simultaneously.

QUnit hooks

Because of its popularity, the library offers a variety of features and, in particular, it exposes some hooks to monitor the tests execution :

  • QUnit.begin : triggers a callback whenever the test suite begins
  • QUnit.testDone : triggers a callback whenever a test ends
  • QUnit.done : triggers a callback whenever the test suite ends

Each hook provides information about the current event. For instance, when the test suite begins, an object containing the number of tests to execute (member totalTests) is passed to the callback. The same way, when a test ends, the parameter contains information about the number of passed (member passed) and failed (member failed) assertions.

As we run the tests by starting a browser with the test page URL, it is possible to monitor the execution by leveraging these hooks.

Using dedicated endpoints, we can send back this information to the runner.

(function () {
'use strict'
function post (url, data) {
const xhr = new XMLHttpRequest()'POST', '/_/' + url)
QUnit.begin(function (details) {
post('QUnit/begin', details)
QUnit.testDone(function (report) {
post('QUnit/testDone', report)
QUnit.done(function (report) {
post('QUnit/done', report)

QUnit hooks to monitor the tests execution

Injecting QUnit hooks

The implemented solution is close to script substitution but with a twist : when the test page requests the qUnit resource, we concatenate it with the hooks.

This part is tricky and involves different mechanisms offered by REserve.

First of all, the qunit resource is part of the UI5 delivery, it may be either :

The regular expression /\/thirdparty\/(qunit(?:-2)?\.js)/ matches both.

Then, REserve can publish any local file using the file handler. The lazy me doesn't want to read a file using fs API so a mapping is declared to make the qunit hooks source available on the URL /_/qunit-hooks.js.

Finally, when a request hits thirdparty/qunit.js or thirdparty/qunit-2.js :

  • Two new requests are created : one to get the qUnit resource (ui5Request, the same URL is used) and the other one to read the hooks (hooksRequest on /_/qunit-hooks.js).
  • They are processed internally with the dispatch helper.
  • To avoid an infinite loop (and ensure that the UI5 resource is being retreived), the request ui5Request is flagged with a member internal set to true. The mapping ignores it when it loops back.
  • Once the internal responses are obtained, the final one is built by concatenating the two results.

This mapping must be applied before the UI5 one(s).

const { Request, Response } = require('reserve')/*...*/{
// QUnit hooks
match: '/_/qunit-hooks.js',
file: join(__dirname, './inject/qunit-hooks.js')
}, {
// Concatenate qunit.js source with hooks
match: /\/thirdparty\/(qunit(?:-2)?\.js)/,
custom: async function (request, response, scriptName) {
if (request.internal) {
return // ignore to avoid infinite loop
const ui5Request = new Request('GET', request.url)
ui5Request.internal = true
const ui5Response = new Response()
const hooksRequest = new Request('GET', '/_/qunit-hooks.js')
const hooksResponse = new Response()
await Promise.all([
this.configuration.dispatch(ui5Request, ui5Response),
this.configuration.dispatch(hooksRequest, hooksResponse)
const hooksLength = parseInt(hooksResponse.headers['content-length'], 10)
const ui5Length = parseInt(ui5Response.headers['content-length'], 10)
response.writeHead(ui5Response.statusCode, {
'content-length': ui5Length + hooksLength,
'cache-control': 'no-store' // for debugging purpose

Mappings to inject the hooks in the qunit resource


Members are :

  • total : the total number of tests
  • failed : the count of failed tests
  • passed : the count of passed tests
  • tests : an array aggregating the information reported by QUnit.testDone
  • report : the information reported by QUnit.done

When the tests are done, the resulting object is serialized for later reuse. Once the file is generated, the browser is shut down using the stop API.

A new parameter is added to the job :

  • tstReportDir : the directory where store reports

It is important to wait for the file to be written before calling stop. Indeed, a promise is returned when calling start and this promise is resolved right after stop is called. The last stop signals the end of all the tests meaning that the runner will possibly access these files.

NOTE : The helper filename converts the URL into a valid filename.

const { promisify } = require('util')
const { writeFile } = require('fs')
const writeFileAsync = promisify(writeFile)
const { filename } = require('./tools')
/* ... */{
// Endpoint to receive QUnit.begin
match: '/_/QUnit/begin',
custom: endpoint((url, details) => {
job.testPages[url] = {
total: details.totalTests,
failed: 0,
passed: 0,
tests: []
}, {
// Endpoint to receive QUnit.testDone
match: '/_/QUnit/testDone',
custom: endpoint((url, report) => {
const page = job.testPages[url]
if (report.failed) {
} else {
}, {
// Endpoint to receive QUnit.done
match: '/_/QUnit/done',
custom: endpoint((url, report) => {
const page = job.testPages[url] = report
const reportFileName = join(job.tstReportDir, `${filename(url)}.json`)
const promise = writeFileAsync(reportFileName, JSON.stringify(page))
promise.then(() => stop(url))

Endpoints keeping track of tests execution

Execution queue

A new parameter is added to the job :

  • parallel : the number of parallel tests allowed (default 2)

As already explained in a previous article, once the runner embedded server started, the probing of tests is triggered. After getting the list of pages to execute, the runner starts the number of tests given by the parameter parallel.

The function runTestPage is a sort of recursive one that calls itself after the test completed.

Two job members are added to keep track of the progress :

  • testPagesStarted : the number of tests already started. It also helps to know which page must be started next.
  • testPagesCompleted : the number of tests completed. When this number equals the number of test pages, the runner knows that the tests are over.

Because of the qUnit hooks, the end of the test will stop the browser which will resolve the promise. It means that once a browser is started, the flow of events will take care of the rest, explaining why the code is so simple.

/* ... */
.on('ready', ({ url, port }) => {
job.port = port
if (!job.logServer) {
console.log(`Server running at ${url}`)
async function extractTestPages () {
await start('/test/testsuite.qunit.html') // fills job.testPageUrls
job.testPagesStarted = 0
job.testPagesCompleted = 0
job.testPages = {}
for (let i = 0; i < job.parallel; ++i) {
async function runTestPage () {
const { length } = job.testPageUrls
if (job.testPagesCompleted === length) {
// Last test completed
return generateReport()
if (job.testPagesStarted === length) {
return // No more tests to run
const index = job.testPagesStarted++
const url = job.testPageUrls[index]
await start(url)
async function generateReport () {
/* ... */

Sequencing the tests inside the runner

Next step

Passionate developer, always keen on finding new ways to do things, welcome to my thoughts.