Testing JavaScript Performance with Benchmark.js

Whenever we want to improve the performance of our code, it is important to first have metrics to measure and compare, to ensure that our changes are having the desired effect. Benchmark.js is a commonly used tool for measuring the execution time of JavaScript code, supporting high-resolution timers for measurement accuracy, and ensuring statistically significant results are returned.

A number of web tools are based on Benchmark.js, allowing you to quickly create test suites that you can then run in browsers. Although jsPerf is directly affiliated with Benchmark.js, it tends to be unreliable (at time of writing, it has been down for days with a “402: Payment required” error), so my personal favourite is JSBench.Me, which allows you to create an account to save your test suites, and share them with others.

For example, this test suite shows the performance of the Array.prototype.somefunction with a handwritten equivalent based on iterating through the array in a for loop. We can put our different test cases into the different boxes and add as many as we choose to compare. Any setup logic should be put into the Setup JavaScript field, to ensure that the test cases are focused purely on running the code we actually want to test. When we’re ready, we just hit the RUN button and JSBench.Me will configure Benchmark.js behind the scenes to run our code and compare the results, cleanly presenting the difference in performance between the different options.

As you can see, the handwritten version performs significantly better, and so JSBench.Me highlights that test case as being the fastest, and identifies how much slower each of the other test cases is.

While the online tools can be useful for quickly comparing fragments of code, you might also want to integrate Benchmark.js into your own applications, for example to include performance tests in your CI pipelines to help prevent regressions.

Getting started is easy; simply install Benchmark.js by running the following:

npm i --save-dev benchmark

Then create a new file (e.g. bench.js) to contain your benchmark suite. You need to import the benchmark library:

const Benchmark = require("benchmark");

Create the Suite object, with a name if you like:

const suite = new Benchmark.Suite("My performance test");

You then need to configure the test suite, which features a nice fluent interface. You can add tests to the suite and a call to run it:

suite
  .add("RegExp#test", () => /o/.test("Hello World!"))
  .add("String#indexOf", () => "Hello World!".indexOf("o") > -1)
  .run();

You can now run the suite by executing the file, e.g. by running:

node bench.js

Do this, and you’ll see the script takes some time to run and then… returns nothing. Unfortunately we need to take some extra steps to actually get some results back!

Benchmark.js publishes a number of events while running which we add listeners for when configuring the test suite:

suite.on("complete", (event) => {
  const suite = event.currentTarget;
  const fastestOption = suite.filter("fastest").map("name");
  console.log(`The fastest option is ${fastestOption}`);
});

The currentTarget on the event is the full Suite object. Running this suite will now output the following:

The fastest option is String#indexOf

Now we know that the String#indexOftest case ran in the quickest time. If we want to also get more output as the test suite progresses, we can listen for the cycle event:

suite.on("cycle", (event) => {
  const benchmark = event.target;
  console.log(benchmark.toString());
});

The targethere on the event is the last run benchmark. Now our output will look something similar to this:

RegExp#test x 48,985,511 ops/sec ±0.73% (90 runs sampled)
String#indexOf x 819,109,711 ops/sec ±0.77% (92 runs sampled)
The fastest option is String#index

We now have some more information about how quickly the different options run, and the variance in those results. Fortunately there is a clear winner here, with the indexOf approach being an order of magnitude faster, with low variance giving us high confidence in the results.

Benchmark.js provides a whole host more options than those we have touched upon here in this introduction: you can find out more in their API documentation.

To finish up, here is the same test we originally ran in JSBench.Me but written directly using Benchmark.js:

const Benchmark = require("benchmark");
const suite = new Benchmark.Suite();

const values = [];

for (let i = 0; i < 1000000; i++) {
  values.push(i);
}

function some(list, predicate) {
  if (list == null) {
    return false;
  }

  for (let i = 0; i < list.length; i++) {
    if (predicate(list[i], i)) {
      return true;
    }
  }

  return false;
}

suite
  .add("Array.prototype.some", () => {
    const processed = values.some((value) => value > 990000);
  })
  .add("for loop", () => {
    const processed = some(values, (value) => value > 990000);
  })
  .on("cycle", (event) => {
    const benchmark = event.target;

    console.log(benchmark.toString());
  })
  .on("complete", (event) => {
    const suite = event.currentTarget;
    const fastestOption = suite.filter("fastest").map("name");

    console.log(`The fastest option is ${fastestOption}`);
  })
  .run();

Output:

Array.prototype.some x 111 ops/sec ±1.33% (78 runs sampled)
for loop x 1,687 ops/sec ±2.01% (91 runs sampled)
The fastest option is for loop

You can also run the suite in a browser by wrapping it in a simple HTML page:

<!DOCTYPE html>
<html>
  <head>
    <title>Benchmark.js test</title>
    <meta charset="utf-8" />
  </head>
  <body>
    <script src="https://cdn.jsdelivr.net/npm/lodash@4.17.21/lodash.min.js"></script>
    <script src="https://cdn.jsdelivr.net/npm/benchmark@2.1.4/benchmark.min.js"></script>
    <script src="bench.js"></script>
  </body>
</html>

And view the output in the developer tools console:

Output from benchmark suite in browser:

Note: you have to remove the require from bench.js to avoid errors in the browser; the Benchmark object will already be available globally.

Now that you know the basics of using Benchmark.js, you can start to make quantitative decisions about the performance of your code, and prove when you are making improvements. Go forth and make your code faster!

Or, before you do, find out how you can use Playwright to run Benchmark.js tests across different browsers.