Search code examples
javascriptbenchmarkingmicrobenchmark

Why do these two identical benchmarks have different results on the same browser but with two different platforms?

I want to understand why these two equal benchmarks have different results on these two different platforms.

  1. Classcat, clsx, obj-str, vanilla JS on perf.link

  2. Classcat, clsx, obj-str, vanilla JS on measurethat

In the first one arr.filter(Boolean).join(" ") is the fastest, in the second one is the slowest!

Why?

The code I'm using is this:

import classcat from "https://unpkg.com/classcat?module"
import clsx from "https://unpkg.com/clsx?module"
import objstr from "https://unpkg.com/obj-str?module"

const obj = {
  one: true,
  abcd: Math.random() <= 0.5,
  efghijkl: Math.random() <= 0.5,
  mnopqrstuvwxyz: Math.random() <= 0.5,
  efghijklmnopqrstuvwxyz: Math.random() <= 0.5,
  abcdefghijklmnopqrstuvwxyz: Math.random() <= 0.5,
  abcd: Math.random() <= 0.5,
  efghijkl: Math.random() <= 0.5,
  mnopqrstuvwxyz: Math.random() <= 0.5,
  efghijklmnopqrstuvwxyz: Math.random() <= 0.5,
  abcdefghijklmnopqrstuvwxyz: Math.random() <= 0.5,
  two: false,
  three: true
}

const arr = [
  true && "one",
  Math.random() <= 0.5 && "abcd",
  Math.random() <= 0.5 && "efghijkl",
  Math.random() <= 0.5 && "mnopqrstuvwxyz",
  Math.random() <= 0.5 && "efghijklmnopqrstuvwxyz",
  Math.random() <= 0.5 && "abcdefghijklmnopqrstuvwxyz",
  Math.random() <= 0.5 && "abcd",
  Math.random() <= 0.5 && "efghijkl",
  Math.random() <= 0.5 && "mnopqrstuvwxyz",
  Math.random() <= 0.5 && "efghijklmnopqrstuvwxyz",
  Math.random() <= 0.5 && "abcdefghijklmnopqrstuvwxyz",
  false && "two",
  true && "three",
]

classcat(arr)
classcat(obj)
clsx(arr)
clsx(obj)
objstr(obj)
arr.filter(Boolean).join(" ")
Solution

  • This is a prime sample why you can't trust js benchmarks. The way the tests are run, the boilerplate code to do it, everything has an impact. The engine optimizes to each situation and the results, well, are garbage in the end.

    Read this: https://medium.com/cacher-app/how-js-benchmarks-lie-fa35fa989ee0
    Or the discussion on: Which JS benchmark site is correct?

    When I come into an issue with my code, I test it in that exact context as this is where it will actually need to go fast. This means that 1 solution does not fit all, many, but not all.