The Hidden Race Conditions That Can Corrupt Browser-Based Experiments

Browser-based experiments rarely fail loudly. The variant renders, the dashboard fills up, and a winner gets declared. Underneath, though, a handful of timing bugs can decide which visitors get counted, which version they actually saw, and how many times their conversion was logged. None of those bugs throws an error, and almost none of them shows up in a code review.

One thread, many clocks

JavaScript in the page runs on a single main thread, which tempts developers into assuming things happen in a tidy order. They don't. Scripts load asynchronously, network responses come back whenever the network allows, timers fire late, and the visitor clicks whenever they please. An experiment depends on four steps landing in sequence: assign a bucket, persist that bucket, render the matching variant, then log exposure and conversion against it. Each step is waiting on something else, often on a script the experiment team doesn't own.

Every one of those steps can lose its race.

When the consent banner wins

Consent management is the first collision point. If the experiment script respects consent properly, it can't write its assignment cookie until the visitor accepts. Visitors who ignore the banner never get bucketed at all, so the test population quietly becomes "people who click Accept," a group that may convert differently from everyone else. If the script writes its bucket before consent and the storage is later cleared or blocked, a returning visitor gets reassigned on the next page view and may see both versions within a single session. The fix is an explicit ordering contract: assignment waits for a consent decision, and the analysis counts only visitors who reached that point in both arms.

Sample ratio mismatch is the smoke alarm

Most of these bugs surface the same way, as a split that doesn't match the configuration. A test set to 50/50 that logs 10,000 visitors in control and 10,400 in the variant looks close enough at a glance. A chi-square test on those counts says otherwise: the gap produces a p-value of about 0.005, far too unlikely to be chance.

That symptom has a name, sample ratio mismatch, and it means the comparison is compromised no matter how good the conversion numbers look.

The mismatch tells you something broke, not what broke. Tracing it back to a particular ordering bug takes request logs, client-side timing data and a fair amount of patience, which is why many teams bring in an A/B testing service to audit the delivery layer before they trust another result. An outside review is useful here because the people who wrote the integration tend to test the happy path they designed, while timing bugs live in the paths nobody designed.

Single-page apps move faster than the variant code

Single-page applications add a second clock. When a visitor navigates inside a React or Vue app, the route changes through the History API without a full page load, so an experiment script that ran once on the first page never runs again. A variant meant for the pricing page simply doesn't appear when the visitor arrives there by clicking through from the homepage. Worse, if the script does patch the DOM after a route change, the framework can re-render the component a few milliseconds later and wipe the change out. Those visitors are logged in the variant arm while looking at control, which drags both results toward each other and hides real effects.

Cookie scope adds its own wrinkle. Assignment usually lives in a first-party cookie, and how the browser stores, sends and isolates that cookie depends on attributes many front-end developers set once and forget. This plain-language piece that covers the basics of cookies, same-origin policy and CORS is a good refresher before anyone touches the assignment logic.

Tag managers and the conversion that counts twice

Asynchronous tag managers are the third source of drift. A conversion tag set to fire on both a button click and the thank-you page view will record two events for one purchase whenever both triggers succeed, and only one when the visitor closes the tab quickly. If one variant moves the thank-you message inline instead of on a new page, the trigger behavior differs by arm and so does the double-count rate. Component re-mounts in single-page apps can produce the same duplication. A unique event ID generated at the moment of conversion, with deduplication downstream, closes most of this gap.

Out-of-order network responses make things murkier still. Network speed, device performance and server load all shift the order in which requests complete, as this AbortController walkthrough explains response ordering in React apps. An exposure event that arrives after its own conversion event can be dropped by pipelines that expect exposure first.

Caches and subdomains that disagree

Server-side assignment is often presented as the cure for flicker and timing bugs, and it does remove several. It also introduces caching risk. If the variant is rendered into the HTML and a CDN caches that HTML without varying on the assignment cookie, every visitor in a region receives whichever version was cached first. The test still reports two arms, because the client-side logger reads the cookie, but the pages served no longer match the buckets logged.

Subdomains create the last trap. Browsers treat www.example.com and shop.example.com as the same site for SameSite purposes, a definition Google's documentation spells out with its own www and static subdomains as the example. Same site doesn't mean shared storage, though. A cookie set without a Domain attribute stays with the exact host that wrote it, so a visitor bucketed on the marketing site can be bucketed again, independently, at checkout.

Run an A/A test before trusting the B

The cheapest defense against all of this is an experiment where both arms are identical. An A/A test run through the full stack, with consent, routing, tag manager, CDN and every subdomain in play, should produce a clean split and no meaningful difference in conversions. If it shows sample ratio mismatch or a "significant" lift, the problem is in the plumbing, and finding it there costs far less than shipping a redesign on the strength of a result that was decided by script load order.