Server-Side A/B Testing SDK

Overview

@growthroadmaps/ab-server is a lightweight Node.js SDK for running server-side A/B experiments from your Growth Roadmaps account. It operates entirely on your server — variant decisions are made before the HTML response is sent to the browser, which means:

• Zero client-side flash — visitors never see content swap because the variant is already baked into the response.
• In-memory config — experiment definitions are fetched once and cached in memory, then refreshed on a configurable poll interval. No network round-trip on each request.
• Synchronous getVariant() — returns a variant assignment instantly, with no await required in your request handlers.
• Background event flushing — impression and conversion events are queued and sent to Growth Roadmaps in batches, so tracking never adds latency to your responses.
• TypeScript-first — full type definitions included, no @types package needed.

Requirements & Installation

The SDK requires Node.js 18+ (fetch API built-in). It works in any framework: Express, Fastify, Next.js, Hono, Koa, and plain HTTP servers.

npm install @growthroadmaps/ab-server

Or with yarn or pnpm:

yarn add @growthroadmaps/ab-server
# or
pnpm add @growthroadmaps/ab-server

Quick Start

Connect once at server startup, then call getVariant() on each request:

import { connect } from "@growthroadmaps/ab-server";

const ab = await connect({
  projectKey: process.env.GR_PROJECT_KEY!,
  apiHost: "https://growthroadmaps.com",
});

// In your request handler:
const variant = ab.getVariant({
  experimentId: "your-experiment-id",
  userId: req.user.id,         // or session ID, anonymous ID, etc.
  userAttributes: {            // optional — for audience targeting
    plan: req.user.plan,
    country: req.headers["cf-ipcountry"],
  },
});

// variant.name is "control", "variant-a", etc.
if (variant.name === "variant-a") {
  // render alternate version
}

// Track a conversion later:
ab.track({
  experimentId: "your-experiment-id",
  userId: req.user.id,
  eventName: "signup_completed",
});

// Graceful shutdown (flushes queued events):
process.on("SIGTERM", async () => {
  await ab.close();
  process.exit(0);
});

Configuration Options

Pass these options to connect():

API: connect()

Initialises the SDK, fetches experiment configurations, and starts background polling and flushing. Returns a client instance.

import { connect, ABClient } from "@growthroadmaps/ab-server";

const ab: ABClient = await connect(options);

Call connect() once during server startup. Store the returned client in a module-level variable or inject it via your framework's DI system. Do not call connect() per-request.

Returns: Promise<ABClient>

API: getVariant()

Assigns a user to a variant. Synchronous — no await needed. Uses deterministic hashing of experimentId + userId so the same user always gets the same variant.

const variant = ab.getVariant({
  experimentId: "hero-headline-test",
  userId: "user_8f3kd92",
  userAttributes: {
    plan: "pro",
    country: "US",
  },
});

console.log(variant.name);       // "control" | "variant-a" | "variant-b" ...
console.log(variant.id);         // slug identifier
console.log(variant.isControl);  // boolean

Returns: Variant

interface Variant {
  id: string;
  name: string;
  isControl: boolean;
  weight: number;  // traffic weight (0–100)
}

API: track()

Queues a conversion or custom event. Events are flushed in the background — this method returns immediately and does not block your response.

ab.track({
  experimentId: "hero-headline-test",
  userId: "user_8f3kd92",
  eventName: "signup_completed",
  properties: {
    plan: "pro",
    revenue: "49.00",
  },
});

Returns: void (event is queued, not sent immediately)

API: close()

Stops background polling and flushing, then sends all queued events before returning. Call this during graceful server shutdown to avoid losing events.

await ab.close();

Returns: Promise<void> — resolves once the final flush completes.

// Example: Express graceful shutdown
const server = app.listen(3000);

process.on("SIGTERM", async () => {
  server.close(async () => {
    await ab.close();
    process.exit(0);
  });
});

Framework Integration Examples

Express

import express from "express";
import { connect } from "@growthroadmaps/ab-server";

const app = express();

let ab: Awaited<ReturnType<typeof connect>>;

async function start() {
  ab = await connect({
    projectKey: process.env.GR_PROJECT_KEY!,
    apiHost: "https://growthroadmaps.com",
  });

  app.get("/", (req, res) => {
    const userId = req.session?.userId ?? req.ip;
    const variant = ab.getVariant({
      experimentId: "homepage-hero",
      userId,
    });

    res.render("home", { heroVariant: variant.name });
  });

  app.listen(3000);
}

start();

Fastify

import Fastify from "fastify";
import { connect } from "@growthroadmaps/ab-server";

const fastify = Fastify();

fastify.decorate("ab", null);

fastify.addHook("onReady", async () => {
  fastify.ab = await connect({
    projectKey: process.env.GR_PROJECT_KEY!,
    apiHost: "https://growthroadmaps.com",
  });
});

fastify.addHook("onClose", async () => {
  await fastify.ab?.close();
});

fastify.get("/", async (request, reply) => {
  const userId = request.session?.userId ?? request.ip;
  const variant = fastify.ab.getVariant({
    experimentId: "homepage-hero",
    userId,
  });

  return reply.view("home", { heroVariant: variant.name });
});

await fastify.listen({ port: 3000 });

Next.js API Routes

// lib/ab.ts — singleton client, shared across requests
import { connect, ABClient } from "@growthroadmaps/ab-server";

let client: ABClient | null = null;

export async function getABClient(): Promise<ABClient> {
  if (!client) {
    client = await connect({
      projectKey: process.env.GR_PROJECT_KEY!,
      apiHost: "https://growthroadmaps.com",
    });
  }
  return client;
}

// app/api/page-data/route.ts
import { NextRequest, NextResponse } from "next/server";
import { getABClient } from "@/lib/ab";
import { cookies } from "next/headers";

export async function GET(request: NextRequest) {
  const ab = await getABClient();
  const userId = cookies().get("userId")?.value ?? crypto.randomUUID();

  const variant = ab.getVariant({
    experimentId: "pricing-page-layout",
    userId,
  });

  return NextResponse.json({ layout: variant.name });
}

Next.js Middleware

Use middleware to assign variants before the page renders, then pass the decision via a response header or cookie so the page component can read it without an extra fetch:

// middleware.ts
import { NextRequest, NextResponse } from "next/server";
import { connect } from "@growthroadmaps/ab-server";

let ab: Awaited<ReturnType<typeof connect>> | null = null;

async function getClient() {
  if (!ab) {
    ab = await connect({
      projectKey: process.env.GR_PROJECT_KEY!,
      apiHost: "https://growthroadmaps.com",
    });
  }
  return ab;
}

export async function middleware(request: NextRequest) {
  const client = await getClient();
  const userId =
    request.cookies.get("userId")?.value ?? crypto.randomUUID();

  const variant = client.getVariant({
    experimentId: "homepage-hero",
    userId,
  });

  const response = NextResponse.next();
  response.headers.set("x-ab-variant", variant.name);
  // Ensure userId is persisted
  response.cookies.set("userId", userId, {
    httpOnly: true,
    maxAge: 60 * 60 * 24 * 365,
    sameSite: "lax",
  });

  return response;
}

export const config = {
  matcher: ["/"],
};

// app/page.tsx — read the variant from the header
import { headers } from "next/headers";

export default function HomePage() {
  const heroVariant = headers().get("x-ab-variant") ?? "control";

  return (
    <main>
      {heroVariant === "variant-a" ? (
        <h1>Your Better Headline Here</h1>
      ) : (
        <h1>Original Headline</h1>
      )}
    </main>
  );
}

Finding Your Experiment ID

Every experiment has a unique slug ID that you pass to getVariant() and track():

1. Log in to Growth Roadmaps and navigate to A/B Testing in the sidebar.
2. Open the experiment you want to connect to.
3. Click Settings (gear icon) or go to the Integration tab.
4. Copy the Experiment ID — it looks like hero-headline-test or a UUID.

Environment Variables

Keep your project key out of source control using environment variables:

# .env.local (never commit to git)
GR_PROJECT_KEY=pk_live_xxxxxxxxxxxxxxxxxxxxx

# .env.example (safe to commit)
GR_PROJECT_KEY=

import { connect } from "@growthroadmaps/ab-server";

const ab = await connect({
  projectKey: process.env.GR_PROJECT_KEY!,   // assert non-null for TypeScript
  apiHost: "https://growthroadmaps.com",
});

How Variant Assignment Works

The SDK uses deterministic hashing to assign users to variants:

1. The experimentId and userId are concatenated and hashed (Fowler–Noll–Vo, 32-bit).
2. The hash is mapped to a value between 0–100.
3. If the value falls within the experiment's trafficPercentage bucket, the user is assigned a variant based on variant weights. Otherwise, they receive the control.

This means:

• The same userId always gets the same variant for the same experiment — no cookies required.
• Variant assignment is computed locally with no network call.
• You can use any stable identifier: database user ID, anonymous visitor GUID, session token, etc.

Traffic Percentage

The traffic percentage setting on an experiment controls what fraction of your users are exposed to the test at all. Users outside the experiment always receive the control variant.

// Example: experiment set to 20% traffic
//
// Users with hash(experimentId + userId) in 0–20 → experiment bucket
//   → assigned to a variant based on variant weights (50/50, etc.)
//
// Users with hash value in 21–100 → not in experiment
//   → variant.name === "control", variant.isControl === true

This allows you to:

• Start small — launch at 10% traffic to reduce risk while collecting data.
• Ramp up — increase the percentage from the dashboard without a code change.
• Roll out — once you pick a winner, set the winning variant's weight to 100% and increase traffic to 100%.

Statistics: CUPED & Sequential

The conversion-rate engine ships two trustworthiness upgrades alongside the existing Bayesian Beta-Binomial outputs. Both are additive — the legacy "probability to beat control" and 95% credible interval still appear, with the new metrics shown next to them.

Sequential testing (mSPRT)

The dashboard exposes an always-valid probability ("AV %") for each treatment. It comes from a mixture Sequential Probability Ratio Test (mSPRT) that treats the difference Δ̂ = p_t − p_c as approximately normal and mixes over the alternative δ ~ N(0, τ²) with τ = 0.01 (≈ 1pp). The always-valid p-value p_av = min(1, 1/Λ) is safe to monitor continuously without inflating false positives — peeking is allowed. A treatment crosses to "win" or "loss" once p_av ≤ α (default α = 0.05). Decisions are sticky per experiment and goal: once a side crosses, the engine will not flip back to "inconclusive" on the next tick. Auto-conclude prefers this sequential decision over the legacy fixed-horizon verdict.

CUPED (variance reduction)

When enough exposed users have a stable userId, the engine applies CUPED with a single pre-period covariate per visitor: the count of prior conversions on the same goal recorded BEFORE the experiment's start across the same project's other experiments. The adjustment Y_adj = Y − θ(X − mean(X))with θ = cov(Y, X)/var(X) shrinks the per-arm variance by roughly 1 − ρ², tightening the always-valid CI and probability. A blue CUPED pill appears next to the AV % when the adjustment is applied. CUPED is gated by sample size (≥ 200 users total, ≥ 100 per arm) and |ρ| ≥ 0.05; below threshold the engine falls back transparently to the un-adjusted path. Anonymous visitors (no userId) are excluded because there is no way to attach pre-period history.

Backwards compatibility

No SDK or API contract changes. The /api/ab/experiments/:id/resultsresponse gains optional per-variant fields (always_valid_probability_to_be_best, always_valid_p_value, always_valid_ci_95 (95% credible interval on the treatment − control rate difference, valid under continuous monitoring), sequential_decision, cuped_applied, cuped_correlation, cuped_variance_reduction) and a top-level sequential + cuped summary. Existing consumers that only read the legacy fields keep working unchanged.

Troubleshooting