A/B Testing

Run an experiment, split players into variants, and let ROLearn measure which one wins. Assignment happens on our side, so every server and client sees the same answer for the same player. Available on the Scale plan and above.

Create an experiment

Make an experiment in the dashboard (SDK, then Experiments) or with the API. You give it a name, two or more variant names, and how much traffic to include.

POST /api/sdk/experiments
{
  "name": "checkout_button_color",
  "variants": ["control", "blue"],
  "traffic_pct": 50,
  "status": "running"
}

Read a player's variant

Ask for the variant wherever you want to branch your game. The value is sticky: the same player always gets the same variant for the life of the experiment.

local RoLearn = require(game:GetService("ReplicatedStorage"):WaitForChild("RoLearn"))

-- Returns the player's sticky variant for this experiment ("control" if off).
local variant = RoLearn.getVariant("checkout_button_color", player)

if variant == "blue" then
    setButtonColor(Color3.fromRGB(10, 94, 255))
else
    setButtonColor(Color3.fromRGB(34, 197, 94))
end

How assignment works

• The variant is a SHA1 hash of the experiment id and the player id, so it is deterministic and needs no network round trip to stay consistent.
• It is sticky. A player keeps their variant even across sessions and servers.
• Players outside the traffic percentage, or when the experiment is paused, get control.

Read the results

As your normal events flow in, ROLearn compares the variants for you. No extra tracking calls are needed beyond the events you already send.

• Conversion style metrics (did they purchase, did they finish the level) use a 2 proportion z test.
• Value style metrics (revenue per player, session length) use a Welch t test, which is safe when the two groups have different spread.
• The console shows each variant's daily active users, ARPDAU, the lift, and whether the result is significant yet.