Skip to content
Open fromusers

Randomization groups

SurveysUsability tests

Randomization groups assign each participant to a study group (for example Group A / Group B) so you can run experiments inside one study. Assignment is balanced with permuted-block randomization, which keeps arms even as sample size grows.

  • A/B or multi-arm concept tests.
  • Prototype comparison where each person should see one design.
  • Content experiments where wording or flow differs by arm.
  • Any study where you need a stable group variable for filters in Results Studio.
  1. You define a randomization variable and its group values.
  2. As participants start, FromUsers assigns the next group from a balanced block sequence.
  3. After each full block (for a 50/50 split, every two assignments), counts stay matched.
  4. The assigned value is available as a variable for logic, piping, and analysis.

This avoids long runs of the same arm that can happen with naive coin-flip randomization, especially in smaller samples.

  1. Open study logic and create or enable a randomization group variable.
  2. Define the group labels (A/B, Control/Variant, or more arms).
  3. Use the variable in:
  4. Publish to Live and collect responses.
  5. Segment results by the group variable.

Two checkout prototypes:

  1. Create groups control and variant.
  2. Map each group to a prototype with compare assets.
  3. Keep the same task success criteria across arms.
  4. Compare completion and ratings by group in Results Studio.

Preview is for authoring and path checks. Live sessions perform real group assignment for participants. Always validate experiment wiring with test Live responses when possible.

  • Group balance assumes participants actually start sessions. Abandoned starts can still leave temporary unevenness until more people join.
  • Changing group definitions mid-field can complicate analysis. Prefer freezing arms before launch.
  • Randomization groups assign membership. They do not by themselves shuffle page order. Combine with section or block randomization when order bias also matters.
  • Keep arm names stable and analysis-friendly.
  • Document which content each arm should see.
  • See Experiment recipes for common patterns.

Common questions

How does balanced assignment work?

FromUsers uses permuted-block assignment so group sizes stay even as responses arrive, instead of drifting with pure random draws.

Can groups drive which prototype someone sees?

Yes. Use the group variable with compare assets, display logic, or branching to vary content by arm.

Do I need a separate study per variant?

Usually no. One study with randomization groups keeps analysis and distribution in a single place.