THEFTSubscribe

01 · UX Design

Shipped. Still in production.

Apple

Internal A/B and multivariate testing tool

Lead UX for an internal marketing experimentation platform

Status
Designed for Apple internal use
Service
UX design / research / product strategy
Role
UX design lead
Scope
Dashboard, experiment setup, variant editor, audiences, offers, shipping groups
Team
Design lead + embedded Apple marketing team
Launched
Internal

Apple was paying a third-party vendor for every experiment its internal marketing teams wanted to run, and the dependency was getting in the way of how the company actually wanted to test.

Jason led UX on an Apple-native A/B and multivariate testing platform, built on the company's internal directory for permissions and surfaced through an internal admin console.

The design covers a calendar dashboard, draft-first experiment states, a gated experiment setup, an IP-granular audience builder, and a multi-variant WYSIWYG editor with live preview. Engineering carried the platform to internal launch.

Method · effort by phase

01
02
03
04
05

01 · Discovery

Picked up mid-stream: audited prior work, reviewed competitor experimentation tools, and worked the system backward from prior results.

02 · Define

Embedded with the internal marketing team to build the requirements document, full sitemap, and navigation map before the first screen.

03 · Ideate

Designed the calendar dashboard, draft-first experiment states, gated setup, offer and shipping tooling, drag-and-drop audiences, and the multi-variant WYSIWYG editor.

04 · Prototype

Whiteboards, grid paper, and comps went through multiple rounds before anything was presented to the team.

05 · Implement

Detailed visual design handed off as the platform moved into build and launched internally.

What the work surfaced

01

Testers want three answers on entry

Which tests are running, how long, and where: the calendar dashboard answers those three questions first and pushes everything else deeper.

02

Drafts are first-class state

Testers run hundreds of experiments a week and pause mid-setup constantly, so drafts can be started, stopped, and edited without losing state.

03

Gates protect a global storefront

Setup is a gated five-step flow (metadata, audience, offer, variants, review) that forces approvals before launch; the gate is the product, not a layer on top.

04

A single directory is the permissions model

Every permission routes through the company's internal directory, so managers control access and ownership with no parallel auth system to maintain.

05

Audience granularity goes to IP

A drag-and-drop attribute palette lets non-technical marketers compose precise segments down to IP-block granularity without asking engineering.

06

Traffic allocation is one slider

A single slider controls the percentage of traffic per variant: faster to learn than a numeric field and impossible to enter invalid values.

07

One experiment, many variants, one mode

The variant editor enforces a single mode (A/B or MVT) per experiment so users cannot ship inconsistent tests or mis-attribute results.

08

Live preview in a browser simulator

The WYSIWYG editor previews each variant across Safari, Chrome, Firefox, and Edge before launch.

The artifacts, as wireframes

Shipped interfaces stay confidential. The structure is the deliverable: these are the wireframes the way they went to engineering.

APL.WIRE.01 · SitemapInternal admin console IA · reconstructed for illustrationAdminConsoleDashboardProof ofConceptsActivity LogAnalyticsArchivesProposalsProjectsSearchSettingsHelpNotificationsProcessChecklistExperimentsActivity LogAnalyticsSearchResultsExperimentSetupActivity LogValidationActivity LogQAStagingProductionRose path traces the flagship flow: Projects to Experiment Setup to Validation to environments
APL.WIRE.01 · Sitemap · Internal admin console through Projects, Experiments, Validation, environments
APL.WIRE.02 · DashboardStore selector plus five experiment state tables · reconstructed for illustrationMainStores › B5 › 1Welcome Back, John ▾DashboardStoresArchivesExperimentRequestRegion AA1A2A3A4A5A6A7Region BB1B2B3B4B5B6B7B8B9B10Region CC1C2Active ExperimentsVisitorsGoalsStart DateEnd DateUserLorem ipsum dolor sit amet14,002409-26-201410-03-2014John DoeLorem ipsum dolor sit amet8,201309-26-201410-03-2014John DoeLorem ipsum dolor sit amet3,880209-26-201410-03-2014John DoePaused ExperimentsVisitorsGoalsStart DateEnd DateUserLorem ipsum dolor sit amet14,002409-26-201410-03-2014John DoeLorem ipsum dolor sit amet8,201309-26-201410-03-2014John DoeDraft ExperimentsVisitorsGoalsStart DateEnd DateUserLorem ipsum dolor sit amet14,002409-26-201410-03-2014John DoeLorem ipsum dolor sit amet8,201309-26-201410-03-2014John DoeLorem ipsum dolor sit amet3,880209-26-201410-03-2014John DoeStopped ExperimentsVisitorsGoalsStart DateEnd DateUserLorem ipsum dolor sit amet14,002409-26-201410-03-2014John DoeCompleted ExperimentsVisitorsGoalsStart DateEnd DateUserLorem ipsum dolor sit amet14,002409-26-201410-03-2014John DoeLorem ipsum dolor sit amet8,201309-26-201410-03-2014John DoeDashboard reads by country then by experiment state; drafts are a first-class surface, not a bucket
APL.WIRE.02 · Dashboard · storefront selector drives five experiment state tables
APL.WIRE.03 · Experiment SetupApproval gate plus full details form · reconstructed for illustrationMainDashboardDraft ExperimentsLaunchpadLive Experiments+ConceptsArchives+AnalyticsAdminManage UsersActivity LogSettingsHelp3JohnFollowVisual EditorApprovaliBoard ApprovedYesNoSchedule KickoffScheduleExperiment DetailsUndefinedProposalEnter URL of proposal hereExperiment NameEnter Name of Experiment hereCountryStore FrontUrl Pathchoose store front above or enter custom url path here...+ Add AnotherStart Date10/01/14Start Time00:00 AMEnd Date10/31/14End Time00:00 AMTracking Var52 ▾Mutually ExclusiveYesExperiment DescriptionEnter experiment description here.NotesEnter any notes related to the experiment here.Requested AssetsRequest new assetsQA RequestRequest QA testingCancelSubmitGated form: Approval must clear, details filled, assets and QA requested before submit unlocks
APL.WIRE.03 · Experiment setup · approval gate, details form, asset and QA requests
APL.WIRE.04 · Experiment DetailsMid-edit: seasonal variant, multi-region storefronts · reconstructed for illustrationMainDashboardDraft ExperimentsLaunchpadLive Experiments+ConceptsArchives+AnalyticsAdminManage UsersActivity LogSettingsHelp3JohnFollowVisual EditorExperiment DetailsExperiment NameSeasonal Variant 1CountryRegion BStore FrontB1, B2, B3, B4, ...Url Pathhttps://storefront.example.com/B1/×https://storefront.example.com/B2/×+ Add AnotherExperiment DescriptionLorem ipsum dolor sit amet, consectetur adipiscing elit. Donec id quam at augue consectetur aliquet. Quisquevitae vehicula nibh. Sed imperdiet lobortis purus, ac pharetra lorem efficitur ac. Vivamus tempus metus aliquetlectus pellentesque, pulvinar ornare purus tincidunt. Sed massa sapien, imperdiet eu leo sit amet, lobortisdapibus justo. Etiam molestie ut libero ut vulputate. Sed rutrum lobortis sapien sit amet aliquam.CancelFilled state: scope (region + two storefronts) is the spine, URLs stack with add-another affordance
APL.WIRE.04 · Experiment details · seasonal variant across multi-region storefronts

Delivered

  • Product requirements document, drafted whiteboard-in, iterated with marketing
  • Full sitemap across three levels with admin-gated leaves
  • Navigation map unifying front-end, role-gating the back-end
  • Calendar dashboard with active, paused, and draft experiments
  • Draft-first experiment state management
  • Gated five-step experiment setup (metadata, audience, offer, variants, review)
  • Asset request flow routed through internal directory groups
  • Offer setup with shipping method and shipping-group configuration
  • Drag-and-drop audience builder with IP-level granularity
  • Single-slider traffic allocation
  • Multi-variant WYSIWYG editor with per-variant page support
  • Live preview browser simulator across Safari, Chrome, Firefox, Edge
  • Internal directory permissions integration
  • Grid system across desktop, tablet landscape, tablet portrait, mobile
  • Detailed visual design through hand-off to engineering

Outcome

An internal, directory-permissioned A/B and multivariate testing platform designed for Apple marketing.

A complete design system and product designed for the internal marketing team, built around the company's existing internal directory permissions and approval flow.

Modes
A/B + multivariate
Status
Designed for Apple internal use
Surface
Web · internal admin console