Demo: fictional airlines, illustrative data.Back to the site
SpinProof
Northmere Switch
Creative TestReleased· Released to you

Premium Economy launch: three routes for the ad.

Which of three Premium Economy ideas should go to production for the spring launch?

Audience
UK adults 25 to 64 who have taken a long-haul return flight in the last three years
Fieldwork
3 Aug to 6 Aug 2026
Sample
200 per cell · detects ±12.8 pts
Primary KPI
Consideration, fixed at Gate 2
Step 7 of 7 · Released3 of 3 gates signed
Page oneResultsOpen endsSample and qualityEvidence File

Method lock

Locked before fieldwork.

The spec, survey version, primary KPI, sub-groups and analysis plan are hashed at Gate 2. If any of it changed afterwards, the hash would not match.

Locked 31 Jul, 11:05nm-premium-economy-survey-v1.1

SHA-256 a346fbfe2354cc495a408f9f37ad59993bfbd33464d2a740874dc2cdce619309

Show the locked plan
{
  "design": "Creative Test",
  "audience": "UK adults 25 to 64 who have taken a long-haul return flight in the last three years",
  "perCell": 200,
  "cells": [
    "control",
    "A",
    "B",
    "C"
  ],
  "primaryKpi": "consideration",
  "secondaryKpis": [
    "favourability",
    "message",
    "intent"
  ],
  "subgroups": [],
  "test": "two-proportion z (pooled), Wald 95% interval, Holm adjustment across sub-groups and creatives",
  "minimumDetectableLift": 0.128,
  "exclusions": [
    "decoy-ad over-claim",
    "attention check",
    "speeder (< one third of median time)",
    "duplicate or fraud flag",
    "exposure not confirmed by media data"
  ]
}

Human gates

Who signed what, and when.

  1. Gate 1 · Researcher approves spec and survey

    Signed by SpinProof Research Desk, lead researcher, 27 Jul, 15:20

  2. Gate 2 · Analysis plan locked and hashed

    Signed by SpinProof Research Desk, lead researcher, 31 Jul, 11:05

  3. Gate 3 · Researcher signs the report

    Signed by SpinProof Research Desk, second reviewer, 11 Aug, 16:40

Agent run log

Every action, with the model or code version used.

Agents draft, recruit, check and write. Deterministic code does the statistics. Real people give every answer.

  1. Intake agent22 Jul, 09:12

    Turned the booking into study spec v1: Creative Test, audience, KPIs and cells.

    claude-sonnet-5-5 · intake prompt v1.4

  2. Intelligence agent22 Jul, 09:31

    Built the context brief: competitor ads from Meta Ad Library, share of search baseline, AI answer visibility baseline, news and social scan.

    claude-sonnet-5-5 · intel prompt v1.2

  3. Questionnaire agent22 Jul, 10:02

    Built the survey from the Creative Test template, with decoy ad, attention checks and consent screen.

    claude-sonnet-5-5 · questionnaire prompt v2.0

  4. Lead researcher27 Jul, 15:20

    Gate 1 signed: spec and survey approved with two wording edits.

    SpinProof Research Desk

  5. Channels agent28 Jul, 10:15

    Set up recruitment: Meta ad sets with screener, QR codes for placements, panel quotas, SpinProof panel invites.

    claude-sonnet-5-5 · channels prompt v1.3

  6. Analysis agent31 Jul, 11:05

    Locked the analysis plan. SHA-256 a346fbfe2354…

    stats-engine 0.9.2, deterministic

  7. Lead researcher31 Jul, 11:06

    Gate 2 signed: plan locked before fieldwork.

    SpinProof Research Desk

  8. Fieldwork agent3 Aug, 08:00

    Fieldwork opened. Invites live on all channels; rewards queue active.

    claude-sonnet-5-5 · fieldwork prompt v1.1

  9. Verification agent4 Aug, 18:00

    Daily quality pass: fraud, speeders, attention and decoy checks run; exclusions logged with reasons.

    claude-sonnet-5-5 · verification prompt v1.6

  10. Verification agent5 Aug, 18:00

    Daily quality pass: fraud, speeders, attention and decoy checks run; exclusions logged with reasons.

    claude-sonnet-5-5 · verification prompt v1.6

  11. Verification agent6 Aug, 18:30

    80 completes excluded in total: decoy-ad over-claim 29, failed attention check 23, speeder 21, duplicate or fraud flag 7.

    claude-sonnet-5-5 · verification prompt v1.6

  12. Fieldwork agent6 Aug, 20:00

    Quotas met in every cell. Fieldwork closed; rewards paid.

    claude-sonnet-5-5 · fieldwork prompt v1.1

  13. Analysis agent10 Aug, 09:00

    Ran the locked plan: every KPI, pre-registered sub-groups, Holm adjustment. No deviations from plan.

    stats-engine 0.9.2, deterministic

  14. Reporting agent10 Aug, 11:30

    Drafted the report from computed tables only. Every number linked to an analysis output.

    claude-sonnet-5-5 · reporting prompt v1.8

  15. Evidence agent10 Aug, 11:45

    Assembled the Evidence File. 0 untraced numbers. Charts re-checked against tested values.

    claude-sonnet-5-5 · evidence prompt v1.0

  16. Second reviewer11 Aug, 16:40

    Gate 3 signed: report checked against the Evidence File.

    SpinProof Research Desk

  17. Norms agent12 Aug, 09:00

    Added the study to the travel benchmark set.

    claude-sonnet-5-5 · norms prompt v0.9

  18. Portal12 Aug, 09:05

    Released to client portal with raw data, crosstabs and Evidence File.

    portal

Results trace

Every reported number, traced to raw answers.

Each number in the report links to the rows it came from and the analysis output that produced it.

ReportedSource rowsOutput
C1Creative A: Consideration +6.0 pts
responses.csv: cell=A vs cell=control → 68/200 vs 56/200
analysis/creatives.json#A
C2Creative B: Consideration +0.5 pts
responses.csv: cell=B vs cell=control → 57/200 vs 56/200
analysis/creatives.json#B
C3Creative C: Consideration +12.0 pts
responses.csv: cell=C vs cell=control → 80/200 vs 56/200
analysis/creatives.json#C

Downloads

Always included, never on request.

responses.csvEvery complete, valid and excluded, with flags and reasonscrosstabs.csvEvery KPI by cell and pre-registered sub-groupevidence.jsonMethod lock, gates, agent run log and results trace

Demo data. Fictional airlines, illustrative results generated for this demo.