The Evidence File
Every number traces back to a raw answer.
The audit trail other brand lift studies leave out. It ships with every SpinProof study, with the raw data and crosstabs, so you, your agency or your auditor can check the work without asking us.
What it is
The Evidence File is the audit trail that ships with every SpinProof study. It proves how the study was designed, who answered, who was excluded and why, what every agent and researcher did, and where each reported number came from. It is always included with the raw data and crosstabs, never available only on request.
What’s in it.
| Part | What it proves |
|---|---|
| Method lock | The spec, survey version, primary KPI, sub-groups and analysis plan, hashed and timestamped before fieldwork |
| Sample flow | Who was reached, started, screened out, dropped out, completed and excluded, by channel, with every reason |
| Quality | Fraud flags, attention failures, speeders, duplicates and the decoy-ad false-recall rate |
| Agent run log | Every agent action, with the model or code version that did it |
| Sign-offs | Who approved each of the three gates, and when |
| Results trace | Each reported number and chart linked to the source rows and the analysis output that produced it |
What it would have caught.
We reviewed three recent brand lift reports from one vendor. None proved its headline on the full sample. An Evidence File would have caught each problem:
- Sub-group headlines. A ‘+26% awareness’ headline came from 39 against 32 people chosen after the data came in. The method lock makes that impossible: sub-groups are fixed before fieldwork.
- Charts that drift from the test. One report drew 85% against 91% as 48% against 52%. The results trace ties every chart to the tested numbers.
- Assumed exposure. The test group was surveyed in the venue and the control came from a separate panel. The sample flow shows exactly how exposure was established.
- Data on request. Significance tests were ‘available upon request’. The Evidence File, raw data and crosstabs ship with every study.
Ask every vendor for one.
You don’t need to use SpinProof to raise the bar. Ask any brand lift vendor these five questions before you trust a headline:
- 1Was the primary KPI fixed before fieldwork, and can you show me when?
- 2What was the smallest lift this study could detect?
- 3How was exposure proven, and what was the false-recall rate on a decoy ad?
- 4How many people were excluded, and for what reasons?
- 5Can I have the raw data and see how each headline number was calculated?
What you can download.
- responses.csv: every complete, valid and excluded, with flags and reasons.
- crosstabs.csv: every KPI by cell and every pre-registered sub-group.
- evidence.json: the method lock, gates, agent run log and results trace in machine-readable form.
Questions
About the Evidence File.
Does the Evidence File cost extra?
No. It is included in every tier, from a single Creative Test to a 12-month Brand Tracker.
Can my agency or auditor check the work without us?
Yes. That is the point. The raw data, crosstabs and results trace let anyone recompute the headline numbers and see how each one was produced.
What does the hash prove?
The method lock records a SHA-256 hash of the spec, survey version, KPI, sub-groups and analysis plan at Gate 2. If any of it changed afterwards, the hash would no longer match, so you can see the plan wasn’t edited after the data came in.
Next step
See an Evidence File for yourself.
Open the demo portal and download the responses, crosstabs and evidence.json for a released study. Then ask your current vendor for the same.