The verdict on the model itself
Believe the direction. Don't quote the decimal.
13 checks ran on this fit. Seven passed, five flagged, one failed. The failure is real, and it is why this page does not say “validated” — but none of it changes which channel worked.
7
passed
5
flagged
1
failed
The one that failed
Some of your channels ran too closely together to be told apart.
Measured 34.0081 against a line at 5.0000. Channel overlap, further down this page, names the pairs and says which one to deal with first.
The five that flagged
All of them are borderline rather than broken — worth knowing before someone argues with a slide, not worth refitting for.
It did not look long enough. Measured 184.7059 against a line at 400.0000.
The edges of the ranges are not settled. Measured 249.9867 against a line at 400.0000.
The search hit rough ground. Measured 0.0030 against a line at 0.0010.
It cannot even explain the history it was shown. Measured 6.0886 against a line at 10.0000.
Your data flatly contradicts something you told the model to expect. Measured 1.0000 against a line at 0.0000.
No out-of-sample test on this fit
Nothing was hidden from this model, so nothing on this page proves it can predict.
Every week of your data was used to fit it. The checks below still apply, but the one that matters most — does it work on weeks it has never seen — was not run. Refit with a holdout to get it.
What passed, in plain words
The full list, for whoever asks
Grouped as did the maths settle down, does it predict, could your data tell the channels apart, did your data actually teach the model anything, on made-up data, did it find the answer we hid. Each row further down carries the measured value, the threshold, and the engine’s own one-line explanation.
failed: Variance inflation (post-transform) · flagged: Bulk effective sample size, Tail effective sample size, Divergent transitions, In-sample accuracy, No prior/data conflict · 1 not run
In the engine’s order, and always the whole list — never a subset.
Open the full list ↓Weeks of evidence
156
Every week in every region the model was allowed to learn from.
Unknowns to solve for
32
Every separate quantity the model had to work out from your data.
Data per unknown
4.9×
How many weeks of evidence there are for each thing being worked out. Below about 4× you are asking more questions than your data can answer.
How wobbly the answer is
2
Above 30, changing one week of spend by a little moves the whole answer by a lot. Overlapping channels are what drive this up.
How this fit was set up
How we chose the way to fit your data, what that changed, which channels were held out of the budget plan, and what was tried but not used — each with what it costs you. Your numbers themselves were never edited: no imputing, no winsorising, no dropped rows.
How we chose the way to fit your data
Chosen by a test on your own spend
For the recordlocked in 28 Oct 2026, 15:58 UTCBefore fitting your data, we built 12 test versions of your sales on your real spend, each hiding returns we chose, and fitted each of 4 candidate setups to all of them. In each test version we also checked how much of the possible gain from moving budget between channels each setup's advice would have captured. Each setup also forecast your latest 8 weeks of sales without seeing them. We hid those weeks for this check only: your fit itself uses every week. A setup that makes 4 changes to your model forecast them worse than your model as you specified it, by more than ordinary week-to-week wobble, so it could not be chosen. Your model as you specified it proved the hardest to fool. It was locked in before any result from your own data existed, so nothing in your data could sway the choice. On 3 of those versions, re-checked with our slower, more thorough fitting method, it found the hidden returns of 1 of your 6 channels with spend.
On the test versions of your sales · your six channels with spend
one found on its ownfive not found
What it means for you
No alternative earned a change to your model. The test shows how a setup does on returns we hid in your own spend. It cannot see demand your data does not record, which could still inflate every channel's return.
Channels held out of the budget plan
The budget planner leaves each of these at today's spend. Each one says why, and what that costs you.
- Paid SocialNot found on its ownApplied
When we hid known returns in your spend, the chosen setup did not find Paid Social's return reliably, so we do not report a return for it. It still counts toward the total effect of your media, and the budget planner will not move it.
What it costs you: You get no return and no budget advice for Paid Social. A test that changes its spend on its own would make it measurable.
- Paid SearchNot found on its ownApplied
When we hid known returns in your spend, the chosen setup did not find Paid Search's return reliably, so we do not report a return for it. It still counts toward the total effect of your media, and the budget planner will not move it.
What it costs you: You get no return and no budget advice for Paid Search. A test that changes its spend on its own would make it measurable.
- Programmatic DisplayNot found on its ownApplied
When we hid known returns in your spend, the chosen setup did not find Programmatic Display's return reliably, so we do not report a return for it. It still counts toward the total effect of your media, and the budget planner will not move it.
What it costs you: You get no return and no budget advice for Programmatic Display. A test that changes its spend on its own would make it measurable.
- AffiliateNot found on its ownApplied
When we hid known returns in your spend, the chosen setup did not find Affiliate's return reliably, so we do not report a return for it. It still counts toward the total effect of your media, and the budget planner will not move it.
What it costs you: You get no return and no budget advice for Affiliate. A test that changes its spend on its own would make it measurable.
- Retail MediaNot found on its ownApplied
When we hid known returns in your spend, the chosen setup did not find Retail Media's return reliably, so we do not report a return for it. It still counts toward the total effect of your media, and the budget planner will not move it.
What it costs you: You get no return and no budget advice for Retail Media. A test that changes its spend on its own would make it measurable.
Tried on your spend, not used
Each block is a setup we tried on the test versions of your sales and did not choose, with the changes it made. Your fit uses none of them.
An alternative we tried
Tried as part of a setup that makes 4 changes to your model, which did not do as well as your model as you specified it on the test versions of your sales, so it was not taken further.
- One combined return for two channels bought togetherTried, not usedraised by two channels bought togetherPaid Social + Paid Search
- A firmer starting assumption for a channel the data barely seesTried, not usedraised by spend that runs in short burstsTV — Broadcast
- A firmer starting assumption for a channel the data barely seesTried, not usedraised by spend that barely movesAffiliate
- A firmer starting assumption for a channel the data barely seesTried, not usedraised by a channel that ran for only part of your dataRetail Media
Nothing: your fit does not use this change.
An alternative we tried
Tried as part of a setup that makes 4 changes to your model, which forecast your latest 8 weeks of sales worse than your model as you specified it, by more than ordinary week-to-week wobble, so it could not be chosen.
- A shared starting assumption for two channels bought togetherTried, not usedraised by two channels bought togetherPaid Social + Paid Search
- An assumed diminishing-returns curve for a channel the data barely seesTried, not usedraised by spend that runs in short burstsTV — Broadcast
- An assumed diminishing-returns curve for a channel the data barely seesTried, not usedraised by spend that barely movesAffiliate
- An assumed diminishing-returns curve for a channel the data barely seesTried, not usedraised by a channel that ran for only part of your dataRetail Media
Nothing: your fit does not use this change.
Did the maths settle down?
The model does not solve your data in one pass — it searches for the answer from several starting points at once, and it is only finished when they all arrive at the same place. If they did not, nothing further down this page means anything.
- PassChain convergence (R-hat)
Worst R-hat 1.0096 at roi. Above 1.01 the chains have not mixed and the posterior is not yet the posterior.
measured 1.010 · threshold 1.010
- WarnBulk effective sample size
Lowest bulk ESS 185 at baseline_knots. Under 400 the point estimates carry visible Monte Carlo error.
measured 184.706 · threshold 400.000
- WarnTail effective sample size
Lowest tail ESS 250 at roi. Under 400 the interval endpoints carry visible Monte Carlo error.
measured 249.987 · threshold 400.000
- WarnDivergent transitions
3 divergences in 1000 draws (0.30%). Divergences mean the sampler could not follow the posterior's curvature, so the region it failed in is under-represented. Raise target_accept_prob (spec section 6) before trusting the numbers.
measured 0.003 · threshold 0.001
Does it predict?
Matching history proves very little: a flexible enough model can trace any past. What counts is the stretch of weeks we hid from it — and whether it beat the free answer of assuming this year looks like last year.
- WarnIn-sample accuracy
MAPE 6.1% (target <10%), R-squared 0.895 (target >0.9). In-sample fit is necessary, not sufficient — a model with enough knots fits anything.
measured 6.089 · threshold 10.000
- SkippedHoldout accuracy
No holdout was reserved. Fit with `data.with_holdout(n)` to get an out-of-sample number; in-sample fit alone cannot detect overfitting.
- PassPosterior-predictive check
Bayesian p-value 0.520 (target 0.05-0.95). High values mean the estimated noise is larger than the residuals — the model is not being held to account by the data.
measured 0.520 · threshold 0.050
- PassBaseline stays positive
P(baseline < 0) = 0.00 (target <0.2). A negative baseline is the classic symptom of over-attribution: the model has given media credit for more than the business would sell with no marketing at all.
measured 0.000 · threshold 0.200
Could your data tell the channels apart?
When two channels go up and down together, no amount of modelling can say which one did the work — the split you get back is the one we assumed going in. These checks measure that rather than hope about it.
- FailVariance inflation (post-transform)
Highest VIF 34.0 on Retail Media (warn 5, hard 10), measured after adstock and saturation because that is what the model sees.
measured 34.008 · threshold 5.000
- PassDesign conditioning
Condition number 2.3 (target <30) on the media block with the baseline and controls projected out.
measured 2.319 · threshold 30.000
- PassObservations per parameter
156 observations, 32 parameters, 4.9x (target >4).
measured 4.875 · threshold 4.000
Did your data actually teach the model anything?
The classic way marketing models mislead is by handing back the assumption they started with, dressed up as a finding. A channel is flagged here only when two separate signals agree the data never budged it.
- PassData moved the prior
Every channel's ROI posterior moved off its prior; the least informed narrowed by -571%. A channel whose posterior is its prior has been assumed, not estimated.
measured -5.713 · threshold 0.200
- WarnNo prior/data conflict
Outside the prior's 1st-99th percentile: Paid Search (its return is withheld). Either the prior is wrong or the data are being over-read; the fit is a compromise between two claims that disagree.
measured 1.000 · threshold 0.000
On made-up data, did it find the answer we hid?
Only possible on synthetic data, where we know the right answer because we chose it. Passing means the ranges came out wide enough to contain the truth on this dataset.
- PassRecovers known ROI on synthetic data
6/6 true ROIs inside the 90% interval (need 6), the median interval spanning 2.3x the channel's true return. The count includes Paid Social, Paid Search, Programmatic Display, Affiliate, Retail Media, whose own returns are withheld.
measured 1.000 · threshold 0.900
Channel overlap
How closely every pair of channels moved together. Measured after we account for advertising that keeps working after it stops running, and for the fact that the tenth million spent does less than the first — so this is the overlap the model has to live with, not the overlap you would see in a spend report.
| TV — Broadcast | — | 0.18 | 0.21 | 0.13 | -0.08 | -0.09 |
|---|---|---|---|---|---|---|
| Paid Social | 0.18 | — | 0.82 | 0.70 | 0.15 | -0.02 |
| Paid Search | 0.21 | 0.82 | — | 0.71 | 0.16 | 0.03 |
| Programmatic Display | 0.13 | 0.70 | 0.71 | — | 0.36 | -0.11 |
| Affiliate | -0.08 | 0.15 | 0.16 | 0.36 | — | 0.11 |
| Retail Media | -0.09 | -0.02 | 0.03 | -0.11 | 0.11 | — |
Read a cell as: when this channel went up, did that one go up too? 1.00 means always, 0.00 means never, −1.00 means the opposite. Anything above 0.80 and the two are one channel as far as your data is concerned — you can trust what the pair did together, but not the split between them.
How each channel spends
A channel that is always on at the same level gives the model almost nothing to learn from, however big its budget — it is the ups and downs that reveal what the spend does. This table shows which of your channels have them.
| Channel | Spend | Pattern | Weeks dark | Variation | VIF |
|---|---|---|---|---|---|
| TV — Broadcast | 33% | bursty | 60% | 1.38 | 1.1 |
| Paid Social | 23% | continuous | 0% | 0.34 | 3.8 |
| Paid Search | 20% | continuous | 0% | 0.44 | 6.8 |
| Programmatic Display | 13% | continuous | 0% | 0.32 | 3.2 |
| Affiliate | 7% | continuous | 0% | 0.01 | 2.4 |
| Retail Media | 4% | continuous since launch | 63% | 1.44 | 34.0 |