6 · Documentation
Noise band and runs
Why a single measurement is a sample, how we calculate the uncertainty and when a movement is a movement.
Whoever builds a score from a single measurement is partly measuring chance. augenmerk states the uncertainty for every project and reports changes only when they exceed it.
Same question, two answers
AI systems do not answer the same way every time. The same question yields two different answers on two days, with other brands in another order. Measured in the rice market across three runs, with nothing having changed about the brands:
| Brand | Run 1 | Run 2 | Run 3 | Span |
|---|---|---|---|---|
| Tilda | 67 % | 83 % | 78 % | 17 pts |
| Reishunger | 50 % | 69 % | 67 % | 19 pts |
| dennree | 56 % | 33 % | 52 % | 22 pts |
The market leader's number is stable, everyone else's is not. That is not a weakness of the measurement but statistics: the spread of a share is largest at 50 % and small at the edges. The brands that need a measurement sit in the middle.
How the noise band is calculated
The uncertainty depends above all on how many counting questions carry a number. From a project's existing questions we draw repeatedly and measure the width of the 90 % range. We draw questions, not measurements: one question produces an answer in every channel, and those are related.
From this follows a rule of thumb the application uses: with eight counting questions the noise band is ±10 percentage points. With more questions it shrinks with the square root of the number of questions; with 32 counting questions it is around ±5 points. The overview and the competition view state the value for the project at hand.
| Counting questions | Noise band |
|---|---|
| 8 | ±10 pts |
| 13 | ±8 pts |
| 20 | ±6 pts |
| 32 | ±5 pts |
This is an extrapolation and rather optimistic, because real questions are not independent of each other. For brands in the middle of the field the actual spread can be higher. The application therefore adds two rules: segment values are shown only from three runs on, and below about twenty questions no score is produced, only a number with its stated uncertainty.
When a movement is a movement
augenmerk reports a change in presence since the last run only when it exceeds the noise band. Everything below is chance between two runs and does not appear as a finding. The same applies to gaps in the ranking:
- A place counts as secure when the lead over the next name exceeds the noise band. Below that, the overview reports a wobbling place: in the next run the order can flip without anything really having changed.
- A gain counts as real when the next run holds it. If the value falls back, it was an outlier.
- Stable means: no movement above the noise band for several runs. What you see then is reliable; changes from here on are real changes.
Runs
A run measures all active questions of a project on all channels. Every answer stays stored, with timestamp, model and source list. Presence across all runs is the reliable value; the latest run shows the direction. A new customer in an already measured market gets the history retroactively, because every answer stays stored.
Two things break a history, and the application says so:
- A reworded question is a new question. The evaluation recognises a question by its wording. The old history stays stored; the new one begins with the next run.
- A rebuild of the question set shifts the overall value when a large part of the counting questions is new. Such a rebuild should be dated and noted under "How we measure".
New and resolved
After every run the application compares the findings with the state of the previous run. New findings appear as "new", vanished ones as "resolved" and stay visible for one run. Resolved is determined by the measurement, not by the user: as soon as the number behind a finding moves, it goes to "Resolved". See Findings catalogue.
Updated: 2026-09-16