How to collect a matched AI visibility baseline
When practical, begin recording a declared prompt-and-engine panel before a planned change. Preserve the prompt wording, engine or surface configuration, run states, and shipment time, then compare the baseline with a later declared window. There is no universal number of days, runs, or citations that makes the comparison valid. If either window is thin, publish its raw counts, denominator, and limits rather than turning it into a causal or confident decision claim.
Why collect more than one pre-change observation?
Answer-system outputs can vary across repeated runs. One pre-change observation records one dated result; it does not describe how often that result appears in the declared panel.
An illustrative protocol might run the same prompt-and-engine panel on three declared dates before a change. Another might use a two-week window with a fixed cadence. Those are examples, not validated universal minima, and scheduled attempts that were unavailable or failed must remain visible.
The objective is to document the observed variation and denominator before the split point. Choose the window and cadence for the decision, cost, and provider limits, then state them with the result.
Why are baselines sometimes reconstructed too late?
If measurement begins only after a content brief is approved, schedule pressure can leave little or no pre-change window. A team may then reach for an older screenshot, a tracker started after publication, or a recollection of what appeared before.
Those records can provide historical context, but they are not a matched baseline unless they used the same prompt wording, engine configuration, outcome rules, and documented run states.
Start the declared panel when the prompt becomes relevant if that is practical. If it is not, label the missing or thin baseline rather than reconstructing a stronger comparison than the records support.
What can a reconstructed before support?
A reconstructed record may use different prompt wording, engines, dates, or outcome rules from the later window. Unless those dimensions match and were recorded before the change, it cannot support a like-for-like movement estimate.
Without matched observations from before the change, a later number describes the later state but not movement across the change. It cannot show that a rewrite worked or that an engine was slow.
A matched timeline can show whether an observed difference occurred after the split point, but it still does not identify a cause. Rival changes, prompt variation, model changes, retrieval variation, and other events remain alternative explanations.
When should measurement actually start?
When practical, start when a team decides a prompt is worth tracking and before the planned change goes live. Planning and review time can then contain pre-change observations without delaying the work solely to satisfy an arbitrary window.
Register the prompt wording, engine or surface, locale when controllable, cadence, outcome rules, and intended split point before collecting. Content work can proceed in parallel. Only completed observations enter a citation-rate denominator, while failed, skipped, and unavailable attempts remain in the collection record.
In RankEcho, a tracked prompt can be scheduled on a daily cadence. Record successful, failed, skipped, and unavailable attempts separately; a schedule is not itself an observation.
What makes a before-and-after comparison interpretable?
At minimum, preserve four parts of the design and disclose any mismatch.
Use the same prompt wording. A paraphrase may be useful as a separate prompt, but it is not a like-for-like continuation of the original series.
Use the same named engines or surfaces and record the version or configuration when available. A before measured on two engines should not be pooled with an after measured on four as though every cell were matched.
Record the split point as the time the change became publicly available, not when it was drafted or approved.
Keep outcome states apart: cited, observed without a citation, failed, skipped, and unavailable. Collapsing an unavailable attempt into an observed non-citation changes the denominator and can bias a period comparison.
How should baseline length and sample size be reported?
There is no universal minimum. Use repeated observations when feasible, choose a declared window and cadence for the decision and provider constraints, and report every scheduled attempt by outcome state. A two-to-four-week window can be an illustrative protocol; it is not a validity threshold or a promise that the baseline is sufficient.
For illustration, one citation in three completed observations and eleven citations in thirty-three are both 33 percent. The different denominators show different evidence depth, but neither count alone supplies a universal decision threshold, representative sample, or statistical-confidence claim.
If a window is thin, show the raw numerator and denominator and label the limitation. Do not convert a small count into a confident direction or causal conclusion.
What this does not fix
It does not establish causation. A prompt measured before and after a rewrite shows what changed alongside the rewrite. Engines change independently, rivals publish, and retrieval varies; an observational before-and-after cannot separate those explanations by itself. The accurate phrasing is observed, not caused.
It does not prescribe when movement should appear. Declare one or more later observation windows that fit the use case, and report what was observed without implying a universal fourteen-, twenty-eight-, or ninety-day response schedule.
If a change must ship before enough pre-change observations exist, preserve the observations that do exist and label the limitation. The stable panel can still provide a reference for a later planned change.
Frequently asked questions
There is no universal duration or run count. Declare a window and cadence that fit the decision and provider constraints, keep unavailable and failed attempts visible, and report the numerator and denominator. Two to four weeks is one possible protocol, not a validity threshold.
You cannot recreate matched pre-change observations after the change. Older records may provide context if their prompt, engine, and outcome definitions match. Otherwise, start a stable panel now and use it as the reference for a later planned change.
One observation is a valid dated observation, but it does not estimate how often the outcome appears across repeated runs. If it is the only record available, publish it as one observation with that limitation rather than as a stable rate.
Changes to prompt wording, engine or surface configuration, outcome definitions, or the split point can make cells unmatched. Collapsing an unavailable or failed attempt into a non-citation also changes the denominator and can distort the comparison.
Cost and usage depend on the product plan, provider requests, cadence, and successful run count. RankEcho can schedule a tracked prompt daily, but teams should confirm current limits and keep failed, skipped, and unavailable attempts distinct from completed observations.