Start with ordinary questions
Build a short set of questions that reflect real customer needs: what do you offer, where do you deliver, how much notice is required, and what happens if the customer leaves out a detail? Include a request that should succeed and one the business should decline or route to staff.
Write the expected behavior before running the test. Otherwise, it is tempting to accept an impressive answer without noticing that it missed a constraint. For a delivery request, a wrong postcode can matter more than polished wording.
Keep the comparison fair
Use the same questions and agent settings when comparing the current information with an improved version. Preserve the source snapshots and inspect the actual answers. If the baseline already answers a question correctly, count that success. Do not make the old website artificially worse to produce a larger improvement.
Anthropic’s evaluation guidance distinguishes tasks, repeated trials, grading and actual outcomes. Its advice is especially relevant here: model behavior can vary, so a single successful attempt is not a reliability guarantee. Simple automated checks also need human review.
Turn feedback into another test
An owner should be able to point to an answer and say which fact needs correcting. Store that feedback with the question and report. After the correction is approved, create a new information version and rerun the relevant cases. Keep the older results for comparison.
Welvia’s Juniper pilot uses controlled source snapshots and isolated inquiry preparation. That helps test the workflow, but it does not establish visibility across public search engines or all consumer assistants. Those require separate checks. Good quality reporting makes the remaining gaps visible instead of turning every green indicator into a claim about the whole internet.
Sources & further reading
Sources checked 2026-10-12. Recommendations and examples reflect Welvia’s approach; they do not imply a partnership or guaranteed result.