Less searching.
Better-supported answers.

01 / The issue

The answer exists. Finding it takes work.

In this fictional support team, 30 staff search 250 policy and product documents to answer recurring internal questions. Similar documents contain different versions, and experienced colleagues become the default search engine for the rest of the team.

A generic chatbot would not solve the underlying trust problem. Staff need an answer tied to current, approved material—and a clear indication when the material does not support an answer. The objective is assisted knowledge retrieval, not autonomous customer decisions.

02 / The solution

One connected way to work.

A curated source collection

Document owners approve content, mark versions, and retire superseded pages before they are searchable.

Permission-aware retrieval

The assistant searches only sources the current user may access. Access restrictions are enforced by the application, not by asking the model to keep secrets.

Answers with supporting passages

The interface places source links and relevant excerpts beside a suggested response so staff can inspect the basis of the answer.

An uncertainty and escalation path

When evidence is missing or conflicting, the assistant asks for clarification or directs the question to a responsible person. It cannot change accounts or execute business actions.

03 / The workflow

From the first step to the final record.

1. Ask in everyday language

   A staff member asks a product or policy question from the internal portal.

2. Find relevant approved material

   The system retrieves permitted, current passages and includes them as reference material for the answer.

3. Inspect the suggestion

   The staff member checks citations, reads source context, and edits any response before using it.

4. Escalate and improve the library

   An unanswered question becomes a content-review item. Document owners decide what to update rather than allowing unreviewed model text into the knowledge base.

04 / Implementation challenges

The details that shape the rollout.

In this fictional implementation, a limited trial comes before wider rollout. These are the obstacles and design responses illustrated by the scenario.

Conflicting and outdated documents

The fictional pilot introduces document ownership and version rules. Ambiguous policies are resolved by the business before they become answer sources.

Plausible but unsupported responses

The design evaluates whether each factual claim is supported by retrieved passages, not just whether the answer sounds helpful. Missing support leads to an explicit limitation.

Malicious instructions in source text

Retrieved documents are treated as data, not system instructions. The assistant has no action-taking tools, and test cases include attempts to override its task or reveal restricted content.

Evaluation bias

The example separates a practice set from held-out questions and includes out-of-scope prompts. Human reviewers use a defined rubric and inspect disagreements.

05 / Business benefits

What changes for the people doing the work.

Support staff

Less time browsing folders, with direct access to the source behind a suggested answer.

Knowledge owners

A visible list of unanswered questions and stale content to prioritize.

Managers

A way to inspect usefulness and unsupported claims separately, rather than relying on a single satisfaction score.

Customers indirectly

Staff may retrieve information more efficiently, but all external replies remain subject to staff review. Customer satisfaction is not measured here.

06 / Sample results & numbers

What improvement could look like.

Illustrative figures, not verified client results. The scenario and numbers are fictional; they are not a forecast or performance promise.

The comparison

All figures are invented. Timings compare matched batches of 100 routine internal questions under manual search and assisted search. Answer-quality rates compare an unconstrained chatbot prototype with the source-linked design on the same illustrative 200-question set, including 40 out-of-scope questions.

How the numbers add up

Research capacity = (6 − 2) minutes × 100 questions = 400 minutes, or six hours and 40 minutes. Supported-answer rate rises from 60% to 87%, a 27-percentage-point change. Unsupported responses decline by (36 − 10) ÷ 36 = 72.2%, rounded. Correct escalation rises from 18 to 34 of the 40 out-of-scope questions.

What the example does not proveThese are synthetic comparison figures, not a model benchmark or a production evaluation. Quality measures can overlap and must not be added together. Ten example responses still contain unsupported claims and six out-of-scope questions are still mishandled; human review remains necessary. A real test must assess access control, adversarial inputs, latency, cost, and performance on representative questions.

07 / Further improvements

A focused next phase.

Expand the evaluation set

Add uncommon wording, conflicting documents, and questions requiring multiple sources. Track unsupported claims and missed escalations separately.

Make freshness visible

Show document review dates and flag retired sources. Test that removing a document also removes it from subsequent retrieval.

Tune cost and response time

Measure retrieval and model latency separately before selecting caching or model-routing changes. Never cache restricted answers across users.

08 / Lessons learned

What this scenario teaches us.

These takeaways are illustrated by the fictional example, not claimed from a completed client engagement.

1. An AI feature starts with content quality

   Retrieval cannot make contradictory policies agree.

2. Citations need verification

   A source link alone does not prove that the linked passage supports the answer.

3. An honest limit is a useful response

   Escalating an unsupported question can be better than supplying a confident guess.

4. Keep permissions outside the model

   Application controls must govern access; prompts are not an authorization system.