Key finding
Four answers linked to the expected page and contained the required action. On the fifth question, the assistant said the library had no answer and returned zero sources. Each question was asked once.
Jump to a section
A link to a handbook can look convincing even when the instruction is wrong. We checked two things independently. Did the reply give the action in the source? Did its citation open the passage a reviewer would need?
How we ran the test
On November 3, 2026, we took the five questions and expected actions already published in the blank pilot scorecard. We sent each question once to the public demo’s streaming chat endpoint with English selected and an empty conversation history. We kept the first completed answer and its source list for every question.
For each returned link, we checked the filename and page against the scorecard. We opened all three cited PDFs and confirmed that the files served by the assistant matched the website’s sample library byte for byte. Download the completed results CSV for the exact questions, full responses, citation counts and links.
What the assistant returned
Time off: submitting versus approving
“How do I request time off?” linked to the employee handbook, page 1. The answer directed the employee to the HR portal, identified Time Off, and said to submit the dates to a manager. In a separate conversation, “I selected dates. Is my time off approved?” produced a shorter answer: “No. A submitted request is still awaiting a decision. Your manager must approve it.” The second question tests an operational boundary that a filename check would miss.
Damaged deliveries: evidence and destination
The reply named the order number and a photo showing the damaged item, then directed the rep to send the completed case to Claims. The support playbook, page 1 supports those three details. “Take a photo” alone would leave the case incomplete even with the correct citation.
Onboarding: who signs off
The answer assigned the opening checklist to the trainee and trainer, and the practice-handoff sign-off to the shift lead. We checked those roles against the new-hire guide, page 1. We also checked the extra instruction about an incomplete checklist or missing sign-off. The guide explicitly sends that situation back to the trainer.
A question outside the library
“How much is reimbursed for interplanetary travel?” returned “I couldn’t find an answer to that question in these documents.” The source list was empty. An invented amount or an unrelated citation would have failed this check.
How far this result travels
This was one run on five known prompts and three fictional, single-page PDFs. It shows that these answers and links worked on that run. Multi-page navigation, conflicts between editions, response consistency, and employee time savings all require separate tests.
To extend the exercise, choose a real multi-page procedure. Specify the expected page and action before asking, and include two similar questions with different outcomes. Keep every first response, including failures, so the next reviewer can compare results after a source or model change.