1. What decision does the system influence?
Name the action, affected person and maximum acceptable impact. “Answers questions” is not precise enough.
2. Which sources are authoritative?
Document retrieval scope, access rights, freshness and how citations are shown.
3. What does a representative evaluation set contain?
Include normal work, edge cases, adversarial inputs and examples where refusal is correct.
4. What happens when confidence is low?
Define a human-review path, timeout and safe default; never hide uncertainty behind fluent text.
5. Can a model action be reversed?
Separate suggestion from execution and keep an audit trail for consequential tool calls.
6. Where does data travel?
Record provider, region, retention, subprocessors and whether prompts or outputs are used for training.
7. What are the latency and cost budgets?
Measure per user journey, not only per API call, and set alerts before margin or UX collapses.
8. How are model changes detected?
Pin versions when possible and run regression evaluations before a routing or prompt change.
9. Who owns an incident?
Name the person who can disable the feature, inspect logs and communicate impact.
10. What evidence allows launch?
Require passing evaluation thresholds, security review, privacy decisions, rollback and monitoring.