Agent Ops
An operator checks the agent’s promises before approving a payout.
Human review
The message and the payout, side by side
Review promises
The agent closed 61% of contacts alone. Then it promised fourteen refunds under an expired promotion: $23,000. The team of 24 would not grow. Reviewing every conversation was impossible.
- 1,770 agent-closed chats per day
- 71 commitments per day · estimate
- 2.4 h daily review time · estimate
Design estimates: 177 hours of daily chat review, requiring 25 reviewers, versus 2.4 hours to review commitments. These are not measured savings.
-
Seventeen chats. One cause.
I put repeating causes above individual chats and ranked them by financial exposure. One expired article becomes one correction at the source, rather than seventeen separate verdicts.
-
The promise and the proof
A hard case once crossed five tools. I brought the conversation, trace and verdict together; the transcript flags the unsupported claim.
-
The promise lacks proof
The conversation stays intact. An unsupported refund claim is flagged at the exact message, so the reviewer sees what the customer was told.
-
Money waits for a person
Before a refund leaves, the reviewer sees the amount, the policy and the customer message. Rejecting remains available even when evidence is incomplete.
Human review
A missing record is part of the evidence
An empty trace can mean an invented answer or missing vendor data. I made the gap explicit, so the reviewer can distinguish a policy match from the article the agent never consulted.
Tested, reworked and accepted
The client is under NDA; the screens use invented data. User testing informed the revisions. The client accepted the prototype: nineteen screens across three roles. Implementation remained with the client.
- 19 screens · accepted prototype
- 3 roles · one oversight system
Let's talk
If the case above answered your question — or raised one.