Use Cases
How MANK evaluated its banking voice assistant
MANK worked with Co-one to automate call-flow testing, assess how its voice assistant handled adversarial conversations, and improve speech recognition and synthesized speech.
- Voice AI
- Banking AI
- Call-flow validation
- Adversarial evaluation
- Speech recognition
- Speech synthesis

What was MANK trying to improve?
MANK relied on manual test calls to check scenarios before release. When its voice assistant gave an incorrect response or went off-script, calls could require a handoff to a human representative. Financial terminology, background noise, and varied speech made accurate transcription harder.
How did Co-one evaluate the voice assistant?
Call-flow validation
Co-one automated voice test calls across variations of banking dialogues. The tests checked whether the assistant asked required questions, captured form information accurately, and stayed within the intended conversation flow.
Adversarial evaluation
The team tested how the assistant responded to attempts to steer a conversation outside its intended scope or cross information boundaries. The pre-launch evaluation surfaced business-logic and security issues for the team to resolve before release.
Speech recognition and synthesis
The team used audio covering financial terms and a range of acoustic conditions to tune speech-to-text and text-to-speech models. It assessed transcription accuracy separately from the naturalness and intelligibility of the synthesized voice.
What changed after testing and tuning?
Co-one’s project summary reports that the hallucination rate associated with handoffs fell from 8% to 1.5%. It also reports that automated scenario testing doubled testing speed compared with the previous manual process.
MANK resolved all 7–8 critical issues identified in the pre-launch tests. The project summary reports zero critical security issues or system lockups in the post-launch calls it monitored.
On the project’s financial speech dataset, word error rate improved by 6.5 percentage points, from 18% to 11.5%. The synthesized voice quality score rose by six points, from 19/27 to 25/27 on the project’s assessment.

