Use Cases

How MANK evaluated its banking voice assistant

MANK worked with Co-one to automate call-flow testing, assess how its voice assistant handled adversarial conversations, and improve speech recognition and synthesized speech.

  • Voice AI
  • Banking AI
  • Call-flow validation
  • Adversarial evaluation
  • Speech recognition
  • Speech synthesis
MANK and Co-one case study cover on evaluating a banking voice assistant.

What was MANK trying to improve?

MANK relied on manual test calls to check scenarios before release. When its voice assistant gave an incorrect response or went off-script, calls could require a handoff to a human representative. Financial terminology, background noise, and varied speech made accurate transcription harder.

How did Co-one evaluate the voice assistant?

Call-flow validation

Co-one automated voice test calls across variations of banking dialogues. The tests checked whether the assistant asked required questions, captured form information accurately, and stayed within the intended conversation flow.

Adversarial evaluation

The team tested how the assistant responded to attempts to steer a conversation outside its intended scope or cross information boundaries. The pre-launch evaluation surfaced business-logic and security issues for the team to resolve before release.

Speech recognition and synthesis

The team used audio covering financial terms and a range of acoustic conditions to tune speech-to-text and text-to-speech models. It assessed transcription accuracy separately from the naturalness and intelligibility of the synthesized voice.

What changed after testing and tuning?

Co-one’s project summary reports that the hallucination rate associated with handoffs fell from 8% to 1.5%. It also reports that automated scenario testing doubled testing speed compared with the previous manual process.

MANK resolved all 7–8 critical issues identified in the pre-launch tests. The project summary reports zero critical security issues or system lockups in the post-launch calls it monitored.

On the project’s financial speech dataset, word error rate improved by 6.5 percentage points, from 18% to 11.5%. The synthesized voice quality score rose by six points, from 19/27 to 25/27 on the project’s assessment.

More in use cases

Co-one and Akbank case study cover on evaluating a banking AI assistant.

Use Cases ·

How Co-one helped Akbank improve its AI assistant

Akbank worked with Co-one to improve how its banking AI assistant recognizes customer intent and responds. The project combined knowledge base management, automated evaluation, and human review.

Talk to Co-one

Bring a workflow. We’ll start there.

Show us where your agent makes decisions or takes action. We’ll discuss what to simulate, what to validate, and where runtime controls belong.

Book a demo

Choose whether to allow optional analytics and marketing tags.