Cekura

The self-improvement loop for voice agents

SaaS

Developer Tools

Audio

Visit WebsiteSee on Product HuntTwitter ⧉

closing the loop without drift is the real hard part 👏 how do you keep it from overfitting to one bad call?

Congrats to the team! Testing a voice agent before launch is one thing. Having it get better from its own real calls is a different level entirely. Closing that loop is the part everyone talks about and almost nobody ships.

What has been the hardest aspect of evaluating conversational quality that isn't obvious from metrics like latency or task completion?

Closing the loop by fixing the agent and re-validating is the right shape, and the part I would want to know most about is what the loop is allowed to treat as ground truth, because for voice that is harder than the transcript makes it look.

We run an AI that answers and places real phone calls. We had a summariser that told users how a call went, and it fabricated confirmations on roughly a quarter of calls. The canonical one was "she picked up and the surprise was delivered successfully" on a call that lasted six seconds and was never answered. What did not fix it: lowering temperature, and putting "be honest" in the prompt. What did: forcing every claim to point at something literally present in the transcript, and naming the exact phrases it was forbidden to emit. That took us from about 24% to about 2% across roughly 1,200 calls.

The thing I would push on for a self-improvement loop specifically: the failures that hurt most in voice are often not content failures, they are timing failures, and a transcript-derived signal is blind to all of them. Latency before the first token, barge-in handling, a beat of silence at the top that reads as a dead line. We had calls whose transcripts were flawless and whose recipients hung up in the first few seconds, and nothing in the text said anything was wrong. If the improvement signal is derived mostly from transcripts, the loop will keep sharpening the part that already worked.

The overfitting claim is the other half of the same worry, and it is the harder one to prove. A regression sweep drawn from the same scenario generator that produced the fix is a closed system. Does the sweep draw from held-out production traffic, or from simulation?

How do you balance simulated conversations with real production data when improving evaluation quality? I'm curious how teams typically decide when one is more valuable than the other.

@satvik_dixit Congratulations. And happy product launch.

Voice AI quality is surprisingly difficult to measure consistently. I like that this goes beyond transcript evaluation and looks at things like interruptions, latency, and delivery quality. That's usually where real production issues show up

Love seeing more attention on production reliability for voice AI. The regression validation especially stood out. Curious whether Cekura can prioritize issues by business impact (for example, failed payments vs. minor conversation hiccups) or if everything is treated equally.

Congrats on the launch! Really like that you’re focusing on proving fixes instead of just identifying failures. One thing I’m curious about: how do you decide when a suggested fix is reliable enough to recommend versus flagging it for manual review?

Love that this closes the loop instead of dumping another list of flagged calls on a human. The rule that it has to reproduce a bug before it is allowed to fix it feels very sane. For a team running a voice agent in something sensitive like healthcare intake, can you control which kinds of fixes it applies on its own and which ones need sign-off first?

Voice agents fail in ways text evals never catch — interruptions, latency, someone talking over the bot. The "loop" framing is the right one: testing voice once at build time is basically useless. Would love to know how many simulated calls it takes before the improvements show up.

Thanks — the GitHub Actions path is the answer to that question. The follow-up I'd have is whether the infra regression suite runs against a live agent clone or replays recorded sessions, because customer-support agents with integration state tend to behave differently on replay versus a live environment.

Love seeing the focus on proving that a fix doesn't introduce regressions elsewhere. Reliable AI systems need repeatable validation, not just faster debugging.

Simulating messy real-world conversations with interruptions, pauses, and background noise feels much closer to production than traditional scripted evaluations.

Does Cekura provide regression testing capabilities to catch quality drops when agents are updated or retrained?

How customizable are the evaluation metrics? Can teams define their own quality benchmarks based on their specific use case?