Auriko

Trading desk for LLM calls

API

Developer Tools

Artificial Intelligence

Visit WebsiteSee on Product HuntTwitter ⧉LinkedIn ⧉

Optimizing for the expected cost of the full session instead of the cheapest individual request is the interesting part here. Does the routing model also account for context continuity beyond cache economics—for example, provider-specific differences that could cause subtle behavioral drift during a long agent run?

llm calls as a trading desk is such a clever framing 📈 30% savings is a real hook, congrats on #1

@zxy_action1 Congratulations and happy product hunt.

The trading desk analogy is compelling but trading desks also have slippage, the cost of execution diverging from the expected price. What's the equivalent for LLM routing, like how often does Auriko route to a provider that then has a latency spike or quality degradation that negates the cost saving, and is there a real-time feedback loop that reroutes mid-session or only adjusts for future requests?

The quant-desk framing lands, and Michael's point that spend hides in cache pricing and routing more than headline token price matches what bit me. Where I'd push: Ridhwik already asked the quality-floor question, and "optimized without compromising quality" can't be the real answer. I run an LLM backend where the output is the product, and two models at the same token price diverge hardest on the one axis a price benchmark never sees — by the time a complaint tells me, the bad output already shipped. So the mechanism I want to understand: can I define what "request quality" means per route — my own golden set or judge — or is it one internal score you calibrate? Whose definition the router optimizes against is the whole risk surface for anything customer-facing.

Love the "trading desk for inference" framing—routing on cache behavior and real-time provider signals instead of just headline prices is exactly the kind of optimization most teams skip, and the zero-markup model makes it a no-brainer to try. Congrats on the launch! 🚀

Nice launch! LLM cost optimization is exactly where a lot of teams need help right now.

This is amazing. Love the concept. Thinking of giving this a go but without signing I can't find quantisation of the models. Also a question for you: whats your process if a provider you have on there suddenly swaps to a different quantisation? Can they do it without notice and do you have fail safes for that? I got burnt a little on OpenRouter where a provider I was using swapped to a lower quantisation and I didn't know about it until things started failing. Now I just pin it 3 levels deep to different providers as a fail safe for me.

Smart angle. LLM costs are getting complex fast when you're routing between multiple providers and models. How do you handle latency tradeoffs when optimizing for cost? Sometimes the cheapest call isn't fast enough for real-time use cases.

As someone routing agent traffic across providers, the cost-arbitrage-as-trading-desk framing lands — but the failure mode I'd test first is a venue going bad mid-run. When the cheapest provider starts erroring or its latency spikes, does Auriko fail over inside the same request (retry to the next-best path transparently), or does the caller eat the error and only re-route on the next call? And does the 30% cost-reduction number account for retry spend, since a cheap-but-flaky path can net out more expensive once you add the retries?

A 30% inference cost reduction that requires zero change to how our teams build is a rare operational win, and treating providers as trading venues is a genuinely clever framing.

This is so good! We are constantly experimenting with different model providers and from testing this out so far, it's worked great, especially compared to other model routers.

the cost angle makes sense but I'd worry about behavioral drift - even at the same nominal price point, different providers running "the same model" can have different quantization, latency profiles, or subtle output differences. if you're routing a request to whichever venue is cheapest at that moment, how do you keep output consistency for something like a customer facing agent where behavior needs to stay predictable

How does Auriko handle providers with different caching rules? Some make caching easy to reason about, while others expose less detail. Does Auriko normalize all of that for developers?

Love that you guys came from the quant trading world and applied real arbitrage logic to LLM routing instead of just defaulting to whatever provider has the shiniest SDK. The benchmarking transparency page is a nice touch too.

Congrats! A trading desk for LLM calls is a framing I haven’t seen before and it clicks immediately, model costs do behave like a market. My question: when Auriko routes a call to a cheaper model to save money, how do I protect quality? Can I set a floor per task type? Saving 40% on inference means nothing if my customer-facing outputs get worse and I find out from a complaint.

The trader instinct behind this makes complete sense to me. Treating those choices like a live market feels like the sort of thing only people who have lived it would ever think to build.

A trading-desk framing for LLM calls makes sense. Once teams have more than one model and more than one workload, the real work becomes routing, cost control, and knowing why a call behaved the way it did. The audit trail matters as much as the cheaper token path.

treating LLM providers as trading venues is a genuinely smart framing from people who understand arbitrage. token price differences between providers are real and most teams just pick one model and stick with it out of inertia. the cache behavior optimization is the part i'd want to dig into more, prompt caching can drop costs dramatically on repetitive agent workloads but only if you're structuring requests to actually hit the cache. does auriko handle that automatically or does it require some setup on how you're sending requests?