⏭️ Forward Deployed: Voice AI on what works in 2026

Alright, we're in the remote studio. This is a special one because we're launching a fourth podcast on Latent Space. People don't necessarily keep track, but we cover different things. Basil came across my radar because you're doing all these dinners and gatherings and panels and podcasts on forward-deployed engineering.

You hosted the FDE track at AIE, and it did super well, so welcome to the pod.

Yeah, thanks.

So you've been running your podcast for a while. What made you decide to focus on FDE, and what's your typical self-intro?

Yeah, so I started as a product manager at Intuit. I was specifically working on Credit Karma for a couple years. I worked at a small venture studio after that, and then I ended up starting a consulting business called Exaflop Labs, where we were working with retailers, insurance companies, that sort of thing.

Just building agents, essentially. This was like 2 years ago. So it just felt like a natural extension of a lot of the product work that I was doing earlier. It was like, hey, I'm working with customers, they have a specific type of project that they want built for some use case, and I'm going to understand what they want, I'm going to make trade-offs, and I'm going to help them build that.

So yeah, it just felt like a natural extension of what I was doing. And so back in January, we started working with a couple private equity firms, and a lot of people were just like, hey, I'm following on what's going on on Twitter, what's going on on LinkedIn, I don't know what's marking BS and what's not.

So I was like, why not just bring on people who are working on the cool stuff at cool companies? We will talk about what they're working on, like we'll do a deep dive, and we'll record it, we'll post it, and hopefully people can take learnings away that they can apply to their businesses.

And that's why I ended up starting doing a lot of the fireside panels and the podcasts that I've been doing since then.

Can you give rattle off just your greatest hits, like what you've covered?

Yeah, so our very first episode was on the future of agentic engineering. So we brought on companies like Factory, Cognition, Composio, Soundgrab. We've done panels on voice agents, which is actually the one that we're going to be showing here.

We've done some on computer-use agents, we've done agents in the enterprise, we've done a ton of stuff just to get nitty-gritty into the details of a lot of this stuff.

Yeah, and I guess you picked the, for our sort of first feature, you picked the voice agents one. What are we about to listen to, and what stood out in particular?

Yeah, so we brought on some people from Decagon, Vapi, Retail, Daily, and a company called Smallest AI. Basically, we just talked about what is the state of the art in building voice agents, because that's a very hot use case.

I think Sierra even talked about this. This is like one of the most competitive markets in AIright now is building voice agents. And so I thought it would be useful to talk about how they're actually built and some of the things that engineers who are building in this space still have to contend with.

So for example, we talk about how the state of the artright now in building voice agents is a cascaded pipeline. So we talk about how it's like a three-step process. You go like speech to text, LLM, then text to speech.

And we talk about like why is that versus why don't we just have a voice-to-voice model? Like why don't we use that? Like the reality is those just are not super reliableright now. We also talk about like the trade-offs that you have to make when you're building voice agents.

So you have to trade off like the intelligence of responses that you're getting versus the latency. So you can get very intelligent responses, but you're also going to trade off it's going to be a slower response. And so for different use cases, that might be good or bad.

We also talk about like reliability. So there are some LLM companies that may not be like super reliable in terms of like their infrastructure. And so like what do you do when Opus goes down? Like you need to have a waterfall of models to basically pick it up so that your voice agents don't just stop working.

We also talk about like turn-taking and how like it seems like a trivial problem where, hey, like you're talking to a voice agent and let's say I pause. So how does the voice agent know that, hey, like I should interject and I should actually like respond to you versus, oh, the person is just taking time to think.

So turn-taking is actually like not a trivial problem to solve. And so we talk a little bit about that. And then also we even talk about like how I think PipeCat built a voice agent for the World's Fair,right?

So we even talked about that a little bit. And how like, I think, I mean, you can talk about this a little bit, but I think like you guys built a voice agent to like if anyone calls the voice agent, it'll answer questions about the conference.

And then you got a ton of like real-world information and turn that into a benchmark.

I mean, to be clear, I have no idea actually. I never looked at the analytics. I have no idea how many people actually called. Probably people are just kicking the tires. Like it's not that serious. Like it's a conference, people know what it is.

But I think it was a good deployment of the use case. Daily is a sponsor, so why not? I would say that the vast majority of voice stuff is support. And like, yeah, oh my God, there's so many call centers out there, hundreds of billions of dollars spent on this stuff, and they're all bad.

So hopefully we can sort of raise the state of the art, which I think that's why Decagon and all these things people exist. Like to me, it's not as interesting if a bunch of people who are obviously selling you voice pipelines telling you that voice pipelines are the state of the art.

Like, yeah, duh. But having the people actually focus on customer support and all those things also basically concludes like, yeah, like the models are not there yet, they may never be, and actually this is just the way that you got to do it.

And the people that really feel it are not the researchers, because researchers just always want bigger models to solve everything. And the product engineers will try to do it, but the FDEs, the people actually dealing with the customers, will be like, dude, they made this horrible mistake.

This can never happen again. How can you guarantee me that? Well.

Yeah, yeah, yeah. So like actually we even talked about that a little bit. So like we talked about like the inbound versus outbound use cases. Because we even did that once. We like ran an outbound use case. And like what you find is whenever somebody like picks up the phone and you do an outbound use case, a lot of the times they just hang up as soon as they realize it's a bot.

So we even talked about that. Like how do you solve for that?

Alright, that's a good teaser. I admire your work. I'm excited to feature on Latent Space. We'll be doing more in the future together, but this is just an intro to FDE for at least the Latent Space audience.

I think this might be a good place to give everyone a one-on-one lesson on what does a simple voice agent architecture look like. So maybe that's a good question for Varun, because you guys run PipeCat.

Yeah, so there are many ways to build one nowadays because the complexity is the nature of things. But the simplest one, which is called a cascade model, is typically voice input. It comes through a transfer. It could be WebRTC, phone call, or web sockets.

It goes into a speech-to-text. The transcription happens. There can be some additional modelsright there for background noise removal, voice isolation. So if there are a few people in the foreground, it will isolate to one person, remove the background noise.

There's a turn detection model. Unlike a text where you know when you're done, you press enter and the message goes and the LLM starts to execute. With voice, there is no, it's not walkie-talkie or push-to-talk. So there is no additional information apart from, like if you're looking at the face, you could actually figure out that the person stopped speaking.

But typically you use something like a voice activity detection and a smart turn model to figure out that the turn is complete. If I pause mid-sentence, that's different from like if I finish speaking and pause. You may use a speech-to-speech modelright there.

Then when you have the text from the transcription, then you send it to an LLM. The LLM may respond with an inference. It may actually tell you, it may infer to tool calls. It may do a bunch of things there.

You get that. When you have the full output, you text to, like you do text-to-speech, TTS, and it's usually faster than real-time. So then you have to figure out how to stream it back to the end user. It will go over the same path, transport that you spoke over, and then it'll play it back on your speakers.

But things are getting more complex as people, this was like 2024, 25, people wanted to do this. If you're doing an outbound use case, like you do an outbound call, debt collection is quite commonly a use case for this.

You get a call, they tell you you have like an unpaid bill. They ask you if you're paid or give you a certain set of options. For that, the bot has really, like if you go off the rails, like the human goes and starts saying random stuff, it can just drop the call.

It's not obligated to stay on the call in any way. So guardrails are easy. It knows what the inputs it can accept. It knows what inputs it doesn't need to respond to. It flips when you have an inbound use case because the bot has no context.

Like it has some context of why it exists, but doesn't know why you, the person who's calling it, is. If you're a mom-and-pop shop, like a flower shop or like a barista or something like that, okay, there's only like finite things that you can do there.

But if you are more complex, let's say you're Amazon,right? You're like a billion products on your platform and you have different policies for different things. Is it a physical item? Is it a cheap physical item? Is it an expensive item?

Is it an electronic item? Like all of that. So it needs to figure. And then when you said something like, you can't put all of that inside a single prompt because it will, like LLMs are goofy in that sense that always remember the first 4% and the last 4% and everything in between.

They kind of forget. So if you have a return policyright in the middle of that context, it's very likely it'll hallucinate. So you have to do more smart things and that's where you have like more models come in.

You can have a compaction model. Like we're all using coding agents now,right? So you know, like if you're using like a 1 million context model, like at 25%, you should become nervous at that 25% mark. It's nowhere near 70% mark.

But at 25% mark, you're like, okay, should I start compacting and like saving my work so that this thing does not go off the rails,right? The same thing, I think voice AI has been like one generation ahead of coding agents.

Like we've all in the last two years solved things that felt like so alien, but very tangible for us, which now when we see coding agents, we're like, man, compaction, we were doing compaction from day one. Like we knew like after five turns or ten turns, when your context was only 250 tokens or 250,000 tokens or 50,000 tokens, models were really small like before.

And at 10,000 tokens, they would like go off the rails. So a lot of our work is, I think, about like making sure that the bot does not hallucinate and trying to like keep track of the conversation as the turns progress.

And the turns are fairly short because either way, the bot is more likely to speak more than the human did. So you have to like kind of keep track of like what did the human say? Where are we in the conversation?

Yeah, so I guess this is a question for one of you three. So do you guys use this like cascading model for your guys' voice agents?

Yes, that is one of the major offerings that we have, cascading and speech-to-speech.

Yeah, I was going to ask, like that sounds too complicated. Why not just go voice to voice? Like why not? Like why do you have to have this like crazy three-step process?

I feel like the easiest way to answer this is you can call into a really nice voice-to-voice demo and you're like, wow, it's like listening to me laugh and it's like responding to my tone and it's so snappy, it's so fast.

But then I tell it that it asked me what day I want to schedule my appointment for and I say, next week. And it says, great, is that June 10th? And I'm like, no, it's the year is 2030.

And it's like, you'reright, it is 2030. So let's schedule this for June 10th, 2030. And I'm like, great, sounds good. And so from that perspective, I am actually very curious, especially on the smaller side, how that's evolved over time because we are very much keeping our eye on how these speech-to-speech models are performing.

But overall, I think our current stance is the cascading model just allows you to enforce so many more rigid guardrails and just tight control over the ability to say, hey, this input is going to go through the same supervisor models to detect for prompt injection or social engineering.

Then it's going to go down the conveyor belt into intent selection. Then we're going to optimize context by checking conditions ahead of time to say, in this complex process that we're going through with this user, maybe this prompt is applicable if they're this type of customer, but this prompt is applicable if they're that type of customer.

So let's figure all this stuff out ahead of time, compact and compile a good system prompt for our message generation model and get a response back. And then we can take that response and we can check a whole bunch of other things,right?

We can check, is this grounded in truth,right? Is it 2030 or is it 2020, whatever year it is, 2026? And then it goes back out over the line,right? And so the obvious constraint to that is how do you make that performant,right?

How do you parallelize as many of those steps in the conveyor belt as possible? I think the last six or so months has been a really amazing feat from our engineering team, at least, to find and shave off like 10 milliseconds at a time across every single part of this pipeline to make it feel snappy, even though there's a lot of things going on behind the scenes that you don't necessarily have to do with a speech-to-speech model.

So that's at least my take. But yeah, I am curious for the rest of the group's thoughts on this.

Do you want to go?

Yeah, no, I think you have brought a question around like cascaded versus speech-to-speech. Like let's talk about like why did people start thinking about speech-to-speech? And so the initial idea was simple that, hey, if you convert speech to text, it's going to lose the emotional information,right?

So if you say, hey, you might be saying it in a sad way or an excited way, the bot is going to answer in the same manner,right? So that was the obvious reason people started. I think at some point of time, at least at smallest, like that has evolved to speech-to-speech is a more natural way in which the human brain operates.

So for example, when you do the cascaded thing, you do speech-to-text, then you send the prompt to an LLM and then it responds,right? So we call that a synchronous architecture. Like it's happening one after the other. But our brain is thinking while listening.

So as I'm speaking to you, you're already forming your thoughts. And if I'm talking for too long, you'll interrupt me,right? Or you might be taking notes in the back end,right? So you might be essentially doing tool calls while I'm speaking to you.

And so the whole idea is that if you ever want to pass the Turing test of how the human brain operates, you need something that is working asynchronously and cannot just understand emotions, but also operate like take in speech natively and give out speech natively asynchronously.

And so that's why we've been sort of building Hydra. Now, Hydra is our speech-to-speech model. Now, in terms of accuracy and interpretability and all those things, I think whenever there is like a new architecture that comes out, it's often sort of good in one parameter and then regressed a little bit in other parameters,right?

Like, so for example, the speech-to-text accuracies for just a speech-to-text model might be way better than the encoder of a speech-to-speech model that you have. And so I think the challenge is that while you made progress on the

making it more natural and operate more human-like, et cetera, et cetera, how could you keep the accuracy bar the same? And so I think a lot of that comes down to interpretability of such models, like because you don't want it to be a black box.

So how can you understand where it is lacking? And so that's a lot of research that we do on how to make speech-to-speech models more interpretable. The other constraint is, I think initially the speech-to-speech models, like I think Sesme, et cetera, that came in, that they were just speech-to-speech.

Like they literally took in speech and gave out speech. Ours is like multi-modal. So it takes in speech and text, gives out speech and text both. So it can do tool call and take in text parallelly. So if you want to put guardrails, if you want to do all those things, that constraint does not go away.

What we are also seeing is there is still, at least in enterprises, a lot more cascaded deployments compared to speech-to-speech model. But speech-to-speech, I think, will be like an eventual future. That's my opinion.

So my take is that when you have like competing things, you end up in a hybrid. And I think the answer for the mid-term will be some form of hybrid because the speech-to-speech are improving. There are parts of the conversation loop where you would say like, oh, my use case of my workflow is complex.

I'm going to use multiple LLMs anyway. So for the active loop, like I'm talking to as a human, you called me, I'm answering your questions. But whenever I have to do some kind of lookup or something, I delegate to another LLM, which will then do the cascade stuff.

So the voice in the first part of the loop keeps running. And then you have like interesting kind of semaphores, or like you can think of these as threads. You want to interrupt the speech-to-speech model because you realize that this is a complex question.

And you want to, before the speech-to-speech response, you say, actually, you cannot answer this question,right? And delegate to the cascade and so on and so forth. So typically, I kind of say there are many use cases where, you know, if you wake me up in the middle of the night and ask me a bunch of questions, there are definitely some class of questions that I can answer without thinking.

And we all do our jobs in a certain way where we can be in autopilot like 50% of our time,right? So as these use cases become emergent and you're fully deployed with a customer doing high volume, you could essentially train specialized models that fully understand like that 50% use case very well.

And you could always ask the model like, hey, what is my return policy? And I'll say like, in the simplest case, this is my return policy. And it applies to like 70%. And I know which SKUs or products it belongs to or it applies to.

And if you ask me outside of that, then I have to like do all the crazy work. And if I don't, then I answer it,right? So.

So on these cascaded pipelines, what models are you guys using? Let's say of the frontier models of the GPTs, of the Opus and Sonnet, are you guys using the latest ones or are you using like GPT-4 because it's like theright balance between speed and intelligence?

If you turn off thinking, then you can actually use any of the like. If you want a fast model which responds to like the prompt and does not need to think, because if you want it to think, you actually start to think about parallelizing the work because you want the first model to be super fast.

I still like my Gemini 2.5 really well. Like it's so fast. It's like the 3.5 which they launch is like, okay, it's slower, but the 2.5 is so good. Even the Haiku is really, really good. They're still slower than what you would expect, but yeah.

How do you parallelize that workflow?

How do you parallelize that pipeline? Because it feels like, oh, well, I do have to first know what text the person said. And then the LLM needs the text to do anything. And then you can only generate speech once the text has been generated that you want it to speak,right?

You just spend the, you know, you can send the speech, like in the speech-to-speech model case, you would send the speech directly to one place, fork it into another place, run the STT there. And if you want multiple models, the text output from the first forks into multiple LLMs.

You can think of it as like a waterfall. Like, you know, it comes down.

But because you don't know what the output exactly will be.

Yeah, you better put a gate at the bottom between XR or whatever fancy thing you want to do, which says like the, if all of them say we areright, then you put another LLM downstream and say, which one is better?

Like you can go really crazy. If you love computer architecture, 20 years we had nothing to do. Now you can do all this crazy stuff from 20 years ago.

Yeah. So how do like accents in different languages all fit into this? Maybe Sudarshan, because you guys are building speech-to-speech.

Yeah, so at least for speech-to-speechright now, we are just fully focused on making it more intelligent in English and getting it really good in English. Like we don't want to do any other, we don't want to introduce any other variables because the technology in itself is, I would say, quite frontier.

Asynchronous is not yet like mainstream, et cetera,right? But in terms of like cascaded, yeah, we've seen like a lot of demand in, so we have a lot of presence in India. So we see like a lot of demand from India.

We see a lot of demand from Latin America, et cetera. I think US is more like mostly English, and then there's some Spanish, and then there's a lot of accents to it. And I think, yeah, US, at least we have not had any troubles in terms of

technology. I think it's, yeah, I mean, noise cancellation is probably the last mile of problem that is pending in terms of handling those things. But yeah, that has nothing to do with accents. I'm curious what you guys have seen.

Maybe I can add some examples about that because on multilingual, I like to definitely leverage cascade model because there are multiple levers you can pull. To talk about an example, it's a customer we would deploy for Japan. So you could typically go to your 4.1 and OpenAI or your DeepGram for transcriber, and that typically worked well.

Now, the challenge became on the voice piece. So it turns out what we thought it was a state-of-the-art for something like English, Spanish, Portuguese, those type of that are typical in the US, didn't work at all. So one of the benefits about this is that you can swap certain pieces that doesn't quite work for you.

At least on Vapi, how we solve it is that we let customers bring their custom text-to-speech server. So they actually have like, there's some startup, there's a lab in there, and there's typically, you will find that in every market, Arabic is also really hard to get itright in pronunciation of brands, addresses.

And that really we don't, cannot build expertise and optimize for every single use case. So that's one of the reasons why I like that as well.

Cool. Also, actually question on the like Sesame model. Yeah, like they came out with this like really insane like demo, and then I've never heard of them since. Like what, do you guys know like what happened and what's going on?

Yeah, I think, so they are building hardware is what I understand. Like they are putting those voice into some sort of a hardware. I'm not sure if it's glasses or what are they working on exactly, but I know a few people who got hired there and they're all sort of focused on the hardware and voice sort of backgrounds.

Yeah, the answer I got was that the CEO had already made a ton of money and he just really wants to play around basically. So I guess he can do whatever he wants. Cool.

So is it standard practice for in this like cascaded pipeline for you to just give, just have one giant like system prompt that you're giving the LLM on these are all the rules for how you should be responding?

Or is it more of a like a workflow approach? How do you guys think about that? What's the standard best practiceright now?

So I think it highly depends upon the use case and also the size of your prompt,right? So I think I have a counter approach to Varun in terms of that. Like usually in inbound, I find, you know, you're usually having dedicated lines and you know specifically where the workflows are going to go,right?

And so in these cases, if you know a little bit more about and you can predict where the conversation is going to go, then you can have a more of a node or graph builder kind of approach. But if you don't know what's going to happen, typically I think in outbound, yes, there is a dedicated like message, but what the caller says back, you never know because they could just be frustrated, they could be angry, or they could be, I don't know, like annoyed.

And in that case, when they're erratic, when a person's erratic, you never know. So having it in a way that is like all in one prompt allows you to have it more like a central brain in terms of I can pick up components from my prompt that may not have been in a dedicated flow, but actually can be utilized to enhance the response.

And then on top of that, you have to think of your knowledge base and knowledge, the things at your expense,right? Like in an inbound flow, maybe you know exactly when you need to pull a specific part of your knowledge.

So you don't need to extract every single piece of context,right? Whether it's your websites and documents. But sometimes in outbound, you should utilize all that information to get a better response. And you will rely more on cosine similarity and stuff to make sure that hopefully your retrieval is good enough to respond to them.

I think from my perspective, the limiting factor has been and will continue to be, although it's getting a lot better over time, just to, you know, some of the earlier points made, the ability for these frontier LLMs to be able to take a massive wall of text and not just read the first three lines and the last three lines, but actually be able to properly reason and like localize themselves into, you know, we are halfway through this procedure and these things have already happened and these things haven't happened yet and these things are true and these things are false about the situation.

And so therefore, like this is the one thing that I should be really zooming in onright now,right? So from our perspective, I think we kind of take an approach that depends on the customer, but we do within our execution engine have the ability to make some of these rules or prompts or guidelines more situational so that we can, you know, check if these are relevantright now before we even include them or don't include them in the system prompt.

And so that was our solution that was especially relevant, you know, like seven months ago when the LLMs really were just like consistently skipping step four A1. But we have found over time now that sometimes the reliability of deciding when these things are relevant or not is actually worse than just giving all of it to the model because the models have improved a lot since that time.

And so now we're going back, I think, swinging a little bit back towards just give the model everything and more or less it will figure it out. But at the end of the day, you know, with non-deterministic systems, it really just ends up being customer by customer.

We have to really understand their use cases and at the end of the day, just build it and then test it a whole bunch of times and then use LLM as a judge to tell us, you know, did it work?

Was it reliable? Or do we need to add a little bit more pre-processing and context optimization ahead of time in order to make it reliable? So the real answer is most likely always just test and find out and then test again and then find out again and, you know, it goes on forever.

I'm assuming that, yeah, maybe the models are able to handle those edge cases now, like just in a giant prompt, but that probably also increases latency. So first, could you talk about like what does a good latency look like for a voice agent?

And is that even true that, yeah, the models are getting better, but they also require more time? And so for the end user, that's actually a bad thing.

Yeah, that is a really good question. I mean, I think the definition of what like benchmarking latency stats look like that are good have changed a lot and expectations are, you know, getting better and better and better from our customers.

Overall, I think it is also important to mention that as much as fillers and contextual fillers is something that we don't like to hear just as a noun, it is normal for in normal conversation for humans to say a few words while they're thinking.

And so we've actually found that when we have zero contextual fillers, sometimes it actually feels more rigid than when we don't. And so we tend to find that having theright amount of fillers that kick in when, you know, we are experiencing some sort of latency because of some very large prompt or some very complex thing that the model is thinking through, most commonly, you know, running a tool call where our customer's API becomes the bottleneck and we're waiting, you know, five seconds for something to come back.

Being able to say, hey, just give me a sec, looking at that, okay, cool, here's your answer,right? There's actually nothing wrong with that when we've tested with our customers. Obviously, the limiting factor in that case then becomes how good are your fillers and how natural do they sound?

And so that's been a place where we've invested a lot of time as well to say, how do we create an execution engine where we really don't need fillers, but in those moments where we do, they're really good?

So that's at least my take. But again, I know folks have probably a lot of opinions here about latency and how to find that trade-off. Because in my opinion, I honestly think that that's the hardest problem to solve in voice deployments is just that trade-off that's always going to be there of like performance versus stability and reliability in general.

So.

Yeah, I agree with all of that. Some of the ways that we think about it in Vapi is offloading some of that instruction. So break it down. Sometimes you don't really need that, you know, 10, 15-step workflow in the prompt.

One example I can talk about is this collections use case. We do allow users to capture, businesses to capture credit card information, but not every call is about that. Some people already have a payment method maybe added or that.

So one way of doing that, offloading into a specialized agent, so we can start thinking about multi-agents and there are many architecture and patterns in there, is to constrain the instructions that you give it to that particular moment of the conversation.

And once that achieves its goal, it can go back to that more freeform approach which has loaded like your longer context. Something we're also been experimenting lately and it's more about the guardrails space is to your point where you can have, think about it as a waterfall, you can actually stream to multiple places.

So what if you have streaming down to a small language model which can do inference in a very little time? So think about classifiers to maybe collect intent while the pipeline is still maybe adding a filler word as well to keep a consistent experience, but kicking off a background process that is still intelligent, still thinking.

So those are still things that we'd like to invest a little bit more research on. So yeah, constantly investing on that.

I think one thing that we haven't also factored in in terms of the latency trade-off is cost,right? Now that you're putting such a big large prompt into your LLM, you have to think about those calls where many people just hang up and predominantly calls just hang up 10 seconds into the call,right?

Now you're having to pay for all those tokens to just get inputted in. So that like Steven mentioned, splitting out your prompt, you are also playing to that strength of lower latency and lower cost.

I'll speak just to the benchmark. So we publish turn-based STT benchmarks, LLM benchmarks, and TTS benchmarks. The whole point of that is that as I think today Nimotron 3.5 launched and there's already an ASR benchmark out, did very well the NVIDIA one.

So the idea there is that all our tooling is open source. So with the STT benchmarks, the TTS benchmarks, and the LLM benchmarks, all the turns are actually open. So you can run it against any new model that comes out.

You can run it locally. You can look at the outputs from our graphs and compare that to what you're getting on your infrastructure if you're self-hosting or, you know, elsewhere. So that's one important aspect to like consider. If you know what your turns will look like, you can just update or, you know, just download our benchmarks, update the turns and see how that like goes for your own flows.

Usually for the LLM stuff, it's really useful because you can throw the STT and the TTS and just run the LLM inference loops to like check against a customer. So yeah, I think evals are really important. And if you have like a, like while talking to customers, if you can evaluate like what type of conversations or like what type of workflow they're going to have, like those are real direct inputs into like what you can directly start testing, even though you're not live with them or even they've not shown any intent.

So I think that's where like the SEs and the forward deployed engineers can actually come to forward like you. We all have Claude running in the background,right? So you can just say, here's what we had a discussion. Like, can you build an eval?

Like based on our eval suite, like can you like just run a smoke test with the conversation information that we already have? And that helps a lot, I think.

Yeah, just adding to that, like we are seeing, so we, while we focus on models, we also have all the models sort of orchestrated on our platform. So you could build a voice agent on Smallest. We are seeing a lot of customers who just pair our models with our own like small language model.

It's called Electron. We fine-tune one of those to make it work for real-time voice use cases and see a lot of folks using that with better knowledge bases, memory, et cetera, over GPT-4.1, you know, the popular real-time models.

Just A, from a cost perspective, it's lower. B, from latency perspective, it's like way lower. And then just more reliable. I think I'm not sure if everyone faces this, but like OpenAI APIs spike and then you have no control on those latencies.

With, if you sort of have a self-hosted model, you could scale up, scale down based on your requirement. So that's one thing we are seeing a lot with our customers.