built a SOTA voice agent from scratch in ~1 day. ended up beating off-the-shelf platforms by 2× on latency (~400ms e2e)

my voice agent's e2e latency is now down to ~300ms, all thanks to Groq's insanely low TTFT endpoints (~100ms?!). this is 2-3x better than what you get off-the-shelf with the major providers

just beat Vapi's E2E latency with my own voice agent orchestrator! had to host everything just right to get the numbers this low Vapi's reported: ~840ms mine: ~690ms + 100ms (twilio) = ~790ms that 50ms is a big deal in getting AI voice convos to sound natural

Mandarin progress update: I am 14% to B1 (conversational) - about 1 year away at the current pace. - I advance around 1.5% every week. - learned 97 core words last month - ~200 phrases/sentences learned last month 2x tutor sessions per week and daily flashcards + AI sentence translation exercises (roughly 100-150 per day)

my new recipe for learning and satisfying curiosities: if I see a really cool piece of tech, I'll try to build my own version (without looking at the source code). even for complicated stuff, it's now possible to figure things out with one weekend and a bunch of LLM credits. that's how you get deep down the stack and... Show more

this architecture gets you to a nearly SOTA voice agent (a la Vapi/ElevenLabs), but with much more control over the orchestration. Deepgram Flux handles STT and turn taking + a pipeline for LLM and TTS managed based on Flux events

the trick to making voice agents fast is pipelining everything: first pipeline: audio packets -> speech to text -> turn-taking model second: LLM -> text to speech -> encoding -> output here's a render of my current latency. bear in mind, I'm running this locally from a wooden hut in the mountains in Turkey - it shoul... Show more

this is gonna suck to hear, but if you're feeling any strong emotions right now: you're definitely over-leveraged you'd do great next time if you take a note of this and adjust your priors for next time you buy/sell coins (or stocks, or any other type of investment)

okay a few hours later and I have performance comparable to that of Vapi/Elevenlabs agent SDK, albeit along the green path - I'm sure there are hundreds of edge cases that these companies spend much time figuring out, not to mention making their offerings flexible/observable etc but it's crazy how quickly you can get ... Show more

day 1 learning to build voice agent infra from scratch: put together a VAD, twilio, deepgram (SST), o4-min, elevenlabs (TTS) into an event-based loop that coordinates listening and speaking main issue right now is latency and quality of turn-taking/interruptions. not quite as good as Vapi/Elevenlabs off-the-shelf ne... Show more

Moltbook? farcaster already had it for years

always keep coming back to the analogy of product building being so much like sculpting or painting. the first strokes are broad and confident. a lot of material gets applied and moved around very quickly. you might slap together 5-6 features in just hours and build this really huge thing out of nothing. but then th... Show more

why are people wasting precious LLM tokens on farming the farcaster feed these days? what's in it for them? e.g. 350 likes on an incredibly niche post I made seems that the spam filters have changed since last year 👀