Your Dictation Feature Doesn't Need a WebSocket

0:00This audio is presented by Hacker Nune, where anyone can learn anything about any technology. Your dictation feature doesn't need a web socket, by assembly AI. Open any tutorial on adding voice input to an app, and you'll end up in the same place. A web socket, a session lifecycle, a partial transcript handler, and some end of turn logic you now have to tune. Then look at what you're actually transcribing. A two-second clip, the user held a button, said, remind me to email Priya about the Q3 numbers, and let go. Tadio finished recording before your first packet went out. There's nothing to stream. You have a complete file, and you want a complete transcript. That's a request and a response. The oldest shape in web development, and reaching for a streaming API here means paying for connection management you never wanted. So let's build the request, response version instead, and then look at where the milliseconds go, because that part turns out to be more interesting than the code. Do you need a web socket for short audio clips? No. If the audio is already finished recording,

1:01a single HTTP request is the right shape, and streaming buys you nothing. Here's the honest version of the three-way choice. Streaming is for audio that doesn't exist yet. Live captions, a voice agent listening for a caller to finish a thought, a meeting note-taker running for 40 minutes. You need partial results as words arrive, so you hold a connection open, and the API pushes text back at you. That's a real problem worth a real web socket. A sync batch is for audio that's long and finished. Upload a podcast, get a job ID, pull or wait for a web hook, retrieve the transcript. The latency is measured in seconds to minutes and nobody mines, because nobody is standing there waiting. Sync is the one people forget exists. The audio is finished, it's short, and somebody is standing there waiting. Push to talk dictation, a voicemail, an IVR menu response, a voice agent turn where you're doing your own turn detection and just want the utterance transcribed. Push to talk lands squarely in the third bucket, and most implementations use the first one.

2:04The tell is when you find yourself writing code to detect the end of a turn. For audio where the user already told you the turn was over by lifting their finger. The sync speech detects the API exists for that third case. You post a clip, you get a transcript back in the same response, and the median round trip is about 134 milliseconds. How do you transcribe a short clip in a single HTTP request? Send the audio to and read the transcript out of the JSON response. There's Nijab ID, no polling loop, and no session to open our close. The SDK's rapid and one method. If you'd rather not add a dependency to a mobile back end or an edge function, it's a multi-part form post. The header is required on every request. The API key goes in with no prefix. What comes back? Two fields worth knowing about. Is the server side processing time, which I ask useful when you're trying to work out whether a slow request was the model or network, and you want that distinction in a minute? And is the first thing support asks for when

3:04something goes wrong, so log it on every request rather than only on failures? Per word timestamps are opt in. Said in you get an in milliseconds on each word, at a small latency cost. Timings are exact or absent, never estimated. A word the model can't align comes back without them. Leave the flag off in words carry and only. For a dictation box that's usually what you want, you're inserting text at a cursor, not building a karaoke player. A few constraints to design around. Audio runs from 80 milliseconds to 120 seconds, files cap at 40 megabytes, and the API takes wave or raw PCMS16LE. 16 bit only. Wave carries its sample rate and channel count in the header. Raw PCM doesn't, so sending PCM means passing and in the config, and both are required. There's no URL ingestion, so you're sending bytes, not a link. And there's a 30 second deadline on the request itself, which is worth knowing if you're pushing clips near the two-minute ceiling. The full list is in the audio requirements.

4:08Error handling deserves more than a try block that swallows everything. The API returns machine readable codes, and a few more, and the 429 and 503 responses carry a value you should actually respect. That's the whole integration. Under 50 meaningful lines, including error handling, which is roughly the point. Where do the milliseconds actually go? Here's the part nobody writes about, and it's the thing that will actually determine whether your dictation feature feels instant. On a streaming API, you open the connection once at the start of a session and advertise the setup across everything that follows. On a single request API, there is no session to amortize against. Every cold request pays for connection setup before a single byte of audio moves. Three things have to happen first. One, DNS resolution, often cached, but a cold lookup is a round trip of its own. Two, TCP handshake, sion, sinac, ag. One round trip, three, TLS handshake,

5:08client hello, certificate, key exchange, both sides derive session keys. One more round trip on TLS one. Three, two on TLS one. Two, none of that work depends on your audio. It's pure plumbing, and it sits in front of every cold request. For a client near the serving region, it's a few minutes of milliseconds. For a client on another continent, it can be well over 100, which, against approximately 134 milliseconds transcription, means you could spend more time shaking hands than transcribing. The fix is a endpoint, and the trick is when you call it, is an inauthenticated no-op that returns. The response body is irrelevant. The entire value of the call is on the wire. Making any request forces your HTTP client to resolve DNS, open the socket, and complete the TLS negotiation, and THE resulting connection lands in your client's pool. The request that follows reuses it and starts uploading audio immediately. The ideal moment to call it is when you know audio

6:09is coming, but don't have it yet, which in a push to talk, Ui is the exact instant the user presses the button. The handshake then runs concurrently with the recording. By the time they let go, the connection is open and warm, and you've moved the entire setup, cost off the critical path, and into time the user was going to spend talking anyway. Two gotchas will quietly undo this. Pooled connections expire, HTTPX, for example, drops idle connections after five seconds by default, and the server closes its side after a few minutes of idle. Warm too early and you pay the handshake anyway, silently, with nothing in your logs to say so. And the warm-in-the-transcribe have to share a connection pool. The same client object, the same base URL. The sync API serves a global endpoint plus regional US and EU endpoints for data residency, and a connection warmed against one of them does nothing for requests and to another. This is also why earns its place in the response. If your end-to-end latency is 400 milliseconds and says 130, you don't have a model problem.

7:13You have a handshake problem, and now you know which one to go fix. There's more on the mechanics in the connection pre-warming docs. How do you improve accuracy on names, ids, and jargon? Short clips are the hardest thing to transcribe well, which is the uncomfortable irony of dictation. A long recording gives the model context to work with. Two seconds of, send it to Shavan at Anthropic, gives it almost nothing. The sync API runs on universal 3, 5 Pro, and it takes three kinds of context, all optional, all in the same config part, none of them costing extra. Key terms, bias the model towards specific tokens, your user's contact list, your product catalog, the acronyms your industry uses and nobody else does. The list caps at 2048 characters across all terms. Contextual prompting describes the audio rather than instructing the model in up to 4,096 characters. Keep it to one short block, a description, not a keyword dump, and not commands about punctuation or formatting, which are already

8:16handled. Note that a custom prompt replaces the managed default entirely, including its language steering, so if your audio isn't English, say so in the description. Conversation context, hands over the dialog that came before this clip. Ids built for multi-turn exchanges where the current utterance only makes sense given the last one. An agent asks, what's your order number? In the caller answers with a string of digits and letters that could go several ways. Turns go in chronological order, oldest first, with no speaker labels. It holds 100 turns or 4,096 characters, and context over either cap is trimmed rather than rejected. The oldest turns drop off the front, so a long running call degrades gracefully instead of airing. The advice that matters most here is counterintuitive. Start with none of it. All three are robust to a relevant input. The model stays grounded in the audio and won't insert words that weren't spoken, but a long keyter list stuffed with common words invites over correction. Ship without them, find the terms your model actually gets wrong, then add only those.

9:19Details are aimed prompting in keyterms and conversation context. Language selection is a separate knob. In the SDKs it's, which always takes a list, one code for monolingual audio, several for code switching. Over raw HTTP the config field is, and it accepts either a single code or an array. 19 languages are supported, from English and Spanish through Japanese, Mandarin, Erdo, and Hebrew. How does this compare to other synchronous transcription endpoints? Most major providers ship something in this shape. They differ mostly in how much audio they'll accept and what they'll take it as. API MaxClip Max Size accepted formats Assembly AI Sync 120s 40 MBWAV. Raw PCMS 16LE Google Cloud Speech to text recognize 60s 10 MB multiple Azure Speech Rest API for short audio 60s. Wave PCM 16 KHz mono. Agopus Open AI V1. Audio, transcriptions, no stated cap 25 MBMP3. MP4. M4A. Wave, WebM. Others Google

10:26enforces both of its limits together. 60 seconds, 10 MB, or both, whichever you hit first. The 60s ceilings on Google and Azure are the practical constraint to watch. They're fine for push to talk, where 2-10 seconds is typical, and they fall over the moment someone dictates a long voice memo or you want to transcribe a full voicemail in one shot. As your short audio rest endpoint ISALSO strict about formats. 16 KHz mono wave or agopus. Nothing else. Which means a resampling step in your pipeline if your recorder doesn't already produce that. The more useful question isn't which ceiling is highest. It's whether the provider treats short clip transcription as a first class path with its own latency work, or as a convenience wrapper on the batch pipeline. The presence of something like a pre-warming endpoint is a decent signal either way. When should you not use a sync API? Reach for something else in four cases, and it's worth being blunt about them. The audio runs over two minutes. Use a sync transcription. Submit the file, get a webhook, move on.

11:29The audio doesn't exist yet. Live captions, an agent that needs to barge in mid-sentence, anything where you need words before the speaker stops. That's streaming, and no amount of clever request batching substitutes for it. You need speaker diurization. The sync API doesn't do it. Two-second dictation clips have one speaker, so this rarely bites, but if you're transcribing a snippet of a conversation and need to know who said what, use a sync or streaming. You need PE reduction, also not available on sync. If you're handling audio that requires reduction at the transcription layer, that's an a sync workflow. The pattern in that list, the sync path is deliberately narrow. It does one thing, short-finished audio, fast, in one round trip, and drops the features that don't fit that shape. That's the trade-off you're accepting, and it's the right one for a dictation box. The broader point, most, real-time, voice features aren't streaming problems. Their air-quest response problems wearing a streaming costume. The user pressed a button.

12:30They said a thing. They let go. Somewhere in the middle of that, we collectively decided the answer was a persistent bidirectional connection with a turn detection model attached. And then spent-or optimization budget tuning end of turn thresholds instead of the thing that was actually costing us 100 milliseconds, which was a TLS handshake we cold-have-run while the user was still talking. Look at your voice feature and ask when the audio finishes. If the answer is, before I send it, you were never streaming anything. Frequently asked questions, do you need a web socket to transcribe short audio clips? No. A web socket is only necessary when audio is still arriving and you need partial transcripts as words are spoken. For a clip that has already finished recording, a push-to-talk utterance, a voicemail, an IVR response, a single HTTP request returns a complete transcript with no session management, no polling, and less code. Assembly i's sync speech to text API returns a finished transcript in about 134 milliseconds at P50. How do I call a speech to text

13:32API from a mobile app? Record the audio locally, then post the bytes to a synchronous transcription endpoint and read the transcript from the response. With the sync API that's ASA multi-part form within part, and header, and in header, no SDK required, which matters on mobile-weared dependency wait counts. Keep your API key on a backend rather than shipping it in the client, and have the app talk to your server. How do I get a transcript without polling for job status? Use a synchronous endpoint instead of an ASync 1. ASync transcription API is return a job ID that you pull or attach a web hook to because they're built for long files where processing takes seconds to minutes. A synchronous API does the transcription inside the same HTTP exchange and returns the text in the response body, which removes job IDs, polling loops, and web hook infrastructure from your application entirely. What is the maximum audio length for a synchronous speech to text request? It varies by provider. Assembly I sync API accepts 80 milliseconds to 120 seconds per request with a 40 megabytes cap.

14:37Google Cloud Speech to text recognize a method caps at 60 seconds or 10 megabytes, and Azure Speech to text rest API for short audio caps at 60 seconds. OpenAI's transcription endpoint has no stated duration limit but caps files at 25 megabytes. For anything longer, every provider expects you to move to an asynchronous or batch endpoint. Why is my speech to text API call slower than the advertised latency? Usually because connection setup is counted in your latency budget but not in the provider's benchmark. A cold HTTPS request pays for DNS resolution, a TCP handshake, and a TLS handshake before your audio starts uploading, tens of milliseconds nearby, over 100 milliseconds intercontinental-ly. Compare the provider's server side timing field against your own end-to-end measurement total the two apart. The sync API returns for exactly this. Pre-warming the connection when recording starts moves that cost off the critical path. How do you improve transcription accuracy for names and technical terms? Give the model

15:40context about vocabulary it can't otherwise know. The sync API accepts a list for explicit vocabulary such as contact names and product names, up to 2048 characters. A describing what the audio is about, up to 4,096 characters, and holding prior turns for multi-turn exchanges, all included on no additional cost. Start with none of them and add only the terms the model demonstrably gets wrong, since large keyter lists padded with common words can't cause over correction. Thank you for listening to this Hackernoun story, read by Artificial Intelligence. Visit Hackernoun.com to read, write, learn and publish.