config.realtime is what discriminates the two. Set it and it is the whole pipeline; leave it null and the cascade runs.
Take the cascade unless latency is the product. Take a realtime model when the conversation itself is the product — a demo, a companion, a practice partner, an interview — and you can live with an opening line that is approximate and a call with no failover.
Setting one
realtime replaces the cascade rather than sitting beside it. Saving a config with realtime set clears stt, llm and tts to null — whatever you sent in them is dropped, not refused. Going back to the cascade means sending realtime: null and the three slots together. Read the response rather than assuming your input survived; see channels.text agent has no audio, so realtime is cleared there the same way stt and tts are. On video the pipeline is unchanged in every other respect: the Anam avatar renders from whatever audio the agent publishes, and a realtime model publishes audio exactly as a text-to-speech model does.
The fields
There is no
fallback here, and no reasoning_effort, priority or builtin_tools — those live on llm, which a realtime agent does not have.
The five models
GET /v1/catalog is authoritative; the exact rates are in the table on models.
The pricing shape is different from every other model kind. A realtime entry carries a seven-way split — text in, cached text in, audio in, cached audio in, text out, audio out, and a per-session minute — because audio and text are priced apart and audio costs several times more: on gpt-realtime-2.1, audio input is eight times text input. A single input rate would be wrong by a large multiple. The two xAI entries use the other half of that shape: they bill by the minute of audio and their token rates are a real zero, not a missing figure.
Two provider charges on this pipeline never reach Talqing’s cost figures. The OpenAI entries run a separate transcription model so the call has a transcript at all — OpenAI bills it apart from the audio tokens and reports no quantity anyone can meter. xAI charges per client text event, which nothing on the wire counts. Both land on your provider bill.
Scripted speech is not scripted
This is the consequence that surprises people most, and it follows from the pipeline having no text-to-speech stage at all. On the cascade, a fixed line is handed to the voice as text and spoken exactly. On a realtime model there is nothing to hand it to, so the line is delivered as an instruction: say the following to the user, word for word, and say nothing else. The model usually complies, in roughly those words. It may also reword, reorder, trim, or add a syllable of its own. Everything the agent says from a script goes through that same path:
The practical consequences:
- A quoted price, a reference number or a legal line is not safe here. If the exact words matter, they belong on the cascade.
- A recording consent notice is refused outright. See below.
- A greeting is a strong suggestion. It still personalizes —
{{userdata.first_name}}resolves before the instruction is built — but the model owns the delivery. - Substitution still happens first, so a template that resolves to nothing still reads correctly.
Recording consent is refused
Recording works normally on this pipeline — the audio the agent publishes is recorded like any other. What is refused isrecording.consent: "disclosure", the mode that makes the agent speak a notice:
on_user_turn_completed never fires
A realtime model detects the end of a turn inside the provider’s own socket, so LiveKit never reaches the point where it would run the per-turn hook. Rather than let a hook sit in a config and quietly do nothing, the combination is rejected when you save:
on_enter and on_exit are unaffected and run normally. See lifecycle hooks.
Turn detection moves inside the provider
On the cascade, the end of a turn is our decision, made from the speech-to-text model, a voice activity detector and — on a batch model in one of the turn detector’s languages — a judgement about whether the sentence sounds finished. On a realtime model the provider’s own server-side voice detection commits the turn, before any transcript exists. What that changes:
Anything else built on interim transcripts is in the same position: the transcript arrives as a finished item, after the model has already started replying. The call’s transcript itself is complete — the OpenAI entries run a transcription model for it, Gemini transcribes both directions inside the Live session, and the xAI plugin sets its own — but nothing consumes it mid-turn.
See turn handling and speech-to-text and turn detection.
Screen share is refused
vision_input.screenshare.enabled with realtime set is refused when you save the config, not at publish. The reason is the turn hook again: the newest frame is injected on each user turn, and that moment does not exist here. The catalog’s vision flag does not rescue it — three of the five entries can read an image, so a capability check would wave through exactly the models where the frame goes nowhere.
Images people send still work, on the three entries that read them. It is the live screen that has nowhere to go. See vision and images.
There is no failover
The cascade lets you name a second model per stage. A realtime agent cannot: there is nofallback field, and the framework’s realtime fallback adapter is deliberately not used, because it merges both models’ capabilities conservatively for the whole call — pairing anything with one of these would cost the primary model abilities it has, on every call, for a failover that may never happen.
So when the provider fails mid-call:
- The failure is recorded on the call as a
session.errorevent carrying what the provider actually said and whether it was recoverable. - An error the provider marks recoverable does not end the call; the session carries on.
- An unrecoverable one ends the call. The cascade tolerates up to three consecutive unrecoverable errors from a stage before closing; a realtime model gets no such grace, because there is nothing left running. The call’s close reason reads
error— “The call ended on an error”. See close reasons.
Post-call analysis
Analysis is on by default and works here, with one substitution you should know about. It normally runs on the agent’s own language model, so turning it on never demands a provider key the agent did not already need. A realtime agent has no language model of its own, and a speech-to-speech model cannot read a transcript over a chat API — so analysis falls back to the first language model in the catalog from the same provider. The key property survives: same provider, same key. Naminganalysis.model explicitly overrides that, and then that provider needs a key too. See call analysis.
Everything a realtime agent is refused
Every rule below is enforced when you save or publish, with the message quoted here or in the section above.
See validation for how errors and warnings come back.
What still works
Everything that is not about the three stages: tools and the whole operation tree, knowledge bases, MCP servers, handoffs to other realtime agents,vars and userdata, conversation memory, background audio, noise cancellation, recording, post-call analysis, webhooks, batch calling, phone numbers, and the video avatar.
When to take the cascade instead
Six answers, and any one of them is enough:- A specific voice. The realtime rosters are the provider’s own; the text-to-speech catalog is far larger, and expressive delivery exists only there.
- Exact scripted lines — a disclosure, a quoted price, a compliance sentence.
- A batch speech-to-text price. Transcribing each utterance in one request is the cheapest way to hear a caller, and it is a cascade-only option.
- A fallback, on any stage.
- A screen share the agent watches while you talk about it.
- A hook on every user turn — checking a policy, writing to your own system, injecting context per turn.
Next
Models
The catalog, both pipelines side by side, and every rate.
Voices
Rosters, defaults and speed.
Validation
How these refusals come back, and how to fix them.