Realtime models

config.realtime names one speech-to-speech model that hears and speaks audio directly, replacing the stt → llm → tts cascade. It gives the shortest time to first sound. In exchange, fixed lines are paraphrasable, there is no failover, and several settings are refused. Voice and video only. Use the cascade unless latency is the product. A realtime model suits a demo, a companion or a practice partner, where approximate wording and no failover are acceptable.

curl -X POST https://api.talqing.com/v1/agents \
  -H "Authorization: Bearer $TALQING_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "config": {
      "name": "Northwind practice line",
      "channel": "voice",
      "prompt": "You are a friendly practice partner for sales reps at Northwind Tools. Ask one question at a time. Keep every reply under three sentences.",
      "greeting": {"message": "Hi, this is the Northwind practice line. What are we working on today?"},
      "language": "en",
      "realtime": { "provider": "openai", "model": "gpt-realtime-2.1", "voice": "marin" }
    }
  }'

const agent = await talqing.agents.create({
  createAgentRequest: {
    config: {
      name: "Northwind practice line",
      channel: "voice",
      prompt:
        "You are a friendly practice partner for sales reps at Northwind Tools. Ask one question at a time. Keep every reply under three sentences.",
      greeting: { message: "Hi, this is the Northwind practice line. What are we working on today?" },
      language: "en",
      realtime: { provider: "openai", model: "gpt-realtime-2.1", voice: "marin" },
    },
  },
});

agent = talqing.agents.create(
    config={
        "name": "Northwind practice line",
        "channel": "voice",
        "prompt": (
            "You are a friendly practice partner for sales reps at Northwind Tools. "
            "Ask one question at a time. Keep every reply under three sentences."
        ),
        "greeting": {"message": "Hi, this is the Northwind practice line. What are we working on today?"},
        "language": "en",
        "realtime": {"provider": "openai", "model": "gpt-realtime-2.1", "voice": "marin"},
    }
)

Saving with realtime set clears stt, llm and tts to null; whatever you sent in them is dropped. To return to the cascade, send realtime: null with the three slots.

Field	Meaning
`provider`, `model`	An entry from the catalog’s `realtime` array. Default `openai/gpt-realtime-2.1`.
`voice`	From the model’s own `voices` roster (`GET /v1/catalog/voices?kind=realtime`). `null` uses `default_voice`.
`speed`	Within the entry’s `speed_min`–`speed_max`; only `1.0` if `supports_speed` is false.
`voice_name`	Dashboard label, never sent.

There is no fallback, reasoning_effort, priority, builtin_tools or expressive.

Fixed lines are instructions

With no text-to-speech stage, a fixed line is delivered to the model as “say this, word for word”. The model usually complies but may reword it. This applies to the greeting, the say operation and a handoff message. {{…}} substitution still happens first. Consequences:

A quoted price, reference number or legal sentence is not guaranteed. Use the cascade.
recording.notice is refused: a recording notice needs a cascade pipeline (STT + LLM + TTS)….
greeting_interruptible: false is refused: the provider detects interruptions itself.

Turn detection moves to the provider

The provider’s server-side voice detection commits each turn.

Setting	On a realtime model
`endpointing.min_silence_duration`	Sent to the provider as its silence window (not on `gpt-live-1`).
`endpointing.max_silence_duration`	Ignored.
`interruption.enabled`, `min_speech_duration`, `resume_false_interruption`, `false_interruption_timeout`	Work as on the cascade.
`interruption.min_words`	No effect: there are no interim transcripts.
`preemptive_generation`	Pinned off.
`on_user_turn_completed`	Refused: the hook never runs. `on_enter` and `on_exit` work.
`vision_input.screenshare.enabled`	Refused: there is no per-turn moment to inject a frame. Images people send still work if the model has `vision`.
`elapsed_time`	Refused: there is no text message to stamp.
`max_steps`	Not enforced.

See turn handling.

No failover

When the provider fails mid-call, the error is recorded on the call as a session.error event. A recoverable error does not end the call; an unrecoverable one ends it with close reason error. If that is unacceptable, use the cascade with an llm.fallback.

Post-call analysis

Analysis normally runs on the agent’s own llm. A realtime agent has none, so it runs on the first catalog language model from the same provider, on the same key. Set analysis.model to choose another. See call analysis.

Per-model differences

Read these from the catalog entry rather than assuming:

supports_instruction_update: false: the model refuses new instructions once connected, so an agent on it cannot have handoffs (refused on save).
vision: false: the model cannot read images. Publishing warns. The xAI Grok Voice models accept an image and answer as if they saw it, so treat the flag as binding.
languages empty: the model has no language setting; steer it from the prompt. See language.
pricing: realtime rates are split into text in, cached text in, audio in, cached audio in, text out, audio out, and per_session_minute. A model priced per session minute has zero token rates.
openai/gpt-live-1 is full duplex: it takes no silence window and no interruption settings, and it runs its reasoning and tool calls on a second OpenAI language model billed at that model’s token rates, as a separate LLM line on the call.

Some provider charges never reach Talqing’s cost figures: OpenAI’s realtime transcription model (no metered quantity) and xAI’s per-client-event charge. Both appear on your provider bill.

Where it is refused

Rule	When	Result
`channel: "text"`	save	`realtime` cleared.
`stt`/`llm`/`tts` sent with `realtime`	save	Cleared.
`vision_input.screenshare.enabled`, `elapsed_time`	save	Refused.
`on_user_turn_completed`, `greeting_interruptible: false`, `recording.notice`	save and publish	Error.
`handoffs` on a model with `supports_instruction_update: false`	save and publish	Error.
A handoff between a realtime and a cascade agent, either way	publish	`a handoff target must use the same pipeline as its source…`
`speed` outside range, or not `1.0` without speed support	save and publish	Error.
An agent `language` the model does not publish	save and publish	Error. Models with no `languages` are exempt.
No avatar on `video`	save and publish	Error.
No key for the model’s provider	publish	Error.
`vision: false`	publish	Warning.

Everything else works: tools, MCP servers, handoffs between realtime agents, vars, userdata, conversation memory, background audio, noise cancellation, recording, analysis, webhooks, batch calling, phone numbers and video avatars.

Models

The catalog and the cascade’s model slots.

Voices

Browsing a realtime model’s voice roster.

Get started

Build

Guides

Realtime models

Fixed lines are instructions

Turn detection moves to the provider

No failover

Post-call analysis

Per-model differences

Where it is refused

Next

Models

Voices

​Fixed lines are instructions

​Turn detection moves to the provider

​No failover

​Post-call analysis

​Per-model differences

​Where it is refused

​Next

Models

Voices

Fixed lines are instructions

Turn detection moves to the provider

No failover

Post-call analysis

Per-model differences

Where it is refused

Next