Speech operations

Three operations put words into the conversation from inside a tool. say speaks a fixed line, generate_reply has the model write and speak one, and add_message adds a system message to the model’s context without anyone hearing it.

[
  { "kind": "say", "config": { "text": "One moment, let me look that up." } },
  {
    "kind": "http",
    "config": {
      "method": "GET",
      "url": "https://api.example.com/accounts/{{args.account_id}}",
      "headers": { "Authorization": "Bearer {{secrets.BILLING_API_KEY}}" },
      "timeout": 10
    },
    "silent": true,
    "publish_fields": [
      { "path": "balance_due", "key": "balance", "store": "tooldata" },
      { "path": "due_date", "store": "tooldata" }
    ]
  },
  {
    "kind": "generate_reply",
    "config": {
      "instructions": "Tell the caller their balance is {{tooldata.balance}}, due on {{tooldata.due_date}}. One sentence, then ask if they want to pay now."
    }
  }
]

The say covers the HTTP latency: the tree moves on while it plays.

Config

Kind	Field	Default	Meaning
`say`	`text`	required	The line, spoken word for word with templates filled in
`say`	`wait_for_playback`	`false`	Hold the tree until the line has played
`generate_reply`	`instructions`	required	What to say. The model writes the words, in the conversation’s language and the agent’s voice
`generate_reply`	`wait_for_playback`	`false`	Hold the tree until the reply has played
`add_message`	`text`	required	A system message added to the model’s context. Nothing is spoken and no reply is triggered
`add_message`	`scope`	`"conversation"`	`conversation` keeps it for the rest of the conversation; `turn` only for the reply to the current turn

All text fields are templated (see templating and data) and a blank or whitespace-only value is refused at save. None of the three returns a result to the model. scope: "turn" works only in a tool wired to the on_user_turn_completed hook; anywhere else it is refused when the agent is published.

Choosing one

say when the wording is fixed: a filler, a confirmation, a reference number, a disclosure. It costs only text-to-speech.
generate_reply when the phrasing depends on what came back. It costs one more model call in the turn. Write instructions as a direction, not as dialogue: “Confirm the booking and read back the date” rather than “Your booking is confirmed”.
add_message to steer the rest of the conversation after a lookup: “The caller is on the Enterprise plan; offer the priority queue.” Values that later operations read belong in tooldata or userdata instead.

A tool whose http, code and frontend_rpc operations are all silent (or that has none) is silent: the model adds nothing after your say. To have it add a line, end with a generate_reply.

`wait_for_playback`

Leave it off for fillers: the line exists to cover the next operation’s latency. Set it when the caller must have heard the line before the next operation runs, such as a disclosure before a side effect.

The wait is transitive. Speech is first in, first out, so it also waits for anything the agent was already saying in the turn.
An interrupted line counts as finished. To make sure the line is heard in full, also set disable_interruptions on the tool.
The wait is bounded. A line that has not finished after 60 seconds fails the operation, and on_error applies.

A say immediately before an end_call or a transfer never needs the flag: both wait for queued speech before they act.

On a realtime (speech-to-speech) agent, say asks the model to speak the line, and it may reword, reorder or trim it. When the wording must be exact — a price, a legal line, a disclosure — use a cascade model stack. See realtime.

End call

Ending the call after a goodbye.

The operation tree

What the model is told after a tool runs.

Get started

Build

Guides

Speech operations

Config

Choosing one

`wait_for_playback`

Next

End call

The operation tree

​Config

​Choosing one

​wait_for_playback

​Next

End call

The operation tree

Config

Choosing one

`wait_for_playback`

Next