TalqingSessionProvider from
web calls; full signatures are on the
React SDK page.
Images
The caller attaches a photo mid-call; the bytes travel over the call’s data channel. There is no upload endpoint.status:
Whether an agent can see images depends on its model’s
vision flag in
GET /v1/catalog. See vision and images.
Screen share
The agent is handed the newest frame of the caller’s screen at the end of each of their turns. The agent side isvision_input.screenshare — see
vision and images.
TALQING_SCREEN_SHARE_CAPTURE and
TALQING_SCREEN_SHARE_PUBLISH: 1920×1080, contentHint: "text", h264, capped at
one frame a second. If you publish a screen share yourself, keep h264 —
livekit-client forces contentHint: "motion" on vp9 and av1, and the agent
starts misreading text on screen.
Userdata stays on the server
The page can neither read nor write the session’suserdata. Seed it with
userdata when your server calls POST /v1/calls/token; tools change it during
the call, and what the call ends with is merged onto the contact’s record. To
show the page something the agent learned, push it with a
frontend RPC. See userdata.
Frontend RPC
A tool’sfrontend_rpc operation calls into your page — to
open a checkout, highlight a row, or read something only the page knows. Register
one handler per method:
- A handler receives
payloadas the JSON string of the operation’spayload("{}"when it sent none) and returns a string. Valid JSON is parsed for the tool; anything else arrives as{ "result": "<string>" }. - A
"*"handler catches every method and receives the whole envelope,methodincluded. - An unhandled method returns RPC error 1404; any other throw returns 1500
with its message; throw
TalqingRpcErrorto pick the code. - The agent waits 5 seconds unless the operation sets
timeout. Do not await a person inside a handler.
useTalqingFrontendRpcs() registers the
wildcard and keeps the latest payload per method in actions.
Avatar video
Achannel: video agent’s Anam avatar is published by a
second participant. useTalqingAvatarTrack() finds it, falling back to the
agent’s own camera or screen-share track:
Typed messages and the transcript
useSessionMessages() returns the live transcript as messages (each with id,
timestamp, message, from, and type of userTranscript, agentTranscript
or chatMessage) and a send(text) that types into a voice call — useful for a
spelling or an order number. After the call, the stored transcript is on
GET /v1/calls/{session_id} (calls).
Next
React SDK
Every export with its full signature.
Frontend RPC
The tool operation that calls these handlers.
Vision and images
Which models can see, and the screen-share rules.