A web call has a page around it, and the page can hand the agent things a phone cannot — a photo, a screen, live page state — and take instructions back. Every hook below runs inside the TalqingSessionProvider from web calls; full signatures are on the React SDK page.

Images

The caller attaches a photo mid-call; the bytes travel over the call’s data channel. There is no upload endpoint.
The agent does not speak when an image lands; it sees it on the caller’s next turn. Silence is therefore no signal of delivery — render each image’s status: Whether an agent can see images depends on its model’s vision flag in GET /v1/catalog. See vision and images.

Screen share

The agent is handed the newest frame of the caller’s screen at the end of each of their turns. The agent side is vision_input.screenshare — see vision and images.
start() must run from a user click — the browser’s picker requires it. The agent can ask for a screen but never open one, so keep the button visible for the whole call.
The hook publishes with TALQING_SCREEN_SHARE_CAPTURE and TALQING_SCREEN_SHARE_PUBLISH: 1920×1080, contentHint: "text", h264, capped at one frame a second. If you publish a screen share yourself, keep h264 — livekit-client forces contentHint: "motion" on vp9 and av1, and the agent starts misreading text on screen.

Userdata stays on the server

The page can neither read nor write the session’s userdata. Seed it with userdata when your server calls POST /v1/calls/token; tools change it during the call, and what the call ends with is merged onto the contact’s record. To show the page something the agent learned, push it with a frontend RPC. See userdata.

Frontend RPC

A tool’s frontend_rpc operation calls into your page — to open a checkout, highlight a row, or read something only the page knows. Register one handler per method:
  • A handler receives payload as the JSON string of the operation’s payload ("{}" when it sent none) and returns a string. Valid JSON is parsed for the tool; anything else arrives as { "result": "<string>" }.
  • A "*" handler catches every method and receives the whole envelope, method included.
  • An unhandled method returns RPC error 1404; any other throw returns 1500 with its message; throw TalqingRpcError to pick the code.
  • The agent waits 5 seconds unless the operation sets timeout. Do not await a person inside a handler.
To just render what the agent asked for, useTalqingFrontendRpcs() registers the wildcard and keeps the latest payload per method in actions.

Avatar video

A channel: video agent’s Anam avatar is published by a second participant. useTalqingAvatarTrack() finds it, falling back to the agent’s own camera or screen-share track:
Frame size depends on the avatar model, so size the container by aspect ratio and crop. The avatar’s voice is already in the room audio — do not add an audio element.

Typed messages and the transcript

useSessionMessages() returns the live transcript as messages (each with id, timestamp, message, from, and type of userTranscript, agentTranscript or chatMessage) and a send(text) that types into a voice call — useful for a spelling or an order number. After the call, the stored transcript is on GET /v1/calls/{session_id} (calls).

Next

React SDK

Every export with its full signature.

Frontend RPC

The tool operation that calls these handlers.

Vision and images

Which models can see, and the screen-share rules.