Prerequisites
- A FaceMode API key (sent as
x-api-key: <key>on every REST call). - An avatar ID.
- A room FaceMode can join:
{"type": "livekit" | "daily", "url", "token"}. The token is a server-side credential that grants FaceMode join, publish, and subscribe permission in your room. Never expose it to clients.
Integration recipe
1
Create a session
POST /api/sessions with the avatar and room. The response returns 201
and an ingestion object.inputProvider is optional (deepgram, gemini, gnani, elevenlabs,
openai, cartesia, sarvam, custom); omit it to use the backend
default. room.name is optional for LiveKit rooms. room.type is
"livekit" or "daily".In the common case the response already has ingestion.ready: true with the
ingestion WebSocket url and a one-time wsToken (about 15 minute expiry):2
Poll until ingestion is ready
When
ingestion.ready is false, poll GET /api/sessions/{id} with bounded
backoff until ready becomes true, for up to 240 seconds. Stop
immediately if the reported workerStatus is FAILED or ENDED. Room
credentials from the create response remain valid; do not expect them to be
repeated in poll responses.3
Open the ingestion WebSocket
Connect to The
ingestion.url and authenticate by sending the wsToken as a
WebSocket subprotocol:facemode. prefix is required and each token is single-use. Disable
WebSocket compression and native ping/pong keepalives; the protocol uses
application-level heartbeats instead. If the session response includes
ingestion.headers, forward them unchanged on the handshake.4
Negotiate with start
start must be the first application message on the socket. Send it, then
wait for started before streaming anything.{"type": "started", "session_id": "...", "server_sample_rate": 48000, "server_channels": 1}. The server values
describe the canonical format FaceMode mixes to internally; your input
format is fixed by the start you sent and cannot change mid-session.5
Stream PCM audio
For
pcm_s16le, send raw little-endian signed 16-bit PCM as binary
WebSocket frames - no JSON or base64 wrapper. Chunk at roughly 100 ms of
audio per frame (for example 3200 bytes at 16 kHz mono), and keep every
frame aligned to 2 * channels bytes.Accepted audio_encoding values are pcm_s16le (recommended, binary
frames), pcm_f32le (binary frames), and base64_json (a JSON
{"type": "audio", "data": "<base64>", "seq": n} message). sample_rate
must be an integer from 8000 to 48000 Hz; preferred rates are 8000, 11025,
12000, 16000, 22050, 24000, 32000, 44100, and 48000. channels is 1 or 2.6
Manage utterances
You have two options - pick one per session and do not mix them:
- Explicit control: send
{"type": "start_utterance", "seq": n}before the first audio chunk of an utterance, thenend_utterancewhen it ends, orcancel_utteranceto abort it (for example on user interruption).end_utteranceis acknowledged withutterance_endedechoing yourseq. Once you send any explicit utterance message, the session stays in explicit mode. - Inferred mode: send audio with no utterance messages and FaceMode
closes each utterance after about 800 ms of silence. Tune the gap with
utterance_silence_msinstart(0disables inference, otherwise 100-5000 ms).
seq is a non-negative integer you increment for every sequenced control
message. Keep it monotonic for the life of the session, including across
reconnects.7
Keep the socket alive
After
started, send {"type": "ping", "ts": <epoch ms>} about every 15
seconds; the server answers with pong. Do not send pings before startedstartmust be the first message.
8
Reconnect on transport drops
If the socket dies after
started, call
POST /api/sessions/{id}/reconnect with the original room object (plus
inputProvider if you set one at creation). The response mints a fresh
one-time wsToken. Open a new socket with facemode.<newToken>, resend the
identical start, and wait for started before resuming. Continue your
seq counter where it left off and do not replay audio that was already
sent. Tokens are never reused.9
End the session
Send
{"type": "end_session", "seq": n} and wait briefly for
{"type": "ended", "session_id": "..."}, then close the socket. ended is
a protocol acknowledgement, not confirmation that backend cleanup is
finished; poll GET /api/sessions/{id} if you need the session’s terminal
state.Message reference
Client to server:
Server to client:
Minimal example (Node.js)
On reconnect, call
POST /api/sessions/{id}/reconnect, open a new socket with
the fresh facemode.<newToken> subprotocol, resend the same start, and
continue your seq counter. Never reuse a wsToken.Frontend
The browser side does not change: join the same room with the LiveKit or Daily client SDK using a token you mint for that end user, and render the avatar video track like any other remote participant’s. FaceMode joins as its own participant (facemode-avatar by default, or the identity fixed in your
publish token).
See also
Sessions API
Create, poll, reconnect, and delete sessions.
WebSocket protocol
Full canonical message reference.