Skip to main content
Use the direct API when your stack is not LiveKit Agents or Pipecat, or when you need Daily rooms or full control over the wire protocol. Your backend drives a short REST exchange, then streams audio to a per-session ingestion WebSocket. Your frontend joins the room with the LiveKit or Daily client SDK as usual. This page is a recipe. Field-level details live in the sessions API reference and the WebSocket protocol reference.

Prerequisites

  • A FaceMode API key (sent as x-api-key: <key> on every REST call).
  • An avatar ID.
  • A room FaceMode can join: {"type": "livekit" | "daily", "url", "token"}. The token is a server-side credential that grants FaceMode join, publish, and subscribe permission in your room. Never expose it to clients.

Integration recipe

1

Create a session

POST /api/sessions with the avatar and room. The response returns 201 and an ingestion object.
inputProvider is optional (deepgram, gemini, gnani, elevenlabs, openai, cartesia, sarvam, custom); omit it to use the backend default. room.name is optional for LiveKit rooms. room.type is "livekit" or "daily".In the common case the response already has ingestion.ready: true with the ingestion WebSocket url and a one-time wsToken (about 15 minute expiry):
2

Poll until ingestion is ready

When ingestion.ready is false, poll GET /api/sessions/{id} with bounded backoff until ready becomes true, for up to 240 seconds. Stop immediately if the reported workerStatus is FAILED or ENDED. Room credentials from the create response remain valid; do not expect them to be repeated in poll responses.
3

Open the ingestion WebSocket

Connect to ingestion.url and authenticate by sending the wsToken as a WebSocket subprotocol:
The facemode. prefix is required and each token is single-use. Disable WebSocket compression and native ping/pong keepalives; the protocol uses application-level heartbeats instead. If the session response includes ingestion.headers, forward them unchanged on the handshake.
4

Negotiate with start

start must be the first application message on the socket. Send it, then wait for started before streaming anything.
The server replies with {"type": "started", "session_id": "...", "server_sample_rate": 48000, "server_channels": 1}. The server values describe the canonical format FaceMode mixes to internally; your input format is fixed by the start you sent and cannot change mid-session.
5

Stream PCM audio

For pcm_s16le, send raw little-endian signed 16-bit PCM as binary WebSocket frames - no JSON or base64 wrapper. Chunk at roughly 100 ms of audio per frame (for example 3200 bytes at 16 kHz mono), and keep every frame aligned to 2 * channels bytes.Accepted audio_encoding values are pcm_s16le (recommended, binary frames), pcm_f32le (binary frames), and base64_json (a JSON {"type": "audio", "data": "<base64>", "seq": n} message). sample_rate must be an integer from 8000 to 48000 Hz; preferred rates are 8000, 11025, 12000, 16000, 22050, 24000, 32000, 44100, and 48000. channels is 1 or 2.
6

Manage utterances

You have two options - pick one per session and do not mix them:
  • Explicit control: send {"type": "start_utterance", "seq": n} before the first audio chunk of an utterance, then end_utterance when it ends, or cancel_utterance to abort it (for example on user interruption). end_utterance is acknowledged with utterance_ended echoing your seq. Once you send any explicit utterance message, the session stays in explicit mode.
  • Inferred mode: send audio with no utterance messages and FaceMode closes each utterance after about 800 ms of silence. Tune the gap with utterance_silence_ms in start (0 disables inference, otherwise 100-5000 ms).
seq is a non-negative integer you increment for every sequenced control message. Keep it monotonic for the life of the session, including across reconnects.
7

Keep the socket alive

After started, send {"type": "ping", "ts": <epoch ms>} about every 15 seconds; the server answers with pong. Do not send pings before started
  • start must be the first message.
8

Reconnect on transport drops

If the socket dies after started, call POST /api/sessions/{id}/reconnect with the original room object (plus inputProvider if you set one at creation). The response mints a fresh one-time wsToken. Open a new socket with facemode.<newToken>, resend the identical start, and wait for started before resuming. Continue your seq counter where it left off and do not replay audio that was already sent. Tokens are never reused.
9

End the session

Send {"type": "end_session", "seq": n} and wait briefly for {"type": "ended", "session_id": "..."}, then close the socket. ended is a protocol acknowledgement, not confirmation that backend cleanup is finished; poll GET /api/sessions/{id} if you need the session’s terminal state.

Message reference

Client to server: Server to client:

Minimal example (Node.js)

On reconnect, call POST /api/sessions/{id}/reconnect, open a new socket with the fresh facemode.<newToken> subprotocol, resend the same start, and continue your seq counter. Never reuse a wsToken.

Frontend

The browser side does not change: join the same room with the LiveKit or Daily client SDK using a token you mint for that end user, and render the avatar video track like any other remote participant’s. FaceMode joins as its own participant (facemode-avatar by default, or the identity fixed in your publish token).

See also

Sessions API

Create, poll, reconnect, and delete sessions.

WebSocket protocol

Full canonical message reference.