LiveKit Agents (JS / TS)

The @facemode/agents-plugin-facemode package connects a LiveKit Agents JS AgentSession to FaceMode. The plugin creates the FaceMode session, streams your TTS audio to FaceMode over the canonical WebSocket protocol, and routes the avatar video track into your LiveKit room.

Install

npm install @facemode/agents-plugin-facemode

Quickstart

import { fileURLToPath } from 'node:url';
import {
  ServerOptions,
  cli,
  defineAgent,
  inference,
  voice,
} from '@livekit/agents';
import { AvatarSession } from '@facemode/agents-plugin-facemode';

export default defineAgent({
  entry: async (ctx) => {
    const session = new voice.AgentSession({
      stt: new inference.STT({ model: 'deepgram/nova-3', language: 'multi' }),
      llm: new inference.LLM({ model: 'openai/gpt-5.4-mini' }),
      tts: new inference.TTS({
        model: 'cartesia/sonic-3',
        voice: '9626c31c-bec5-4cca-baa8-f8ba9e84c8bc',
      }),
    });
    const avatar = new AvatarSession({
      apiKey: process.env.FACEMODE_API_KEY ?? '',
      avatarId: process.env.FACEMODE_AVATAR_ID,
      apiUrl: process.env.FACEMODE_API_URL,
      inputProvider: 'deepgram',
    });
    await ctx.connect();
    await avatar.start(session, ctx.room, {
      room: {
        type: 'livekit',
        url: process.env.LIVEKIT_URL ?? '',
        token: process.env.LIVEKIT_PUBLISH_TOKEN ?? '',
        name: process.env.LIVEKIT_ROOM_NAME,
      },
    });
    await session.start({
      agent: new voice.Agent({ instructions: 'You are a helpful voice assistant.' }),
      room: ctx.room,
      outputOptions: { audioEnabled: false },
    });
    await avatar.waitForJoin();
  },
});

cli.runApp(new ServerOptions({
  agent: fileURLToPath(import.meta.url),
  agentName: 'facemode-basic',
}));

Create the AvatarSession

Construct AvatarSession with apiKey and avatarId.

Connect and start the avatar

Call await ctx.connect(), then avatar.start(session, ctx.room, { room }). The room option is the credential object sent to FaceMode; it is separate from the already-connected ctx.room the agent uses for media. Call avatar.start() before session.start().

Start the agent with audio output disabled

Start the AgentSession with outputOptions: { audioEnabled: false }. The plugin replaces the session’s audio output so TTS frames go to FaceMode instead of being published directly.

Wait for the avatar to join

await avatar.waitForJoin() resolves when the avatar signals audio_ready or its avatar video track is subscribed. It rejects with FaceModeProtocolError after the timeout (30000 ms by default; pass { timeout: null } to wait indefinitely).

AvatarSession options

Option	Default	Description
`apiKey`	required	FaceMode API key. Required.
`avatarId`	`''`	FaceMode avatar ID.
`apiUrl`	`https://api.facemode.io/api`	FaceMode API root. Override for self-hosted or local backends.
`inputProvider`	`undefined`	Optional session input provider. Sent on session creation only when set.
`avatarParticipantIdentity`	`'facemode-avatar'`	Expected LiveKit identity of the avatar participant. When left at the default, the plugin adopts the identity embedded in the room token.
`avatarParticipantName`	`'FaceMode Avatar'`	Display name for the avatar participant.

Useful properties: avatar.sessionId returns the active FaceMode session ID and avatar.avatarIdentity returns the resolved participant identity.

start() signature

await avatar.start(agentSession, ctx.room, {
  room: {
    type: 'livekit',
    url: 'wss://project.livekit.cloud',
    token: '<server-side publish token>',
    name: 'my-room', // optional; derived from ctx.room or the token when omitted
  },
});

Room token grants

The room.token value is used by FaceMode to join your room. It must grant the avatar participant join, publish, and subscribe permission on the exact room name (room_join, can_publish, can_subscribe). If the token fixes a participant identity, set avatarParticipantIdentity to match. Keep the token on your server; never expose it to browser clients or log it.

inputProvider

Set inputProvider to pin the upstream speech input provider for the session. Supported values: 'deepgram', 'gemini', 'gnani', 'elevenlabs', 'openai', 'cartesia', 'sarvam', 'custom'. When omitted, the field is not sent and the backend default applies.

Audio format

The plugin negotiates the canonical protocol using your TTS output format and locks it for the session:

Encoding is pcm_s16le sent as binary WebSocket frames.
Sample rate must be an integer between 8000 and 48000 Hz. Preferred rates are 8000, 11025, 12000, 16000, 22050, 24000, 32000, 44100, and 48000; other in-range rates are accepted and logged once.
Channels must be 1 or 2.
The declared rate must match the actual PCM data. A format change mid-session raises FaceModeProtocolError.

Auto-reconnect

If the WebSocket transport drops after started, the plugin reconnects in the background:

It posts to /api/sessions/{id}/reconnect, receives a fresh one-time wsToken (tokens are never reused), and opens a new socket with the facemode.<newToken> subprotocol.
It resends the identical start message and waits for started before resuming.
Session identity and outgoing sequence numbers are preserved. Sends made while the transport is down wait for the barrier and are delivered once on the new socket; nothing already sent is replayed.

At most one reconnect runs at a time, with up to 2 attempts and a short backoff. No reconnect is attempted after stop(), after FaceMode reports ended, or after a fatal protocol error. If all attempts fail, pending and future sends raise FaceModeProtocolError.

Lifecycle and shutdown

Call await avatar.stop() (or aclose()) to shut down. On a negotiated session the plugin sends end_session and waits up to 3 seconds for the canonical ended acknowledgement before closing the socket. Interruptions send cancel_utterance; a normal end of speech sends end_utterance.

Error types

All errors derive from FaceModeError:

Exception	Raised when
`FaceModeApiError`	A FaceMode REST call fails, the response is malformed, ingestion times out (240s budget), or the session enters a `FAILED` / `ENDED` state.
`FaceModeProtocolError`	The WebSocket protocol fails: negotiation errors, format changes, server `error` messages, unexpected socket close, `waitForJoin()` timeout, or reconnect exhaustion.

import {
  FaceModeError,
  FaceModeApiError,
  FaceModeProtocolError,
} from '@facemode/agents-plugin-facemode';

The JS package exports FaceModeApiError (lowercase “pi”), while the Python package exports FaceModeAPIError.

Timeouts

Budget	Value
Ingestion readiness polling (`GET /api/sessions/{id}` when `ready: false`)	Up to 240s with bounded backoff; `FAILED` / `ENDED` short-circuit.
WebSocket handshake	240s.
`start` -> `started` acknowledgement	15s.
`waitForJoin()`	30000 ms default, configurable or `null` for no timeout.

The plugin disables WebSocket compression and uses an application-level ping every 15 seconds after negotiation instead of native keepalives.

API URL override

The default API URL is https://api.facemode.io/api. Set apiUrl (or the FACEMODE_API_URL environment variable in your own config) for a self-hosted or local backend.

Get Started

Concepts

Integrations

LiveKit Agents (JS / TS)

Install

Quickstart

AvatarSession options

start() signature

Room token grants

inputProvider

Audio format

Auto-reconnect

Lifecycle and shutdown

Error types

Timeouts

API URL override

​Install

​Quickstart

​AvatarSession options

​start() signature

​Room token grants

​inputProvider

​Audio format

​Auto-reconnect

​Lifecycle and shutdown

​Error types

​Timeouts

​API URL override

Install

Quickstart

AvatarSession options

start() signature

Room token grants

inputProvider

Audio format

Auto-reconnect

Lifecycle and shutdown

Error types

Timeouts

API URL override