LiveKit Agents (Python)

The livekit-plugins-facemode package connects a LiveKit Agents AgentSession to FaceMode. The plugin creates the FaceMode session, streams your TTS audio to FaceMode over the canonical WebSocket protocol, and routes the avatar video track into your LiveKit room.

Install

pip install livekit-plugins-facemode

Quickstart

A complete agent looks like this:

import os

from livekit.agents import Agent, AgentSession, AgentServer, JobContext, cli, inference, room_io
from livekit_plugins_facemode import AvatarSession

server = AgentServer()


@server.rtc_session(agent_name="facemode-basic")
async def entrypoint(ctx: JobContext):
    session = AgentSession(
        stt=inference.STT(model="deepgram/nova-3", language="multi"),
        llm=inference.LLM(model="openai/gpt-5.4-mini"),
        tts=inference.TTS(model="cartesia/sonic-3", voice="9626c31c-bec5-4cca-baa8-f8ba9e84c8bc"),
    )
    avatar = AvatarSession(
        api_key=os.environ["FACEMODE_API_KEY"],
        avatar_id=os.environ.get("FACEMODE_AVATAR_ID", ""),
        api_url=os.environ.get("FACEMODE_API_URL", "https://api.facemode.io/api"),
        input_provider=os.environ.get("FACEMODE_INPUT_PROVIDER") or None,
    )
    await ctx.connect()
    await avatar.start(
        session,
        ctx.room,
        room={
            "type": "livekit",
            "url": os.environ["LIVEKIT_URL"],
            "token": os.environ["LIVEKIT_PUBLISH_TOKEN"],
        },
    )
    await session.start(
        agent=Agent(instructions="Greet the user and ask how you can help."),
        room=ctx.room,
        room_options=room_io.RoomOptions(audio_output=False),
    )
    await avatar.wait_for_join()


if __name__ == "__main__":
    cli.run_app(server)

Create the AvatarSession

Construct AvatarSession with your api_key and avatar_id. Both can come from environment variables.

Connect and start the avatar

Call await ctx.connect(), then avatar.start(session, ctx.room, room=...). The room argument is the room credential object sent to FaceMode; it is separate from the already-connected ctx.room the agent uses for media. avatar.start() must be called before session.start(), and only once per AvatarSession instance.

Start the agent with audio output disabled

Start the AgentSession with room_io.RoomOptions(audio_output=False). The plugin replaces the session’s audio output so TTS frames go to FaceMode instead of being published directly.

Wait for the avatar to join

await avatar.wait_for_join() resolves when the avatar video track appears in the room. It raises FaceModeNotReadyError if the track does not arrive within the timeout (30 seconds by default; pass timeout=None to wait indefinitely).

AvatarSession options

Option	Default	Description
`api_key`	required	FaceMode API key. Required.
`avatar_id`	`""`	FaceMode avatar ID.
`api_url`	`https://api.facemode.io/api`	FaceMode API root. Override for self-hosted or local backends.
`input_provider`	`None`	Optional session input provider. Sent on session creation only when set.
`avatar_participant_identity`	`"facemode-avatar"`	Expected LiveKit identity of the avatar participant. When left at the default, the plugin adopts the identity embedded in the room token or returned by the session response.
`avatar_participant_name`	`"FaceMode Avatar"`	Display name for the avatar participant.

Useful properties: avatar.session_id returns the active FaceMode session ID, and avatar.avatar_identity returns the resolved participant identity.

start() signature

await avatar.start(
    agent_session,            # your AgentSession
    ctx.room,                 # connected rtc.Room used for media
    room={                    # credentials sent to FaceMode
        "type": "livekit",
        "url": "wss://project.livekit.cloud",
        "token": "<server-side publish token>",
        "name": "my-room",    # optional; derived from ctx.room or the token when omitted
    },
)

room accepts a LiveKitRoom instance or a mapping with type, url, token, and optional name. room_config= is an explicit alias for room= if you prefer to avoid the overloaded name.

Room token grants

The room.token value is used by FaceMode to join your room. It must grant:

room_join for the exact room name,
can_publish so the avatar video and audio tracks can be published,
can_subscribe so FaceMode can participate in the room’s media session.

If the token fixes a participant identity, set avatar_participant_identity to match and make sure your room policy allows that identity. Keep the token on your server; never expose it to browser clients or log it.

Input provider

Pass input_provider to pin the upstream speech input provider for the session. Supported values: deepgram, gemini, gnani, elevenlabs, openai, cartesia, sarvam, custom. When omitted, the field is not sent and the backend default applies. An invalid value raises ValueError at construction time.

Audio format

The plugin negotiates the canonical protocol using your TTS output format and locks it for the session:

Encoding is pcm_s16le sent as binary WebSocket frames.
Sample rate must be an integer between 8000 and 48000 Hz. Preferred rates are 8000, 11025, 12000, 16000, 22050, 24000, 32000, 44100, and 48000; other in-range rates are accepted and logged once.
Channels must be 1 or 2.
A format change mid-session raises FaceModeProtocolError.

Reconnect behavior

If the WebSocket transport drops after the protocol has started, the plugin reconnects automatically in the background:

It calls POST /api/sessions/{id}/reconnect for a fresh one-time wsToken (a consumed token is never reused).
It opens a new socket with the facemode.<newToken> subprotocol, resends the identical start message, and waits for started before resuming.
Outgoing control seq numbers stay monotonic across the reconnect. Audio already sent is not replayed; sends made during the gap are delivered once on the new socket.

Reconnect is bounded to 2 attempts with backoff and runs single-flight. Exhaustion surfaces a typed FaceModeNotReadyError or FaceModeAPIError. Intentional stop(), a canonical ended message, fatal protocol errors, and drops before the first negotiation do not reconnect.

Lifecycle and shutdown

Call await avatar.stop() (or aclose()) to shut down. On a negotiated session the plugin sends end_session and waits briefly for the canonical ended acknowledgement before closing the socket. Note that ended is a protocol acknowledgement, not confirmation that backend cleanup is complete. When LiveKit interrupts the agent’s audio output, the plugin sends cancel_utterance; a normal end of speech sends end_utterance.

Error types

All errors derive from FaceModeError (a RuntimeError):

Exception	Raised when
`FaceModeAPIError`	A FaceMode REST call fails, the response is malformed, ingestion times out (240s budget), or the session enters a `FAILED` / `ENDED` state.
`FaceModeProtocolError`	The WebSocket protocol fails: negotiation errors, format changes, server `error` messages, unexpected socket close.
`FaceModeNotReadyError`	`wait_for_join()` times out waiting for the avatar video track, or reconnect negotiation is not acknowledged.

from livekit_plugins_facemode import (
    FaceModeAPIError,
    FaceModeNotReadyError,
    FaceModeProtocolError,
)

Timeouts

Budget	Value
Ingestion readiness polling (`GET /api/sessions/{id}` when `ready: false`)	Up to 240s with bounded backoff; `FAILED` / `ENDED` short-circuit.
WebSocket open handshake	240s.
`start` -> `started` acknowledgement	15s.
`wait_for_join()`	30s default, configurable.

The plugin disables WebSocket compression and native keepalives; after negotiation it sends an application-level ping every 15 seconds.

API URL override

The default API URL is https://api.facemode.io/api. Set api_url (or the FACEMODE_API_URL environment variable in your own config) for a self-hosted or local backend.

Get Started

Concepts

Integrations

LiveKit Agents (Python)

Install

Quickstart

AvatarSession options

start() signature

Room token grants

Input provider

Audio format

Reconnect behavior

Lifecycle and shutdown

Error types

Timeouts

API URL override

​Install

​Quickstart

​AvatarSession options

​start() signature

​Room token grants

​Input provider

​Audio format

​Reconnect behavior

​Lifecycle and shutdown

​Error types

​Timeouts

​API URL override

Install

Quickstart

AvatarSession options

start() signature

Room token grants

Input provider

Audio format

Reconnect behavior

Lifecycle and shutdown

Error types

Timeouts

API URL override