# LiveKit Agents

Use Svara as the TTS service in a LiveKit Agents pipeline.

## Overview

[LiveKit Agents](https://docs.livekit.io/agents/) builds voice agents out of three swappable legs — STT, LLM, TTS — plus VAD and end-of-turn detection. Svara plugs into the TTS leg through the official Python SDK: `from svara.livekit import TTS`. By default it rides the [input-streaming WebSocket](https://docs.kenpathlabs.com/input-streaming.md), so the agent starts speaking while the LLM is still writing its sentence.

```
mic ─▶ VAD ─▶ STT ─▶ LLM ─┬▶  Svara TTS (eager WS)  ─▶ speaker
                          └── token stream, not sentences
```

The plugin emits **24 kHz, 16-bit, mono PCM** straight into LiveKit’s `AudioEmitter` — no decode, no resample on the hot path — and pins the platform-certified sampling defaults so the agent sounds exactly like the API it fronts.

## The TTS plugin

Two synthesis paths, chosen by `mode`:

- `mode="eager"` (default) — forwards the LLM token stream into the input-streaming WebSocket; speech starts a few words in and prosody stays continuous across the whole turn. Measured roughly **2× faster to first audio** than the sentence-buffered path in the same pipeline.
- `mode="http"` — buffers whole sentences and synthesizes each over streaming HTTP. Used automatically for fixed-text `say()`, and available as the conservative fallback for full turns.

```bash
pip install "svara-voice[livekit] @ git+https://github.com/kenpath-labs/svara-python.git"
```

```python
from svara.livekit import TTS

tts = TTS(voice="sv_enhdbrj5")            # Aanya; eager WS mode by default
# tts = TTS(voice="sv_enhdbrj5", mode="http")   # sentence-buffered fallback
# tts = TTS(voice="sv_enhdbrj5", language="hi", pronunciation_dictionary_id="pd_...")
```

> The plugin’s source is small and readable — [svara/livekit/tts.py](https://github.com/kenpath-labs/svara-python/blob/main/src/svara/livekit/tts.py) in the SDK repo — if you need to fork its behavior rather than configure it.

## Wiring the agent

Pass the plugin as `tts=` on the `AgentSession`. Every other leg is a stock plugin, and each is independently swappable — nothing below is Svara-specific.

```bash
pip install 'livekit-agents[openai,silero,turn-detector]'
```

```python
from livekit import agents
from livekit.agents import Agent, AgentServer, AgentSession, JobContext, JobProcess
from livekit.plugins import openai, silero

from svara.livekit import TTS

server = AgentServer(setup_fnc=lambda proc: proc.userdata.__setitem__(
    "vad", silero.VAD.load()))


@server.rtc_session(agent_name="svara-agent")
async def entrypoint(ctx: JobContext) -> None:
    await ctx.connect()

    session = AgentSession(
        vad=ctx.proc.userdata["vad"],
        stt=openai.STT(model="gpt-4o-mini-transcribe"),
        llm=openai.LLM(model="gpt-4.1-mini"),
        tts=TTS(voice="sv_enhdbrj5"),      # any id from GET /v1/voices
    )
    await session.start(
        Agent(instructions="You are a helpful voice assistant. "
                           "Reply in one or two short sentences."),
        room=ctx.room,
    )
    await session.generate_reply(instructions="Greet the caller in one sentence.")


if __name__ == "__main__":
    agents.cli.run_app(server)
```

| Parameter | Type | Default | Description |
|---|---|---|---|
| `SVARA_API_KEY` | env | - | Your Svara key (the SDK reads it automatically). |
| `SVARA_BASE_URL` | env | `https://api.kenpathlabs.com` | The gateway origin; override for a regional pin (api-in, api-us, api-eu). |
| `LIVEKIT_URL / _API_KEY / _API_SECRET` | env | - | Your LiveKit server or Cloud project. |

Run `python agent.py download-files` once (it fetches the VAD and turn-detector models), then `python agent.py dev`.

## Streaming from the LLM

With `mode="eager"` the plugin declares `streaming=True` and LiveKit hands it the raw token stream; the plugin forwards each delta over one WebSocket and pushes the binary PCM frames that come back. Nothing to implement — the protocol details in [Input streaming](https://docs.kenpathlabs.com/input-streaming.md) are what the plugin speaks for you.

Prefer sentence-at-a-time synthesis for a specific agent? Construct the plugin with `mode="http"`, or flip it at runtime with `tts.update_options(mode="http")`.

## Gotchas

- **Use the gateway host.** `https://api.kenpathlabs.com` serves the API; the console host does not.
- **Don’t resample.** The plugin declares `sample_rate=24000` and LiveKit’s emitter takes the raw PCM; LiveKit itself resamples for an 8 kHz telephony leg. Anything you add on top only brings latency and artifacts.
- **429s are already mapped.** The plugin converts Svara’s ElevenLabs-shaped errors into LiveKit’s `APIStatusError` with the right retryability, so the agent backs off on `rate_limit_exceeded` and `too_many_concurrent_requests` instead of dropping the turn. `insufficient_quota` is terminal until your quota resets.
- **Use `sv_` voice ids in code.** Ids like `sv_enhdbrj5` come from [GET /v1/voices](https://docs.kenpathlabs.com/voices.md) and are stable across renames.
- **Leave the sampling knobs alone.** The plugin pins the platform-certified sampling; overriding it per-agent is how long utterances start to ramble.