> ## Documentation Index
> Fetch the complete documentation index at: https://docs.camb.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Realtime WebSocket events

> Client and server event contracts for the Realtime WebSocket API.

The Realtime WebSocket API creates a real-time speech translation session over a single WebSocket connection. The client sends a session configuration first, then streams microphone audio with JSON events that contain base64-encoded audio. The server responds with session lifecycle events, transcripts, translated text, and translated audio.

For the interactive WebSocket playground, see [Realtime (WebSocket)](https://docs.camb.ai/api-reference/websockets/realtime.md).

## Endpoint

```http theme={null}
GET /v1/realtime?mode=fast
Host: realtime.camb.ai
x-api-key: <YOUR_API_KEY>
```

The request upgrades to a WebSocket connection. The `mode` query parameter is optional.

| Mode | Notes |
| - | - |
| `fast` | Default. Accepts audio almost immediately and translates with the lowest latency, but supports fewer languages. |
| `slow` | Supports the full language list and translates more accurately, at the cost of a 30s+ cold boot before the session is ready. |

> [!NOTE]
>
> If `session.update.session.mode` is omitted, the server uses the `mode` query parameter. If both are omitted, the server uses `fast`.

> [!NOTE]
>
> **Migrating from `model`.** `mode` replaced an earlier `model` parameter that took the engine codenames `lilac`, `violet`, `iris`, and `orchid`. Both the old parameter and the old values are still accepted, so existing integrations keep working — they resolve as follows, and the server logs a deprecation warning.
>
> | Retired value | Resolves to |
> | - | - |
> | `iris` | `fast` |
> | `lilac`, `violet`, `orchid` | `slow` |
>
> `mode` wins if you send both. A value that was never valid is still rejected — the compatibility mapping covers the four codenames above, not arbitrary strings.

## Session lifecycle

**1. Open the WebSocket**

Connect to `wss://realtime.camb.ai/v1/realtime`. Send your API key in the `x-api-key` request header when your WebSocket client supports custom headers.

**2. Send `session.update` first**

The first WebSocket message must be a JSON `session.update` event. The server waits up to 10 seconds for this initial event.

**3. Wait for session activation**

After authorization and activation, the server sends `session.created`, followed by `session.updated`.

**4. Stream microphone audio**

Send base64-encoded audio chunks with `input_audio_buffer.append`. The server forwards decoded audio bytes into the realtime pipeline.

**5. Read realtime output**

The server emits completed input transcripts, translated text deltas, final translated text, translated audio chunks, and audio completion events.

Only text WebSocket messages are parsed as realtime events. Binary messages and Pong frames are ignored. Ping frames receive Pong replies.

## Authentication

Prefer the `x-api-key` WebSocket request header.

```http theme={null}
x-api-key: <YOUR_API_KEY>
```

You can also send credentials in the initial `session.update` event.

```json API key auth theme={null}
{
  "type": "session.update",
  "session": {
    "mode": "fast",
    "source_language": "en-US",
    "target_language": "de-DE",
    "output_modalities": ["text", "audio"]
  },
  "auth": {
    "api_key": "<YOUR_API_KEY>"
  }
}
```

> [!NOTE]
>
> If the request header and `auth` object are both present, the request header credential is used.

## Limits

| Limit | Value |
| - | - |
| Initial `session.update` timeout | 10 seconds |
| Maximum client event text size | 1 MiB |
| Maximum decoded audio payload per `input_audio_buffer.append` | 256 KiB |
| Audio encoding in `input_audio_buffer.append.audio` | Base64-encoded audio bytes |

## Session configuration

Send the active session settings in `session.update.session`.

```json theme={null}
{
  "mode": "fast",
  "source_language": "en-US",
  "target_language": "de-DE",
  "output_modalities": ["text", "audio"]
}
```

| Field | Type | Required | Notes |
| - | - | - | - |
| `mode` | string | No | Defaults to the `mode` query parameter, then `fast`. Must be `fast` or `slow`. |
| `source_language` | string | Yes | Source language tag, for example `en-US`, or `auto` to [detect it](#automatic-source-language-detection). |
| `target_language` | string | Yes, unless `source_language` is `auto` | Target language tag, for example `de-DE`. With `auto` in `slow` mode it may be omitted until you know it; `fast` mode always needs one. |
| `source_language_candidates` | string\[] | No | Only with `auto`: up to 10 languages detection may choose from. |
| `speaker_setup` | string | No | Only with `auto`: `changing_speakers` (default) or `single_known_speaker`. |
| `output_modalities` | string\[] | No | Defaults to `["text", "audio"]`. |

## Client events

| Event | When to send it | Support |
| - | - | - |
| `session.update` | First message on the connection. In a session started with `auto`, again to change languages. | Required for activation. |
| `input_audio_buffer.append` | After the session is active. | Supported. |
| `input_audio_buffer.ignore_for_language_detection` | In a session started with `auto`, to keep an utterance from counting towards a language switch. | Supported. |
| `input_audio_buffer.clear` | Not available in this version. | Recognized, returns `error`. |
| `input_audio_buffer.commit` | Not available in this version. | Recognized, returns `error`. |
| `response.cancel` | Not available in this version. | Recognized, returns `error`. |

### `session.update`

Initializes the realtime session. This must be the first client event.

```json theme={null}
{
  "type": "session.update",
  "session": {
    "mode": "fast",
    "source_language": "en-US",
    "target_language": "de-DE",
    "output_modalities": ["text", "audio"]
  }
}
```

In a session started with `source_language: "auto"`, a later `session.update` can change `source_language`, `target_language`, or both; see [Automatic source language detection](#automatic-source-language-detection). Any other change after activation is refused with an `error` event, and the session carries on:

```json theme={null}
{
  "type": "error",
  "error": {
    "type": "invalid_request_error",
    "code": "session_update_field_not_supported",
    "message": "Languages can only change in sessions started with `source_language` set to \"auto\".",
    "param": null
  }
}
```

### `input_audio_buffer.append`

Appends audio bytes to the realtime input stream.

```json theme={null}
{
  "type": "input_audio_buffer.append",
  "audio": "<base64_audio_bytes>"
}
```

The server base64-decodes `audio` and forwards the decoded bytes into the realtime pipeline. Each decoded payload can be up to 256 KiB.

### `input_audio_buffer.ignore_for_language_detection`

Only in a session started with `source_language: "auto"`. If your client can tell that an utterance was echo or crosstalk, send its `item_id` (from `conversation.item.input_audio_transcription.completed`) within about half a second, before the switch applies, so it doesn't count towards a language switch.

```json theme={null}
{
  "type": "input_audio_buffer.ignore_for_language_detection",
  "item_id": "7b0d5c1e-..."
}
```

### Unsupported client events

These events are recognized so clients can receive a structured error instead of silent failure.

> [accordion] input\_audio\_buffer.clear
>
> ```json theme={null}
> {
>   "type": "input_audio_buffer.clear"
> }
> ```
>
> Server response:
>
> ```json theme={null}
> {
>   "type": "error",
>   "error": {
>     "message": "input_audio_buffer.clear is not supported in this version"
>   }
> }
> ```

> [accordion] input\_audio\_buffer.commit
>
> ```json theme={null}
> {
>   "type": "input_audio_buffer.commit"
> }
> ```
>
> Server response:
>
> ```json theme={null}
> {
>   "type": "error",
>   "error": {
>     "message": "input_audio_buffer.commit is not supported in this version"
>   }
> }
> ```

> [accordion] response.cancel
>
> ```json theme={null}
> {
>   "type": "response.cancel"
> }
> ```
>
> Server response:
>
> ```json theme={null}
> {
>   "type": "error",
>   "error": {
>     "message": "response.cancel is not supported in this version"
>   }
> }
> ```

## Server events

| Event | Description |
| - | - |
| `session.created` | Sent after the server authorizes, starts, and activates the realtime session. |
| `session.updated` | Sent immediately after `session.created` with the active session configuration. |
| `conversation.item.input_audio_transcription.completed` | Sent when the pipeline produces a completed user transcript. |
| `session.source_language_changed` | Only with `source_language: "auto"`: the detected source language changed. |
| `session.source_language_switch_confirmation_requested` | Only for a `single_known_speaker` session: it heard its own target language and asks before switching. |
| `response.text.delta` | Sent for incremental assistant translation text. Deltas are additive for the current response. |
| `response.text.done` | Sent when the final assistant translation text is available. |
| `response.audio.delta` | Sent when synthesized output audio bytes are available. |
| `response.audio.done` | Sent when the current assistant audio response is complete. |
| `error` | Sent for recognized unsupported client events and billing stop decisions. |

### Session events

`session.created` includes the durable realtime session ID.

```json theme={null}
{
  "type": "session.created",
  "session": {
    "id": "2a4d3cb0-ff62-4e02-a37c-9fcf4e49c8cc",
    "mode": "fast",
    "source_language": "en-US",
    "target_language": "de-DE",
    "output_modalities": ["text", "audio"]
  }
}
```

`session.updated` confirms the active session configuration.

```json theme={null}
{
  "type": "session.updated",
  "session": {
    "mode": "fast",
    "source_language": "en-US",
    "target_language": "de-DE",
    "output_modalities": ["text", "audio"]
  }
}
```

| Field | Type | Notes |
| - | - | - |
| `session.id` | UUID string | Durable realtime session ID. Present on `session.created`. |
| `session.mode` | string | Active mode, `fast` or `slow`. |
| `session.source_language` | string | Source language tag, or `auto` while the source language is detected automatically. |
| `session.target_language` | string | Target language tag. Empty while an `auto` session has no target yet. |
| `session.output_modalities` | string\[] | Active output modalities. |

### Transcript and response events

```json theme={null}
{
  "type": "conversation.item.input_audio_transcription.completed",
  "transcript": "Hello, how are you?"
}
```

```json theme={null}
{
  "type": "response.text.delta",
  "delta": "Guten"
}
```

```json theme={null}
{
  "type": "response.text.done",
  "text": "Guten Tag, wie geht es Ihnen?"
}
```

```json theme={null}
{
  "type": "response.audio.delta",
  "delta": "<base64_audio_bytes>"
}
```

```json theme={null}
{
  "type": "response.audio.done"
}
```

### Automatic source language detection

With `source_language: "auto"`, the server works out which language is being spoken and keeps listening for changes. It reports each change, and labels every completed transcript with the language it was identified as:

```json theme={null}
{
  "type": "session.source_language_changed",
  "language": "de-DE",
  "previous_language": "en-US"
}
```

```json theme={null}
{
  "type": "conversation.item.input_audio_transcription.completed",
  "transcript": "Können Sie mir bitte den Bericht schicken?",
  "item_id": "7b0d5c1e-...",
  "language": "de-DE",
  "passthrough": false
}
```

| Field | Type | Notes |
| - | - | - |
| `language` | string | On `session.source_language_changed`: the new source language. On a transcript: the language it was identified as, absent when it couldn't be identified. |
| `previous_language` | string | The source language before the change. Absent on the first detection. |
| `item_id` | string | Identifies the utterance for `input_audio_buffer.ignore_for_language_detection`. |
| `passthrough` | boolean | `true` when the speech was already in the target language, so nothing was translated or spoken for it. |

A `single_known_speaker` session that hears its own target language for a sustained time asks before switching:

```json theme={null}
{
  "type": "session.source_language_switch_confirmation_requested",
  "language": "de-DE"
}
```

To accept, send `{"type": "session.update", "session": {"source_language": "de-DE"}}`. To reject, do nothing.

Languages can change mid-session with `session.update`:

| Send | Effect |
| - | - |
| `{"target_language": "fr-FR"}` | Translate into French from the next utterance on. |
| `{"target_language": ""}` | Stop translating and only transcribe (`slow` mode only). |
| `{"source_language": "de-DE"}` | Fix the source language. Detection keeps labelling transcripts but stops switching. |
| `{"source_language": "auto"}` | Go back to detecting the source language. |

The server answers with `session.updated`, and with `session.source_language_changed` when the source language changed. A target the session's mode can't translate into is refused with `error.code` `target_language_unsupported_for_mode`, and the previous target stays.

For switching thresholds, speaker setups, and linking two sessions into a call, see [Automatic source language detection](https://docs.camb.ai/api-reference/websockets/realtime.md#automatic-source-language-detection) in the API reference.

### Errors and billing stops

The server sends `error` for recognized but unsupported client events.

```json theme={null}
{
  "type": "error",
  "error": {
    "message": "input_audio_buffer.commit is not supported in this version"
  }
}
```

Billing can also stop a session. In that case, the server sends an `error` event whose `error.message` is the billing close reason, then ends the realtime loop. Active sessions are charged in billing windows and finalized on close, failure, or billing stop.

## Example message flow

```json Client: session.update theme={null}
{
  "type": "session.update",
  "session": {
    "mode": "fast",
    "source_language": "en-US",
    "target_language": "de-DE",
    "output_modalities": ["text", "audio"]
  }
}
```

```json Server: session.created theme={null}
{
  "type": "session.created",
  "session": {
    "id": "2a4d3cb0-ff62-4e02-a37c-9fcf4e49c8cc",
    "mode": "fast",
    "source_language": "en-US",
    "target_language": "de-DE",
    "output_modalities": ["text", "audio"]
  }
}
```

```json Server: session.updated theme={null}
{
  "type": "session.updated",
  "session": {
    "mode": "fast",
    "source_language": "en-US",
    "target_language": "de-DE",
    "output_modalities": ["text", "audio"]
  }
}
```

```json Client: append audio theme={null}
{
  "type": "input_audio_buffer.append",
  "audio": "<base64_audio_bytes>"
}
```

```json Server: transcript theme={null}
{
  "type": "conversation.item.input_audio_transcription.completed",
  "transcript": "Hello, how are you?"
}
```

```json Server: text delta theme={null}
{
  "type": "response.text.delta",
  "delta": "Guten Tag"
}
```

```json Server: text done theme={null}
{
  "type": "response.text.done",
  "text": "Guten Tag, wie geht es Ihnen?"
}
```

```json Server: audio delta theme={null}
{
  "type": "response.audio.delta",
  "delta": "<base64_audio_bytes>"
}
```

```json Server: audio done theme={null}
{
  "type": "response.audio.done"
}
```


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.