Realtime WebSocket events

The Realtime WebSocket API creates a real-time speech translation session over a single WebSocket connection. The client sends a session configuration first, then streams microphone audio with JSON events that contain base64-encoded audio. The server responds with session lifecycle events, transcripts, translated text, and translated audio. For the interactive WebSocket playground, see Realtime (WebSocket).

Endpoint

GET /v1/realtime?mode=fast
Host: realtime.camb.ai
x-api-key: <YOUR_API_KEY>

The request upgrades to a WebSocket connection. The mode query parameter is optional.

Mode	Notes
`fast`	Default. Accepts audio almost immediately and translates with the lowest latency, but supports fewer languages.
`slow`	Supports the full language list and translates more accurately, at the cost of a 30s+ cold boot before the session is ready.

If session.update.session.mode is omitted, the server uses the mode query parameter. If both are omitted, the server uses fast.

Migrating from model. mode replaced an earlier model parameter that took the engine codenames lilac, violet, iris, and orchid. Both the old parameter and the old values are still accepted, so existing integrations keep working — they resolve as follows, and the server logs a deprecation warning.

Retired value	Resolves to
`iris`	`fast`
`lilac`, `violet`, `orchid`	`slow`

mode wins if you send both. A value that was never valid is still rejected — the compatibility mapping covers the four codenames above, not arbitrary strings.

Session lifecycle

Open the WebSocket

Connect to wss://realtime.camb.ai/v1/realtime. Send your API key in the x-api-key request header when your WebSocket client supports custom headers.

Send session.update first

The first WebSocket message must be a JSON session.update event. The server waits up to 10 seconds for this initial event.

Wait for session activation

After authorization and activation, the server sends session.created, followed by session.updated.

Stream microphone audio

Send base64-encoded audio chunks with input_audio_buffer.append. The server forwards decoded audio bytes into the realtime pipeline.

Read realtime output

The server emits completed input transcripts, translated text deltas, final translated text, translated audio chunks, and audio completion events.

Only text WebSocket messages are parsed as realtime events. Binary messages and Pong frames are ignored. Ping frames receive Pong replies.

Authentication

Prefer the x-api-key WebSocket request header.

x-api-key: <YOUR_API_KEY>

You can also send credentials in the initial session.update event.

{
  "type": "session.update",
  "session": {
    "mode": "fast",
    "source_language": "en-US",
    "target_language": "de-DE",
    "output_modalities": ["text", "audio"]
  },
  "auth": {
    "api_key": "<YOUR_API_KEY>"
  }
}

If the request header and auth object are both present, the request header credential is used.

Limits

Limit	Value
Initial `session.update` timeout	10 seconds
Maximum client event text size	1 MiB
Maximum decoded audio payload per `input_audio_buffer.append`	256 KiB
Audio encoding in `input_audio_buffer.append.audio`	Base64-encoded audio bytes

Session configuration

Send the active session settings in session.update.session.

{
  "mode": "fast",
  "source_language": "en-US",
  "target_language": "de-DE",
  "output_modalities": ["text", "audio"]
}

Field	Type	Required	Notes
`mode`	string	No	Defaults to the `mode` query parameter, then `fast`. Must be `fast` or `slow`.
`source_language`	string	Yes	Source language tag, for example `en-US`, or `auto` to detect it.
`target_language`	string	Yes, unless `source_language` is `auto`	Target language tag, for example `de-DE`. With `auto` in `slow` mode it may be omitted until you know it; `fast` mode always needs one.
`source_language_candidates`	string[]	No	Only with `auto`: up to 10 languages detection may choose from.
`speaker_setup`	string	No	Only with `auto`: `changing_speakers` (default) or `single_known_speaker`.
`output_modalities`	string[]	No	Defaults to `["text", "audio"]`.

Client events

Event	When to send it	Support
`session.update`	First message on the connection. In a session started with `auto`, again to change languages.	Required for activation.
`input_audio_buffer.append`	After the session is active.	Supported.
`input_audio_buffer.ignore_for_language_detection`	In a session started with `auto`, to keep an utterance from counting towards a language switch.	Supported.
`input_audio_buffer.clear`	Not available in this version.	Recognized, returns `error`.
`input_audio_buffer.commit`	Not available in this version.	Recognized, returns `error`.
`response.cancel`	Not available in this version.	Recognized, returns `error`.

`session.update`

Initializes the realtime session. This must be the first client event.

{
  "type": "session.update",
  "session": {
    "mode": "fast",
    "source_language": "en-US",
    "target_language": "de-DE",
    "output_modalities": ["text", "audio"]
  }
}

In a session started with source_language: "auto", a later session.update can change source_language, target_language, or both; see Automatic source language detection. Any other change after activation is refused with an error event, and the session carries on:

{
  "type": "error",
  "error": {
    "type": "invalid_request_error",
    "code": "session_update_field_not_supported",
    "message": "Languages can only change in sessions started with `source_language` set to \"auto\".",
    "param": null
  }
}

`input_audio_buffer.append`

Appends audio bytes to the realtime input stream.

{
  "type": "input_audio_buffer.append",
  "audio": "<base64_audio_bytes>"
}

The server base64-decodes audio and forwards the decoded bytes into the realtime pipeline. Each decoded payload can be up to 256 KiB.

`input_audio_buffer.ignore_for_language_detection`

Only in a session started with source_language: "auto". If your client can tell that an utterance was echo or crosstalk, send its item_id (from conversation.item.input_audio_transcription.completed) within about half a second, before the switch applies, so it doesn’t count towards a language switch.

{
  "type": "input_audio_buffer.ignore_for_language_detection",
  "item_id": "7b0d5c1e-..."
}

Unsupported client events

These events are recognized so clients can receive a structured error instead of silent failure.

input_audio_buffer.clear

{
  "type": "input_audio_buffer.clear"
}

Server response:

{
  "type": "error",
  "error": {
    "message": "input_audio_buffer.clear is not supported in this version"
  }
}

input_audio_buffer.commit

{
  "type": "input_audio_buffer.commit"
}

Server response:

{
  "type": "error",
  "error": {
    "message": "input_audio_buffer.commit is not supported in this version"
  }
}

response.cancel

{
  "type": "response.cancel"
}

Server response:

{
  "type": "error",
  "error": {
    "message": "response.cancel is not supported in this version"
  }
}

Server events

Event	Description
`session.created`	Sent after the server authorizes, starts, and activates the realtime session.
`session.updated`	Sent immediately after `session.created` with the active session configuration.
`conversation.item.input_audio_transcription.completed`	Sent when the pipeline produces a completed user transcript.
`session.source_language_changed`	Only with `source_language: "auto"`: the detected source language changed.
`session.source_language_switch_confirmation_requested`	Only for a `single_known_speaker` session: it heard its own target language and asks before switching.
`response.text.delta`	Sent for incremental assistant translation text. Deltas are additive for the current response.
`response.text.done`	Sent when the final assistant translation text is available.
`response.audio.delta`	Sent when synthesized output audio bytes are available.
`response.audio.done`	Sent when the current assistant audio response is complete.
`error`	Sent for recognized unsupported client events and billing stop decisions.

Session events

session.created includes the durable realtime session ID.

{
  "type": "session.created",
  "session": {
    "id": "2a4d3cb0-ff62-4e02-a37c-9fcf4e49c8cc",
    "mode": "fast",
    "source_language": "en-US",
    "target_language": "de-DE",
    "output_modalities": ["text", "audio"]
  }
}

session.updated confirms the active session configuration.

{
  "type": "session.updated",
  "session": {
    "mode": "fast",
    "source_language": "en-US",
    "target_language": "de-DE",
    "output_modalities": ["text", "audio"]
  }
}

Field	Type	Notes
`session.id`	UUID string	Durable realtime session ID. Present on `session.created`.
`session.mode`	string	Active mode, `fast` or `slow`.
`session.source_language`	string	Source language tag, or `auto` while the source language is detected automatically.
`session.target_language`	string	Target language tag. Empty while an `auto` session has no target yet.
`session.output_modalities`	string[]	Active output modalities.

Transcript and response events

{
  "type": "conversation.item.input_audio_transcription.completed",
  "transcript": "Hello, how are you?"
}

{
  "type": "response.text.delta",
  "delta": "Guten"
}

{
  "type": "response.text.done",
  "text": "Guten Tag, wie geht es Ihnen?"
}

{
  "type": "response.audio.delta",
  "delta": "<base64_audio_bytes>"
}

{
  "type": "response.audio.done"
}

Automatic source language detection

With source_language: "auto", the server works out which language is being spoken and keeps listening for changes. It reports each change, and labels every completed transcript with the language it was identified as:

{
  "type": "session.source_language_changed",
  "language": "de-DE",
  "previous_language": "en-US"
}

{
  "type": "conversation.item.input_audio_transcription.completed",
  "transcript": "Können Sie mir bitte den Bericht schicken?",
  "item_id": "7b0d5c1e-...",
  "language": "de-DE",
  "passthrough": false
}

Field	Type	Notes
`language`	string	On `session.source_language_changed`: the new source language. On a transcript: the language it was identified as, absent when it couldn’t be identified.
`previous_language`	string	The source language before the change. Absent on the first detection.
`item_id`	string	Identifies the utterance for `input_audio_buffer.ignore_for_language_detection`.
`passthrough`	boolean	`true` when the speech was already in the target language, so nothing was translated or spoken for it.

A single_known_speaker session that hears its own target language for a sustained time asks before switching:

{
  "type": "session.source_language_switch_confirmation_requested",
  "language": "de-DE"
}

To accept, send {"type": "session.update", "session": {"source_language": "de-DE"}}. To reject, do nothing. Languages can change mid-session with session.update:

Send	Effect
`{"target_language": "fr-FR"}`	Translate into French from the next utterance on.
`{"target_language": ""}`	Stop translating and only transcribe (`slow` mode only).
`{"source_language": "de-DE"}`	Fix the source language. Detection keeps labelling transcripts but stops switching.
`{"source_language": "auto"}`	Go back to detecting the source language.

The server answers with session.updated, and with session.source_language_changed when the source language changed. A target the session’s mode can’t translate into is refused with error.code target_language_unsupported_for_mode, and the previous target stays. For switching thresholds, speaker setups, and linking two sessions into a call, see Automatic source language detection in the API reference.

Errors and billing stops

The server sends error for recognized but unsupported client events.

{
  "type": "error",
  "error": {
    "message": "input_audio_buffer.commit is not supported in this version"
  }
}

Billing can also stop a session. In that case, the server sends an error event whose error.message is the billing close reason, then ends the realtime loop. Active sessions are charged in billing windows and finalized on close, failure, or billing stop.

Example message flow

{
  "type": "session.update",
  "session": {
    "mode": "fast",
    "source_language": "en-US",
    "target_language": "de-DE",
    "output_modalities": ["text", "audio"]
  }
}

{
  "type": "session.created",
  "session": {
    "id": "2a4d3cb0-ff62-4e02-a37c-9fcf4e49c8cc",
    "mode": "fast",
    "source_language": "en-US",
    "target_language": "de-DE",
    "output_modalities": ["text", "audio"]
  }
}

{
  "type": "session.updated",
  "session": {
    "mode": "fast",
    "source_language": "en-US",
    "target_language": "de-DE",
    "output_modalities": ["text", "audio"]
  }
}

{
  "type": "input_audio_buffer.append",
  "audio": "<base64_audio_bytes>"
}

{
  "type": "conversation.item.input_audio_transcription.completed",
  "transcript": "Hello, how are you?"
}

{
  "type": "response.text.delta",
  "delta": "Guten Tag"
}

{
  "type": "response.text.done",
  "text": "Guten Tag, wie geht es Ihnen?"
}

{
  "type": "response.audio.delta",
  "delta": "<base64_audio_bytes>"
}

{
  "type": "response.audio.done"
}

Getting Started

Models

Tutorials

SDK Guides

Hosting Platforms

Integrations

API Reference

Other Products

Release Logs

Realtime WebSocket events

Endpoint

Session lifecycle

Authentication

Limits

Session configuration

Client events

`session.update`

`input_audio_buffer.append`

`input_audio_buffer.ignore_for_language_detection`

Unsupported client events

Server events

Session events

Transcript and response events

Automatic source language detection

Errors and billing stops

Example message flow

​Endpoint

​Session lifecycle

​Authentication

​Limits

​Session configuration

​Client events

​session.update

​input_audio_buffer.append

​input_audio_buffer.ignore_for_language_detection

​Unsupported client events

​Server events

​Session events

​Transcript and response events

​Automatic source language detection

​Errors and billing stops

​Example message flow

Endpoint

Session lifecycle

Authentication

Limits

Session configuration

Client events

`session.update`

`input_audio_buffer.append`

`input_audio_buffer.ignore_for_language_detection`

Unsupported client events

Server events

Session events

Transcript and response events

Automatic source language detection

Errors and billing stops

Example message flow