> ## Documentation Index > Fetch the complete documentation index at: https://docs.camb.ai/llms.txt > Use this file to discover all available pages before exploring further. # Realtime Speech To Speech > Stream microphone audio into a realtime translation session and receive transcripts, translated text, and translated audio. **Beta.** The Speech to Speech WebSocket is generally available for testing but session events, configuration, and audio formats may change in backwards-incompatible ways before GA. Bidirectional WebSocket endpoint for real-time speech translation. This endpoint is served by `realtime-api-server` at `wss://realtime.camb.ai/v1/realtime`, separate from the `/apis/live-tts/ws` and `/streaming-transcription/listen` WebSocket endpoints. ```http theme={null} GET /v1/realtime Host: realtime.camb.ai x-api-key: ``` The realtime endpoint uses the low-latency `iris` model. Authenticate with the `x-api-key` WebSocket request header. If your client cannot set WebSocket headers, send credentials in the first `session.update` event instead. ## Quickstart Use the **SDK** (Python or TypeScript) — it handles the session lifecycle (including the `session.starting` cold-boot wait), normalizes binary and base64 audio frames, and exposes typed events. Input and output audio are PCM16, mono, 24 kHz. The example below streams a WAV file and writes the translated speech to another WAV. ```python Python theme={null} import asyncio import os import wave from camb.client import CambAI from camb.realtime import ServerEventType from camb.live_transcription import FileAudioSource async def main(): client = CambAI(api_key=os.environ["CAMB_API_KEY"]) session = await client.realtime.connect( source_language="en-us", target_language="de-de", ) out_audio = bytearray() @session.on(ServerEventType.TEXT_DONE) def _(event): print("translation:", event.text) @session.on(ServerEventType.AUDIO_DELTA) def _(event): out_audio.extend(event.data) # raw PCM16 mono 24 kHz async with session: await session.wait_until_ready() # Input WAV must be 16-bit PCM, mono, 24 kHz. await session.stream_audio(FileAudioSource("input_24k_mono.wav", real_time=True)) with wave.open("translated.wav", "wb") as out: out.setnchannels(1) out.setsampwidth(2) out.setframerate(24000) out.writeframes(bytes(out_audio)) asyncio.run(main()) ``` ```typescript TypeScript theme={null} import fs from "node:fs"; import { CambClient, RealtimeServerEventType } from "@camb-ai/sdk"; const client = new CambClient({ apiKey: process.env.CAMB_API_KEY }); const session = await client.realtime.connect({ sourceLanguage: "en-us", targetLanguage: "de-de", }); const outChunks = []; let resolveDone; const audioDone = new Promise((r) => (resolveDone = r)); session.on(RealtimeServerEventType.TextDone, (event) => console.log("translation:", event.text), ); session.on(RealtimeServerEventType.AudioDelta, (event) => outChunks.push(Buffer.from(event.data)), // raw PCM16 mono 24 kHz ); session.on(RealtimeServerEventType.AudioDone, () => resolveDone()); await session.waitUntilReady(); // Input WAV must be 16-bit PCM, mono, 24 kHz. This skips the standard 44-byte // header and streams the PCM data at real-time pace in 100 ms slices. const data = fs.readFileSync("input_24k_mono.wav").subarray(44); const chunkSize = 24000 * 2 * 0.1; for (let i = 0; i < data.length; i += chunkSize) { await session.sendAudio(data.subarray(i, i + chunkSize)); await new Promise((r) => setTimeout(r, 100)); } await Promise.race([audioDone, new Promise((r) => setTimeout(r, 30_000))]); await session.close(); // Raw PCM16 mono 24 kHz; wrap in a WAV header to play it (see the tutorial). fs.writeFileSync("translated.pcm", Buffer.concat(outChunks)); ``` See the [Realtime Speech Translation tutorial](/tutorials/realtime-translation-with-sdk) for the microphone quickstart, the full event list, and configuration. The sections below document the underlying wire protocol for reference (for example, if you are building a client in a language without an SDK). ## Integration in 4 steps Connect to `wss://realtime.camb.ai/v1/realtime`. This endpoint is not under the `client.camb.ai/apis` namespace used by the other WebSocket API references. ```python theme={null} async with websockets.connect( "wss://realtime.camb.ai/v1/realtime", additional_headers=[("x-api-key", "YOUR_API_KEY")], ) as ws: ... ``` The first WebSocket message must be a JSON `session.update` event. The server waits up to 10 seconds for it. ```json theme={null} { "type": "session.update", "session": { "source_language": "en-us", "target_language": "de-de", "output_modalities": ["text", "audio"] } } ``` The server responds with `session.created`, then `session.updated`. Send microphone audio as base64-encoded bytes in `input_audio_buffer.append`. Only text WebSocket messages are parsed as realtime events. ```json theme={null} { "type": "input_audio_buffer.append", "audio": "" } ``` Each decoded audio payload can be up to 256 KiB. Listen for transcript, translated text, and translated audio events. `response.text.delta` values are additive for the current response, and `response.audio.delta` contains base64-encoded synthesized audio bytes. ## Authentication Prefer the WebSocket request header: ```http theme={null} x-api-key: ``` The initial `session.update` event can also carry credentials: ```json API key auth theme={null} { "type": "session.update", "session": { "source_language": "en-us", "target_language": "de-de", "output_modalities": ["text", "audio"] }, "auth": { "api_key": "" } } ``` If both the request header and `auth` object are present, the request header credential is used. ## Reference The AsyncAPI spec above documents every client and server event. Quick lookup: ### Session configuration | Field | Type | Required | Notes | | :------------------ | :-------- | :------- | :---------------------------------------------------------------------------------------------------- | | `source_language` | string | Yes | Source language tag, for example `en-us`. Must be a [supported language](#supported-languages). | | `target_language` | string | Yes | Target language tag, for example `de-de`. Must be a [supported language](#supported-languages). | | `output_modalities` | string\[] | No | Defaults to `["text", "audio"]`. | | `voice` | object | No | Output voice selection. Defaults to `{ "type": "default" }`. See [Voice selection](#voice-selection). | ### Voice selection By default, translated speech is synthesized with a built-in voice for the target language. To synthesize the translation with one of your own cloned voices, include a `voice` object in the session configuration: ```json theme={null} { "type": "session.update", "session": { "source_language": "en-us", "target_language": "de-de", "output_modalities": ["text", "audio"], "voice": { "type": "cloned", "voice_id": 147320 } } } ``` | Field | Type | Required | Notes | | :--------- | :------ | :------------------------ | :----------------------------------------------------------------------------------------------------------------------------------------------------------- | | `type` | string | Yes | `"default"` to use the built-in voice, or `"cloned"` to use one of your voices. | | `voice_id` | integer | When `type` is `"cloned"` | ID of a voice you own. Get it from [List Voices](/api-reference/endpoint/list-voices) or [Create Custom Voice](/api-reference/endpoint/create-custom-voice). | The voice must belong to your account — stock/built-in voice IDs are rejected with an `error` event. Omitting `voice` (or sending `{ "type": "default" }`) uses the built-in voice. The resolved selection is echoed back in `session.created`. If you use the SDK, pass `voice_id` (Python) or `voiceId` (TypeScript) to `realtime.connect()` and it builds this `voice` object for you: ```python Python theme={null} session = await client.realtime.connect( source_language="en-us", target_language="de-de", voice_id=147320, ) ``` ```typescript TypeScript theme={null} const session = await client.realtime.connect({ sourceLanguage: "en-us", targetLanguage: "de-de", voiceId: 147320, }); ``` For the most natural-sounding results, choose a voice whose reference language matches your `target_language`. A large mismatch between the voice's native language and the translation language can reduce clarity and accent accuracy. ### Supported languages `source_language` and `target_language` accept the BCP-47 tags below (case-insensitive). Pick any supported language as the source and any supported language as the target. | Code | Language | | :------ | :----------------------------- | | `ar-ae` | Arabic (United Arab Emirates) | | `ar-eg` | Arabic (Egypt) | | `ar-sa` | Arabic (Saudi Arabia) | | `de-de` | German (Germany) | | `en-gb` | English (United Kingdom) | | `en-us` | English (United States) | | `es-es` | Spanish (Spain) | | `fr-ca` | French (Canada) | | `fr-fr` | French (France) | | `hi-in` | Hindi (India) | | `ja-jp` | Japanese (Japan) | | `ko-kr` | Korean (Korea) | | `pt-br` | Portuguese (Brazil) | | `zh-cn` | Chinese (Mandarin, Simplified) | ### Client events | Event | Support | | :-------------------------- | :--------------------------------------------- | | `session.update` | Required first message. | | `input_audio_buffer.append` | Supported after activation. | | `input_audio_buffer.clear` | Recognized but not supported; returns `error`. | | `input_audio_buffer.commit` | Recognized but not supported; returns `error`. | | `response.cancel` | Recognized but not supported; returns `error`. | ### Server events | Event | Description | | :------------------------------------------------------ | :----------------------------------------------------- | | `session.created` | Session has been authorized, started, and activated. | | `session.updated` | Active session configuration. | | `conversation.item.input_audio_transcription.completed` | Completed user transcript. | | `response.text.delta` | Additive translated text delta. | | `response.text.done` | Final translated text. | | `response.audio.delta` | Base64-encoded translated audio bytes. | | `response.audio.done` | Current translated audio response is complete. | | `error` | Unsupported recognized event or billing stop decision. | ### Limits | Limit | Value | | :------------------------------------------------------------ | :------------------------- | | Initial `session.update` timeout | 10 seconds | | Maximum client event text size | 1 MiB | | Maximum decoded audio payload per `input_audio_buffer.append` | 256 KiB | | Audio encoding in `input_audio_buffer.append.audio` | Base64-encoded audio bytes | ### Billing Active sessions are charged in billing windows and finalized on close, failure, or billing stop. If billing stops a session, the server sends an `error` event whose `error.message` is the billing close reason, then ends the realtime loop. ## AsyncAPI ````yaml api-reference/websockets/realtime-asyncapi.json realtime id: realtime title: Realtime description: '' servers: - id: production protocol: wss host: realtime.camb.ai bindings: [] variables: [] address: /v1/realtime parameters: [] bindings: [] operations: - &ref_3 id: clientSendEvents title: Client send events description: Client sends session configuration and audio input events type: send messages: - &ref_12 id: SessionUpdate contentType: application/json payload: - name: Update Session description: First client event. Authorizes and activates the realtime session. type: object properties: - name: type type: string description: session.update required: true - name: session type: object required: true properties: - name: source_language type: string description: Source language tag, for example `en-US`. required: true - name: target_language type: string description: Target language tag, for example `de-DE`. required: true - name: output_modalities type: array required: false properties: - name: item type: string enumValues: - text - audio required: false - name: voice type: object description: >- Output voice selection. Use the built-in voice or one of your cloned voices. required: false - name: auth type: object required: false properties: - name: api_key type: string required: true headers: [] jsonPayloadSchema: type: object properties: type: type: string const: session.update x-parser-schema-id: session: &ref_0 type: object properties: source_language: type: string description: Source language tag, for example `en-US`. example: en-US x-parser-schema-id: target_language: type: string description: Target language tag, for example `de-DE`. example: de-DE x-parser-schema-id: output_modalities: type: array items: type: string enum: - text - audio x-parser-schema-id: default: - text - audio x-parser-schema-id: voice: type: object description: >- Output voice selection. Use the built-in voice or one of your cloned voices. oneOf: - type: object title: Default voice properties: type: type: string const: default x-parser-schema-id: required: - type x-parser-schema-id: - type: object title: Cloned voice properties: type: type: string const: cloned x-parser-schema-id: voice_id: type: integer format: int64 description: >- ID of a voice you own, from List Voices or Create Custom Voice. Stock voice IDs are rejected. example: 12345 x-parser-schema-id: required: - type - voice_id x-parser-schema-id: x-parser-schema-id: VoiceConfig required: - source_language - target_language x-parser-schema-id: SessionConfig auth: type: object properties: api_key: type: string x-parser-schema-id: required: - api_key x-parser-schema-id: Auth required: - type - session x-parser-schema-id: SessionUpdatePayload title: Update Session description: First client event. Authorizes and activates the realtime session. example: No examples found bindings: [] extensions: - id: x-parser-unique-object-id value: SessionUpdate - &ref_13 id: InputAudioBufferAppend contentType: application/json payload: - name: Append Input Audio description: >- Append base64-encoded microphone audio bytes to the realtime input stream. type: object properties: - name: type type: string description: input_audio_buffer.append required: true - name: audio type: string description: >- Base64-encoded audio bytes. The decoded payload can be up to 256 KiB. required: true headers: [] jsonPayloadSchema: type: object properties: type: type: string const: input_audio_buffer.append x-parser-schema-id: audio: type: string contentEncoding: base64 description: >- Base64-encoded audio bytes. The decoded payload can be up to 256 KiB. x-parser-schema-id: required: - type - audio x-parser-schema-id: InputAudioBufferAppendPayload title: Append Input Audio description: >- Append base64-encoded microphone audio bytes to the realtime input stream. example: |- { "type": "", "audio": "" } bindings: [] extensions: - id: x-parser-unique-object-id value: InputAudioBufferAppend - &ref_14 id: InputAudioBufferClear contentType: application/json payload: - name: Clear Input Audio Buffer description: >- Recognized but not supported in this version. The server responds with an error event. type: object properties: - name: type type: string description: input_audio_buffer.clear required: true headers: [] jsonPayloadSchema: type: object properties: type: type: string const: input_audio_buffer.clear x-parser-schema-id: required: - type x-parser-schema-id: InputAudioBufferClearPayload title: Clear Input Audio Buffer description: >- Recognized but not supported in this version. The server responds with an error event. example: |- { "type": "" } bindings: [] extensions: - id: x-parser-unique-object-id value: InputAudioBufferClear - &ref_15 id: InputAudioBufferCommit contentType: application/json payload: - name: Commit Input Audio Buffer description: >- Recognized but not supported in this version. The server responds with an error event. type: object properties: - name: type type: string description: input_audio_buffer.commit required: true headers: [] jsonPayloadSchema: type: object properties: type: type: string const: input_audio_buffer.commit x-parser-schema-id: required: - type x-parser-schema-id: InputAudioBufferCommitPayload title: Commit Input Audio Buffer description: >- Recognized but not supported in this version. The server responds with an error event. example: |- { "type": "" } bindings: [] extensions: - id: x-parser-unique-object-id value: InputAudioBufferCommit - &ref_16 id: ResponseCancel contentType: application/json payload: - name: Cancel Response description: >- Recognized but not supported in this version. The server responds with an error event. type: object properties: - name: type type: string description: response.cancel required: true headers: [] jsonPayloadSchema: type: object properties: type: type: string const: response.cancel x-parser-schema-id: required: - type x-parser-schema-id: ResponseCancelPayload title: Cancel Response description: >- Recognized but not supported in this version. The server responds with an error event. example: |- { "type": "" } bindings: [] extensions: - id: x-parser-unique-object-id value: ResponseCancel bindings: [] extensions: &ref_1 - id: x-parser-unique-object-id value: realtime - &ref_2 id: serverSendEvents title: Server send events description: Server sends session lifecycle, transcript, text, audio, and error events type: receive messages: - &ref_4 id: SessionCreated contentType: application/json payload: - name: Session Created description: Sent after authorization, startup, and activation complete. type: object properties: - name: type type: string description: session.created required: true - name: session type: object required: true properties: - name: source_language type: string description: Source language tag, for example `en-US`. required: true - name: target_language type: string description: Target language tag, for example `de-DE`. required: true - name: output_modalities type: array required: false properties: - name: item type: string enumValues: - text - audio required: false - name: voice type: object description: >- Output voice selection. Use the built-in voice or one of your cloned voices. required: false - name: id type: string description: Durable realtime session ID. required: true headers: [] jsonPayloadSchema: type: object properties: type: type: string const: session.created x-parser-schema-id: session: allOf: - *ref_0 - type: object properties: id: type: string format: uuid description: Durable realtime session ID. x-parser-schema-id: required: - id x-parser-schema-id: x-parser-schema-id: required: - type - session x-parser-schema-id: SessionCreatedPayload title: Session Created description: Sent after authorization, startup, and activation complete. example: No examples found bindings: [] extensions: - id: x-parser-unique-object-id value: SessionCreated - &ref_5 id: SessionUpdated contentType: application/json payload: - name: Session Updated description: >- Sent immediately after session.created with the active session configuration. type: object properties: - name: type type: string description: session.updated required: true - name: session type: object required: true properties: - name: source_language type: string description: Source language tag, for example `en-US`. required: true - name: target_language type: string description: Target language tag, for example `de-DE`. required: true - name: output_modalities type: array required: false properties: - name: item type: string enumValues: - text - audio required: false - name: voice type: object description: >- Output voice selection. Use the built-in voice or one of your cloned voices. required: false headers: [] jsonPayloadSchema: type: object properties: type: type: string const: session.updated x-parser-schema-id: session: *ref_0 required: - type - session x-parser-schema-id: SessionUpdatedPayload title: Session Updated description: >- Sent immediately after session.created with the active session configuration. example: No examples found bindings: [] extensions: - id: x-parser-unique-object-id value: SessionUpdated - &ref_6 id: InputAudioTranscriptionCompleted contentType: application/json payload: - name: Input Audio Transcription Completed description: Completed user transcript produced by the realtime pipeline. type: object properties: - name: type type: string description: conversation.item.input_audio_transcription.completed required: true - name: transcript type: string required: true headers: [] jsonPayloadSchema: type: object properties: type: type: string const: conversation.item.input_audio_transcription.completed x-parser-schema-id: transcript: type: string x-parser-schema-id: required: - type - transcript x-parser-schema-id: InputAudioTranscriptionCompletedPayload title: Input Audio Transcription Completed description: Completed user transcript produced by the realtime pipeline. example: |- { "type": "", "transcript": "" } bindings: [] extensions: - id: x-parser-unique-object-id value: InputAudioTranscriptionCompleted - &ref_7 id: ResponseTextDelta contentType: application/json payload: - name: Response Text Delta description: >- Incremental translated text. The delta is additive for the current response. type: object properties: - name: type type: string description: response.text.delta required: true - name: delta type: string description: Additive translated text delta for the current response. required: true headers: [] jsonPayloadSchema: type: object properties: type: type: string const: response.text.delta x-parser-schema-id: delta: type: string description: Additive translated text delta for the current response. x-parser-schema-id: required: - type - delta x-parser-schema-id: ResponseTextDeltaPayload title: Response Text Delta description: >- Incremental translated text. The delta is additive for the current response. example: |- { "type": "", "delta": "" } bindings: [] extensions: - id: x-parser-unique-object-id value: ResponseTextDelta - &ref_8 id: ResponseTextDone contentType: application/json payload: - name: Response Text Done description: Final translated text for the current response. type: object properties: - name: type type: string description: response.text.done required: true - name: text type: string description: Final translated text. required: true headers: [] jsonPayloadSchema: type: object properties: type: type: string const: response.text.done x-parser-schema-id: text: type: string description: Final translated text. x-parser-schema-id: required: - type - text x-parser-schema-id: ResponseTextDonePayload title: Response Text Done description: Final translated text for the current response. example: |- { "type": "", "text": "" } bindings: [] extensions: - id: x-parser-unique-object-id value: ResponseTextDone - &ref_9 id: ResponseAudioDelta contentType: application/json payload: - name: Response Audio Delta description: Base64-encoded synthesized output audio bytes. type: object properties: - name: type type: string description: response.audio.delta required: true - name: delta type: string description: Base64-encoded synthesized output audio bytes. required: true headers: [] jsonPayloadSchema: type: object properties: type: type: string const: response.audio.delta x-parser-schema-id: delta: type: string contentEncoding: base64 description: Base64-encoded synthesized output audio bytes. x-parser-schema-id: required: - type - delta x-parser-schema-id: ResponseAudioDeltaPayload title: Response Audio Delta description: Base64-encoded synthesized output audio bytes. example: |- { "type": "", "delta": "" } bindings: [] extensions: - id: x-parser-unique-object-id value: ResponseAudioDelta - &ref_10 id: ResponseAudioDone contentType: application/json payload: - name: Response Audio Done description: Current assistant audio response is complete. type: object properties: - name: type type: string description: response.audio.done required: true headers: [] jsonPayloadSchema: type: object properties: type: type: string const: response.audio.done x-parser-schema-id: required: - type x-parser-schema-id: ResponseAudioDonePayload title: Response Audio Done description: Current assistant audio response is complete. example: |- { "type": "" } bindings: [] extensions: - id: x-parser-unique-object-id value: ResponseAudioDone - &ref_11 id: Error contentType: application/json payload: - name: Error description: >- Structured error for unsupported recognized events and billing stop decisions. type: object properties: - name: type type: string description: error required: true - name: error type: object required: true properties: - name: message type: string required: true headers: [] jsonPayloadSchema: type: object properties: type: type: string const: error x-parser-schema-id: error: type: object properties: message: type: string x-parser-schema-id: required: - message x-parser-schema-id: required: - type - error x-parser-schema-id: ErrorPayload title: Error description: >- Structured error for unsupported recognized events and billing stop decisions. example: |- { "type": "", "error": { "message": "" } } bindings: [] extensions: - id: x-parser-unique-object-id value: Error bindings: [] extensions: *ref_1 sendOperations: - *ref_2 receiveOperations: - *ref_3 sendMessages: - *ref_4 - *ref_5 - *ref_6 - *ref_7 - *ref_8 - *ref_9 - *ref_10 - *ref_11 receiveMessages: - *ref_12 - *ref_13 - *ref_14 - *ref_15 - *ref_16 extensions: - id: x-parser-unique-object-id value: realtime securitySchemes: [] ````