Realtime

Overview

Realtime voice uses WebRTC for microphone input and synthesized audio output. One established session remains conversational across multiple turns; clients do not submit a separate HTTP request for every utterance.

Realtime model

Model	Transport	Input	Output
GPT Live amanai/gpt-live	WebRTC	Microphone audio and session events	Streaming audio and transcript events

Session object

const session = {
  type: "realtime",
  model: "amanai/gpt-live",
  language: "id-ID",
  instructions: "Speak naturally and keep answers concise.",
  audio: { output: { voice: "cove" } },
  delegation: { type: "client", ack_filler: false }
};

Language presets: English (US) (en-US), English (UK) (en-GB), Indonesian (id-ID), Malay (ms-MY), Arabic (ar-SA), Czech (cs-CZ), Danish (da-DK), Dutch (nl-NL), Filipino (fil-PH), Chinese (Simplified) (zh-CN), Japanese (ja-JP), Korean (ko-KR), Spanish (es-ES), French (fr-FR), German (de-DE), Italian (it-IT), Macedonian (mk-MK), Persian (fa-IR), Polish (pl-PL), Portuguese (Brazil) (pt-BR), Romanian (ro-RO), Russian (ru-RU), Swedish (sv-SE), Thai (th-TH), Turkish (tr-TR), Vietnamese (vi-VN), Hindi (hi-IN). Any other valid bounded BCP-47 tag can be sent directly.

The Playground exposes instructions, voice, and acknowledgement. API clients may also send language directly; the edge compiles it into the session language policy.

Voices: alloy, arbor, ash, ballad, breeze, cedar, coral, cove, echo, ember, juniper, maple, marin, sage, shimmer, sol, spruce, vale, verse.

WebRTC flow

const peer = new RTCPeerConnection();
const stream = await navigator.mediaDevices.getUserMedia({ audio: true });
for (const track of stream.getTracks()) peer.addTrack(track, stream);

peer.ontrack = (event) => {
  remoteAudio.srcObject = event.streams[0];
};

const offer = await peer.createOffer();
await peer.setLocalDescription(offer);

const response = await fetch("https://api.amanai.dev/v1/realtime/calls", {
  method: "POST",
  headers: {
    Authorization: "Bearer " + AMANAI_API_KEY,
    "Content-Type": "application/json"
  },
  body: JSON.stringify({
    sdp: peer.localDescription.sdp,
    session
  })
});

const established = await response.json();
await peer.setRemoteDescription({ type: "answer", sdp: established.sdp });

The response includes the SDP answer, a private session_id, and a one-use session_control_token. Do not expose the control token in logs or URLs.

Conversation loop

1. Acquire microphone permission and add the audio track to the peer connection.
2. Negotiate the session once with POST /v1/realtime/calls.
3. Play the remote audio track and consume transcript or state events from the data channel.
4. Keep the same connection open for every turn so context is preserved.
5. Reconnect as a new session only after terminal failure or expiry.

Connect a Chat model to Live

Your application owns its conversation history and selects the Chat model. For a request that must use Chat, send recent turns to POST /v1/chat/completions from your backend, then append the verified answer to the open Live data channel as speakable context. This uses the existing endpoints and needs no Amanai conversation database.

// Application backend: use your own Amanai key and selected Chat model.
const chat = await fetch("https://api.amanai.dev/v1/chat/completions", {
  method: "POST",
  headers: { Authorization: "Bearer " + process.env.AMANAI_API_KEY,
             "Content-Type": "application/json" },
  body: JSON.stringify({ model: selectedChatModel, messages: recentTurns })
});
const answer = (await chat.json()).choices[0].message.content;

// WebRTC client: send the verified Chat answer to the open Live data channel.
events.send(JSON.stringify({
  type: "session.context.append", channel: "speakable",
  content: [{ type: "input_text", text: answer }]
}));

Live may also emit delegation.created for a client task; your application can call Chat and return that result with delegation.context.append. The model decides when to delegate, so an application that requires Chat for a specific request should call it explicitly. Keep API keys and Live control tokens on your backend. The current Amanai data-channel event format is distinct from the newer official OpenAI Live SDK event format.

Download the complete Live + Chat example. It includes a localhost demo and a private backend mode. Each successful Live setup and each Chat request is billed under your own Amanai key.

Cleanup

curl -X DELETE https://api.amanai.dev/v1/realtime/sessions/$SESSION_ID \
  -H "Authorization: Bearer $AMANAI_API_KEY" \
  -H "X-Amanai-Realtime-Control-Token: $SESSION_CONTROL_TOKEN"

Stop local microphone tracks, close the peer connection, and release the server session when the user ends the call.

Billing

Realtime is billed per successfully established session. Failed negotiation is not billed. See Billing & pricing for the current rate.