One socket. Audio in, voice out.

How it works

A conversation is a single WebSocket at /v1/realtime. Your client decides when the speaker starts and stops talking, and sends their audio in between. When they stop, Oya transcribes the turn, writes a reply in the speaker's style, and speaks it back in whole sentences, so the first sentence plays while the rest is still being written.

Every connection needs an access key: an X-Oya-Key header, or ?key= on the URL from a browser, where WebSockets can't set headers. Without a valid key the handshake fails with HTTP 401, and a key that has used its allowance gets 429. Each key has a cap on session length, sessions open at once and minutes per day.

• You send: JSON control events and binary PCM16 audio at 16 kHz.
• You receive: JSON events (transcript, reply text, timings) and binary PCM16 audio at 24 kHz, tagged with a turn id.
• Talking over Oya cancels the reply in flight. That's barge-in, and it's built into the protocol.

A working conversation in the browser

Two files: an AudioWorklet that turns the mic into 20 ms PCM16 frames at 16 kHz, and a page script that connects, gates the mic with a simple voice detector, and plays Oya's voice. Run it after a click (browsers only start audio on a user gesture), from https:// or localhost so the mic is allowed.

// Mic → 20 ms PCM16 frames at 16 kHz, plus each frame's loudness (RMS)
class Capture extends AudioWorkletProcessor {
  constructor() {
    super();
    this.ratio = sampleRate / 16000;
    this.pos = 0;
    this.frame = new Int16Array(320);
    this.n = 0;
    this.sumSq = 0;
  }
  process(inputs) {
    const ch = inputs[0][0];
    if (!ch) return true;
    for (; this.pos < ch.length; this.pos += this.ratio) {
      const s = Math.max(-1, Math.min(1, ch[Math.floor(this.pos)]));
      this.sumSq += s * s;
      this.frame[this.n++] = s * 0x7fff;
      if (this.n === 320) {
        this.port.postMessage({ pcm: this.frame.buffer, rms: Math.sqrt(this.sumSq / 320) }, [this.frame.buffer]);
        this.frame = new Int16Array(320);
        this.n = 0;
        this.sumSq = 0;
      }
    }
    this.pos -= ch.length;
    return true;
  }
}
registerProcessor("capture", Capture);

const ws = new WebSocket("wss://YOUR_OYA_HOST/v1/realtime?key=YOUR_ACCESS_KEY");
ws.binaryType = "arraybuffer";
const send = (msg) => ws.send(JSON.stringify(msg));

// 1. Configure the session: "" means auto-detect, including code-switching
ws.onopen = () => send({ type: "session.update", session: { language: "", voice: "boma" } });

// 2. Play Oya's voice: PCM16 at 24 kHz, each frame prefixed with its turn id
const out = new AudioContext({ sampleRate: 24000 });
let currentTurn = 0, playHead = 0, sources = [];

function play(pcm16) {
  const buf = out.createBuffer(1, pcm16.length, 24000);
  const ch = buf.getChannelData(0);
  for (let i = 0; i < pcm16.length; i++) ch[i] = pcm16[i] / 0x8000;
  const src = out.createBufferSource();
  src.buffer = buf;
  src.connect(out.destination);
  playHead = Math.max(out.currentTime + 0.02, playHead);
  src.start(playHead);
  playHead += buf.duration;
  sources.push(src);
  src.onended = () => (sources = sources.filter((s) => s !== src));
}
function stopPlayback() {
  sources.forEach((s) => s.stop());
  sources = [];
}

ws.onmessage = ({ data }) => {
  if (data instanceof ArrayBuffer) {
    const turn = new DataView(data).getUint32(0, true);   // little-endian
    if (turn === currentTurn) play(new Int16Array(data, 4));
    return;
  }
  const e = JSON.parse(data);
  if (e.type === "response.started") currentTurn = e.turn;
  if (e.type === "input.transcript") console.log("You:", e.text);
  if (e.type === "response.text.delta") console.log("Oya:", e.text);
  if (e.type === "error") console.error(e.message);
};

// 3. Capture the mic and gate it with a simple voice detector
const mic = await navigator.mediaDevices.getUserMedia({
  audio: { echoCancellation: true, noiseSuppression: true, channelCount: 1 },
});
const ctx = new AudioContext();
await ctx.audioWorklet.addModule("capture.js");
const node = new AudioWorkletNode(ctx, "capture");
ctx.createMediaStreamSource(mic).connect(node);

let speaking = false, voiced = 0, silent = 0;
node.port.onmessage = ({ data: { pcm, rms } }) => {
  const loud = rms > 0.015;
  if (!speaking) {
    voiced = loud ? voiced + 1 : 0;
    if (voiced < 3) return;                    // ~60 ms of voice starts a turn
    speaking = true;
    silent = 0;
    stopPlayback();                             // barge-in: silence Oya locally at once
    send({ type: "input.speech_started" });     // …and cancel the reply on the server
  }
  ws.send(pcm);                                 // 20 ms of PCM16 at 16 kHz
  silent = loud ? 0 : silent + 1;
  if (silent * 20 >= 700) {                     // 700 ms of quiet ends the turn
    speaking = false;
    voiced = 0;
    send({ type: "input.speech_stopped" });
  }
};

That's a complete conversation. The reference client adds three refinements worth copying: it keeps the last 300 ms of audio before speech is detected and sends it after input.speech_started, so the first syllable isn't clipped; it adapts the threshold to the room's noise floor; and it uses a stricter threshold while Oya is talking, so Oya's own voice through the speaker doesn't count as an interruption.

Connect, then configure

On connect, Oya sends session.created with the current settings and the audio formats it expects and produces:

{
  "type": "session.created",
  "session": { "voice": "boma", "instructions": "You are Oya, a warm, quick-witted Nigerian friend…" },
  "audio": {
    "input":  { "format": "pcm_s16le", "sample_rate": 16000 },
    "output": { "format": "pcm_s16le", "sample_rate": 24000 }
  }
}

Change settings at any time with session.update. Only the fields you send change; Oya replies with session.updated and the full settings.

Conversation history lives for the life of the connection: Oya remembers earlier turns until the socket closes.

Raw PCM, both ways

• Audio only counts between input.speech_started and input.speech_stopped; frames sent outside a turn are ignored.
• Turns shorter than 250 ms are treated as noise and get no reply.
• A single turn is capped at 30 seconds; audio past that is dropped.

Event reference

Client → Oya

Oya → client

Every audio frame carries its turn

Each binary message from Oya starts with the turn id as an unsigned 32-bit little-endian integer, followed by the PCM16 samples. Compare it with the turn from the latest response.started and drop anything else: after a barge-in, a few frames from the old reply can still be in flight.

const turn = new DataView(data).getUint32(0, true);
const samples = new Int16Array(data, 4);   // 24 kHz, mono

One turn, start to finish

1. You
   input.speech_started

   The speaker begins. Any reply still playing is cancelled.

2. You
   binary frames

   20 ms of PCM16 at a time, while they talk.

3. You
   input.speech_stopped

   Your voice detector decides they've finished.

4. Oya
   input.transcript

   What Oya heard, with the new turn id.

5. Oya
   response.started

   The reply begins; accept audio for this turn.

6. Oya
   response.text.delta + audio

   Words stream as they're written; voice arrives a sentence at a time.

7. Oya
   metrics

   Timings for the turn, sent with the first audio.

8. Oya
   response.done

   Finished, or interrupted if the speaker cut in.

Talking over Oya

Real conversations overlap. When the person starts talking during a reply, do two things at once: stop playback locally (so the person hears Oya stop immediately) and send input.speech_started. The server cancels the reply, and Oya remembers what it had said with a note that it was cut off, so it can pick up naturally.

To stop a reply without starting a new turn, for example from a "stop" button, send response.cancel.

See what callers feel

Every turn reports its timings in milliseconds, measured from the moment Oya receives input.speech_stopped. Add your own voice detector's silence window to get the full "you stop → Oya speaks" time.

Codes

Pin a language when you know your callers' language; leave it on auto when they mix.

Twenty voices, five accents

Use the lowercase name as the voice id, e.g. "sade". Every voice can speak every language; the accent is what changes.

Honest numbers

Today a reply's first audio arrives around three seconds after the speaker stops, plus your voice detector's silence window. That's slower than a person, and closing the gap is the point of the roadmap: streaming recognition, then our own instant voices, then one native speech model.

What it doesn't do yet

• No API keys or auth yet. Preview access is by invitation; keys and per-account limits come before general access.
• No rate limits are enforced yet, so please be gentle.
• Voice detection runs on your side. You decide when a turn starts and stops. Server-side detection, including knowing when someone has finished a thought rather than just paused, is planned.
• Batch transcription per turn. Oya transcribes after the speaker stops, not while they speak. Streaming recognition is Stage 2.
• Preview API. Event names and fields may change before general access.

Questions