Streaming

Set "stream": true to receive server-sent events. Use fetch and ReadableStream — not native EventSource, which cannot send an Authorization header.

POST/v1/chat/completions

Auth · API key

application/json → text/event-stream

Enable streaming

json

{
  "model": "modelrail-chat",
  "stream": true,
  "stream_options": { "include_usage": true },
  "messages": [{ "role": "user", "content": "Hello" }]
}

stream_options.include_usage is only valid when stream is true. The final usage chunk may have an empty choices array.

SSE protocol

Behavior	Detail
Content-Type	`text/event-stream`
Frames	`data: <json>`
End	`data: [DONE]`
Object	`chat.completion.chunk`
Content	Accumulate `choices[].delta.content`

Each event is a data: frame followed by a blank line.
Streaming ends with data: [DONE].
Ignore keep-alive comments if present; parse only data: lines.

TypeScript example

TypeScript

async function streamChat(prompt: string) {
  const res = await fetch("https://api.modelrail.dev/v1/chat/completions", {
    method: "POST",
    headers: {
      Authorization: `Bearer ${process.env.MODELRAIL_API_KEY}`,
      "Content-Type": "application/json",
      "x-request-id": crypto.randomUUID(),
    },
    body: JSON.stringify({
      model: "modelrail-chat",
      stream: true,
      messages: [{ role: "user", content: prompt }],
    }),
  });

  if (!res.ok || !res.body) {
    throw new Error(`HTTP ${res.status}`);
  }

  const reader = res.body.getReader();
  const decoder = new TextDecoder();
  let buffer = "";
  let text = "";

  while (true) {
    const { done, value } = await reader.read();
    if (done) break;

    buffer += decoder.decode(value, { stream: true });
    const parts = buffer.split("\n\n");
    buffer = parts.pop() ?? "";

    for (const part of parts) {
      const line = part.split("\n").find((l) => l.startsWith("data: "));
      if (!line) continue;

      const data = line.slice(6).trim();
      if (data === "[DONE]") return text;

      const json = JSON.parse(data);
      if (json.error) {
        throw new Error(json.error.message ?? "stream error");
      }

      const delta = json.choices?.[0]?.delta?.content;
      if (typeof delta === "string") text += delta;
    }
  }

  return text;
}

Stream errors

Things to note

Mid-stream provider failure may emit an error-shaped SSE event after HTTP 200. Check for an error object on each parsed frame. See Errors.