Testing LLM API integrations: streaming, tool calls and failures

An LLM integration fails in ways ordinary API clients rarely meet: an answer that arrives in hundreds of server-sent events, a stream that stops halfway through a sentence, a 429 during a traffic spike, an overloaded provider, a tool call the client has to execute and answer. The Mock LLM API reproduces each of these deterministically, in the OpenAI and Anthropic request shapes, so the handling can be tested on every run instead of discovered in production.

LLM API testing — request and response sequence Sequence diagram for LLM API testing: GET /streams/v1/events answers 200 OK. GET /streams/v1/websocket answers 401 Unauthorized. POST /llm/v1/chat/completions answers 429 Too Many Requests. POST /auth/v1/firebase-token answers 503 Service Unavailable. Your client Fun API Playground GET /streams/v1/events 200 OK Read events as JSON or text/event-stream GET /streams/v1/websocket 401 Unauthorized WebSocket lab connection and resume protocol POST /llm/v1/chat/completions 429 Too Many Requests Chat Completions shape: streaming (SSE), tools, JSON mode… POST /auth/v1/firebase-token 503 Service Unavailable Mint a Firebase Auth token whose role custom claim grants…
Real endpoints from the catalog — every request above is served, and every status code shown is one that endpoint actually returns.

How to practise it

  1. Send a streamed request to mock-echo, concatenate every delta, and assert both the final text and that the stream ended with a stop reason and the final event.
  2. Stream from mock-stream-cut and assert your client reports an incomplete answer instead of showing half a sentence as the final result.
  3. Call mock-429 and verify your client waits for Retry-After, retries a bounded number of times and then gives up with a clear error.
  4. Send a request to mock-tool-call with one tool declared, execute the tool call it returns, send the result back and assert the final answer uses it.
  5. Call mock-slow with a client timeout shorter than eight seconds and assert your fallback runs instead of the request hanging.

APIs to try it on

Status codes involved

Other techniques

Getting started guide · All 55 mock REST APIs

Last updated