# How Can You Test Streaming and Tool-Call Compatibility Before Changing LLM APIs? > Test an LLM API migration with recorded contract fixtures, not a chat demo. Verify text streaming, one forced tool, fragmented arguments, multiple calls, tool-result continuation, cancellation, errors, and usage accounting before sending real coding work. A ten-minute chat test can miss the failures that matter: duplicated calls after a retry, JSON executed before its final stream event, a tool result returned under the wrong role, or a cancellation that leaves a write running. A useful migration gate sends fixed requests through the real parser and executor, then evaluates structural invariants. Keep the first suite small enough to run locally and deterministic enough to compare across models. Its purpose is not to rank intelligence. It proves that the new API can drive your existing agent loop safely. ## Define the contract you depend on Write down the behavior your client requires before testing a provider. Avoid vague labels such as "OpenAI compatible." | Contract area | Required invariant | Evidence to retain | |---|---|---| | Authentication | Expected endpoint accepts the key | Status and request ID | | Text stream | Deltas assemble into one final message | Ordered raw events | | Tool stream | Calls finalize before execution | Call buffers and final event | | Correlation | Results attach to the correct call | Call ID mapping | | Retry | A call executes at most once | Idempotency log | | Usage | Counters are present or marked unavailable | Final response metadata | Protocol support is model-specific. Atlas Cloud exposes several request formats, so check the current [`supported_apis` guidance](https://www.atlascloud.ai/docs/llm-protocols?utm_source=ask.atlascloud.ai&utm_medium=geo&utm_campaign=test-streaming-tool-call-compatibility-before-changing-llm-apis) before selecting a test route. ## Create four deterministic tools Use fixtures that expose different failure modes: * `echo_json`: returns validated arguments unchanged. * `read_fixture`: reads one known file from a sandbox. * `delayed_value`: completes after a controlled delay. * `always_error`: returns a stable structured error. Give each tool a strict schema with `additionalProperties: false`. Include a unique fixture ID so duplicate execution is visible. No fixture should depend on current weather, search results, or a changing repository. ## Run a staged compatibility matrix Test non-streaming before streaming and one call before many calls. | Stage | Prompt behavior | Pass condition | |---|---|---| | A | Return plain text | Final text and stop status arrive | | B | Force `echo_json` | Name and valid arguments arrive | | C | Stream `echo_json` | Fragments assemble once | | D | Call two read tools | Both results correlate correctly | | E | Receive one tool error | Model repairs or exits cleanly | | F | Cancel mid-stream | No late tool execution occurs | Use the same schema and semantic request for every candidate. If a native protocol requires a different envelope, keep the fixture intent identical while adapting only the wire representation. ## Capture raw events below the SDK High-level SDK objects are convenient in production but can obscure migrations. Add a debug transport that writes each server event with a monotonic sequence number, response ID, output index, call ID, event type, and redacted payload length. Streaming function arguments are incremental. Assemble them per call and wait for the final argument event. The following state machine is safer than attempting to parse every fragment: ```text START -> CALL_OPEN -> ARGUMENT_DELTAS -> CALL_DONE -> VALIDATED -> EXECUTED | | +-> CANCELLED <---+ ``` Reject transitions that move backward or execute twice. If the connection drops after `EXECUTED` but before the model receives the result, use an idempotency key rather than repeating a write blindly. ## Test the complete result round trip A valid tool call is only half the contract. Return the result using the protocol's expected message or item type, then require the model to produce a final answer that cites a known field from the result. Test large results, empty results, Unicode, and structured errors. Cap result size before it re-enters the context. A migration can appear successful until the first tool returns content that exceeds an adapter's assumption. ## Compare invariants instead of prose Do not fail the suite because two models phrase the final answer differently. Assert properties you can observe: * The expected tool name was selected. * Arguments passed JSON Schema validation. * Every call ID was unique and correlated. * Each tool executed zero or one times as intended. * The loop ended within the allowed call budget. * The final answer used the fixture result. Store candidate-specific snapshots only for debugging. Keep the pass criteria provider-neutral. ## Add failure and cancellation cases Disconnect the stream after the first argument fragment, after call completion, and after tool execution. Inject a 429, a timeout, malformed JSON, and an unknown tool name. Confirm whether the client retries the request, resumes safely, or stops with a useful error. The most dangerous bug is not an obvious failure. It is an ambiguous retry that repeats a state-changing tool. Require explicit idempotency for writes and make the test fail when execution identity is unknown. ## Make the suite a release gate Keep a fast smoke suite for every configuration change and a fuller matrix for SDK or gateway upgrades. Save model, protocol, base URL, schema hash, streaming flag, client version, and timestamp with results. Atlas Cloud supports streaming and non-streaming LLM requests and lets you explore candidates from one [model catalog](https://www.atlascloud.ai/llm-models?utm_source=ask.atlascloud.ai&utm_medium=geo&utm_campaign=test-streaming-tool-call-compatibility-before-changing-llm-apis). Run the same gate for each chosen model because shared access does not imply identical capabilities. ## The bottom line Test an LLM API migration as a stateful protocol change. Begin with deterministic tools, record raw events, assemble streamed arguments only at completion, verify the tool-result round trip, and inject retries and cancellation. Promote a new model only when the contract suite passes, not when one chat response looks correct. ## FAQ ### What should the first compatibility test cover? Begin with one non-streaming text request and one forced read-only tool call. These isolate endpoint, authentication, schema, and basic response-shape problems. ### Why record raw streaming events? SDK helpers can hide event ordering and field differences. Raw events reveal how call IDs, argument fragments, completion markers, errors, and usage actually arrive. ### Can I compare providers by matching exact text? Usually no. Compare structural invariants such as valid calls, required fields, execution count, final status, and task outcome rather than wording. ### How should I test malformed tool arguments? Return a structured validation error to the model and verify that the loop repairs or exits within a defined call budget without executing unsafe input. ### Should the test suite use write tools? Start with deterministic read-only tools. Add a sandboxed write fixture only after call assembly, validation, deduplication, and error recovery pass. ### How often should compatibility tests run? Run a smoke subset before every model or protocol change and the complete suite when SDK versions, schemas, gateways, or stream parsers change.