Blog

Guide

Kimi K3 Response Handling in Practice: Complete JSON, Stream Interruption, and Retry Boundaries

Nov 18, 2026
An abstract geometric cover image featuring two sets of flat woven straps interwoven into an open lattice on a deep ink-green background, accented with restrained brass tones.
On this page

First, distinguish the two response paths

With the same Kimi K3 API, what you receive may be one of two completely different things: a complete JSON response, or a sequence of streaming chunks arriving in order. The former is a "question-and-answer" document delivery; the latter is an incremental sequence pushed as generation proceeds. The root cause of many response-handling bugs is not that parsing was written incorrectly, but that you never figured out at the start which path you are on.

Both paths share the same request protocol (openai_chat_completions, contract version 2); the differences are concentrated in how the response is consumed:

  • Non-streaming: after the request returns, you get a complete response body containing the main text, reasoning content, and usage, which can be read in one go.
  • Streaming: the response is enabled as an optional mode; the server continuously pushes chunks and finally ends the stream with the SSE [DONE] marker. Along the way you may receive dozens or hundreds of chunks, or the stream may break midway; [DONE] only indicates the stream has ended, not that the content is complete.

Confirm this first, and everything discussed later becomes meaningful. If your business pipeline contains both types of calls (for example, a backend batch job and a frontend chat sharing one wrapper), it is recommended to explicitly label the path on the client rather than relying on implicit judgment such as "whether there is a callback" to distinguish them—that will plant landmines of deserialization failures during refactoring.

A simple rule of thumb: use streaming when you need to display output to the user character by character or when the output may be very long; use non-streaming when you need to obtain a structured result in one go and subsequently persist it or process it further. The parsing code for the two paths should be written separately; only request construction and error mapping should be shared.

The same request entry point splits into two consumption paths: one returns the entire response block, the other returns segment-by-segment fragments.

Parsing complete JSON: main text, reasoning content, and structured fields

A non-streaming response follows the OpenAI Chat Completions response body format, but there is one easily overlooked point: the main text and the reasoning content are two different fields.

The main text is in the standard message content position, while Kimi K3's reasoning process is delivered through the reasoning_content field. This means your parsing layer must do at least three things:

  • Take the final user-facing answer from the message content;
  • Determine whether reasoning_content exists, and if so decide whether to log it, show it to the user, or discard it;
  • Store usage separately for billing and quota accounting.

Displaying reasoning_content as main text is a common accident: it is intended for debugging and explainability scenarios, and putting it directly into product copy or customer service replies produces inappropriate tone and content. The correct approach is to give it an independent consumption path, such as writing it to debug logs or exposing it through an optional entry point like "view reasoning process".

How to use structured output

This contract supports the response_format parameter and the jsonObject structured output capability. You can ask the model to return a JSON object, thereby eliminating a fragile regex extraction layer. But two prerequisites need to be stated clearly: the jsonObject capability does not equal support for JSON Schema mode; and structured output is not a guarantee that it will "definitely succeed".

Only attempt to parse JSON after the response has completed normally and the content is complete. Even after parsing, you still need to handle three types of failure:

  • Parse failure: the returned content is not valid JSON, or fields are missing or of the wrong type. In this case you should not throw the raw string directly to the upstream business layer, but should enter a degradation branch.
  • Schema validation failure: the JSON syntax is valid but does not conform to the structure you declared. jsonObject does not guarantee JSON Schema-level constraints; the business side still needs to validate fields, types, and required items itself.
  • Truncation: when the output length reaches this model's output token limit (131072), finish_reason may be length, and the JSON may end midway. You must not treat length as a complete JSON success. A flawed JSON parser will report an error, but the more dangerous case is that it "barely parses successfully" and you continue running business logic with half an object.

It is recommended to add a business-field completeness and evidence validation step after parsing, rather than relying only on JSON syntax validity. The context window is 1000000, so in long-document scenarios input is not the bottleneck, but output truncation remains a real boundary.

The response envelope is split into layers: main text, reasoning content, and usage each form a branch flowing to their own consumers.

Streaming reads: [DONE], finish_reason, and usage

In streaming mode, the server pushes chunks in order, and the SSE network termination marker is [DONE]. That sentence sounds bland, but it is often misread: [DONE] only means this stream has ended; it does not mean generation succeeded, nor that the content is complete.

To determine the terminal state, look at choices[].finish_reason in each chunk. Currently, the frontend identifies the terminal state by a non-empty finish_reason; even if [DONE] has been received, a missing finish_reason should still be treated as an interruption. An in-stream error frame indicates that this request failed.

A stream may end in several states:

  1. Normal completion: finish_reason is a successful terminal state, followed by [DONE], and the accumulated content can be regarded as a complete result;
  2. Length truncation: finish_reason=length; the output was truncated due to the length limit, the content may be incomplete, and it cannot be treated as complete JSON success;
  3. Other non-success reasons: such as content filtering, which must be handled separately and must not be treated as normal completion;
  4. Midway disconnection: the connection closes, times out, or an in-stream error frame is received; at this point only partial content exists and it should be marked as failed or incomplete.

The last case is most easily written as "whatever we got is what we use". For chat display this may be acceptable (the user sees half a sentence and can resend), but for persistence, billing, or triggering downstream actions, it is wrong data. Please use "whether a successful terminal finish_reason was received" as the criterion for completeness, not just [DONE].

Usage is aggregated after the stream ends

In streaming mode, usage may appear in a choices:[] frame after finish_reason. The client should finish reading the entire stream before aggregating usage; there is no guarantee that [DONE] itself carries usage.

This brings two direct coding constraints:

  • Do not perform billing or quota settlement on the first chunk. At that point you neither know how long the final output will be nor have the usage field.
  • Do not accumulate usage across intermediate chunks, and do not treat [DONE] as a usage carrier. Intermediate chunks contain no reliable usage data, and any "estimate" will deviate from the final value.

The correct order is: during streaming, only accumulate text and state; after reading the stream, aggregate usage from the choices:[] frame to complete one settlement. If the stream breaks midway and you have already shown content to the user, then the usage status of this call is unknown—you need an "incomplete" status flag rather than defaulting to 0 or booking an estimated value.

Also distinguish internal metadata from network fields: things like response.stream.terminal=done and response.usage.delivery=final_chunk are platform-internal metadata labels, not SSE network fields, nor termination strings that the client should match. The client should rely only on [DONE], finish_reason, error frames, and the usage field itself.

Another practical detail: when accumulating text, append to a list and then join, rather than repeatedly concatenating strings; the performance difference is noticeable with long outputs.

The upper timeline reaches a brass-colored terminal block with a successful finish_reason; the lower timeline breaks midway and does not reach a successful terminal state, so one cannot assume a complete result has been obtained.

Why POST retries are not free idempotency

Many retry implementations are written like this: the request throws a timeout exception, you catch it, wait a while, and resend. This article discusses only the semantics of POST generation operations such as chat completions: such retries may cause duplicate billing and duplicate side effects and cannot be treated as free idempotency.

The reason is simple: a timeout is the client's judgment, not the server's promise. When the timer fires on your side, the request may have already fully reached the server, started inference, or even completed and been billed, with only the response lost on the way back. The client cannot distinguish these two situations based on the exception type alone:

  • The request never reached the server, so retrying will not produce a second execution;
  • The request was accepted and produced effects, so retrying is a second call.

The "effects" of a chat completion usually include token consumption and billing, as well as actions your business side triggers based on the response (database writes, form submissions, message sending). Replaying a POST with an uncertain result is equivalent to performing these side effects twice. This is not a theoretical risk: network jitter, gateway timeouts, and connections reset by intermediate devices all reliably create this kind of ambiguous state.

More troublesome is that a complete 5xx response, a timeout, a disconnection, or a JSON parse failure cannot prove that the request did not generate, was not charged, or had no side effects. The error you see only means that the client did not obtain a usable result this time.

Therefore, the correct precondition for retrying is not "an error occurred" but "this request definitely did not execute". Before writing any retry code, answer this question; if you cannot, do not automatically replay.

Defining retry boundaries: which cases can be replayed

Making retries a graded table is more reliable than writing a uniform for retry in range(3). The following is a handling strategy divided by error type.

Definitely not executed, safe to retry

  • Connection establishment failure (DNS failure, connection refused): the request clearly did not reach the server, so it is explicitly unsent, and retrying will not produce a second execution.
  • Rate-limit response (such as 429): only when the API documentation or response semantics explicitly guarantee rejection before execution should you back off and retry according to that semantics; a 429 or any other status code alone should not be used to infer non-execution or non-billing. Backoff should include random jitter to prevent multiple clients from retrying synchronously.
  • A deduplication mechanism explicitly provided by the API and actually used by the server: only when the API documentation explicitly states it and the server actually deduplicates accordingly should you retry according to its semantics. Request identifiers generated by the client itself are only for tracing; you must not assume that a particular request header or field provides idempotency.

Unknown result, requires explicit handling

  • Timeout: this is the most dangerous category. A timeout cannot prove the request did not execute, and resending may produce a new paid generation. Request identifiers generated by the client itself can only be used for tracing; only when the server explicitly supports and actually uses a deduplication mechanism can duplicate inference and duplicate charging possibly be prevented. Business-side deduplication can at best prevent duplicate downstream actions; it cannot stop the model from generating again and charging. If you cannot confirm the server's deduplication semantics, do not automatically replay; instead report an error to the user or enter a manual/asynchronous compensation flow.
  • Complete 5xx response: even if the response body is fully readable, it cannot prove that nothing was generated, nothing was charged, or there were no side effects. Retry only when it is clearly not executed, or when the API documentation explicitly states that the server will actually deduplicate based on a deduplication identifier; otherwise treat it as an unknown result.
  • Response parse failure: includes JSON parse failure and Schema validation failure. The request may have succeeded and been billed; a parse failure cannot prove that nothing was generated. Retrying may cause duplicate charges and duplicate side effects, so it is recommended to explicitly mark it as an unknown result and record it for later usage reconciliation.
  • Stream midway disconnection or receiving an in-stream error frame: content already pushed cannot be concatenated with the new result; replay will be treated as a brand-new generation.

Do not automatically retry

  • Request parameter errors (parameter-type errors among 4xx): retrying any number of times yields the same result and only amplifies the error.
  • Content moderation rejections: replay will not change the conclusion.
  • Any case where you cannot determine "whether it has taken effect" and there is no server-side deduplication semantics.

Write the boundaries into the code

Two engineering constraints are worth固化: set an upper limit on retry count (usually 1 to 2 is enough), and apply retries only to cases that are explicitly unsent, or where the API documentation explicitly guarantees rejection before execution; log the request identifier, error type, and elapsed time for each retry, because "how many times it retried" directly determines your usage bill and troubleshooting path. Streaming request retries require extra care: replaying after a midway disconnection produces a brand-new paid generation, and content already pushed cannot be concatenated with the new result.

If you are still in the model-selection stage, you can first look at the Kimi K3 model page to understand the contract details, and then decide the granularity of your retry strategy.

Implementation checklist

Condense the preceding rules into a checklist that can be compared item by item during review:

  • Does the client explicitly distinguish the streaming and non-streaming consumption paths, rather than relying on implicit judgment?
  • Does non-streaming parsing handle the main text and reasoning_content separately, and is reasoning_content kept out of user-visible copy?
  • Is usage stored separately and linkable to business records?
  • Is structured output parsed only after the response completes normally and the content is complete, with independent fallbacks for parse failures, Schema validation failures, business and evidence validation failures, and finish_reason=length?
  • Does streaming reading identify the terminal state by a non-empty finish_reason, rather than only waiting for [DONE]?
  • Is [DONE] treated only as a stream-end marker, and is a missing finish_reason marked as an interruption?
  • Is usage aggregated from the choices:[] frame after the stream is fully read, without assuming that [DONE] carries usage?
  • Are in-stream error frames recognized as failures rather than treated as normal endings?
  • Are retries graded by error type and applied only to cases that are explicitly unsent, or where the API documentation explicitly guarantees rejection before execution?
  • Are timeouts, disconnections, complete 5xx responses, or JSON parse failures treated explicitly as "unknown result" rather than promising that retries are safe?
  • Is there an upper limit on retry count, with logs for each retry that can be used to reconcile usage?
  • Is a replay after a stream interruption treated as a new generation rather than a continuation?

This checklist does not need to be fully satisfied all at once, but each item corresponds to a real production issue that will actually occur. Handle the finish_reason terminal-state criterion and the retry boundaries first; the payoff is the most direct.

Related articles

View all articles →

Start building with persistent AI artifacts

Get your API key and call image, video and audio models through one task-based API.

Read the API docs