Streaming delivers generated content incrementally instead of waiting for the complete response.
Enable streaming
bash
curl -N http://localhost:3187/v1/chat/completions \
-H "Authorization: Bearer $YUNHAI_API_KEY" \
-H "Content-Type: application/json" \
-d '{"model":"YOUR_MODEL_ID","stream":true,"messages":[{"role":"user","content":"Introduce yourself"}]}'Read events
Parse the response as Server-Sent Events. One network chunk may not contain a complete event. Chat Completions commonly sends incremental data through data: events and ends with [DONE].
Handle interruptions
Use a reasonable read timeout and allow cancellation. Before retrying tool calls or tasks with side effects, check their state to avoid duplicate execution.