Vision and reasoning requests reused the chat model's 10s first-chunk
timeout, but both legitimately need longer before the first chunk:
vision must download the image server-side first, and reasoning has a
thinking warmup. Logs showed frequent TimeoutCancellationException at
10s/15s for imageRecognition and reasoning.
Add separate visualFirstChunkTimeout (120s) and reasoningFirstChunkTimeout
(90s) config, and raise each service's socket timeout to at least its
first-chunk budget so the socket layer doesn't sever the connection
before the first-chunk timeout can apply. Chat endpoints are unchanged.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
Configure a list of fallback chat endpoints (chatFallbacks); each blank
field inherits the primary, so you can swap just the API key, just the
model, or the whole vendor. On LLM streaming failure the retry loop
advances to the next endpoint, and a failed endpoint enters a cooldown
(fallbackCooldownMinutes) so a dead primary is skipped instead of
wasting a timeout on every message.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
jie65535andClaude Opus 4.8