Capture DeepSeek's prompt_cache_hit_tokens (dropped before by the
openai-kotlin Usage parser) via a raw-JSON extractor in ModelService,
and persist it plus the group name on each daily record.
Collapse the six /jgpt tokens* subcommands into a single /jgpt tokens
dashboard showing cache-hit rate, input/output split, daily trend and
top users/groups. Groups are shown by name only, never by group id.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
The streaming refactor moved post()+body() inside withTimeout, so a
first-chunk timeout threw before `channel` was bound and the finally
guard never ran, leaking the connection on every slow-API retry. Hold
the channel in an outer nullable var and wrap the whole flow in
try/finally so an acquired channel is always cancelled.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
The firstChunkTimeout only wrapped the response-body read, but when the
upstream (e.g. DeepSeek under load) stalls before sending response
headers, httpClient.post() itself blocks and the withTimeout block is
never reached. Every slow request fell through to Ktor's
requestTimeoutMillis (120s) and was retried up to retryMax times,
causing multi-minute waits before any reply.
- Move post() inside withTimeout(firstChunkTimeout) so the entire
request-to-first-data-chunk window is bounded and fails fast.
- Apply withTimeout(firstChunkTimeout) to each streaming read so a
mid-stream stall is also caught quickly instead of waiting on the
socket/request backstop.
- Drop requestTimeoutMillis so legitimately long streams are no longer
killed at 120s; TTFT and inter-token gaps are now governed at the
application layer.
Co-Authored-By: Claude Opus 4.8 <[email protected]>
jie65535andClaude Opus 4.8