Ollama v0.40.14 Update: Fixes Cloud Model Output Limits and Sampling Defaults

Ollama has released version v0.40.14. This update focuses primarily on the stability and compatibility of cloud model integrations, fixing several real-world issues affecting Claude Code, ChatGPT, and the OpenAI-compatible API.

Core Fix: Cloud Model Output Token Limits

ollama launch claude now correctly applies the model's own output token limit. Previously, cloud models running in Claude Code would often hit the default 128k cap, causing output to be unexpectedly truncated. This change makes the launch command carry the model's actual output limit, so cloud models are no longer constrained by the default value.

For users relying on Claude Code for long-context code generation or complex reasoning, this is a fix that directly improves the experience.

Multiple Compatibility Improvements at the API Layer

Cloud Search Response Decoding in /api/chat

Responses returned by cloud web search can now be decoded correctly, including compressed response content. Previously, such responses could fail to parse in /api/chat, affecting the reliability of search-based applications.

Common forms of HTTP response compression include gzip, deflate, and br. Clients must decompress based on the Content-Encoding header before tokenizing and parsing. If the decoding layer only handles uncompressed plaintext, compressed cloud web search results would be treated as garbage and throw errors. Fixing this means the decoding pipeline must first normalize decompression before JSON parsing, and correctly handle chunked transfer to avoid intermittent failures in search-based applications.

OpenAI-Compatible Endpoint No Longer Overrides Sampling Defaults

The OpenAI-compatible endpoint no longer forcibly overrides the sampler defaults packaged with the model. When the client does not pass temperature or top_p, the endpoint uses the model's own defaults instead of uniformly treating them as 1.0, bringing its behavior in line with /api/chat. This is especially important for scenarios that rely on the model's factory parameters to achieve stable output quality.

Sampling parameters determine the randomness and coverage of the generation distribution: lower temperature means more determinism, while top_p controls the nucleus sampling range; together they affect output stability. If the OpenAI-compatible layer uniformly fills in 1.0 when the client doesn't pass values, it effectively treats the requester's silence as "maximum randomness," which is inconsistent with the model's factory-tuned parameter distribution. By using the model's own defaults instead, the same request yields more similar outputs on the OpenAI-compatible endpoint and the native /api/chat, which also helps with reproducibility and evaluation.

/v1/responses Preserves Readable Input Content

/v1/responses now preserves readable input content, including previous assistant refusal replies, instead of flattening them. This change helps maintain context integrity in multi-turn conversations, allowing the model to see input that more closely reflects the real interaction history.

/v1/responses corresponds to OpenAI's Responses API, which uses structured input and output items (such as message and refusal). The previous flattening process compressed multiple input segments into plain text, thereby losing special type information such as assistant refusal replies. Preserving readable content means distinguishing roles and content types when returning data, so the model can be aware of which replies were previously refused in multi-turn history, reducing context misalignment and repeated triggering.

Application Layer and Performance Optimizations

The Ollama app can now retry failures caused by ChatGPT disconnections instead of leaving the connection in a broken state. ollama launch codex now validates the ChatGPT route before configuration and gives a clear error message when the route is unavailable, preventing users from discovering the problem only after configuration is complete.

On the performance side, the MLX backend has optimized top-k sampling speed for large-vocabulary scenarios. For users running large-vocabulary models on Apple Silicon, sampling latency will be somewhat reduced.

Top-k sampling takes the k highest candidates from the vocabulary probabilities and then performs normalized sampling; with large vocabularies (hundreds of thousands of tokens), comparing each item or fully sorting is expensive. A common optimization is to maintain a fixed-size min-heap for partial selection, reducing complexity from full sorting to O(n log k). The MLX backend leverages Apple Silicon's unified memory and Metal acceleration to reduce sorting and copying overhead in the sampling hotspot, thereby shortening per-step decoding latency.

Summary

v0.40.14 is not a major release introducing new features, but rather a dense round of stability fixes: from token limits and sampling defaults to response decoding and connection retries, almost all of them point to the same goal—making the integration behavior of cloud models and third-party clients more predictable. For developers using Claude Code, Codex, and OpenAI-compatible clients simultaneously, this version is worth upgrading to.