Ornith 1.5 Open-Source Model Series Released: 9B, 35B MoE, and 397B Now Available on HuggingFace

Ornith 1.5 Series Quietly Drops

Recently, an active HuggingFace community member shared a discovery on Reddit: the Ornith AI team released three brand-new open-source models at once — the Ornith 1.5 series. The poster jokingly admitted to being "addicted to refreshing HuggingFace every 30 minutes looking for new models," and explicitly disclaimed any affiliation with the Ornith team.

This kind of grassroots, community-driven model release is a genuine reflection of how vibrant the open-source LLM ecosystem has become. Compared to the flashy launch events of major tech companies, an increasing number of models are quietly landing on HuggingFace, spreading organically through community word-of-mouth and hands-on testing. HuggingFace, currently the world's largest AI model hosting and collaboration platform, is often called "the GitHub of AI." It offers not only a Model Hub for storage and version management but also an integrated toolchain including datasets, inference APIs, and Spaces for app hosting. As of 2025, the platform hosts over a million models spanning natural language processing, computer vision, speech recognition, and more. For open-source model developers, publishing on HuggingFace has become the de facto standard workflow — simply upload model weights and config files, and community "model hunters" like the Reddit poster will organically discover and spread the word.

Three Parameter Scales Covering Different Use Cases

Based on the published model repositories, Ornith 1.5 comes in three parameter scales, forming a complete product matrix:

Ornith 1.5 9B: The Sweet Spot for Local Inference

This is the smallest dense model in the series. At 9B parameters, it's perfectly suited for running on consumer-grade GPUs or local environments. A dense model refers to the traditional architecture where all parameters participate in computation during every inference pass, as opposed to sparse architectures like MoE (Mixture of Experts). For a 9B dense model in FP16 (half-precision floating point) format, roughly 18GB of VRAM is needed to load the weights — which falls neatly within the capacity of consumer flagship GPUs like the NVIDIA RTX 4090 (24GB). With 4-bit quantization, VRAM requirements can be further compressed to around 5–6GB, enabling smooth operation even on mid-range GPUs like the RTX 4060 (8GB).

For individual developers and hobbyists, 9B is precisely the "sweet spot" size that runs smoothly on a single 24GB GPU — it strikes the optimal balance between model capability and hardware accessibility, making it the most popular scale for local experimentation.

Ornith 1.5 35B-A3B: The Best Bang-for-Buck with MoE Architecture

The "A3B" in the name is the key detail — it indicates the model uses a MoE (Mixture of Experts) architecture with approximately 35B total parameters, but only about 3B parameters are activated per inference pass.

The core idea behind MoE architecture originated from the conditional computation theory proposed by Jacobs et al. in 1991, but was revitalized in the large model era by Google's Switch Transformer (2021) and later the Mixtral series. Here's how it works: the model contains multiple "expert" sub-networks (typically feed-forward network layers), and during each inference pass, a gating network (Router/Gating Network) dynamically selects a small number of experts for computation based on the input token's features, while the rest remain dormant. In the case of Ornith 1.5 35B-A3B, the 35B total parameters mean the model has large-model-level knowledge storage capacity, but each token only activates about 3B parameters for computation, making the actual computational cost (FLOPs) equivalent to that of a 3B dense model.

The benefits of this design are obvious: the model enjoys the knowledge capacity that comes with a larger parameter count while maintaining inference speed and cost close to that of a small model. However, MoE also presents significant engineering challenges: first, all 35B parameters still need to be loaded into VRAM, so memory usage doesn't shrink as dramatically as compute does; second, load balancing among experts is a training difficulty — if the gating network tends to always select the same few experts (the "expert collapse" problem), the model's effective capacity is severely diminished.

The playful remark in the post title — "We have Q3.8 35B at home" — is a classic community quip acknowledging this type of MoE model, implying you can get a near-flagship experience with a much lower hardware barrier.

Ornith 1.5 397B: The Flagship Model for Research Institutions

This is the flagship of the series. At nearly 400B parameters, it enters the realm of ultra-large models. A 397B parameter model requires approximately 794GB of VRAM just to load weights in FP16 precision — far beyond the capacity of any single consumer GPU. Deploying such a model typically requires multi-node clusters, for example using 8 NVIDIA H100 GPUs (80GB VRAM each, 640GB total) with tensor parallelism, often combined with pipeline parallelism to handle remaining memory demands. Even with 4-bit quantization, VRAM requirements hover around 200GB, still requiring multiple high-end GPUs working in concert.

This is why models of this scale are typically aimed at research institutions or teams with substantial compute resources — not only because of the steep hardware costs (a single 8×H100 cluster costs over $250,000), but also because optimizing inference throughput for large models is itself a complex systems engineering challenge involving model parallelism strategies, communication topology optimization, KV Cache management, and more. Its purpose is to explore the upper limits of model capability.

GGUF Quantized Format: Paving the Way for Local Deployment

One notable detail: the Ornith team simultaneously released GGUF quantized versions for every model.

Model quantization is the technique of converting model weights from high precision (e.g., FP32/FP16) to lower precision (e.g., INT8, INT4, or even lower bit-widths), with the goal of dramatically reducing model storage size and inference compute within an acceptable accuracy loss margin. GGUF (GPT-Generated Unified Format) is a quantized model format designed by Georgi Gerganov, the core developer of the llama.cpp project. Introduced in the second half of 2023, it replaced the earlier GGML format. GGUF's key advantage is that it packages model weights, tokenizer configuration, and model architecture metadata all into a single file, enabling true "out-of-the-box" usability. It also supports multiple quantization schemes (such as Q4_K_M, Q5_K_S, Q8_0, etc.), allowing users to choose different precision-performance tradeoffs based on their hardware capabilities. The Q3.8 mentioned in the post refers to an average quantization bit-width of approximately 3.8 bits — an aggressive quantization strategy for running large-parameter models under extremely limited VRAM conditions.

GGUF is one of the most important formats in the current local inference ecosystem, widely supported by mainstream tools including llama.cpp, Ollama, and LM Studio. llama.cpp itself is a pure C/C++ inference engine supporting CPU inference with optional GPU acceleration, while Ollama and LM Studio are more user-friendly interface tools built on top of it. Together, these three form the mainstream tech stack for local LLM inference today.

This means users can directly download and run these models on local CPUs or consumer GPUs without going through the tedious quantization conversion process themselves. The simultaneous release of quantized versions also reflects the Ornith team's commitment to the local deployment user base — this is precisely the core value proposition that distinguishes open-source models from closed-source API services.

The Rapid Iteration of the Open-Source LLM Ecosystem

The release of Ornith 1.5 is a microcosm of the current wave of dense iteration in the open-source LLM space. From the poster's self-described "addictive" HuggingFace refreshing habit, you can sense just how rapid the pace of new model releases has become — so fast that even dedicated trackers can barely keep up.

For the community, this boom is both a blessing and a challenge:

• Blessing: Users have more choices than ever, with models of different scales and architectures matching various hardware conditions and application needs;
• Challenge: Model quality varies widely, and without authoritative cross-model benchmarks, average users struggle to quickly assess whether a new model is worth the time investment to deploy.

Current open-source LLM evaluation primarily relies on several categories of benchmarks: general knowledge tests like MMLU (Massive Multitask Language Understanding), reasoning benchmarks like GSM8K (math reasoning) and ARC (science Q&A), code generation benchmarks like HumanEval and MBPP, and comprehensive conversational benchmarks like MT-Bench and Chatbot Arena. However, these evaluation systems have notable limitations: on one hand, some models may have intentionally or inadvertently "leaked" benchmark data during training, resulting in inflated scores (the data contamination problem); on the other hand, standardized benchmarks struggle to capture performance differences in real-world scenarios, such as long-context understanding, multi-turn conversation consistency, and instruction-following robustness. Chatbot Arena's human blind-evaluation Elo ranking is considered the closest to real-world experience, but its data collection speed is limited — newly released models often need weeks to accumulate enough voting samples.

As of now, systematic hands-on feedback about Ornith 1.5's actual performance is still lacking in the community. The original poster was also seeking others' experiences, reminding us that when facing a constant stream of new models, the rational approach is to wait for community testing and benchmark results before deciding whether to dive deeper.

Summary: A Comprehensive Open-Source Release Strategy Worth Watching

The Ornith 1.5 series, with its three scales — 9B, 35B-A3B, and 397B — plus comprehensive GGUF quantization support, demonstrates a fairly mature open-source release strategy. The 35B-A3B MoE model in particular represents the mainstream exploration direction in the open-source community's pursuit of balancing performance with cost.

That said, a model's true value ultimately needs to be validated through real-world testing. Interested developers are encouraged to head over to HuggingFace to try it out firsthand, and we look forward to the community delivering more objective benchmark data soon.