Aleph Alpha Puts Kolibri-1’s 78B MoE Weights on Hugging Face for German Workloads

On October 3, 2026—German Unity Day—Aleph Alpha released Kolibri-1 as open weights under Apache 2.0 on Hugging Face, shipping a bilingual English–German mixture-of-experts transformer with 78.1 billion total parameters, about 3.46 billion active per token, and a context path validated up to 1,048,576 tokens. The company blog and model card frame Kolibri as a sovereign, deployable generalist for regulated German and European settings—public administration, industrials, aerospace—rather than a multilingual tourist model. Serving goes through Aleph Alpha’s aleph-alpha-inference vLLM plugin (or the published container), with explicit reasoning-effort levels and Hermes-style tool calling enabled in the recommended serve command.

Why a sparse 78B sits on the cost frontier

Aleph Alpha’s pitch is Pareto quality versus serving cost at roughly 4.4% activation. The architecture is a 50-layer MoE with 384 experts per layer, six of them routed and one shared, and a 4:1 sliding-window to full-attention pattern that keeps most layers on a 512-token window so decode cost stays bounded as context grows. The card lists an FP8 weight footprint near 78 GB and minimum hardware of two A100 80 GB, two H100 SXM5, or a single H200/B200/B300 class GPU. Recommended sampling is temperature 1.0, top_p 0.97, top_k 128. For contexts beyond the 262,144-token native training length, operators pass --max-model-len 1048576 with an HF override; Aleph Alpha still recommends ≤262K for latency-sensitive or complex work.

Company-reported scores place Kolibri near denser peers on math and code while trailing some dense baselines on overall English averages—AIME 2025 at 96.9, GPQA Diamond at 84.3, LiveCodeBench v6 at 85.9, SWE-Bench Verified at 66.4, with German AIME and GPQA rows published alongside English. The card is explicit that dense models activating many more parameters per token sit in a different compute class; Kolibri’s claim is competitive MoE efficiency, not beating every 27B dense checkpoint on every axis. Independent third-party composite indexes were still catching up at publish time, so treat leaderboard screenshots as company-harness evidence until external labs republish.

Training stack, German depth, and Origin

Kolibri is not a continued finetune of an outside base: the card states random initialization and a full Model Factory pipeline validated earlier on Kolibri Origin (30.6B total / ~3.3B active, 65K context, no public release). Pre-training consumed 20T tokens over 21 days on 768 NVIDIA B200 GPUs, then 3.44T mid-training and about 201B long-context tokens—nearly 24T across stages. The pre-training mix is roughly 62.5% English, 23.9% German, and 13.6% code on the card’s training-data summary; the blog stresses organic German curation and rephrasing over heavy translation so the model is bilingual by design. A UniBPE tokenizer at 128K vocabulary is tuned for German morphology while keeping English efficiency. Knowledge cutoff is June 18, 2026 for both languages.

Post-training combines SFT on filtered bilingual open and synthetic mixes with reinforcement learning across more than 1.2 million curated tasks spanning reasoning, agents, instruction following, code, and retrieval. Users can set reasoning effort to none, low, medium, or high through chat-template kwargs. Grounding work includes abstention data and Aleph Alpha’s Merlin-Arthur protocol so the model is trained to decline when context lacks evidence—metrics the blog tracks on AA-Omniscience non-hallucination and RGB abstention rates versus Origin.

Serving path and specialization claims

The blog positions Kolibri for on-prem and sovereign deployments where sending regulated text to a third-party API is off the table. Internal customer-proxy evals—automotive supplier, semiconductors, German public sector, industrial drives, aerospace—are company-reported hill-climbs from Origin to Kolibri without training on customer data; they are directional for vertical buyers, not public leaderboards. Builders should run the published vLLM serve flags with --reasoning-parser kolibri1 and --tool-call-parser kolibri1, then validate German and English tasks against their own corpora before treating card averages as SLAs.

License boundary and operator checklist

Apache 2.0 covers the published weights and configuration files only. Aleph Alpha’s card states the grant does not extend to underlying code, the model architecture, parameter settings, or training methods outside the repository. Operators still need the inference plugin for correct parsers. For Writeble’s opensource desk, Kolibri-1 is the clearest October 3 European open-weights story: a production-shaped bilingual MoE with documented sparsity, long context, and a Unity Day ship date—not a placeholder card waiting on later weights.