Technical report on Kolibri describes training on 24 trillion tokens
The technical report on Kolibri describes training on 24 trillion tokens and reinforcement learning on more than 1.2 million tasks. The German-English model from Aleph Alpha has open weights under the Apache 2.0 license.
The technical report on Kolibri provides further training details: according to Aleph Alpha, training involved a total of 24 trillion tokens. German accounted for over 20 % of the training data measured in bytes. To prepare the data, the company collected German tokens from the web and generated them synthetically, producing over two trillion German tokens in total. Subsequent post-training included reinforcement learning on more than 1.2 million internally prepared tasks focused on reasoning, tool use, instruction following, code and information retrieval.
Aleph Alpha released Kolibri for German and English with a mixture-of-experts architecture. The technical report lists 78.1 billion parameters in total and 3.46 billion active parameters per token; an earlier article rounded these figures. According to the article in The Decoder, the model supports a context window of up to one million tokens and was trained on 768 B200 GPUs in Germany and Finland.
The weights are available on Hugging Face under the Apache 2.0 license. According to the company, the model is intended primarily for public administration, industry and aviation. The company also claims that Kolibri achieves a leading balance of quality and operating costs in German and English among the open models evaluated in the technical report.
Why it matters
Open weights expand self-deployment options for teams working with German and English texts. The new details make it easier to assess the model's language focus and how it was developed; claims about quality and operating costs come from the company's evaluation.
Release card
Kolibri
Aleph Alpha
- Specifications
- The model has 78.1 billion parameters in total and 3.46 billion active parameters per token.
- Context
- The model supports a context window of up to one million tokens.
- Inputs
- The model works with text in German and English.
- Licence
- Apache 2.0
- Availability
- The weights are publicly available for download on Hugging Face under the Apache 2.0 license.
- Deployment in public administration, in line with the developer's stated focus.
- Deployment in industry, in line with the developer's stated focus.
- Deployment in aviation, in line with the developer's stated focus.
According to the technical report, Kolibri lies on the Pareto frontier for quality and operating costs in German and English among the open models evaluated.
The card summarizes information from the article and any dated corrections, with a link to the original source. It is not our assessment of the model. It does not yet have a dedicated editorial profile. Model selection and other announcements →
What was added since the original report
Verified updates
-
Training involved 24 trillion tokens.; The training data includes over two trillion German tokens obtained through an in-house web pipeline and synthetic generation.; Post-training included reinforcement learning on more than 1.2 million internally prepared tasks.
- Training involved 24 trillion tokens.
- The training data includes over two trillion German tokens obtained through an in-house web pipeline and synthetic generation.
- Post-training included reinforcement learning on more than 1.2 million internally prepared tasks.
Relevant practical impact
What this means
For a business
For companies developing their own language solutions for public administration, industry or aviation, Kolibri represents another candidate for technical evaluation. The available weights and Apache 2.0 license provide concrete terms to consider when deciding whether to use it.
DevelopmentCheck the original
Event sources
confirmed by 2 independent sources · 2 publishers, 2 independent. We count feeds from the same owner only once.