Release of the open-source model Dory1.0-4B with a looped architecture and elastic depth

A research team has released Dory1.0-4B, a language model with 4 billion parameters built on a so-called looped (latent-recurrent) architecture with elastic depth. This means a single checkpoint can be used with a variable compute budget at inference time — the model's shared recurrent block is repeated (looped) as many times as needed depending on task difficulty. The model is based on a normalized transformer (nGPT) with interleaved sliding-window and full attention layers, and is divided into an input block, a shared recurrent block, and an output block.

According to the authors, the model was pretrained on 6 trillion tokens, of which the first 4.5 trillion were in NVFP4 format; further training then extended the context to 256 thousand tokens. This was followed by fine-tuning using supervised fine-tuning and reinforcement learning with verifiable rewards, with a so-called difficulty-aware loop curriculum, which routes harder tasks to a higher number of loops.

According to the authors, Dory1.0-4B, at two or more loops, outperforms on average across 12 benchmarks the larger models Qwen3.5-9B, Gemma 4 12B, and Gemma 4 26B-A4B. At four loops, it generates 1.3 to 2.5 times fewer tokens on agentic benchmarks than at one loop, because the additional compute happens in latent space instead of in longer output. The size of the KV cache, according to the authors, remains the same as in a non-recurrent model regardless of the number of loops.

The model checkpoints are released as open-source in BF16, MXFP8, NVFP4, and GGUF formats. This is an article published on arXiv (2611.03998v1); for details on methodology and exact benchmark figures, see the source article.