Hugging Face describes decision models – a new type of language model for structured decisions
Hugging Face has introduced the concept of decision models: an LLM that, instead of generating text, returns probabilities for predefined options. It uses the "decision" pipeline in the transformers library and the GEV-26B-Decide model.
Hugging Face described on its blog decision models – a type of large language model that, unlike regular LLMs, does not generate text but instead directly returns a probability distribution over a predefined set of options. The model receives as input a context (state) and a question with defined answers, and returns a structured output containing the selected answer, a confidence level, and probabilities for all the options. An example from the article shows routing a complaint message to the billing/technical/sales department using the "decision" pipeline from the transformers library and the model autotrust/GEV-26B-Decide.
The authors argue that this approach solves problems with the common use of LLMs for classification tasks: standard models offer no guarantee of adhering to the required JSON schema, constrained decoding (tools such as Outlines or XGrammar) mitigates this problem but adds computational overhead, and generating a full text response is, in their view, unnecessary if the application only needs to select from a finite set of answers. The article further compares decision models with traditional classification (a fixed set of classes built into the model's head) and with zero-shot classification (candidate labels specified only at inference time, demonstrated on the model facebook/bart-large-mnli).
According to the article, the naming of the concept has a history: the company TypeSafe AI introduced it under the name Jev as a "System One model" – referring to the distinction between fast, intuitive (System 1) and slow, deliberative (System 2) thinking. According to Hugging Face, the community has settled on the neutral term "decision models" because it better describes the model's computational role without being tied to a specific theory of cognition.
The source text was only partially available and does not continue further into implementation details or further comparisons. See the source article for details.
Why it matters
For developers who currently use LLMs for classification, routing, or deciding among a finite set of options (e.g. routing support tickets, categorizing documents), decision models offer an alternative to unreliable JSON output parsing or to costly constrained decoding. Instead of generating text token by token, the model directly evaluates the probability of each predefined answer, which can simplify integration and increase the reliability of structured outputs in production applications.
Two audiences, two different impacts
What this means
For individuals
Developers and data engineers are getting a new tool (the "decision" pipeline in transformers) for implementing classification and decision-making tasks without relying on generating and parsing JSON from an LLM.
For a business
Companies operating request-routing systems (e.g. customer support) may in the future consider decision models as an alternative to current LLM-based classification solutions, which could affect the reliability and cost of processing structured decisions.
DevelopmentCheck the original
Event sources
only one source so far · 1 publisher, 0 independent. We count feeds from the same owner only once.