Fastino Releases GLiNER2.5-Decide: A 340M Open-Weight Decision Model That Runs on CPU

Fastino Labs has released GLiNER2.5-Decide, a 340M-parameter open-weight decision model. It takes text and a schema of typed questions and returns structured answers. Each answer comes with a probability distribution, a confidence score, and constraint-feasibility metadata. It targets the frequent judgment calls inside agent pipelines: routing, triage, tool selection, and guardrails.
Is it deployable? Yes, the weights ship under Apache 2.0 and install with pip install gliner2. They run on CPU, GPU, or in air-gapped environments. Fastino team also offers hosted inference and fine-tuning through its GLiNER API.
What GLiNER2.5-Decide Actually Does
GLiNER2.5-Decide is a non-generative classifier. It uses a DeBERTa-v3-large encoder and is fine-tuned from gliner2-large-v1. It produces no generated tokens and needs no prompt template.
Label sets are passed at call time. Each question in the schema declares its permitted answers. It also declares whether it expects one answer, multiple answers, or an ordered value. Schemas can carry instructions, examples, label descriptions, and rules linking answers across questions.
The pipeline has 2 stages. The encoder reads the text and schema together and scores every permitted answer. A constrained decoder then searches for the highest-scoring joint assignment the declared rules allow.
Fastino is explicit about scope. The model does not reason, explain, or answer open questions. It is a specialist for operational decisions.
Why Joint Decoding Matters
Fastino team illustrates the value with a guardrail example. Decoded independently, the model flagged prompt injection at 0.82. It also labeled the same prompt safe at 0.52. The attack was detected, but the 2 outputs conflicted.
Joint decoding applies a rule that any detected harm requires an unsafe verdict. The model then returns safety=unsafe and harm_type=prompt_injection together. Downstream code can use those scores to block, route, or escalate.
Schemas can express implications, exclusions, cardinality limits, and ordinal bounds. The same encoder can also extract entities, relations, and structured records with character-level offsets in 1 forward pass. Classification answers do not return evidence spans.

Benchmark Results on Fast Decisions
Fastino team evaluated the model on Fast Decisions, an internally generated, held-out suite. It contains 5,100 test examples across 17 datasets. The tasks cover customer operations, domain routing (banking, clinical, travel, benefits), and general content understanding. The metric is exact-match accuracy: a prediction counts only if its label set matches the reference exactly.

Model
Type
Average

GLiNER2.5-Decide
340M encoder
60.1%

JevK5
4B-class Qwen3.5 decoder
57.5%

SemIf
Qwen3.5-4B decoder
56.4%

GLiFormer large-v1
Single-pass encoder
49.0%

Laya
421M ModernBERT encoder
46.6%

GLiNER2.5-Decide led 9 of the 17 datasets. Intent routing was its strongest area. It scored 75.3% on support intent and 64.3% on banking intent. Those are 18.6 and 8.6 points ahead of the next-best models.
Latency: Practical on CPU
Fastino team benchmarked the checkpoint end to end at batch 1 with a 2-head, 15-label schema. At 64 tokens, p50 latency was:

167.3 ms on a 48-vCPU Intel Xeon Platinum 8581C
43.6 ms on an NVIDIA T4
43.4 ms on an NVIDIA L4
38.3 ms on an NVIDIA V100
47.3 ms on an NVIDIA A100

Short requests are dominated by fixed preprocessing and kernel-launch overhead. That keeps the GPUs within 9 ms of each other. At 1,024 tokens, the A100 pulls ahead at 52.6 ms, versus 75.6 ms on V100 and 131.4 ms on L4.
Using It in Code
This multi-head call is adapted from the model card:

Copy CodeCopiedUse a different Browser

from gliner2 import AutoExtractor

model = AutoExtractor.from_pretrained(“fastino/GLiNER2.5-Decide”)

model.classify_text(
“Please confirm the new retention rule is applied before Friday’s audit.”,
{
“intent”: [“fyi”, “request”, “approval”, “complaint”],
“urgency”: [“low”, “normal”, “high”, “critical”],
“route”: [“support”, “billing”, “legal”, “security”],
},
)

Single-label heads return one string. Multi-label heads return every label above a cls_threshold. Labels can carry descriptions, and ordinal scales are passed as ordinary strings like “0” to “10”.
Use Cases and the Model Family
Fastino targets model routing, tool calling, browser and computer use, guardrails, context pruning, LLM-as-a-judge, and simulations. Fine-tuning works locally, fully or with LoRA, through the GLiNER2 trainer. A SKILL.md file exposes the hosted workflow to coding agents.
Fastino also published GLiNER2.5-Decide-1B, built on the Ettin 1B encoder. It scores 59.6% on the same suite, just under the 340M model. For multilingual input, Fastino points to GLiNER2.5-multi-Decide, a 287M model scoring 56.7%. The underlying architecture is described in the GLiNER2 paper.
Key Takeaways

340M open-weight encoder, Apache 2.0, runs on CPU or air-gapped.
Returns answers, probabilities, confidence, and feasibility metadata.
Joint decoding enforces rules across related answers.
60.1% average on Fastino’s internal 17-dataset suite, leading 9.
167.3 ms p50 on CPU, 38.3 ms on V100 for short inputs.

Check out the Technical Details and Model Card. All credit goes to the researcher of this project. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well.
Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us
The post Fastino Releases GLiNER2.5-Decide: A 340M Open-Weight Decision Model That Runs on CPU appeared first on MarkTechPost.