Guidance
An open-source Microsoft library for constrained generation with large language models. It controls the decoding process to enforce structural constraints on model outputs, enabling deep integration of program logic with the generation process.
Timeline (last 90 days)
Mainstream inference frameworks vLLM, Outlines, and Guidance all support constrained decoding, which typically uses finite state machines or pushdown automata to dynamically compute the set of valid tokens.
In mainstream LLM engineering contexts, Constrained Decoding refers to forcing the model at inference time via logit masking to only output token sequences that conform to a specific grammar or schema, with representative tools including Outlines, XGrammar, Guidance, and others.
Open-source frameworks such as Outlines, Guidance, and LMQL have already implemented constrained output/restricted decoding mechanisms.
Token Healing is a concept introduced by Microsoft's Guidance framework.
Outlines, Guidance, and LMQL are representative implementation libraries for hard constraints in the decoding phase.
Constrained decoding tools such as Outlines and Guidance allow models to be forced at inference time to produce text conforming to a specific grammar or schema, but potentially at the cost of some generation quality.
约束解码技术生态包括OpenAI的JSON Mode、Outlines库、LMQL、Microsoft的Guidance库、SGLang和Instructor等工具
All Facts (9)
Microsoft的Guidance库通过模板语法约束模型输出结构
85%Unverified业界已有成熟的约束解码实现，包括微软的Guidance库和Outlines项目
85%UnverifiedMainstream inference frameworks vLLM, Outlines, and Guidance all support constrained decoding, which typically uses finite state machines or pushdown automata to dynamically compute the set of valid tokens.
50%UnverifiedIn mainstream LLM engineering contexts, Constrained Decoding refers to forcing the model at inference time via logit masking to only output token sequences that conform to a specific grammar or schema, with representative tools including Outlines, XGrammar, Guidance, and others.
50%UnverifiedOpen-source frameworks such as Outlines, Guidance, and LMQL have already implemented constrained output/restricted decoding mechanisms.
50%UnverifiedToken Healing is a concept introduced by Microsoft's Guidance framework.
50%UnverifiedOutlines, Guidance, and LMQL are representative implementation libraries for hard constraints in the decoding phase.
50%UnverifiedConstrained decoding tools such as Outlines and Guidance allow models to be forced at inference time to produce text conforming to a specific grammar or schema, but potentially at the cost of some generation quality.
50%Unverified约束解码技术生态包括OpenAI的JSON Mode、Outlines库、LMQL、Microsoft的Guidance库、SGLang和Instructor等工具
50%