Special Tokens
Symbols reserved in the vocabulary specifically for structural control, such as <|im_start|> and [INST], which delimit role boundaries and turn starts and ends within the input sequence, serving as an explicit encoding of conversational syntax.
Timeline (last 90 days)
The "instruction-conversation genre" is jointly shaped by three key elements: instruction fine-tuning, the system prompt, and a structured input format with special tokens and role assignments.
Special tokens are symbols reserved in the vocabulary specifically for structural control, such as `<|im_start|>`, `<|im_end|>`, or `[INST]`, `[/INST]`.
Different model families use different special token schemes, which is one reason the same prompt performs differently across models.
All Facts (3)
The "instruction-conversation genre" is jointly shaped by three key elements: instruction fine-tuning, the system prompt, and a structured input format with special tokens and role assignments.
50%UnverifiedSpecial tokens are symbols reserved in the vocabulary specifically for structural control, such as `<|im_start|>`, `<|im_end|>`, or `[INST]`, `[/INST]`.
50%UnverifiedDifferent model families use different special token schemes, which is one reason the same prompt performs differently across models.
50%