Instruction Hierarchy
Instruction Hierarchy (IH) is a mechanism for ensuring the safe and reliable operation of large language models, requiring the model to distinguish instruction priorities and to obey higher-priority instructions when lower-priority instructions conflict with them.
Timeline (last 90 days)
The concept of the instruction hierarchy (IH) was first systematically proposed by Wallace et al. in 2024 in the paper "The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions".
Fine-tuning with only goal-goal conflict samples significantly improved IH performance on a large number of real-world tasks unseen during training.
On the conflict-free tasks across the eight benchmarks, the average utility change brought by the IH improvement was negligible.
The authors advocate incorporating the goal-goal conflict type into IH training, even though these samples do not reflect typical LLM usage scenarios.
The framework of decomposing tasks into goals and constraints provides an actionable tool for identifying blind spots in IH training.
All Facts (5)
The concept of the instruction hierarchy (IH) was first systematically proposed by Wallace et al. in 2024 in the paper "The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions".
50%UnverifiedFine-tuning with only goal-goal conflict samples significantly improved IH performance on a large number of real-world tasks unseen during training.
50%UnverifiedOn the conflict-free tasks across the eight benchmarks, the average utility change brought by the IH improvement was negligible.
50%UnverifiedThe authors advocate incorporating the goal-goal conflict type into IH training, even though these samples do not reflect typical LLM usage scenarios.
50%UnverifiedThe framework of decomposing tasks into goals and constraints provides an actionable tool for identifying blind spots in IH training.
50%