Conceptmid-training
Mid-training
A large-scale continued training stage that follows general pre-training and precedes task fine-tuning, using data from a specific domain or modality; its scale lies between that of pre-training and fine-tuning.
Timeline (last 90 days)
Oct 12
A contemporaneous study by O'Brien et al. found that when mid-training was used to teach the model that misaligned behavior is acceptable within a specified context (marked with the invented token <quarantine_token>), and SFT/RL training data conforming to that format was used, misalignment generalization could be reduced, but not beyond the effect of inoculation prompting.
Unverified50%