Adapter
A lightweight neural network module inserted between layers of a pretrained model or at its output. During fine-tuning only a small number of parameters are updated while the original model weights remain unchanged, achieving parameter-efficient adaptive transfer.
Timeline (last 90 days)
By fine-tuning with LoRA over a shared, frozen backbone and maintaining only two lightweight adapters, one for the student and one for the teacher, memory usage can be reduced from two full models to one backbone plus two small adapters.
Adapters insert small feed-forward modules between Transformer layers and train only these modules, introducing additional serial computation, so inference latency is slightly higher than that of LoRA.
Method selection recommendations: prioritize LoRA for low latency, easy deployment, and a mature ecosystem; consider adapters when multi-task isolation and modular mounting are needed; if the parameter budget is extremely tight and tasks are relatively simple, try prefix or prompt tuning.
SCLATE provides an open event scheduler into which both benchmarks and unmodified agents can register their respective events via adapters.
All Facts (4)
By fine-tuning with LoRA over a shared, frozen backbone and maintaining only two lightweight adapters, one for the student and one for the teacher, memory usage can be reduced from two full models to one backbone plus two small adapters.
50%UnverifiedAdapters insert small feed-forward modules between Transformer layers and train only these modules, introducing additional serial computation, so inference latency is slightly higher than that of LoRA.
50%UnverifiedMethod selection recommendations: prioritize LoRA for low latency, easy deployment, and a mature ecosystem; consider adapters when multi-task isolation and modular mounting are needed; if the parameter budget is extremely tight and tasks are relatively simple, try prefix or prompt tuning.
50%UnverifiedSCLATE provides an open event scheduler into which both benchmarks and unmodified agents can register their respective events via adapters.
50%