Parameter Efficient Finetuning

SPoT: Better Frozen Model Adaptation through Soft Prompt Transfer

Vu, Constant, Al-Rfou, Cer, and Lester (Google Research/UMass, 2022) show that initializing soft prompts from a related source task dramatically improves prompt tuning, achieving near fine-tuning performance while keeping the base model frozen. The result reveals that prompt initialization is a critical and underappreciated factor in parameter-efficient adaptation.