Sparse fine-tuning adapts a language model by training only a small, carefully selected subset of parameters while leaving the rest fixed. The aim is to reduce the memory and computation required for adaptation without treating every model weight as equally important.
This project connects activation-aware parameter selection with sparse optimization. The public Super-Tuning codebase and accompanying preprint contain the verified implementation and research details; no private experimental results are presented here.