Today I passed my PhD proposal defense at KAUST and advanced to PhD candidacy.
My proposal committee was chaired by Peter Richtárik; its other members were Suhaib Fahmy, Rolf Krause, and Panos Kalnis.
You can watch the full proposal-defense recording below or open it on YouTube.
The proposal is titled “Efficient Optimization and Compression Methods for Large Language Models.” It brings together three lines of work that initially looked like separate efficiency problems:
- Thanos studies post-training pruning, using block-wise second-order reconstruction to remove weights while updating the surviving ones.
- Super and Supra use pruning-inspired scores to choose a fixed sparse set of trainable weights, with Supra combining that sparse update with LoRA under a matched parameter budget.
- Randomized PipeDream studies the optimization behavior of pipeline-parallel training and the structured delays created by a PipeDream-style schedule.
The connection is the cost of working with large models at three different stages. Pruning targets inference and storage. Sparse and low-rank updates target adaptation. Pipeline parallelism becomes necessary when training itself no longer fits comfortably on one accelerator.
Putting these projects into one proposal forced me to state what the research program is really about. The common theme is not a single compression technique; it is the interaction between model structure and optimization. Efficiency improvements become more useful when the algorithm reflects where computation, memory, and delay actually come from.
Passing the proposal is an important checkpoint, but it is still a checkpoint. The next stage is to strengthen the empirical evidence, extend the theory, and make the accompanying implementations easier to reproduce and use.