This month I completed my master’s research in Computer Science at KAUST. The thesis, Efficient and Fast Pruning of Large Language Models, is now available through the KAUST repository with a permanent DOI.
My thesis committee was chaired by Peter Richtárik; its other members were Marco Canini, Panagiotis Kalnis, and Mikhail Moshkov.
The work asks a practical question: how can we remove parameters from a large language model without treating every weight as an isolated number? Magnitude pruning is fast, but it ignores both the activations that pass through a layer and the way surviving weights can compensate for removed ones. My thesis studies increasingly informed alternatives, from activation-aware scores to second-order reconstruction.
That line of work led to Thanos, a block-wise approach to pruning. Instead of deleting one weight and repairing the layer one coordinate at a time, Thanos handles a selected group of weights together and uses second-order information to coordinate the update. The goal is to make richer reconstruction steps manageable at the scale of transformer layers.
Today I presented this project in the KAUST SANDS seminar series. A seminar and a thesis require different explanations: the thesis preserves the derivations and experiments, while the talk has to expose the central idea quickly enough for a broader systems-and-algorithms audience. You can watch the recording below or open it on YouTube.
This is not the end of the project. It is a useful checkpoint: the thesis records the path that led to the method, and the seminar helped turn that path into a clearer research story.