← All projects

Research / LLM Pruning

Thanos

A method for pruning large language models using second-order information and coordinated weight compensation.

Preprint2025★ 4 GitHub stars
Diagram of coordinated weight pruning and compensation in a model weight matrix

Thanos studies how multiple weights can be removed while compensating for their joint effect on a layer’s output. Instead of treating each removed weight in isolation, the method uses Hessian-derived second-order information to coordinate weight updates.

The project explores efficient, block-wise layer optimization for both unstructured and structured sparsity. It is presented here as an active research project; no publication status or benchmark claims are implied.

Related work

Publications

2025
PreprintarXiv

Thanos: A Block-wise Pruning Algorithm for Efficient Large Language Model Compression

Ivan Ilin, Peter Richtárik

A block-wise pruning method that removes multiple weights jointly and coordinates their compensation using second-order information.