Local Work, Global Progress
A convergence theory for AI agent teams. Agents improve their own parts with corrections for the others, and a coordinator keeps only verified progress.
Hi, I’m
Ivan Ilin
Machine Learning ResearcherPhD Candidate
I work on efficient optimization and compression methods for large language models, including pruning, sparse fine-tuning, quantization, and pipeline parallelism.
Open to machine learning research internships
ivan.ilin@kaust.edu.sa
View CV
StatusPhD Candidate
AffiliationKAUST · Computer Science
Google Scholar
158Citations
5h-index
4i10
GitHub
625Stars
451Followers
27Repos
Research areas
Making large models more efficient
My research focuses on making large models more efficient through compression, adaptation, and distributed training.
LLM Pruning
Methods for removing redundant model parameters while preserving model quality.
Sparse Fine-Tuning
Training a small, carefully selected subset of model parameters efficiently.
Quantization
Reducing model precision and memory requirements while controlling quality degradation.
Pipeline Parallelism
Optimization and convergence analysis for training models across pipeline stages with delayed updates.
Featured projects
Here are some of my favorite projects, from research implementations to open-source tools.
QK-Wanda
We couple query and key pruning by extending Wanda scores with opposite-projection activation energies and sharing the pruning budget across Q and K.
First Theory for PipeDream
Randomized PipeDream captures PipeDream's stale-weight behavior in an analyzable block-SGD model, revealing how pipeline depth affects convergence.
Pruning LLMs with Thanos
A block-wise algorithm for pruning large language models using second-order information and coordinated weight compensation.
Selected publications
2026
2026
2026
2024
Personal blog
Research milestones, project updates, thoughts, and talks from recent years.
I passed my PhD proposal defense
My proposal connects LLM pruning, sparse fine-tuning, and pipeline-parallel optimization into one research program.
Our TRACE project received a KAUST sTRG
A USD 100,000 Small Translational Research Grant will support TRACE, a project for turning textbooks into verified, interactive courses.
Starting as a TA for CS 260 at KAUST
This fall I am supporting Design and Analysis of Algorithms through office hours, assignments, grading, and exams.