Convergence Theory of AI Agent Collaboration
A convergence theory for AI agent teams. Agents improve their own parts with corrections for the others, and a coordinator keeps only verified progress.
Hi, I’m
Ivan Ilin
Machine Learning ResearcherPhD Candidate
I work on efficient optimization and compression methods for large language models, including pruning, sparse fine-tuning, quantization, and pipeline parallelism. I also develop convergence theory for teams of collaborating AI agents.
Open to machine learning research internships
ivan.ilin@kaust.edu.sa
View CV
StatusPhD Candidate
AffiliationKAUST · Computer Science
Google Scholar
164Citations
5h-index
4i10
GitHub
628Stars
452Followers
28Repos
Research areas
Model Compression
Pruning and quantization methods that make large models smaller while preserving their quality.
AI Agent Collaboration
Convergence theory for teams of AI agents that each improve their own part of a shared solution.
Pipeline Parallelism
Optimization and convergence analysis for training models across pipeline stages with delayed updates.
Sparse Fine-Tuning
Training a small, carefully selected subset of model parameters efficiently.
Featured projects
Here are some of my favorite projects, from research implementations to open-source tools.
Pruning LLMs with QK-Wanda
We couple query and key pruning by extending Wanda scores with opposite-projection activation energies and sharing the pruning budget across Q and K.
First Theory for PipeDream
Randomized PipeDream captures PipeDream's stale-weight behavior in an analyzable block-SGD model, revealing how pipeline depth affects convergence.
Pruning LLMs with Thanos
A block-wise algorithm for pruning large language models using second-order information and coordinated weight compensation.
Selected publications
2026
2026
2026
2024
Personal blog
Research milestones, project updates, thoughts, and talks from recent years.
I passed my PhD proposal defense
My proposal connects LLM pruning, sparse fine-tuning, and pipeline-parallel optimization into one research program.
Our TRACE project received a KAUST sTRG
A USD 100,000 Small Translational Research Grant will support TRACE, a project for turning textbooks into verified, interactive courses.
Starting as a TA for CS 260 at KAUST
This fall I am supporting Design and Analysis of Algorithms through office hours, assignments, grading, and exams.