From efficient models to collaborating agents

01 · Compression

Model compression

Pruning removes model weights that contribute less to a chosen objective. The difficult part is not producing zeros; it is deciding what to remove and accounting for the combined effect. My work couples queries and keys when choosing attention pruning masks, and uses second-order information to compensate for removals block by block.

Quantization represents model values with fewer bits, reducing storage and memory traffic at the cost of approximation error. I study how that error propagates through the model and how to fine-tune heavily compressed models without relying only on the straight-through estimator.

02 · Multi-agent systems

AI agent collaboration

When a task is too large for one agent, it is split into parts, such as the modules of a program or groups of robots. Each part becomes easier, but the parts are coupled: an agent can improve its own part while breaking someone else's. I model agent teams as block optimization, where each agent searches its own block with a correction for the others and a coordinator merges only verified progress, and prove when local work adds up to global progress.

03 · Distributed training

Pipeline parallelism

Pipeline parallelism divides a model into stages that can work on different microbatches concurrently. The resulting delays make its optimization dynamics different from ordinary synchronous training. I study theoretical models that make these delayed updates explicit and allow convergence to be analyzed.

04 · Adaptation

Sparse fine-tuning

Sparse fine-tuning updates a selected subset of model parameters while keeping the rest fixed. This turns parameter selection into part of the optimization problem and can reduce the amount of trainable state needed to adapt a model.