I work on constrained deep learning: scalable methods for training neural networks under explicit requirements such as fairness, sparsity, and safety.
My research spans algorithms, theory, and applications of Lagrangian methods for large-scale constrained deep learning. I also work on Feasible Learning and co-develop Cooper, an open-source PyTorch library for constrained deep learning. I am supervised by Simon Lacoste-Julien.
Before my PhD, I completed a BSc in Mathematical Engineering at Universidad EAFIT and held research internships at Mila and McKinsey.
Learning paradigm
Feasible Learning trains models by solving a feasibility problem that bounds the loss on every training example, rather than optimizing for average performance. It is a sample-centric alternative to ERM for settings where tail behavior and per-example reliability matter.
Open-source software
Cooper is an open-source package for solving constrained optimization problems in deep learning. It implements Lagrangian-based first-order update schemes and makes it easy to combine constrained optimization algorithms with PyTorch models, autograd, and modern training pipelines. See the code and docs.
Theory
We show that optimistic gradient ascent (PI control) on the dual variables of a Lagrangian is equivalent to gradient descent-ascent on the Augmented Lagrangian in the single-step, first-order regime commonly used in constrained deep learning. The result helps explain the practical success of dual optimism and establish its limits.
Perspective
We argue that fixed penalty terms are the wrong default for enforcing explicit requirements in deep learning. When a problem naturally specifies targets to satisfy, we should formulate it as a constrained optimization problem and use methods designed to enforce those requirements directly.
* denotes equal contribution. ^ denotes equal supervision.