I work on constrained deep learning: scalable constrained optimization methods for training neural networks that must satisfy explicit requirements such as fairness, sparsity, and safety.
I am supervised by Simon Lacoste-Julien. My research spans algorithms, theory, and applications of Lagrangian methods for constrained optimization in large-scale deep learning. I also work on Feasible Learning and co-develop Cooper, an open-source PyTorch library for constrained deep learning.
Before my PhD, I completed a BSc in Mathematical Engineering at Universidad EAFIT and held research internships at Mila, McKinsey, and Tuya.
Learning paradigm
Feasible Learning trains models by solving a feasibility problem that bounds the loss on every training example, rather than optimizing for average performance. It is a sample-centric alternative to ERM for settings where tail behavior and per-example reliability matter.
Open-source software
Cooper is an open-source package for solving constrained optimization problems in deep learning. It implements Lagrangian-based first-order update schemes and makes it easy to combine constrained optimization algorithms with PyTorch models, autograd, and modern training pipelines. See the code and docs.
Theory
We show that optimistic gradient ascent (PI control) on the dual variables of a Lagrangian is equivalent to gradient descent-ascent on the Augmented Lagrangian in the single-step, first-order regime commonly used in constrained deep learning. The result helps explain the practical success of dual optimism and establish its limits.
Perspective
We argue that fixed penalty terms are the wrong default for enforcing explicit requirements in deep learning. When a problem naturally specifies targets to satisfy, we should formulate it as a constrained optimization problem and use methods designed to enforce those requirements directly.
* denotes equal contribution. ^ denotes equal supervision.