I work on constrained deep learning: scalable methods for training neural networks under explicit requirements such as fairness, sparsity, and safety.
My research focuses on constrained optimization, feasible learning, and Lagrangian methods. I am supervised by Simon Lacoste-Julien and co-develop Cooper, an open-source PyTorch library for constrained optimization in deep learning.
Perspective
This paper argues that fixed penalty terms are the wrong default for enforcing explicit requirements in deep learning. Instead, when a problem naturally has targets to satisfy, we should solve it as a constrained optimization problem with tailored methods rather than hope that penalty tuning recovers the right trade-off.
Open-source software
Cooper is an open-source package for solving constrained optimization problems in deep learning. It implements Lagrangian-based first-order update schemes and makes it easy to combine constrained optimization algorithms with PyTorch models, autograd, and modern training pipelines.
Learning paradigm
Feasible Learning trains models by solving a feasibility problem that bounds the loss on every training example, rather than optimizing for average performance. It is a sample-centric alternative to ERM for settings where tail behavior and per-example reliability matter.
Theory
This paper shows that dual optimistic ascent on the dual variables of the Lagrangian is equivalent to gradient descent-ascent on the Augmented Lagrangian in the single-step, first-order regime commonly used in constrained deep learning. The result explains the practical success of dual optimism and helps establish its limits.
* denotes equal contribution. ^ denotes equal supervision.