Juan Camilo Ramirez — Constrained Deep Learning

About

Juan Ramirez is a fourth-year PhD candidate at Mila and the University of Montreal, supervised by Simon Lacoste-Julien.

He works on constrained learning—developing scalable algorithms that enforce requirements such as fairness, sparsity, and safety in neural networks via Lagrangian methods.

He develops Cooper, an open-source PyTorch library for constrained learning, and co-organized the NeurIPS 2025 Workshop on Constrained Learning.

Research interests: Constrained Learning, Feasible Learning, Reliable & Trustworthy AI, Min-max Optimization, Sparsity.

Contact: juan.ramirez@mila.quebec

Featured Projects

Cooper: A PyTorch Library for Constrained Deep Learning

A practical toolbox for training deep nets under constraints.

I co-develop Cooper, an open-source PyTorch library for non-convex constrained optimization that lets you train deep models under explicit constraints. The goal is to make it practical to state requirements (fairness, sparsity, safety) as constraints and actually enforce them during training — the only assumption is that the constraints are (sub-)differentiable in PyTorch. [Docs] [Paper]

Position: Adopt Constraints Over Penalties in Deep Learning

Most models are only encouraged to satisfy requirements. We argue they should be guaranteed to.

This position paper argues that adding penalty terms to the loss — which nudges models toward desirable behavior but cannot reliably enforce those requirements — is fundamentally unreliable in deep learning. We advocate for constrained optimization instead: directly enforcing requirements (fairness, robustness, sparsity, safety) during training through explicit constraints, with enforceable guarantees.

Feasible Learning: A Sample-Centric Paradigm

Instead of optimizing the average, we enforce per-sample performance.

In high-stakes settings, failure on a single example can be unacceptable. Even in lower-stakes settings, average metrics can hide systematic harm — a model can look good on paper while still failing specific users or groups. Feasible Learning trains models by solving a feasibility problem: enforce a target loss on every training sample, rather than just optimizing the average. Accepted at AISTATS 2025.

News

2025

Sep 29: New preprint! Check out Dual Optimistic Ascent (PI Control) is the Augmented Lagrangian Method in Disguise, where we show that the popular PI controller for updating Lagrange multipliers in constrained optimization is equivalent to the method of multipliers.
Jul 4: We are organizing a NeurIPS 2025 workshop on Constrained Optimization for Machine Learning! See you in San Diego!
May 31: We released a preprint of our position paper Position: Adopt Constraints Over Penalties in Deep Learning.
Apr 1: We just released version 1.0.0 of Cooper, a PyTorch library for non-convex constrained optimization, along with a companion paper.
Jan 22: Our latest paper Feasible Learning has been accepted at AISTATS 2025! Feasible Learning is a novel sample-centric learning paradigm where models are trained by solving a feasibility problem that bounds the loss for each training sample.
Jan 1: Serving as Associate Program Chair for ICML 2025.

Nov 1: I have been awarded the Excellence Scholarship by the Association des Diplômés de l'Université de Montréal.
Sep 6: I succeeded in my predoctoral exam. I am now a PhD candidate!
Sep 1: I will be a TA for Simon Lacoste-Julien's course on Probabilistic Graphical Models for a second time.
May 1: On PI Controllers for Updating Lagrange Multipliers in Constrained Optimization has been accepted at ICML 2024! Vienna, here we come again.
Mar 4: Delighted to receive Université de Montréal's AI scholarship 2023–2024!
Jan 29: Excited to be a TA at the first edition of the Tropical Probabilistic AI School in Rio de Janeiro, Brazil! I am happy to see more AI events popping up around Latin America.
Jan 16: Balancing Act: Constraining Disparate Impact in Sparse Models has been accepted at ICLR 2024! Vienna, here we come.
Jan 12: Quentin Bertrand and I will be TAing Gauthier Gidel's Adversarial Machine Learning course at Mila and the University of Montreal.

2023

Dec 7: Excited to give a talk at Montréal Machine Learning and Optimization (MTL MLOpt) about our recent work Balancing Act: Constraining Disparate Impact in Sparse Models.
Sep 4: Together with António Góis, I will be TAing Simon Lacoste-Julien's graduate course on Probabilistic Graphical Models at Mila.
Jul 23: Arriving in Honolulu, Hawaii, where I will have the pleasure to present Omega: Optimistic EMA Gradients. Omega, our method for stochastic min–max optimization, was awarded an oral at the LatinX in AI workshop at ICML 2023!
Jan 27: I will attend Khipu 2023 in Montevideo, Uruguay. Glad to be a part of the Latin American AI community.

2022

Oct 14: See you at the PyTorch conference 2022 in New Orleans! Cooper will be making an appearance (poster).
Sep 14: I will be presenting my first NeurIPS paper: Controlled Sparsity via Constrained Optimization or: How I Learned to Stop Tuning Penalties and Love Constraints in New Orleans.
Sep 5: Excited to start a PhD in Artificial Intelligence under the supervision of Simon Lacoste-Julien at Mila and the University of Montreal.
Jul 8: Attending the Sparsity in Neural Networks workshop. We are presenting our latest work on implicit neural representations: L₀onie: Compressing COINs with L₀-constraints.

2021

Dec 17: Flexible Learning of Sparse Neural Networks via Constrained L0 Regularization, won the best poster award at the LatinX in AI Workshop @ NeurIPS 2021!
Nov 1: Starting an internship at Mila. I will join Simon Lacoste-Julien's group under the supervision of Jose Gallego-Posada.
Jul 1: I graduated from the BSc in Mathematical Engineering at Universidad EAFIT! Plus, got a mention for my contributions to the student organization. Check out my LinkedIn post about it.
Jan 14: I am auditing Ioannis Mitliagkas' graduate course on Deep Learning Theory at Université de Montréal.

2020 and earlier

Jul 2020: Jose Gallego-Posada, PhD student at Mila will be my supervisor for my undergraduate thesis. I will be working on deep generative models.
Jun 2019: Now part of McKinsey & Co. in Belgium as a research intern. Working with Antoine Stevens and Patrick Dehout in ML for the agricultural and chemical industries.
Jan 2019: Arrived in Louvain-la-Neuve, Belgium for an exchange semester at Université Catholique de Louvain.
Nov 2017: I have been appointed as president of CIGMA-OE, the Mathematical Engineering chapter of the Student Organization at Universidad EAFIT.
Dec 2015: I was awarded a full scholarship for the Bachelor's degree in Mathematical Engineering at Universidad EAFIT.
Dec 2015: Scored amongst the best 0.1% on the Colombian high school examination ICFES.
Dec 2015: Ranked first in the National Chemistry Olympiads of Universidad de Antioquia.

Publications

* denotes equal contribution. ^ denotes equal supervision.

Preprints

Juan Ramirez and S. Lacoste-Julien. Dual Optimistic Ascent (PI Control) is the Augmented Lagrangian Method in Disguise. arXiv:2509.22500, 2025.
Juan Ramirez, M. Hashemizadeh, and S. Lacoste-Julien. Position: Adopt Constraints Over Penalties in Deep Learning. arXiv:2505.20628, 2025.
J. Gallego-Posada*, Juan Ramirez*, M. Hashemizadeh*, and S. Lacoste-Julien. Cooper: A Library for Constrained Optimization in Deep Learning. arXiv:2504.01212, 2025.

Conference

Juan Ramirez*, I. Hounie*, J. Elenter*, J. Gallego-Posada*, M. Hashemizadeh, A. Ribeiro^, and S. Lacoste-Julien^. Feasible Learning. In AISTATS, 2025.
M. Sohrabi*, Juan Ramirez*, T. H. Zhang, S. Lacoste-Julien, and J. Gallego-Posada. On PI Controllers for Updating Lagrange Multipliers in Constrained Optimization. In ICML, 2024.
M. Hashemizadeh*, Juan Ramirez*, R. Sukumaran, G. Farnadi, S. Lacoste-Julien, and J. Gallego-Posada. Balancing Act: Constraining Disparate Impact in Sparse Models. In ICLR, 2024.
J. Gallego-Posada, Juan Ramirez, A. Erraqabi, Y. Bengio, and S. Lacoste-Julien. Controlled Sparsity via Constrained Optimization or: How I Learned to Stop Tuning Penalties and Love Constraints. In NeurIPS, 2022.

Workshop

Juan Ramirez, R. Sukumaran, Q. Bertrand, and G. Gidel. Omega: Optimistic EMA Gradients. LatinX in AI workshop at ICML, 2023.
Juan Ramirez and J. Gallego-Posada. L₀onie: Compressing COINs with L₀-constraints. Sparsity in Neural Networks Workshop, 2022.
J. Gallego-Posada, Juan Ramirez, and A. Erraqabi. Flexible Learning of Sparse Neural Networks via Constrained L0 Regularization. LatinX in AI Workshop at NeurIPS, 2021.

Service

Associate Program Chair for ICML 2025.
Organizer of the first Constrained Optimization for Machine Learning workshop at NeurIPS 2025.
Reviewer for TMLR, ICLR 2026, ICML 2025, AISTATS 2025, and NeurIPS 2024. Reviewer for the LatinX in AI workshops at NeurIPS 2024, ICML 2024, and ICML 2023.
Helped review 2025 applicants for Mila and Khipu.
Social chair for the LatinX in AI workshop at NeurIPS 2024 and visa chair for the LatinX in AI workshop at ICML 2024.
Co-supervised (with Jose) the undergraduate research projects of Daniel Otero and Isabel Urrego, BSc students in mathematical engineering in Colombia (2023).

Teaching Assistantships

Fall 2023 and 2024: Probabilistic Graphical Models by Simon Lacoste-Julien.
Winter 2024: Adversarial Machine Learning by Gauthier Gidel.
Summer schools: Tropical Probabilistic AI School (2024), Lisbon Machine Learning School (2024).

Juan Camilo Ramirez — Research, Projects, Publications