Juan Camilo Ramirez De Los Rios

About

I am a third year PhD candidate at Mila and DIRO under the supervision of Simon Lacoste-Julien.

I develop constrained optimization algorithms for training deep learning models, which enable enforcing requirements such as fairness, sparsity and harmlessness in LLMs.

My research aims to ensure that training models with constraints is as easy as training without them.

Check Cooper out, a library for nonconvex constrained optimization in PyTorch that I co-founded.

Previously, I was an intern in Simon's group at Mila under the supervision of Jose Gallego-Posada. Before that, I completed a BSc in Mathematical Engineer at Universidad EAFIT. During the BSc, I spent a summer at McKinsey & Co. as a research intern.

Research interests: Nonconvex Constrained Optimization, Min-Max Optimization, Deep Learning, Sparsity, Fairness.

Contact: juan43ramirez (at) gmail (dot) com

News

2024

Sep 6: I succeded in my predoctoral exam. I am now a PhD candidate!
Sep 1: I will be a TA for Simon Lacoste-Julien's course on Probabilistic Graphical Models for a second time.
May 1: On PI Controllers for Updating Lagrange Multipliers in Constrained Optimization has been accepted at ICML 2024! Vienna, here we come again.
Mar 04: Delighted to receive Université de Montréal's AI scholarship 2023-2024!
Jan 29: Excited to be a TA at the first edition of the Tropical Probabilistic AI School in Rio de Janeiro, Brazil! I am happy to see more AI events popping up around Latin America.
Jan 16: Balancing Act: Constraining Disparate Impact in Sparse Models has been accepted at ICLR 2024! Vienna, here we come.
Jan 12: Quentin Bertrand and I will be TAing Gauthier Gidel's Adversarial Machine Learning course at Mila and the University of Montreal.

2023

Dec 7: Excited to give a talk at Montréal Machine Learning and Optimization (MTL MLOpt) about our recent work Balancing Act: Constraining Disparate Impact in Sparse Models.
Sep 4: Together with António Góis, I will be TAing Simon Lacoste-Julien's graduate course on Probabilistic Graphical Models at Mila.
Jul 23: Arriving in Honolulu, Hawaii, where I will have the pleasure to present Omega: Optimistic EMA Gradients. Omega, our method for stochastic min-max optimization, was awarded an oral at the LatinX in AI workshop at ICML 2023!
Jan 27: I will attend Khipu 2023 in Montevideo, Uruguay. Glad to be a part of the Latin American AI community.

Oct 14: See you at the PyTorch conference 2022 in New Orleans! Cooper will be making an appearance (poster).
Sep 14: I will be presenting my first NeurIPS paper: Controlled Sparsity via Constrained Optimization or: How I Learned to Stop Tuning Penalties and Love Constraints in New Orleans.
Sep 05: Excited to start a PhD in Artificial Intelligence under the supervision of Simon Lacoste-Julien at Mila and the University of Montreal.
Jul 25: Taking part in CIFAR's Deep Learning Reinforcement Learning Summer School.
Jul 8: Attending the Sparsity in Neural Networks workshop. We are presenting our latest work on implicit neural representations: L₀onie: Compressing COINs with L₀-constraints.

2021

Dec 17: Flexible Learning of Sparse Neural Networks via Constrained L0 Regularization, won the best poster award at the LatinX in AI Workshop @ NeurIPS 2021!
Nov 1: Starting an internship at Mila. I will join Simon Lacoste-Julien's group under the supervision of Jose Gallego-Posada.
Aug 2: during the next three weeks, I will attend (virtually) Neuromatch's Deep Learning Summer School.
Jul 1: I graduated from the BSc in Mathematical Engineering at Universidad EAFIT! Plus, got a mention for my contributions to the student organization. Check out my Linkedin post about it.
Jan 14: I am auditing Ioannis Mitliagkas' graduate course on Deep Learning Theory at Université de Montréal.

2020 and beyond

Jul 2020: Jose Gallego-Posada, PhD student at Mila will be my supervisor for my undergraduate thesis. I will be working on deep generative models.
Jun 2019: Now part of McKinsey & Co. in Belgium as a research intern. Working with Antoine Stevens and Patrick Dehout in ML for the agricultural and chemical industries.
Jan 2019: Arrived in Louvain-la-Neuve, Belgium for an exchange semestrer at the Université Catholique de Louvain.
Nov 2017: I have been appointed as president of CIGMA-OE, the Mathematical Engineering chapter of the Student Organization at Universidad EAFIT.
Dec 2015: I was awarded a full scholarship for the Bachelor's degree in Mathematical Engineering at Universidad EAFIT.
Dec 2015: Scored amongst the best 0.1% on the Colombian high school examination ICFES. Unfortunately, I did not obtain an Andrés Bello prize.
Dec 2015: Ranked first in the National Chemistry Olympiads of Universidad de Antioquia.

Publications

Conference

On PI Controllers for Updating Lagrange Multipliers in Constrained Optimization. M. Sohrabi*, Juan Ramirez*, T. H. Zhang, S. Lacoste-Julien and J. Gallego-Posada. ICML, 2024.
Balancing Act: Constraining Disparate Impact in Sparse Models. M. Hashemizadeh*, Juan Ramirez*, R. Sukumaran, G. Farnadi, S. Lacoste-Julien and J. Gallego-Posada. ICLR, 2024.
Controlled Sparsity via Constrained Optimization or: How I Learned to Stop Tuning Penalties and Love Constraints. J. Gallego-Posada, Juan Ramirez, A. Erraqabi, Y. Bengio and S. Lacoste-Julien. NeurIPS, 2022.

Workshop

Omega: Optimistic EMA Gradients. Juan Ramirez, R. Sukumaran, Q. Bertrand and G. Gidel. LatinX in AI workshop at ICML, 2023.
L₀onie: Compressing COINs with L₀-constraints. Juan Ramirez and J. Gallego-Posada. Sparsity in Neural Networks Workshop, 2022.
Flexible Learning of Sparse Neural Networks via Constrained L0 Regularization. J. Gallego-Posada, Juan Ramirez and A. Erraqabi. LatinX in AI Workshop at NeurIPS, 2021.

Libraries

Cooper: Constrained Optimization for Deep Learning. J. Gallego-Posada*, Juan Ramirez*, M. Hashemizadeh and Simon Lacoste-Julien. Computer software, 2024.

Teaching Assistantships

Fall 23 and 24: Probabilistic Graphical Models by Simon Lacoste-Julien
Winter 24: Adversarial Machine Learning by Gauthier Gidel
Summer schools: Tropical Probabilistic AI School 2024, Lisbon Machine Learning School 2024.

Service

Reviewer for NeurIPS 2024, for Khipu 2025 applications, and for the LatinX in AI workshops at NeurIPS 2024, ICML 2024, and ICML 2023.
Social chair for the LatinX in AI workshop at NeurIPS 2024 and visa chair for the LatinX in AI workshop at ICML 2024.
Cosupervised (with Jose) the undergraduate research projects of Daniel Otero and Isabel Urrego, BSc students in mathematical engineering in Colombia (2023).