How I Overcame the Optimization Wall by Implementing the Augmented Lagrangian Method (ALM) with AI

How I Overcame the Optimization Wall by Implementing the Augmented Lagrangian Method (ALM) with AI as My Sounding Board I am currently developing "co-opt", a web-based optical design software that runs entirely in the browser. In this post, I want to share the insights I gained while building a custom "constrained optimization engine" for lens design from scratch. Before we dive in, a quick disclaimer: the methods introduced in this article are likely well-known, standard approaches to experts in mathematical optimization. Also, to prioritize clarity, I will be breaking down the concepts so that anyone who has touched basic calculus in high school or early college can understand them. Please note that this means the article contains expressions that are not strictly mathematically rigorous . 1. Stumbling at the Starting Line: Trying to Optimize with Only the LM Method In lens design, we have specific goals, such as "I want the focal length to be exactly 50mm" or "I want to minimize opt