I can explain why Lagrange multipliers work for scalar functions by vector calculus. Consider optimizing $f(\vec{x})$ subjected to the constraint $g(\vec{x}) = c$.

At the optima, we can move infinitesimally within the feasible region without changing the value of $f(\vec{x})$. This infinitesimal movement (and any movement in the feasible region) must be along the constraint, which is one level curve of $g$. Since the infinitesimal movement doesn't change the value of $f$, the infinitesimal movement is also along the level curve of $f$.

As the gradient of a function at any point is perpendicular to the function's level curve, we have $\nabla f = \lambda \nabla g$.

Where can I find a similar explanation for KKT conditions?

  • 1
    $\begingroup$ Take a look at chapters 4 and 5 of this book. The link is to the book Non-linear programming by Bazarra. $\endgroup$
    – batwing
    Jul 3 '21 at 14:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.