# Differentiability of solution of optimization problem with respect to constraint (properties, conditions etc)

Assume you have the following simple constrained optimization problem:

$$$$\begin{split} &\min_x f(x) \\ & s.t \quad g(x) = c \end{split}$$$$ where $$f,g$$ are both differentiable.

What are the standard way to show that the problem solution (here I am referring to the objective minimum, and not the decision variable) is/is not differentiable with respect to $$c$$? I have run a few concrete examples and found that numerically the solution seems to be differentiable, but I have no idea how to show it more formally.

• What do you mean by: "solution is/is not differentiable with respect to $c$"? Jan 10 at 11:43
• Note that " $f,g$ are both differentiable for a given $c$" should be "$f,g$ are both differentiable" since neither depends on $c.$
– prubin
Jan 10 at 16:36
• @MatheusDiógenesAndrade i mean the objective minimum (i.e solution of the problem; note that I am not referring to the decision variable here) of this optimization problem would change w.r.t $c$, thus one can also define a certain notion of differentiability for the minimum with respect to $c$.
– c zl
Jan 15 at 5:12
• @prubin thanks for pointing it out, i edited that phrase away.
– c zl
Jan 15 at 5:12

There is a rich literature on parametric optimization, which deals with questions of when the optimal value/solution of an optimization problem is (sub)differentiable with respect to its parameters.

Classical references are:

1. Fiacco's book "Introduction to Sensitivity and Stability Analysis in Nonlinear Programming", available for purchase here
2. Bonnans and Shapiro's book "Perturbation Analysis of Optimization Problems", available for download/purchase here

A more approachable reference is:

In particular, Theorem 4.4 in Still's lecture notes provides general conditions under which the optimal value function of your problem is differentiable with respect to the parameter $$c$$.
In general, the optimization problem you have stated may not have a unique optimum, i.e, there may exist multiple optimal solutions. Consider, for instance, $$\min_{x,y} \{ x | x=c, 0 \leq y \leq 1\}$$: any solution $$(c,y)$$ is optimal.
In that case, the mapping from $$c$$ to an optimal solution of the problem is one-to-many, and is therefore not differentiable. This is often the case when $$f$$ and/or $$g$$ are non-convex: the optimum may not be unique, or it may jump to a distant point even under small perturbation.