Inconsistent solutions to linear optimal control problem

Question

Consider the following optimal control problem: \begin{align} J(t) = \inf_{u(t)} \ & \frac{1}{2} \int_0^\infty e^{-\delta t} \left( x(t)^2 + \lambda y(t)^2 \right) dt \\ s.t. \ &u(t) \geq - \xi \end{align} with state dynamics: \begin{align} \dot{x}(t) &= u(t) - y(t) - a \\ \dot{y}(t) &= \delta y(t) - c x(t) \end{align} where $x(0)$ given, and $\xi$, $\lambda$, $a$, $c$ are constants.

Question

Can you explain why these two approaches yield different results about the value of the optimal control when the constraint is not binding? In the first case it is $u^\ast(t) = + \infty$; in the second case it is $u^\ast(t) = a + (1 - c \lambda) y(t)$. The two approaches are those outlines in chapter 2 and 3 respectively of Sethi's book Optimal Control Theory.

Standard approach

The current value Hamiltonian is: \begin{align} H = \frac{1}{2} \Big( x(t)^2 + \lambda y(t)^2 \Big) + \mu_x(t) \big( u(t) - y(t) - a \big) + \mu_y(t) \big( \delta y(t) - c x(t) \big) \end{align} Because H is linear in the control, the optimal control is bang-bang: \begin{align} u^\ast(t) = \begin{cases} - \xi & if \ \mu_x(t) < 0 \\ any & if \ \mu_x(t) = 0\\ + \infty & if \ \mu_x(t) > 0 \end{cases} \end{align}

Mixed inequality constraint approach

The Lagrangean is: \begin{align} L &= H + \mu(t) \big(u(t) + \xi \big) \\ &= \frac{1}{2} \Big( x(t)^2 + \lambda y(t)^2 \Big) + \mu_x(t) \big( u(t) - y(t) - a \big) + \mu_y(t) \big( \delta y(t) - c x(t) \big) + \mu(t) \big(u(t) + \xi \big) \end{align} The lagrange multiplier satisfies the complementary slackness condition: \begin{align} \mu(t) &\geq 0 \\ \mu(t) \big(u(t) + \xi \big) &= 0 \end{align} The costate variables satisfy: \begin{align} \dot{\mu}_x(t) &= - x(t) + c \mu_y(t) + \delta \mu_x(t) \\ \dot{\mu}_y(t) &= - \lambda y(t) + \mu_x(t) \end{align} Suppose the control constraint is not binding for $t \in [t_1,t_2]$. Then it must be that $\mu(t) = 0$ for $t \in [t_1,t_2]$. It follows: \begin{align} x(t) &= c \mu_y(t) \\ \dot{\mu}_y(t) &= - \lambda y(t) \end{align} Differentiating the first equation and equating it to the second one: \begin{align} \dot{x}(t) &= - c \lambda y(t) = u(t) - y(t) - a \end{align} It follows that: \begin{align} u^\ast(t) = a + (1 - c \lambda) y(t) \end{align}

Answer 1

One of the necessary conditions is $L_u = 0$. If you assume that $\mu(t) = 0$, it follows from $L_u = \mu_x(t) + \mu(t) = 0$ that $\mu_x(t) = 0$. Because $u(t)$ can take any value when $\mu_x(t) = 0$, the finding that $u(t) = a + (1 - c \lambda) y(t)$ is not a contradiction.