1
$\begingroup$

Let's consider the following deterministic constrained optimisation problem, where $c(t)$ is the control, and $x(t)$ and $y(t)$ are the state variables: \begin{align} J(t) = \inf_{c(t)} \ &\int_0^\infty e^{-\delta u} \left( x(u)^2 + \lambda y(u)^2 \right) du \\ s.t. \ &c(t) \geq - \xi \end{align} The dynamics of the states are: \begin{align} x'(t) &= c(t) - y(t) - a \\ y'(t) &= \delta y(t) - b x(t) \end{align} where $x(0)=x_0$ and where the following transversality condition holds: $\lim_{T \to \infty} E_t [e^{-\delta T} y(T)]=0$

To solve this problem, I write the extended Hamiltonian: \begin{align} \mathcal{H} = x(t)^2 + \lambda y(t)^2 + J_{x}(t) ( c(t) - y(t) - a ) + J_{y}(t) \left( \delta y(t) - b x(t) \right) - \mu_{c}(t) (c(t) + \xi) \end{align} where $J_{x}(t)$ and $J_{y}(t)$ denote the derivatives of the value function with respect to the states (i.e. the co-state variables), and $\mu_{c}(t)$ is the Lagrange multiplier.

Next, I write the three optimality conditions: \begin{align} 0 &= \mathcal{H}_{c}(t) \rightarrow J_{x}(t) = \mu_{c}(t) \nonumber \end{align} \begin{align} 0 &= \mathcal{H}_{x}(t) + \frac{dJ_{x}(t)}{dt} - \delta J_{x}(t) \rightarrow 0 = 2 x(t) - J_{y}(t) b + \frac{dJ_{x}(t)}{dt} - \delta J_{x}(t) \nonumber \end{align} \begin{align} 0 &= \mathcal{H}_{y}(t) + \frac{dJ_{y}(t)}{dt} - \delta J_{y}(t) \rightarrow 0 = 2 \lambda y(t) - J_{x}(t) + \frac{dJ_{y}(t)}{dt} \nonumber \end{align} with the complementary slackness conditions: \begin{align} 0 &= \mu_{c}(t) (c(t) + \xi) \rightarrow 0 = J_{x}(t) (c(t) + \xi) \nonumber \end{align} \begin{align} 0 \leq \mu_{c}(t) \rightarrow 0 \leq J_{x}(t) \nonumber \end{align}

I proceed to solve the problem as follows:

  1. I postulate the existence of a value of $x$, called $\hat{x}$, below which the constraint on the control is binding and above which it is not binding. This is reasonable given the context of the problem.
  2. I express $y(t)$ as a function of $x(t)$
  3. I solve the problem in the region where the constraint is not binding (i.e. where $J_x(t)=0$) finding a solution for $x(t)$ and $y(t)$ (it turns out $y(t) = k_1 x(t)$, where $k_1$ is a constant). In this region, the optimal control is a function of $x(t)$, i.e. $c(t) = f(x(t))$
  4. I solve the problem in the region where the constraint is binding (i.e. where $J_x(t) > 0$) finding a solution for $x(t)$ and $y(t)$ (it turns out $y(t) = k_2 + k_3 x(t)$ where $k_2$ and $k_3$ are constants). In this region, the optimal control is simply $c(t) = - \xi$

Question:

Can you suggest how I can determine the value of $\hat{x}$?

Attempts:

  1. I argue that since $c(t)$ is a regular control, there cannot be a jump in the state variables, so that: \begin{align} \lim_{x_t \to \hat{x}^-} y(x(t)) &= \lim_{x_t \to \hat{x}^+} y(x(t)) \\ \lim_{x_t \to \hat{x}^-} k_2 + k_3 x_t &= \lim_{x_t \to \hat{x}^+} k_1 x_t \\ \hat{x} &= \frac{k_2}{k_1 - k_3} \end{align}
  2. Alternatively, I argue that there cannot be a jump in the control, so that: \begin{align} - \xi &= \lim_{x_t \to \hat{x}^+} c(t) \\ - \xi &= \lim_{x_t \to \hat{x}^+} f(x(t)) \\ \hat{x} &= f^{-1}(-\xi) \end{align}

However, these two approaches give different values of $\hat{x}$.

I understand that without seeing the full solution it is hard to give an answer to this question, but I would appreciate also links to examples that show how to solve a problem of this type.

$\endgroup$

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.