Skip to main content
found an approximate representation for -log(t)log(1-t)
Source Link

I haven't been able yet to represent those using DCP, but I found some interesting things:

This is the tricky one. First note that:I had given up already, when I found this wikipedia page on the Dilogarithm function.

\begin{equation} \log(t)\log(1-t) = \frac{1}{2}\left(\log^2(t(1-t)) - \log^2\left(\frac{t}{1-t}\right)\right), \end{equation}\begin{equation} \text{Li}_2(z) = -\int_{0}^{z}\frac{\log(1-t)}{t}dt = -\int_{0}^{1}\frac{\log(1-z t)}{t}dt, \end{equation}

since $2cd = (c+d)^2 - (c-d)^2$, for any $c,d$.and:

Now form the optimization problem: \begin{align} \min_{z}\,-\frac{z}{2} &\qquad\text{s.t.: }\\ &2z \leq w-v,\\ &\,\sqrt{v} \geq \left\lvert\log\left(\frac{t}{1-t}\right)\right\rvert,\tag{1}\\ &\,\sqrt{w} \leq \lvert\log(t(1-t))\rvert\tag{2}, \end{align}\begin{equation} \text{Li}_2(z) + \text{Li}_2(1-z) = \frac{1}{6}\pi^2 - \log(z)\log(1-z) \end{equation}

Now we dealcan approximate the integral with constraints (1) and (2):

Fornumerical integration (1trapezoidal rule) Note that: \begin{equation} \lvert\log(t)-\log(1-t)\rvert = \text{ptp}{([-\log(t),-\log(1-t)])}, \end{equation}\begin{equation} -\int_{0}^{z}\frac{\log(1-t)}{t}dt \approx -\frac{1}{N_t}\sum_{k=1}^{N_t}\frac{1}{2}\left(\frac{\log(1-z t_k)}{t_k} + \frac{\log(1-z t_{k-1})}{t_{k-1}}\right), \end{equation} where $\text{ptp}$ is the peak-to-peak atom in CVX, which is convex. Since$t_k$ for $-\log(\cdot)$ is a convex atom, we're set. I can't seem to find$k\in\{0,N_t\}$ discretizes the interval cp.ptp when I try to use it in CVXPY, though$[0,1]$. Is it really implemented?

Now for (2), note that, since $t \in (0,1)$, we have that : \begin{equation} \lvert\log(t(1-t))\rvert = -\log(t(1-t)). \end{equation}

Thus, we can write (2) as:

\begin{align} &\sqrt{w} + \log(t(1-t)) &\leq 0 &\iff\\ \sqrt{w}t + \sqrt{w}(1-t) &+\log(t) +\log(1-t) &\leq0&\iff\\ \frac{t}{t}\log\left(\frac{t}{e^{-\sqrt{w}t}}\right)&+ \frac{1-t}{1-t}\log\left(\frac{1-t}{e^{-\sqrt{w}(1-t)}}\right)&\leq0 &\iff\\ \frac{1}{t}D_\text{KL}(t \lvert\rvert e^{t h(w)}) &+ \frac{1}{1-t}D_\text{KL}((1-t) \lvert\rvert e^{(1-t)h(w)})&\leq0 &\iff\\ \mathcal{P}\left(g(x,y),s\right)\bigg\rvert_{x=1, y=h(w), s=\frac{1}{t}} &+ \mathcal{P}\left(g(x,y),s\right)\bigg\rvert_{x=1, y=h(w), s=\frac{1}{1-t}}&\leq0, \end{align}

where $h(w) = -\sqrt{w}$, $D_\text{KL}(x \lvert\rvert y)$ is the relative entropy between x and y, $g(x,y) = D_\text{KL}(x \lvert\rvert e^y)$, and $\mathcal{P}$this approximation is the perspective transform of a functionconvex DCP expression.

I think this might work, it's allguess the compositiononly thing missing now is to write the remaining constraint for the change of simpler convex functionsvariable.

I haven't been able yet to represent those using DCP, but I found some interesting things:

This is the tricky one. First note that:

\begin{equation} \log(t)\log(1-t) = \frac{1}{2}\left(\log^2(t(1-t)) - \log^2\left(\frac{t}{1-t}\right)\right), \end{equation}

since $2cd = (c+d)^2 - (c-d)^2$, for any $c,d$.

Now form the optimization problem: \begin{align} \min_{z}\,-\frac{z}{2} &\qquad\text{s.t.: }\\ &2z \leq w-v,\\ &\,\sqrt{v} \geq \left\lvert\log\left(\frac{t}{1-t}\right)\right\rvert,\tag{1}\\ &\,\sqrt{w} \leq \lvert\log(t(1-t))\rvert\tag{2}, \end{align}

Now we deal with constraints (1) and (2):

For (1) Note that: \begin{equation} \lvert\log(t)-\log(1-t)\rvert = \text{ptp}{([-\log(t),-\log(1-t)])}, \end{equation} where $\text{ptp}$ is the peak-to-peak atom in CVX, which is convex. Since $-\log(\cdot)$ is a convex atom, we're set. I can't seem to find cp.ptp when I try to use it in CVXPY, though. Is it really implemented?

Now for (2), note that, since $t \in (0,1)$, we have that : \begin{equation} \lvert\log(t(1-t))\rvert = -\log(t(1-t)). \end{equation}

Thus, we can write (2) as:

\begin{align} &\sqrt{w} + \log(t(1-t)) &\leq 0 &\iff\\ \sqrt{w}t + \sqrt{w}(1-t) &+\log(t) +\log(1-t) &\leq0&\iff\\ \frac{t}{t}\log\left(\frac{t}{e^{-\sqrt{w}t}}\right)&+ \frac{1-t}{1-t}\log\left(\frac{1-t}{e^{-\sqrt{w}(1-t)}}\right)&\leq0 &\iff\\ \frac{1}{t}D_\text{KL}(t \lvert\rvert e^{t h(w)}) &+ \frac{1}{1-t}D_\text{KL}((1-t) \lvert\rvert e^{(1-t)h(w)})&\leq0 &\iff\\ \mathcal{P}\left(g(x,y),s\right)\bigg\rvert_{x=1, y=h(w), s=\frac{1}{t}} &+ \mathcal{P}\left(g(x,y),s\right)\bigg\rvert_{x=1, y=h(w), s=\frac{1}{1-t}}&\leq0, \end{align}

where $h(w) = -\sqrt{w}$, $D_\text{KL}(x \lvert\rvert y)$ is the relative entropy between x and y, $g(x,y) = D_\text{KL}(x \lvert\rvert e^y)$, and $\mathcal{P}$ is the perspective transform of a function.

I think this might work, it's all the composition of simpler convex functions.

I found some interesting things:

This is the tricky one. I had given up already, when I found this wikipedia page on the Dilogarithm function.

\begin{equation} \text{Li}_2(z) = -\int_{0}^{z}\frac{\log(1-t)}{t}dt = -\int_{0}^{1}\frac{\log(1-z t)}{t}dt, \end{equation}

and:

\begin{equation} \text{Li}_2(z) + \text{Li}_2(1-z) = \frac{1}{6}\pi^2 - \log(z)\log(1-z) \end{equation}

Now we can approximate the integral with numerical integration (trapezoidal rule): \begin{equation} -\int_{0}^{z}\frac{\log(1-t)}{t}dt \approx -\frac{1}{N_t}\sum_{k=1}^{N_t}\frac{1}{2}\left(\frac{\log(1-z t_k)}{t_k} + \frac{\log(1-z t_{k-1})}{t_{k-1}}\right), \end{equation} where $t_k$ for $k\in\{0,N_t\}$ discretizes the interval $[0,1]$.

Now this approximation is a convex DCP expression.

I guess the only thing missing now is to write the remaining constraint for the change of variable.

fixed typo
Source Link

I have the following objective that I want to maximize:

\begin{equation} \max_{U_T\in \mathbb{R}, x\in\mathbb{R}^T} J_\alpha(U_T) = \frac{\alpha}{\alpha-1}\log\left(\frac{\cosh(U_T)}{\cosh(\alpha U_T)^\frac{1}{\alpha}}\right) \,,\qquad \text{s.t:} \qquad U_T= Ax+b. \end{equation} where $A \in \mathbb{R}^{1\times T}$, $b\in\mathbb{R}$ and $\alpha>1$.

It is easy to show that this is, in fact, a pseudo-concave function by checking that: \begin{equation} \eta'_\alpha(u)(v-u)\leq0 \implies \eta_\alpha(v)\leq \eta_\alpha(u), \end{equation} where $\eta_\alpha(u) = \frac{\cosh(u)}{\cosh(\alpha u)^\frac{1}{\alpha}}$.

This, of course, implies the quasiconcavity of $J_\alpha(U_T)$.

Here's a plot of the function that I made with Wolfram Mathematica: Plot of <span class=$J_\alpha(x)$ for $\alpha=2$." />

-My questions are:

  1. Is there a way to write this objective function as a DQCP compliant program in CVXPY?
  2. When $\alpha \to 1^+$, $J_1(u) := -\log[\frac{e^{u\tanh(u)}}{\cosh(u)} ]$ pointwise. Is there a way to write that as a DQCP compliant program in CVXPY as well?
  3. If it can't be put into a DCP compliant formulation, what would you recommend that I do to solve this problem numerically (especially question 2)?

I'm using CVXPY since I'll add more complicated constraints later, but maybe the problem has a simple analytical solution as well.


-What I have so far:

I am especially interested in question 2, so I'm trying to solve that first.

Using the definitions of $\cosh$, write: \begin{align} J_1(u) &:= \lim_{\alpha\to 1^+}J_\alpha(u)\\ &= \log(\cosh(u)) -u \tanh(u)\\ &= \log\left(e^u + e^{-u}\right)-u \tanh(u) + K\\ &= -u\left(\log\left(\frac{1}{1 + e^{-2u}}\right)+ \tanh(u)\right) + K\\ &= -\frac{1}{2}f^{-1}(f(2u))(\log(f(2u))- \tanh(u)) + K, \end{align} where $K$ is a constant and $f(y) := 1/(1+e^{-y})$ is the logistic function, with inverse given by $f^{-1}(y) = \log(\frac{y}{1-y})$. By the well-known relation (Wiki): \begin{equation}\tanh(y) = 2f(2y) - 1,\end{equation} and we can conclude that: \begin{equation} J_1(u) = T(f(2u)) + K, \end{equation} where $T:[0,1]\to\mathbb{R}$ is: \begin{equation}T(z) = -\frac{1}{2}\log\left(\frac{z}{1-z}\right)\left(\log(z) - 2z-1\right).\end{equation}

T is in fact concave, as we can see from the graph of its second derivative in Mathematica:

Plot of <span class=$T(z)$" />

By further manipulation of $T$, we get the expression:

\begin{align} -2T(z) &= \log\left(\frac{z}{1-z}\right)(z + z - 1 + \log(z))\\ &= z\log\left(\frac{z}{1-z}\right) + (1-z)\log\left(\frac{1-z}{z}\right) + \log\left(\frac{z}{1-z}\right)\log(z)\\ &= 2\text{JS}(t \vert\vert 1-t) + \log^2(z) - \log(z)\log(1-z), \end{align} where $\text{JS}$ is the Jensen-Shannon entropy (simmetrized KL divergence).

Note the right-hand side is a sum of convex functions, as $\log^2(z)$ and $-\log(z)\log(1-z)$ are convex in $(0,1)$).

Therefore, I think I would be able to write the original program by optimizing $T$ and adding a (quasilinear) constraint to account for the change of variable. Question 2 finally reduces to finding a DCP compliant representation of $T$.


Update: Progress on trying to find a DCP representation of $\log^2(t)$ and $-\log(t)\log(1-t)$:

I haven't been able yet to represent those using DCP, but I found some interesting things:

a. $\log^2(t)$:

We can write this as: \begin{equation} \min_{z} z \qquad\text{s.t.: } w\geq -\log(t),\,w\geq 0,\,z\geq w^2, \end{equation} for fixed $t \in (0,1)$.

b. $-\log(t)\log(1-t)$:

This is the tricky one. First note that:

\begin{equation} \log(t)\log(1-t) = \frac{1}{2}\left(\log^2(t(1-t)) - \log^2\left(\frac{t}{1-t}\right)\right), \end{equation}

since $2cd = (c+d)^2 - (c-d)^2$, for any $c,d$.

Now form the optimization problem: \begin{align} \min_{z}\,-z &\qquad\text{s.t.: }\\ &2z \leq w-v,\\ &\,\sqrt{v} \geq \left\lvert\log\left(\frac{t}{1-t}\right)\right\rvert,\tag{1}\\ &\,\sqrt{w} \leq \lvert\log(t(1-t))\rvert\tag{2}, \end{align}\begin{align} \min_{z}\,-\frac{z}{2} &\qquad\text{s.t.: }\\ &2z \leq w-v,\\ &\,\sqrt{v} \geq \left\lvert\log\left(\frac{t}{1-t}\right)\right\rvert,\tag{1}\\ &\,\sqrt{w} \leq \lvert\log(t(1-t))\rvert\tag{2}, \end{align}

Now we deal with constraints (1) and (2):

For (1) Note that: \begin{equation} \lvert\log(t)-\log(1-t)\rvert = \text{ptp}{([-\log(t),-\log(1-t)])}, \end{equation} where $\text{ptp}$ is the peak-to-peak atom in CVX, which is convex. Since $-\log(\cdot)$ is a convex atom, we're set. I can't seem to find cp.ptp when I try to use it in CVXPY, though. Is it really implemented?

Now for (2), note that, since $t \in (0,1)$, we have that : \begin{equation} \lvert\log(t(1-t))\rvert = -\log(t(1-t)). \end{equation}

Thus, we can write (2) as:

\begin{align} &\sqrt{w} + \log(t(1-t)) &\leq 0 &\iff\\ \sqrt{w}t + \sqrt{w}(1-t) &+\log(t) +\log(1-t) &\leq0&\iff\\ \frac{t}{t}\log\left(\frac{t}{e^{-\sqrt{w}t}}\right)&+ \frac{1-t}{1-t}\log\left(\frac{1-t}{e^{-\sqrt{w}(1-t)}}\right)&\leq0 &\iff\\ \frac{1}{t}D_\text{KL}(t \lvert\rvert e^{t h(w)}) &+ \frac{1}{1-t}D_\text{KL}((1-t) \lvert\rvert e^{(1-t)h(w)})&\leq0 &\iff\\ \mathcal{P}\left(g(x,y),s\right)\bigg\rvert_{x=1, y=h(w), s=\frac{1}{t}} &+ \mathcal{P}\left(g(x,y),s\right)\bigg\rvert_{x=1, y=h(w), s=\frac{1}{1-t}}&\leq0, \end{align}

where $h(w) = -\sqrt{w}$, $D_\text{KL}(x \lvert\rvert y)$ is the relative entropy between x and y, $g(x,y) = D_\text{KL}(x \lvert\rvert e^y)$, and $\mathcal{P}$ is the perspective transform of a function.

I think this might work, it's all the composition of simpler convex functions.

I have the following objective that I want to maximize:

\begin{equation} \max_{U_T\in \mathbb{R}, x\in\mathbb{R}^T} J_\alpha(U_T) = \frac{\alpha}{\alpha-1}\log\left(\frac{\cosh(U_T)}{\cosh(\alpha U_T)^\frac{1}{\alpha}}\right) \,,\qquad \text{s.t:} \qquad U_T= Ax+b. \end{equation} where $A \in \mathbb{R}^{1\times T}$, $b\in\mathbb{R}$ and $\alpha>1$.

It is easy to show that this is, in fact, a pseudo-concave function by checking that: \begin{equation} \eta'_\alpha(u)(v-u)\leq0 \implies \eta_\alpha(v)\leq \eta_\alpha(u), \end{equation} where $\eta_\alpha(u) = \frac{\cosh(u)}{\cosh(\alpha u)^\frac{1}{\alpha}}$.

This, of course, implies the quasiconcavity of $J_\alpha(U_T)$.

Here's a plot of the function that I made with Wolfram Mathematica: Plot of <span class=$J_\alpha(x)$ for $\alpha=2$." />

-My questions are:

  1. Is there a way to write this objective function as a DQCP compliant program in CVXPY?
  2. When $\alpha \to 1^+$, $J_1(u) := -\log[\frac{e^{u\tanh(u)}}{\cosh(u)} ]$ pointwise. Is there a way to write that as a DQCP compliant program in CVXPY as well?
  3. If it can't be put into a DCP compliant formulation, what would you recommend that I do to solve this problem numerically (especially question 2)?

I'm using CVXPY since I'll add more complicated constraints later, but maybe the problem has a simple analytical solution as well.


-What I have so far:

I am especially interested in question 2, so I'm trying to solve that first.

Using the definitions of $\cosh$, write: \begin{align} J_1(u) &:= \lim_{\alpha\to 1^+}J_\alpha(u)\\ &= \log(\cosh(u)) -u \tanh(u)\\ &= \log\left(e^u + e^{-u}\right)-u \tanh(u) + K\\ &= -u\left(\log\left(\frac{1}{1 + e^{-2u}}\right)+ \tanh(u)\right) + K\\ &= -\frac{1}{2}f^{-1}(f(2u))(\log(f(2u))- \tanh(u)) + K, \end{align} where $K$ is a constant and $f(y) := 1/(1+e^{-y})$ is the logistic function, with inverse given by $f^{-1}(y) = \log(\frac{y}{1-y})$. By the well-known relation (Wiki): \begin{equation}\tanh(y) = 2f(2y) - 1,\end{equation} and we can conclude that: \begin{equation} J_1(u) = T(f(2u)) + K, \end{equation} where $T:[0,1]\to\mathbb{R}$ is: \begin{equation}T(z) = -\frac{1}{2}\log\left(\frac{z}{1-z}\right)\left(\log(z) - 2z-1\right).\end{equation}

T is in fact concave, as we can see from the graph of its second derivative in Mathematica:

Plot of <span class=$T(z)$" />

By further manipulation of $T$, we get the expression:

\begin{align} -2T(z) &= \log\left(\frac{z}{1-z}\right)(z + z - 1 + \log(z))\\ &= z\log\left(\frac{z}{1-z}\right) + (1-z)\log\left(\frac{1-z}{z}\right) + \log\left(\frac{z}{1-z}\right)\log(z)\\ &= 2\text{JS}(t \vert\vert 1-t) + \log^2(z) - \log(z)\log(1-z), \end{align} where $\text{JS}$ is the Jensen-Shannon entropy (simmetrized KL divergence).

Note the right-hand side is a sum of convex functions, as $\log^2(z)$ and $-\log(z)\log(1-z)$ are convex in $(0,1)$).

Therefore, I think I would be able to write the original program by optimizing $T$ and adding a (quasilinear) constraint to account for the change of variable. Question 2 finally reduces to finding a DCP compliant representation of $T$.


Update: Progress on trying to find a DCP representation of $\log^2(t)$ and $-\log(t)\log(1-t)$:

I haven't been able yet to represent those using DCP, but I found some interesting things:

a. $\log^2(t)$:

We can write this as: \begin{equation} \min_{z} z \qquad\text{s.t.: } w\geq -\log(t),\,w\geq 0,\,z\geq w^2, \end{equation} for fixed $t \in (0,1)$.

b. $-\log(t)\log(1-t)$:

This is the tricky one. First note that:

\begin{equation} \log(t)\log(1-t) = \frac{1}{2}\left(\log^2(t(1-t)) - \log^2\left(\frac{t}{1-t}\right)\right), \end{equation}

since $2cd = (c+d)^2 - (c-d)^2$, for any $c,d$.

Now form the optimization problem: \begin{align} \min_{z}\,-z &\qquad\text{s.t.: }\\ &2z \leq w-v,\\ &\,\sqrt{v} \geq \left\lvert\log\left(\frac{t}{1-t}\right)\right\rvert,\tag{1}\\ &\,\sqrt{w} \leq \lvert\log(t(1-t))\rvert\tag{2}, \end{align}

Now we deal with constraints (1) and (2):

For (1) Note that: \begin{equation} \lvert\log(t)-\log(1-t)\rvert = \text{ptp}{([-\log(t),-\log(1-t)])}, \end{equation} where $\text{ptp}$ is the peak-to-peak atom in CVX, which is convex. Since $-\log(\cdot)$ is a convex atom, we're set. I can't seem to find cp.ptp when I try to use it in CVXPY, though. Is it really implemented?

Now for (2), note that, since $t \in (0,1)$, we have that : \begin{equation} \lvert\log(t(1-t))\rvert = -\log(t(1-t)). \end{equation}

Thus, we can write (2) as:

\begin{align} &\sqrt{w} + \log(t(1-t)) &\leq 0 &\iff\\ \sqrt{w}t + \sqrt{w}(1-t) &+\log(t) +\log(1-t) &\leq0&\iff\\ \frac{t}{t}\log\left(\frac{t}{e^{-\sqrt{w}t}}\right)&+ \frac{1-t}{1-t}\log\left(\frac{1-t}{e^{-\sqrt{w}(1-t)}}\right)&\leq0 &\iff\\ \frac{1}{t}D_\text{KL}(t \lvert\rvert e^{t h(w)}) &+ \frac{1}{1-t}D_\text{KL}((1-t) \lvert\rvert e^{(1-t)h(w)})&\leq0 &\iff\\ \mathcal{P}\left(g(x,y),s\right)\bigg\rvert_{x=1, y=h(w), s=\frac{1}{t}} &+ \mathcal{P}\left(g(x,y),s\right)\bigg\rvert_{x=1, y=h(w), s=\frac{1}{1-t}}&\leq0, \end{align}

where $h(w) = -\sqrt{w}$, $D_\text{KL}(x \lvert\rvert y)$ is the relative entropy between x and y, $g(x,y) = D_\text{KL}(x \lvert\rvert e^y)$, and $\mathcal{P}$ is the perspective transform of a function.

I think this might work, it's all the composition of simpler convex functions.

I have the following objective that I want to maximize:

\begin{equation} \max_{U_T\in \mathbb{R}, x\in\mathbb{R}^T} J_\alpha(U_T) = \frac{\alpha}{\alpha-1}\log\left(\frac{\cosh(U_T)}{\cosh(\alpha U_T)^\frac{1}{\alpha}}\right) \,,\qquad \text{s.t:} \qquad U_T= Ax+b. \end{equation} where $A \in \mathbb{R}^{1\times T}$, $b\in\mathbb{R}$ and $\alpha>1$.

It is easy to show that this is, in fact, a pseudo-concave function by checking that: \begin{equation} \eta'_\alpha(u)(v-u)\leq0 \implies \eta_\alpha(v)\leq \eta_\alpha(u), \end{equation} where $\eta_\alpha(u) = \frac{\cosh(u)}{\cosh(\alpha u)^\frac{1}{\alpha}}$.

This, of course, implies the quasiconcavity of $J_\alpha(U_T)$.

Here's a plot of the function that I made with Wolfram Mathematica: Plot of <span class=$J_\alpha(x)$ for $\alpha=2$." />

-My questions are:

  1. Is there a way to write this objective function as a DQCP compliant program in CVXPY?
  2. When $\alpha \to 1^+$, $J_1(u) := -\log[\frac{e^{u\tanh(u)}}{\cosh(u)} ]$ pointwise. Is there a way to write that as a DQCP compliant program in CVXPY as well?
  3. If it can't be put into a DCP compliant formulation, what would you recommend that I do to solve this problem numerically (especially question 2)?

I'm using CVXPY since I'll add more complicated constraints later, but maybe the problem has a simple analytical solution as well.


-What I have so far:

I am especially interested in question 2, so I'm trying to solve that first.

Using the definitions of $\cosh$, write: \begin{align} J_1(u) &:= \lim_{\alpha\to 1^+}J_\alpha(u)\\ &= \log(\cosh(u)) -u \tanh(u)\\ &= \log\left(e^u + e^{-u}\right)-u \tanh(u) + K\\ &= -u\left(\log\left(\frac{1}{1 + e^{-2u}}\right)+ \tanh(u)\right) + K\\ &= -\frac{1}{2}f^{-1}(f(2u))(\log(f(2u))- \tanh(u)) + K, \end{align} where $K$ is a constant and $f(y) := 1/(1+e^{-y})$ is the logistic function, with inverse given by $f^{-1}(y) = \log(\frac{y}{1-y})$. By the well-known relation (Wiki): \begin{equation}\tanh(y) = 2f(2y) - 1,\end{equation} and we can conclude that: \begin{equation} J_1(u) = T(f(2u)) + K, \end{equation} where $T:[0,1]\to\mathbb{R}$ is: \begin{equation}T(z) = -\frac{1}{2}\log\left(\frac{z}{1-z}\right)\left(\log(z) - 2z-1\right).\end{equation}

T is in fact concave, as we can see from the graph of its second derivative in Mathematica:

Plot of <span class=$T(z)$" />

By further manipulation of $T$, we get the expression:

\begin{align} -2T(z) &= \log\left(\frac{z}{1-z}\right)(z + z - 1 + \log(z))\\ &= z\log\left(\frac{z}{1-z}\right) + (1-z)\log\left(\frac{1-z}{z}\right) + \log\left(\frac{z}{1-z}\right)\log(z)\\ &= 2\text{JS}(t \vert\vert 1-t) + \log^2(z) - \log(z)\log(1-z), \end{align} where $\text{JS}$ is the Jensen-Shannon entropy (simmetrized KL divergence).

Note the right-hand side is a sum of convex functions, as $\log^2(z)$ and $-\log(z)\log(1-z)$ are convex in $(0,1)$).

Therefore, I think I would be able to write the original program by optimizing $T$ and adding a (quasilinear) constraint to account for the change of variable. Question 2 finally reduces to finding a DCP compliant representation of $T$.


Update: Progress on trying to find a DCP representation of $\log^2(t)$ and $-\log(t)\log(1-t)$:

I haven't been able yet to represent those using DCP, but I found some interesting things:

a. $\log^2(t)$:

We can write this as: \begin{equation} \min_{z} z \qquad\text{s.t.: } w\geq -\log(t),\,w\geq 0,\,z\geq w^2, \end{equation} for fixed $t \in (0,1)$.

b. $-\log(t)\log(1-t)$:

This is the tricky one. First note that:

\begin{equation} \log(t)\log(1-t) = \frac{1}{2}\left(\log^2(t(1-t)) - \log^2\left(\frac{t}{1-t}\right)\right), \end{equation}

since $2cd = (c+d)^2 - (c-d)^2$, for any $c,d$.

Now form the optimization problem: \begin{align} \min_{z}\,-\frac{z}{2} &\qquad\text{s.t.: }\\ &2z \leq w-v,\\ &\,\sqrt{v} \geq \left\lvert\log\left(\frac{t}{1-t}\right)\right\rvert,\tag{1}\\ &\,\sqrt{w} \leq \lvert\log(t(1-t))\rvert\tag{2}, \end{align}

Now we deal with constraints (1) and (2):

For (1) Note that: \begin{equation} \lvert\log(t)-\log(1-t)\rvert = \text{ptp}{([-\log(t),-\log(1-t)])}, \end{equation} where $\text{ptp}$ is the peak-to-peak atom in CVX, which is convex. Since $-\log(\cdot)$ is a convex atom, we're set. I can't seem to find cp.ptp when I try to use it in CVXPY, though. Is it really implemented?

Now for (2), note that, since $t \in (0,1)$, we have that : \begin{equation} \lvert\log(t(1-t))\rvert = -\log(t(1-t)). \end{equation}

Thus, we can write (2) as:

\begin{align} &\sqrt{w} + \log(t(1-t)) &\leq 0 &\iff\\ \sqrt{w}t + \sqrt{w}(1-t) &+\log(t) +\log(1-t) &\leq0&\iff\\ \frac{t}{t}\log\left(\frac{t}{e^{-\sqrt{w}t}}\right)&+ \frac{1-t}{1-t}\log\left(\frac{1-t}{e^{-\sqrt{w}(1-t)}}\right)&\leq0 &\iff\\ \frac{1}{t}D_\text{KL}(t \lvert\rvert e^{t h(w)}) &+ \frac{1}{1-t}D_\text{KL}((1-t) \lvert\rvert e^{(1-t)h(w)})&\leq0 &\iff\\ \mathcal{P}\left(g(x,y),s\right)\bigg\rvert_{x=1, y=h(w), s=\frac{1}{t}} &+ \mathcal{P}\left(g(x,y),s\right)\bigg\rvert_{x=1, y=h(w), s=\frac{1}{1-t}}&\leq0, \end{align}

where $h(w) = -\sqrt{w}$, $D_\text{KL}(x \lvert\rvert y)$ is the relative entropy between x and y, $g(x,y) = D_\text{KL}(x \lvert\rvert e^y)$, and $\mathcal{P}$ is the perspective transform of a function.

I think this might work, it's all the composition of simpler convex functions.

Added some advances on the dcp objective
Source Link

I have the following objective that I want to maximize:

\begin{equation} \max_{U_T\in \mathbb{R}, x\in\mathbb{R}^T} J_\alpha(U_T) = \frac{\alpha}{\alpha-1}\log\left(\frac{\cosh(U_T)}{\cosh(\alpha U_T)^\frac{1}{\alpha}}\right) \,,\qquad \text{s.t:} \qquad U_T= Ax+b. \end{equation} where $A \in \mathbb{R}^{1\times T}$, $b\in\mathbb{R}$ and $\alpha>1$.

It is easy to show that this is, in fact, a pseudo-concave function by checking that: \begin{equation} \eta'_\alpha(u)(v-u)\leq0 \implies \eta_\alpha(v)\leq \eta_\alpha(u), \end{equation} where $\eta_\alpha(u) = \frac{\cosh(u)}{\cosh(\alpha u)^\frac{1}{\alpha}}$.

This, of course, implies the quasiconcavity of $J_\alpha(U_T)$.

Here's a plot of the function that I made with Wolfram Mathematica: Plot of <span class=$J_\alpha(x)$ for $\alpha=2$." />

-My questions are:

  1. Is there a way to write this objective function as a DQCP compliant program in CVXPY?
  2. When $\alpha \to 1^+$, $J_1(u) := -\log[\frac{e^{u\tanh(u)}}{\cosh(u)} ]$ pointwise. Is there a way to write that as a DQCP compliant program in CVXPY as well?
  3. If it can't be put into a DCP compliant formulation, what would you recommend that I do to solve this problem numerically (especially question 2)?

I'm using CVXPY since I'll add more complicated constraints later, but maybe the problem has a simple analytical solution as well.


-What I have so far:

I am especially interested in question 2, so I'm trying to solve that first.

Using the definitions of $\cosh$, write: \begin{align} J_1(u) &:= \lim_{\alpha\to 1^+}J_\alpha(u)\\ &= \log(\cosh(u)) -u \tanh(u)\\ &= \log\left(e^u + e^{-u}\right)-u \tanh(u) + K\\ &= -u\left(\log\left(\frac{1}{1 + e^{-2u}}\right)+ \tanh(u)\right) + K\\ &= -\frac{1}{2}f^{-1}(f(2u))(\log(f(2u))- \tanh(u)) + K, \end{align} where $K$ is a constant and $f(y) := 1/(1+e^{-y})$ is the logistic function, with inverse given by $f^{-1}(y) = \log(\frac{y}{1-y})$. By the well-known relation (Wiki): \begin{equation}\tanh(y) = 2f(2y) - 1,\end{equation} and we can conclude that: \begin{equation} J_1(u) = T(f(2u)) + K, \end{equation} where $T:[0,1]\to\mathbb{R}$ is: \begin{equation}T(z) = -\frac{1}{2}\log\left(\frac{z}{1-z}\right)\left(\log(z) - 2z-1\right).\end{equation}

T is in fact concave, as we can see from the graph of its second derivative in Mathematica:

Plot of <span class=$T(z)$" />

By further manipulation of $T$, we get the expression:

\begin{align} -2T(z) &= \log\left(\frac{z}{1-z}\right)(z + z - 1 + \log(z))\\ &= z\log\left(\frac{z}{1-z}\right) + (1-z)\log\left(\frac{1-z}{z}\right) + \log\left(\frac{z}{1-z}\right)\log(z)\\ &= 2\text{JS}(t \vert\vert 1-t) + \log^2(z) - \log(z)\log(1-z), \end{align} where $\text{JS}$ is the Jensen-Shannon entropy (simmetrized KL divergence).

Note the right-hand side is a sum of convex functions, as $\log^2(z)$ and $-\log(z)\log(1-z)$ are convex in $(0,1)$(0,1)$).

Therefore, I think I would be able to write the original program by optimizing for T1$T$ and adding a (quasilinear) constraint to account for the change of variable. Question 2 finally reduces to finding a DCP compliant representation of $T$.


Update: Progress on trying to find a DCP representation of $\log^2(t)$ and $-\log(t)\log(1-t)$:

I haven't been able yet to represent those using DCP, but I found some interesting things:

a. $\log^2(t)$:

We can write this as: \begin{equation} \min_{z} z \qquad\text{s.t.: } w\geq -\log(t),\,w\geq 0,\,z\geq w^2, \end{equation} for fixed $t \in (0,1)$.

b. $-\log(t)\log(1-t)$:

This is the tricky one. First note that:

\begin{equation} \log(t)\log(1-t) = \frac{1}{2}\left(\log^2(t(1-t)) - \log^2\left(\frac{t}{1-t}\right)\right), \end{equation}

since $2cd = (c+d)^2 - (c-d)^2$, for any $c,d$.

Now form the optimization problem: \begin{align} \min_{z}\,-z &\qquad\text{s.t.: }\\ &2z \leq w-v,\\ &\,\sqrt{v} \geq \left\lvert\log\left(\frac{t}{1-t}\right)\right\rvert,\tag{1}\\ &\,\sqrt{w} \leq \lvert\log(t(1-t))\rvert\tag{2}, \end{align}

Now we deal with constraints (1) and (2):

For (1) Note that: \begin{equation} \lvert\log(t)-\log(1-t)\rvert = \text{ptp}{([-\log(t),-\log(1-t)])}, \end{equation} where $\text{ptp}$ is the peak-to-peak atom in CVX, which is convex. Since $-\log(\cdot)$ is a convex atom, we're set. I can't seem to find cp.ptp when I try to use it in CVXPY, though. Is it really implemented?

Now for (2), note that, since $t \in (0,1)$, we have that : \begin{equation} \lvert\log(t(1-t))\rvert = -\log(t(1-t)). \end{equation}

Thus, we can write (2) as:

\begin{align} &\sqrt{w} + \log(t(1-t)) &\leq 0 &\iff\\ \sqrt{w}t + \sqrt{w}(1-t) &+\log(t) +\log(1-t) &\leq0&\iff\\ \frac{t}{t}\log\left(\frac{t}{e^{-\sqrt{w}t}}\right)&+ \frac{1-t}{1-t}\log\left(\frac{1-t}{e^{-\sqrt{w}(1-t)}}\right)&\leq0 &\iff\\ \frac{1}{t}D_\text{KL}(t \lvert\rvert e^{t h(w)}) &+ \frac{1}{1-t}D_\text{KL}((1-t) \lvert\rvert e^{(1-t)h(w)})&\leq0 &\iff\\ \mathcal{P}\left(g(x,y),s\right)\bigg\rvert_{x=1, y=h(w), s=\frac{1}{t}} &+ \mathcal{P}\left(g(x,y),s\right)\bigg\rvert_{x=1, y=h(w), s=\frac{1}{1-t}}&\leq0, \end{align}

where $h(w) = -\sqrt{w}$, $D_\text{KL}(x \lvert\rvert y)$ is the relative entropy between x and y, $g(x,y) = D_\text{KL}(x \lvert\rvert e^y)$, and $\mathcal{P}$ is the perspective transform of a function.

I think this might work, it's all the composition of simpler convex functions.

I have the following objective that I want to maximize:

\begin{equation} \max_{U_T\in \mathbb{R}, x\in\mathbb{R}^T} J_\alpha(U_T) = \frac{\alpha}{\alpha-1}\log\left(\frac{\cosh(U_T)}{\cosh(\alpha U_T)^\frac{1}{\alpha}}\right) \,,\qquad \text{s.t:} \qquad U_T= Ax+b. \end{equation} where $A \in \mathbb{R}^{1\times T}$, $b\in\mathbb{R}$ and $\alpha>1$.

It is easy to show that this is, in fact, a pseudo-concave function by checking that: \begin{equation} \eta'_\alpha(u)(v-u)\leq0 \implies \eta_\alpha(v)\leq \eta_\alpha(u), \end{equation} where $\eta_\alpha(u) = \frac{\cosh(u)}{\cosh(\alpha u)^\frac{1}{\alpha}}$.

This, of course, implies the quasiconcavity of $J_\alpha(U_T)$.

Here's a plot of the function that I made with Wolfram Mathematica: Plot of <span class=$J_\alpha(x)$ for $\alpha=2$." />

-My questions are:

  1. Is there a way to write this objective function as a DQCP compliant program in CVXPY?
  2. When $\alpha \to 1^+$, $J_1(u) := -\log[\frac{e^{u\tanh(u)}}{\cosh(u)} ]$ pointwise. Is there a way to write that as a DQCP compliant program in CVXPY as well?
  3. If it can't be put into a DCP compliant formulation, what would you recommend that I do to solve this problem numerically (especially question 2)?

I'm using CVXPY since I'll add more complicated constraints later, but maybe the problem has a simple analytical solution as well.


-What I have so far:

I am especially interested in question 2, so I'm trying to solve that first.

Using the definitions of $\cosh$, write: \begin{align} J_1(u) &:= \lim_{\alpha\to 1^+}J_\alpha(u)\\ &= \log(\cosh(u)) -u \tanh(u)\\ &= \log\left(e^u + e^{-u}\right)-u \tanh(u) + K\\ &= -u\left(\log\left(\frac{1}{1 + e^{-2u}}\right)+ \tanh(u)\right) + K\\ &= -\frac{1}{2}f^{-1}(f(2u))(\log(f(2u))- \tanh(u)) + K, \end{align} where $K$ is a constant and $f(y) := 1/(1+e^{-y})$ is the logistic function, with inverse given by $f^{-1}(y) = \log(\frac{y}{1-y})$. By the well-known relation (Wiki): \begin{equation}\tanh(y) = 2f(2y) - 1,\end{equation} and we can conclude that: \begin{equation} J_1(u) = T(f(2u)) + K, \end{equation} where $T:[0,1]\to\mathbb{R}$ is: \begin{equation}T(z) = -\frac{1}{2}\log\left(\frac{z}{1-z}\right)\left(\log(z) - 2z-1\right).\end{equation}

T is in fact concave, as we can see from the graph of its second derivative in Mathematica:

Plot of <span class=$T(z)$" />

By further manipulation of $T$, we get the expression:

\begin{align} -2T(z) &= \log\left(\frac{z}{1-z}\right)(z + z - 1 + \log(z))\\ &= z\log\left(\frac{z}{1-z}\right) + (1-z)\log\left(\frac{1-z}{z}\right) + \log\left(\frac{z}{1-z}\right)\log(z)\\ &= 2\text{JS}(t \vert\vert 1-t) + \log^2(z) - \log(z)\log(1-z), \end{align} where $\text{JS}$ is the Jensen-Shannon entropy (simmetrized KL divergence).

Note the right-hand side is a sum of convex functions, as $\log^2(z)$ and $-\log(z)\log(1-z)$ are convex in $(0,1)).

Therefore, I think I would be able to write the original program by optimizing for T1 and adding a (quasilinear) constraint to account for the change of variable. Question 2 finally reduces to finding a DCP compliant representation of $T$.

I have the following objective that I want to maximize:

\begin{equation} \max_{U_T\in \mathbb{R}, x\in\mathbb{R}^T} J_\alpha(U_T) = \frac{\alpha}{\alpha-1}\log\left(\frac{\cosh(U_T)}{\cosh(\alpha U_T)^\frac{1}{\alpha}}\right) \,,\qquad \text{s.t:} \qquad U_T= Ax+b. \end{equation} where $A \in \mathbb{R}^{1\times T}$, $b\in\mathbb{R}$ and $\alpha>1$.

It is easy to show that this is, in fact, a pseudo-concave function by checking that: \begin{equation} \eta'_\alpha(u)(v-u)\leq0 \implies \eta_\alpha(v)\leq \eta_\alpha(u), \end{equation} where $\eta_\alpha(u) = \frac{\cosh(u)}{\cosh(\alpha u)^\frac{1}{\alpha}}$.

This, of course, implies the quasiconcavity of $J_\alpha(U_T)$.

Here's a plot of the function that I made with Wolfram Mathematica: Plot of <span class=$J_\alpha(x)$ for $\alpha=2$." />

-My questions are:

  1. Is there a way to write this objective function as a DQCP compliant program in CVXPY?
  2. When $\alpha \to 1^+$, $J_1(u) := -\log[\frac{e^{u\tanh(u)}}{\cosh(u)} ]$ pointwise. Is there a way to write that as a DQCP compliant program in CVXPY as well?
  3. If it can't be put into a DCP compliant formulation, what would you recommend that I do to solve this problem numerically (especially question 2)?

I'm using CVXPY since I'll add more complicated constraints later, but maybe the problem has a simple analytical solution as well.


-What I have so far:

I am especially interested in question 2, so I'm trying to solve that first.

Using the definitions of $\cosh$, write: \begin{align} J_1(u) &:= \lim_{\alpha\to 1^+}J_\alpha(u)\\ &= \log(\cosh(u)) -u \tanh(u)\\ &= \log\left(e^u + e^{-u}\right)-u \tanh(u) + K\\ &= -u\left(\log\left(\frac{1}{1 + e^{-2u}}\right)+ \tanh(u)\right) + K\\ &= -\frac{1}{2}f^{-1}(f(2u))(\log(f(2u))- \tanh(u)) + K, \end{align} where $K$ is a constant and $f(y) := 1/(1+e^{-y})$ is the logistic function, with inverse given by $f^{-1}(y) = \log(\frac{y}{1-y})$. By the well-known relation (Wiki): \begin{equation}\tanh(y) = 2f(2y) - 1,\end{equation} and we can conclude that: \begin{equation} J_1(u) = T(f(2u)) + K, \end{equation} where $T:[0,1]\to\mathbb{R}$ is: \begin{equation}T(z) = -\frac{1}{2}\log\left(\frac{z}{1-z}\right)\left(\log(z) - 2z-1\right).\end{equation}

T is in fact concave, as we can see from the graph of its second derivative in Mathematica:

Plot of <span class=$T(z)$" />

By further manipulation of $T$, we get the expression:

\begin{align} -2T(z) &= \log\left(\frac{z}{1-z}\right)(z + z - 1 + \log(z))\\ &= z\log\left(\frac{z}{1-z}\right) + (1-z)\log\left(\frac{1-z}{z}\right) + \log\left(\frac{z}{1-z}\right)\log(z)\\ &= 2\text{JS}(t \vert\vert 1-t) + \log^2(z) - \log(z)\log(1-z), \end{align} where $\text{JS}$ is the Jensen-Shannon entropy (simmetrized KL divergence).

Note the right-hand side is a sum of convex functions, as $\log^2(z)$ and $-\log(z)\log(1-z)$ are convex in $(0,1)$).

Therefore, I think I would be able to write the original program by optimizing $T$ and adding a (quasilinear) constraint to account for the change of variable. Question 2 finally reduces to finding a DCP compliant representation of $T$.


Update: Progress on trying to find a DCP representation of $\log^2(t)$ and $-\log(t)\log(1-t)$:

I haven't been able yet to represent those using DCP, but I found some interesting things:

a. $\log^2(t)$:

We can write this as: \begin{equation} \min_{z} z \qquad\text{s.t.: } w\geq -\log(t),\,w\geq 0,\,z\geq w^2, \end{equation} for fixed $t \in (0,1)$.

b. $-\log(t)\log(1-t)$:

This is the tricky one. First note that:

\begin{equation} \log(t)\log(1-t) = \frac{1}{2}\left(\log^2(t(1-t)) - \log^2\left(\frac{t}{1-t}\right)\right), \end{equation}

since $2cd = (c+d)^2 - (c-d)^2$, for any $c,d$.

Now form the optimization problem: \begin{align} \min_{z}\,-z &\qquad\text{s.t.: }\\ &2z \leq w-v,\\ &\,\sqrt{v} \geq \left\lvert\log\left(\frac{t}{1-t}\right)\right\rvert,\tag{1}\\ &\,\sqrt{w} \leq \lvert\log(t(1-t))\rvert\tag{2}, \end{align}

Now we deal with constraints (1) and (2):

For (1) Note that: \begin{equation} \lvert\log(t)-\log(1-t)\rvert = \text{ptp}{([-\log(t),-\log(1-t)])}, \end{equation} where $\text{ptp}$ is the peak-to-peak atom in CVX, which is convex. Since $-\log(\cdot)$ is a convex atom, we're set. I can't seem to find cp.ptp when I try to use it in CVXPY, though. Is it really implemented?

Now for (2), note that, since $t \in (0,1)$, we have that : \begin{equation} \lvert\log(t(1-t))\rvert = -\log(t(1-t)). \end{equation}

Thus, we can write (2) as:

\begin{align} &\sqrt{w} + \log(t(1-t)) &\leq 0 &\iff\\ \sqrt{w}t + \sqrt{w}(1-t) &+\log(t) +\log(1-t) &\leq0&\iff\\ \frac{t}{t}\log\left(\frac{t}{e^{-\sqrt{w}t}}\right)&+ \frac{1-t}{1-t}\log\left(\frac{1-t}{e^{-\sqrt{w}(1-t)}}\right)&\leq0 &\iff\\ \frac{1}{t}D_\text{KL}(t \lvert\rvert e^{t h(w)}) &+ \frac{1}{1-t}D_\text{KL}((1-t) \lvert\rvert e^{(1-t)h(w)})&\leq0 &\iff\\ \mathcal{P}\left(g(x,y),s\right)\bigg\rvert_{x=1, y=h(w), s=\frac{1}{t}} &+ \mathcal{P}\left(g(x,y),s\right)\bigg\rvert_{x=1, y=h(w), s=\frac{1}{1-t}}&\leq0, \end{align}

where $h(w) = -\sqrt{w}$, $D_\text{KL}(x \lvert\rvert y)$ is the relative entropy between x and y, $g(x,y) = D_\text{KL}(x \lvert\rvert e^y)$, and $\mathcal{P}$ is the perspective transform of a function.

I think this might work, it's all the composition of simpler convex functions.

Added what can be done about it question. Fixed typos
Source Link
Loading
Slight progress: We can write $T$ as sum of simpler concave functions, maybe now we do it?
Source Link
Loading
Added my work thus far
Source Link
Loading
edited body
Source Link
Loading
deleted 13 characters in body
Source Link
Loading
Changed title to CVXPY, because CVX is a different tool, and does not support DQCP
Link
Mark L. Stone
  • 13.9k
  • 1
  • 34
  • 70
Loading
added 164 characters in body
Source Link
Loading
added 73 characters in body
Source Link
Loading
Source Link
Loading