3
$\begingroup$

Suppose we wanted to solve the following optimization problem: $$\inf_{x \geq 0}\sup_{y \in [0, 1];\ z > 0} f(x, y, z),$$ where $f(x, y, z)$ is some objective function with a closed form that can be specified in terms of parameters $x$, $y$, $z$. How would we implement this optimization, say using the Python programming language?

I am only vaguely familiar with implementing optimization in the Bayesian setting (eg using variational inference) or by a grid search. But I do not think Bayesian optimization works here as there are any priors here (I could be wrong though) and I'm not quite sure how to discretize these parameters in a reasonable way.

(The motivation for the optimization problem above comes from my personal study of robust optimization models from the paper https://arxiv.org/pdf/1908.05659.pdf.)

$\endgroup$
5
  • $\begingroup$ If you want to implement a solver for that, you could think about using a metaheuristics-based approach. There are several packages that allow you to build your own metaheuristics, like Mealpy. $\endgroup$ Feb 9 at 20:43
  • $\begingroup$ Also, if you want to code your own solver, you could take a look to the Julia language. $\endgroup$ Feb 9 at 20:44
  • $\begingroup$ Thank you very much for the suggestions. I'm quite unfamiliar with using metaheuristics - can you please direct me to some resources for how I might set up a solver for $\inf$-$\sup$ optimization problems? $\endgroup$ Feb 9 at 21:51
  • $\begingroup$ Do you know anything about the function f , convex in x,z etc? $\endgroup$
    – skr
    Feb 10 at 1:59
  • $\begingroup$ For simplicity, let us assume that it is convex. I haven't evaluated its closed form, but I know for sure that it has a closed form. $\endgroup$ Feb 10 at 3:52

1 Answer 1

3
$\begingroup$

The first thing to do is not to write your own solver, but to use an existing one. In Python, you can use the modeler Pyomo to model the problem and then select which solver to use to solve it.

If you have the closed-form of the function to optimize and the problem is non-convex, you can first try a global optimization solver. If the problem is easy enough, it may find a provably globally optimal solution within reasonable time.

If it is not the case, i.e. the problem is convex, or the resolution takes too much time and the current solution is not feasible or not good enough, then you can try a local optimization solver. If the problem is non-convex, these solvers do not guarantee finding the optimal solution. But they may find a good one quickly.

You can find a list of solvers here In particular, you can look at the "Global Optimization" and "Nonlinearly Constrained Optimization" sections.

Only, if none of these solutions are satisfying, you can consider implementing another algorithm yourself.

$\endgroup$
1
  • 1
    $\begingroup$ "The first thing to do is not to write your own solver, but to use an existing one. ... Only, if none of these solutions are satisfying, you can consider implementing another algorithm yourself." (+1) $\endgroup$ Feb 10 at 12:49

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.