Question about implementation method for optimization problem

Question

Suppose we wanted to solve the following optimization problem: $$\inf_{x \geq 0}\sup_{y \in [0, 1];\ z > 0} f(x, y, z),$$ where $f(x, y, z)$ is some objective function with a closed form that can be specified in terms of parameters $x$, $y$, $z$. How would we implement this optimization, say using the Python programming language?

I am only vaguely familiar with implementing optimization in the Bayesian setting (eg using variational inference) or by a grid search. But I do not think Bayesian optimization works here as there are any priors here (I could be wrong though) and I'm not quite sure how to discretize these parameters in a reasonable way.

(The motivation for the optimization problem above comes from my personal study of robust optimization models from the paper https://arxiv.org/pdf/1908.05659.pdf.)

Answer 1

The first thing to do is not to write your own solver, but to use an existing one. In Python, you can use the modeler Pyomo to model the problem and then select which solver to use to solve it.

If you have the closed-form of the function to optimize and the problem is non-convex, you can first try a global optimization solver. If the problem is easy enough, it may find a provably globally optimal solution within reasonable time.

If it is not the case, i.e. the problem is convex, or the resolution takes too much time and the current solution is not feasible or not good enough, then you can try a local optimization solver. If the problem is non-convex, these solvers do not guarantee finding the optimal solution. But they may find a good one quickly.

You can find a list of solvers here In particular, you can look at the "Global Optimization" and "Nonlinearly Constrained Optimization" sections.

Only, if none of these solutions are satisfying, you can consider implementing another algorithm yourself.