Difference between stochastic optimization and robust optimization

Question

I would like to know whether stochastic optimization and robust optimization are the same and if not, what is the main difference between them. I did an Internet search and I found the following conversation: https://www.quora.com/What-are-the-main-differences-between-stochastic-optimization-and-robust-optimization Here some say it is in fact the same and others say it is not.

My guess would be that in stochastic optimization the distribution of input parameters are known or can be estimated while in robust optimization this is generally not the case. At least you do not use random variables with certain distributions when defining the optimization problem. The goal of robust optimization is that the solutions should remain feasible even if the input parameters of the model vary.

I know from Wikipedia (https://en.wikipedia.org/wiki/Robust_optimization) that there is also something called "Probabilistically robust optimization models". This can be basically regarded as stochastic optimization so the borders between stochastic and robust optimization are not fully clear.

What are your takes on that?

Answer 1

I think there is no single, uniformly accepted answer. But there are two main factors that distinguish them:

1. In stochastic optimization, it is nearly always assumed that we know the probability distribution (possibly in the form of discrete probabilities of each scenario) of the random parameters. In robust optimization it is usually (but not always) assumed that we do not know the distribution.

2. In stochastic optimization, the goal is usually to optimize the expected value of the objective function (min expected cost, max expected profit, etc.). In robust optimization, because we don't know the probabilities, we instead optimize some other measure. Common measures are to optimize the worst-case outcome -- e.g., minimize the maximum cost, maximize the minimum profit, etc. -- sometimes over only a subset of the possible scenarios. There are lots of other common objectives, too.

Although it's often true that in robust optimization feasibility is required in every scenario, this is often true in stochastic optimization, as well, so I wouldn't consider this to be a major distinguishing factor.

Answer 2

As Larry said, there is no single, uniformly accepted answer, so I'll make things even more interesting. In mechanical engineering, specifically in aircraft design where I used to work, we used the following terminology:

• Stochastic optimisation was to solve problems using any non-deterministic methods, e.g., particle swarm algorithms or evolutionary algorithms.
• Robust optimisation was to optimise for stability, i.e., to minimise the second derivatives of certain flow properties so that wings will behave roughly the same even under fluctuating conditions. The tradeoff for that was huge in other nominal properties of the aircraft such as range, weight, maximum speed, etc. This is solved as a multiobjective problem.

What's fascinating here is that we can really see how in different fields we adapt terminology depending on the nature of the problems we are solving. In aircraft design, computational cost is orders of magnitude beyond most OR problems, e.g., a single evaluation of $f(x)$ for the flow equations could take 1-2 weeks. Thus, the term "robust" that we use to describe simulating multiple scenarios in other OR fields becomes meaningless because it's impossible to do that. Nevertheless, the concept of "robustness" is universal, so people still use the term, just in a way that is meaningful for the use-case.

It's also quite interesting that these are possibly the original meanings of these terms as, historically, aircraft design was among the very first real-world applications of optimisation methods.

Answer 3

Let's make it more clear.

Stochastic Programming is not Stochastic Optimization. When you say Stochastic Programming the above answer of @Larry is explaining more about "stochastic programming". This is when your problem of study has some uncertainty (e.g. demand uncertainty, arrival times uncertainty). The robust optimization sets with Stochastic Programming under the classification of optimization problems with uncertainties.

However, when you say stochastic optimization, this is about the technique/solution methodology you use to solve either a deterministic optimization model/problem or a stochastic optimization model/problem. In this case, the above answer of @Nikos Kazazakis describes this.

Therefore, @LarrySnyder610 tried to answer the following question: "Difference between stochastic programming and robust optimization?" whereas @Nikos Kazazakis highlighted the: Difference between stochastic optimization and deterministic optimization approaches?