# Difference between stochastic optimization and robust optimization

I would like to know whether stochastic optimization and robust optimization are the same and if not, what is the main difference between them. I did an Internet search and I found the following conversation: https://www.quora.com/What-are-the-main-differences-between-stochastic-optimization-and-robust-optimization Here some say it is in fact the same and others say it is not.

My guess would be that in stochastic optimization the distribution of input parameters are known or can be estimated while in robust optimization this is generally not the case. At least you do not use random variables with certain distributions when defining the optimization problem. The goal of robust optimization is that the solutions should remain feasible even if the input parameters of the model vary.

I know from Wikipedia (https://en.wikipedia.org/wiki/Robust_optimization) that there is also something called "Probabilistically robust optimization models". This can be basically regarded as stochastic optimization so the borders between stochastic and robust optimization are not fully clear.

What are your takes on that?

• I believe you gave a partial answer yourself. Both "worlds" work with scenarios of uncertain parameters. What distinguishes them is the intended output: in robust optimization you are guaranteed to be feasible in all scenarios. – Marco Lübbecke Jun 28 '20 at 14:31
• Thanks Marco for your comment. According to Wikipedia there are also robust optimizaiton techniques that do not work with probabilistc assumptions and thus with scenarios. They are called "Non-probabilistic robust optimization models". So you can also use robust optimization without scenarios. This is/was my guess about the differences between stochastic and robust optimization. In stochastic optimization (as far as I understood) you need to have distributions of the inputparameters. – PeterBe Jun 28 '20 at 17:01
• One is deterministic and one is stochastic? – Nike Dattani Jun 28 '20 at 21:36
• thank you @PeterBe, I will also look into this (it surprises me, because, when there are no "scenarios" (in a broad sense) why would one have to be "robust" against anything? :)) I like your question, many people seem to have this question. – Marco Lübbecke Jun 29 '20 at 7:29

I think there is no single, uniformly accepted answer. But there are two main factors that distinguish them:

1. In stochastic optimization, it is nearly always assumed that we know the probability distribution (possibly in the form of discrete probabilities of each scenario) of the random parameters. In robust optimization it is usually (but not always) assumed that we do not know the distribution.

2. In stochastic optimization, the goal is usually to optimize the expected value of the objective function (min expected cost, max expected profit, etc.). In robust optimization, because we don't know the probabilities, we instead optimize some other measure. Common measures are to optimize the worst-case outcome -- e.g., minimize the maximum cost, maximize the minimum profit, etc. -- sometimes over only a subset of the possible scenarios. There are lots of other common objectives, too.

Although it's often true that in robust optimization feasibility is required in every scenario, this is often true in stochastic optimization, as well, so I wouldn't consider this to be a major distinguishing factor.

• (Distributionally) Robust optimization over a Wasserstein ball seems to be the "in" thing nowadays. Here to stay, or flavor of the day? – Mark L. Stone Jun 28 '20 at 20:52

As Larry said, there is no single, uniformly accepted answer, so I'll make things even more interesting. In mechanical engineering, specifically in aircraft design where I used to work, we used the following terminology:

• Stochastic optimisation was to solve problems using any non-deterministic methods, e.g., particle swarm algorithms or evolutionary algorithms.
• Robust optimisation was to optimise for stability, i.e., to minimise the second derivatives of certain flow properties so that wings will behave roughly the same even under fluctuating conditions. The tradeoff for that was huge in other nominal properties of the aircraft such as range, weight, maximum speed, etc. This is solved as a multiobjective problem.

What's fascinating here is that we can really see how in different fields we adapt terminology depending on the nature of the problems we are solving. In aircraft design, computational cost is orders of magnitude beyond most OR problems, e.g., a single evaluation of $$f(x)$$ for the flow equations could take 1-2 weeks. Thus, the term "robust" that we use to describe simulating multiple scenarios in other OR fields becomes meaningless because it's impossible to do that. Nevertheless, the concept of "robustness" is universal, so people still use the term, just in a way that is meaningful for the use-case.

It's also quite interesting that these are possibly the original meanings of these terms as, historically, aircraft design was among the very first real-world applications of optimisation methods.