In the GAMS documentation concerning non-smooth optimization I found the following statement:

A smooth approximation for $\max(f(x),g(y))$ is as in the following example code:

[f(x) + g(y) + sqrt(sqr(f(x)-g(y)) + sqr(delta))] /2 where delta a small number.

Does something similar exist for $\max(f_1(x),f_2(x),\cdots,f_n(x))$?

  • 1
    $\begingroup$ Depending on how you want to use the $\max$, you might be able to linearize exactly. $\endgroup$
    – RobPratt
    Feb 21, 2021 at 0:49
  • 1
    $\begingroup$ $\max\left(x_0,x_1,\ldots,x_n\right) = \max\left(x_0,\max\left(x_1,\max\left(\ldots,x_n\right)\right)\right) .$ $\endgroup$
    – Nat
    Feb 21, 2021 at 5:29
  • $\begingroup$ @RobPratt Hi Rob, I would like to avoid ending up with an MILP because of the difficulties associated with MILPs. The question is if I can win anything out of staying in the NLP domain. Probably not, because I will have to deal with a global optimization problem, but I would like to give it a try. $\endgroup$
    – Clement
    Feb 21, 2021 at 9:49
  • $\begingroup$ @Nat That is going to be a nightmare when it comes to implemention. $\endgroup$
    – Clement
    Feb 21, 2021 at 9:54
  • 1
    $\begingroup$ @RobPratt It is a set of constraints in the form $Max (x_1,..,x_n) = f(x_1,x_2,...,x_n)$ where $f(x_1,x_2,...,x_n)$ is a linear function. I can linearize these constraints as explained for example here getting a MILP. The formulation doesn´t help me in avoiding an MILP model. $\endgroup$
    – Clement
    Feb 21, 2021 at 18:11

1 Answer 1


Indeed, there exists numerous smooth approximations for the max function. One of the most well known approximation is the Kreisselmeier-Steinhauser (KS) functional, that approximates the non-smooth functional $$ \max(f_1(x), \cdots, f_n(x)) $$ with the smooth functional $$ KS(f_1, \cdots, f_n; \rho) = \frac{1}{\rho} \log{\sum_{i=1}^n w_i e^{\rho f_i} } = \max_i f_i \; + \; \frac{1}{\rho} \log{\sum_{i=1}^n w_i e^{\rho (f_i - \max_i(f_i))} } $$ with $\rho>0$ and $w_1, \cdots, w_n$ given (positive) weights.

This functional satisfies:

  • $KS(f_1, \cdots, f_n; \rho) > \max_i(f_i)$ for all $\rho >0$
  • $\lim_{\rho \to +\infty} KS(f_1, \cdots, f_n; \rho) \to \max_i(f_i)$
  • $\max_i(f_i) < KS(f_1, \cdots, f_n; \rho) \leq \max_i(f_i) + \frac{\log(n)}{\rho} $

You could find more details about the KS functional in

Polyak, R. A. "Smooth optimization methods for minimax problems." SIAM Journal on Control and Optimization 26.6 (1988): 1274-1286.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.