19
$\begingroup$

I am solving a highly constrained (large number of constraints and large number of variables, but small degree of freedom) NLP problem, and for start, I was using Matlab's fmincon - SQP algorithms. This works wonders, and in $95\%$ of the cases finds an optimal solution even with poor initial guesses.

Now - Matlab does not have built-in Automatic Differentiation (AD) technology (there are packages, but I was not always successful in getting my constraint set AD'd, so I use numerical derivatives), so as the size of the problems expands, I get slower and slower convergence. Then I started to re-implement the problem in Pyomo.

To my surprise, apart from ipopt, I did not find many open-source/freeware solvers for NLP problems. Ipopt performs very poorly as it struggles to find a feasible solution. It finds the right optimum in maybe $5\%$ of the cases. Now both ipopt and SQP are local solvers, so the solutions should not be that much different. I tried Matlab's own ipopt, as well as ipopt supplied by opti-toolbox. Both perform miserably compared to Matlab's SQP.

I am not sure if it's the ipopt settings - do you know any handles that could work for the highly constrained problems? I found that filterSQP could be called from pyomo, but not much literature outside. Also, it's not publicly available for use - one needs to get a license from Dundee university. Also, I saw basic_sqp.py developed for numpy, but no reference on how to call it from pyomo.

If anyone has any experience with any sort of SQP algorithms (or non-ipopt solvers, especially active set implementation that would hopefully work in my case), it would really help a lot.

Many thanks!

Improve this question
$\endgroup$
7
  • $\begingroup$ Have you tried using the ADiMat automatic differentiator under MATLAB sc.informatik.tu-darmstadt.de/res/sw/adimat/general/… . It should work well to provide you derivatives of objective function and constraints for use in FMINCON. Of note, i think you are getting killed on the numerical differentiation under FMINCON, because I believe it does numerical differntiation with respect to the original variables, not in the reduced space which has much lower dimension in your problem due to large number of constraints and few degrees of freedom. $\endgroup$
    – Mark L. Stone
    Jul 18, 2019 at 15:52
  • $\begingroup$ Hi there, reason why I did not use AD is I have to admit my lazyness :) (at least in Matlab). Since I have external matlab calls, and also number of expressions first that go to constraint function, AD breaks down. I used AutoDiff2016 - and it works well, but since I have DAE system embedded in the optimization system in constraints, and there are strict ways to write constraints to be able to use AutoDiff2016. Since the SQP solver works even with the numerical derivatives, I think this should not be the case. Re-coding to pyomo should help with AD, since pyomo gives AD'd derivatives natively $\endgroup$
    – Stanny_boy
    Jul 19, 2019 at 6:58
  • $\begingroup$ Hence, if the same problem crashes/diverges with AD'd derivatives in pyomo, and Matlab (with numerical derivatives), both with ipopt, and converges with SQP with numerical derivatives in Matlab, my best guess is that the issue is not with numerical derivatives vs analytical derivatives, but for SQP vs ipopt as a solver. To be able to judge, I will need to try SQP type of algorithm in pyomo though. $\endgroup$
    – Stanny_boy
    Jul 19, 2019 at 7:01
  • $\begingroup$ You were complaining about speed with numerical derivatives for large problem sizes. AD can address that, at least for first derivatives, using reverse mode, for which time for entire gradient is independent of dimension, You are getting "killed" with numerical derivatives because FMINCON executes in original dimension, not reduced dimension (small number of degrees of freedom).. $\endgroup$
    – Mark L. Stone
    Jul 19, 2019 at 10:15
  • $\begingroup$ AD will defenetly help speed. But it won't solve the problem of poor convergence/struggle of ipopt to find a feasable solution. Hence I am more looking to learn about alternative solvers. $\endgroup$
    – Stanny_boy
    Aug 18, 2019 at 14:04

4 Answers 4

Reset to default
7
$\begingroup$

This post is of my interest. I also required a global solver for my problem. I found out that Pyomo has python interface for an opensource global solver called SCIP for nonlinear optimization problems. You might want to check that out. The process of getting SCIP installed and ready to work on Pyomo is slightly non-trivial and might take some (for which I can help you, this link might be useful) time, but I think this can be something to look out. Hope that helps!

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ Hi Gorja, thanks for the reply. I've checked the SCIP documentation. It seems SCIP when solving pure NLPs depends on the underlying NLP solvers. By default, ipopt is used, which brings me to square 1. That I am stuck with ipopt. I can link FilterSQP and WORHP solvers, provided I have licenses for them. $\endgroup$
    – Stanny_boy
    Aug 18, 2019 at 14:13
  • $\begingroup$ Hi Stanny_boy. Thanks for letting me know this valuable information. $\endgroup$
    – chupa_kabra
    Aug 19, 2019 at 15:58
9
$\begingroup$

I think it is a good idea to have a look at this question on StackOverflow. In addition to that, in the Pyomo manual, it is stated that:

"Pyomo supports a wide variety of solvers. Pyomo has specialized interfaces to some solvers (for example, BARON, CBC, CPLEX, and Gurobi). It also has generic interfaces that support calling any solver that can read AMPL “.nl” and write “.sol” files and the ability to generate GAMS-format models and retrieve the results. The following Pyomo command will give you the current list of supported solvers:"

pyomo help --solvers

The following two papers may also help in choosing a solver for your problem:

  • Gill, P. E., Murray, W., and Saunders, M. A. (2005). SNOPT: An SQP algorithm for large-scale constrained optimization. SIAM review, 47(1):99–131.
  • Kronqvist, Jan, et al. "A review and comparison of solvers for convex MINLP." Optimization and Engineering 20.2 (2019): 397-455.
Improve this answer
$\endgroup$
3
  • $\begingroup$ Thanks Oguz. Indeed, pyomo comes with interfaes with many solvers. Problem I have with NLPs in the open source community is that appart from ipopt, there are no freely available NLP solvers. It's all pretty much limited to ipopt, at least by my search so far. I am wondering if there are any NLP solvers other than ipopt that are freely/widely available for which I do not need to buy a license. Of course there are strong solvers like SNOPT, KNITRO, CONOPT, BARON, but all of these of course cost money. I can test on neos library, but this is not a sustainable solution $\endgroup$
    – Stanny_boy
    Jul 19, 2019 at 7:05
  • $\begingroup$ You are very welcome @Stanny_boy, I will keep looking for open source solver for NLP and I will let you know if I have something useful. $\endgroup$
    – Oguz Toragay
    Jul 19, 2019 at 7:15
  • $\begingroup$ @Oguz Toragay, try out solver SCIP. Pyomo has a python interface for it. $\endgroup$
    – chupa_kabra
    Aug 1, 2019 at 8:18
8
$\begingroup$

APOPT is another NLP (and MINLP) solver that works with Pyomo by reading .nl files and producing .sol files. The solver is apopt.py and called with Python to send the .nl file to a compute server and then return the .sol file back to Python and returned to Pyomo. Here is the source code on GitHub with instructions on use. Please note that we are still developing this solver and I'd appreciate any feedback on the performance. Here are some benchmark results that we shared in 2012 at INFORMS.

NLP Benchmark Results

Improve this answer
$\endgroup$
3
  • 1
    $\begingroup$ Thanks John. I see APOPT is sending the mathematical program file to the server, and sending results back. In case I want own architecture, cloud deployment, many instances running in parallel etc. or I have confidential models/parameters, I would need a local version. Is there such arrangement? Do you plan to have APOPT as a truly open source solver, or you want later on to make it a commercial solver? What is "behind the hood" behind APOPT - i.e. what kind of method is behind it? $\endgroup$
    – Stanny_boy
    Aug 18, 2019 at 13:59
  • $\begingroup$ APOPT is freely available with Python Gekko and APMonitor. It is included with the local executables here: github.com/BYU-PRISM/GEKKO/tree/master/gekko/bin For pyomo / AMPL, I run the web-service with the stand-alone APOPT to collect benchmark problems that further improve the solver. While it is under development, this is very valuable to have cases to improve the solver. It is an MINLP solver and I'm working on combined interior point / active-set method for the NLP. It is not ready for commercial deployment yet. $\endgroup$
    – John Hedengren
    Aug 18, 2019 at 14:08
  • 1
    $\begingroup$ Many thanks John. I will surely give it a try. Would be great to know if/when you plan to have compiled or local version of the solver for pyomo. Till then i will test the web service for performance. $\endgroup$
    – Stanny_boy
    Aug 18, 2019 at 14:20
6
$\begingroup$

Pyomo has an ASL interface, hence any solver that is equipped with one will work out of the box. Commercial options that have free variable limited demos would be KNITRO, BARON, or Octeract Engine, and open source options include Couenne, or MINOTAUR. For some of the commercial solvers you might need to request a special version from the vendor that comes with the interface (or buy the AMPL) version, whereas other provide it by default.

In general, ipopt is so widespread among open source that many other nonlinear solvers invoke it anyway. Couenne and MINOTAUR are interesting options because they come with computational graph-based automatic differentiation which their version of ipopt uses.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.