# Tool/Editor to visualize optimization problem files and solutions

Is there a tool with a graphical user interface which helps to visualize optimization problem files (e.g. lp/mps) and solutions?

Let's say you have an optimization problem and a solution and want to be able to track individual variables, solution values, maybe even dual values and so on. For example, to click on a variable and see all containing rows. Or to select a row and highlight all variables which are in the row. Or even to fuse the information of an solution with the lp file so that you can see all variables and their solution value in the problem.

My working method is to use Notepad++, highlight variable names or searching for variable names and clicking through the files and copying individual rows to a new text file...

AFAIK, some software such as GAMS, Lingo, AMPL and SCIP have had facilities to deal with it. For example, GAMS has a list file which contains much useful information about the problem and its solution. Also, Lingo has a graphical interface to depict the problem specifications.

For example, Lingo matrix picture:

This might be not exactly what you are looking for but GCG (based on SCIP) has a nice visualize command that you can call after it found a decomposition (Dantzig-Wolfe or Benders). It visualizes the decomposition in its master and subproblems but also highlights the non-zero entries of the matrix. If there is no such decomposition it still shows you the non-zeros.

MIPLIB and strIPlib use these (or similar) plots for problem visualization.

What you are asking for sounds somewhat similar to Alteryx (which starts at \$5,195/user per year). Let's see how we can duplicate some of the look and functionality that with free tools. A demo video "Alteryx Designer in 20 Minutes" (titled "Alteryx Demo" on YouTube) is available.

Here's an overview of the workflow:

"Let's say you have an optimization problem and a solution and want to be able to track individual variables, solution values, maybe even dual values and so on. For example, to click on a variable and see all containing rows. Or to select a row and highlight all variables which are in the row. Or even to fuse the information of an solution with the lp file so that you can see all variables and their solution value in the problem.".

Here's the Alteryx user interface:

How can we obtain that sort of functionality for less than the cost of Alteryx?

Using Gekko and APMonitor, along with pyFMI for FMI support (that provides drag and drop), you end up with (description from Gekko's webpage):

"What does GEKKO do?

GEKKO is optimization software for mixed-integer and differential algebraic equations. It is coupled with large-scale solvers for linear, quadratic, nonlinear, and mixed integer programming (LP, QP, NLP, MILP, MINLP). Modes of operation include data reconciliation, real-time optimization, dynamic simulation, and nonlinear predictive control. The client or server is freely available with interfaces in MATLAB, Python, or from a web browser.

GEKKO is a high-level abstraction of mathematical optimization problems. Values in the models are defined by Constants, Parameters, and Variables. The values are related to each other by Intermediates or Equations. Objective functions are defined to maximize or minimize certain values. Objects are built-in collections of values (constants, parameters, and variables) and relationships (intermediates, equations, and objective functions). Objects can build upon other objects with object-oriented relationships.

The APMonitor executable on the back-end compiles a model to byte-code and performs model reduction based on analysis of the sparsity structure (incidence of variables in equations or objective function) of the model. For differential and algebraic equation systems, orthogonal collocation on finite elements is used to transcribe the problem into a purely algebraic system of equations. APMonitor has several modes of operation, adjustable with the imode parameter. The core of all modes is the nonlinear model. Each mode interacts with the nonlinear model to receive or provide information. The 9 modes of operation are:

1. Steady-state simulation (SS)
2. Model parameter update (MPU)
3. Real-time optimization (RTO)
4. Dynamic simulation (SIM)
5. Moving horizon estimation (EST)
6. Nonlinear control / dynamic optimization (CTL)
7. Sequential dynamic simulation (SQS)
8. Sequential dynamic estimation (SQE)
9. Sequential dynamic optimization (SQO)


Modes 1-3 are steady state modes with all derivatives set equal to zero. Modes 4-6 are dynamic modes where the differential equations define how the variables change with time. Modes 7-9 are the same as 4-6 except the solution is performed with a sequential versus a simultaneous approach. Each mode for simulation, estimation, and optimization has a steady state and dynamic option.

APMonitor provides the following to a Nonlinear Programming Solver (APOPT, BPOPT, IPOPT, MINOS, SNOPT) in sparse form:

- Variables with default values and constraints
- Objective function
- Equations
- Evaluation of equation residuals
- Sparsity structure
- Gradient of the objective function
- Hessian of the Lagrangian (2nd derivatives)
- 2nd Derivative of the equations
- 2nd Derivative of the objective function


Once the solution is complete, APMonitor writes the results in results.json that is loaded back into the python variables by GEKKO

When the system of equations does not converge, APMonitor produces a convergence report in ‘infeasibilities.txt’. There are other levels of debugging that help expose the steps that APMonitor is taking to analyze or solve the problem. Setting DIAGLEVEL to higher levels (0-10) gives more output to the user. Setting COLDSTART to 2 decomposes the problem into irreducible sets of variables and equations to identify infeasible equations or properly initialize a model.".

Of course you can extend those capabilities with any number of other Python packages for Operation Research.

The pyFMI GUI is a fair bit more modern looking than Alteryx:

I know a guy that is working on this project:

https://github.com/gleraromero/kaleidoscope

it supports some common optimization algorithms like Column Generation, Branch & Cut and Branch Cut & Price among others.

It's based on the output of another project he's working on, but maybe it's useful for you.

I liked this question because some years ago I found that solvers (for instance CPLEX) don't have, as far as I know, good visualisation tools. It could be an interesting side project something that takes some simple CPLEX output and builds a simple dashboard with that (call it CPLEXboard ;) ).

It could be useful to analyse how the tree is explored, the value of the variables, branching variables, number of cuts, time, nodes and stuff, all live in an interactive environment, instead of just digging a log file...