What you are asking for sounds somewhat similar to Alteryx (which starts at $5,195/user per year). Let's see how we can duplicate some of the look and functionality that with free tools. A demo video "Alteryx Designer in 20 Minutes" (titled "Alteryx Demo" on YouTube) is available.
Here's an overview of the workflow:
You asked for:
"Let's say you have an optimization problem and a solution and want to be able to track individual variables, solution values, maybe even dual values and so on. For example, to click on a variable and see all containing rows. Or to select a row and highlight all variables which are in the row. Or even to fuse the information of an solution with the lp file so that you can see all variables and their solution value in the problem.".
Here's the Alteryx user interface:
How can we obtain that sort of functionality for less than the cost of Alteryx?
Using Gekko and APMonitor, along with pyFMI for FMI support (that provides drag and drop), you end up with (description from Gekko's webpage):
"What does GEKKO do?
GEKKO is optimization software for mixed-integer and differential algebraic equations. It is coupled with large-scale solvers for linear, quadratic, nonlinear, and mixed integer programming (LP, QP, NLP, MILP, MINLP). Modes of operation include data reconciliation, real-time optimization, dynamic simulation, and nonlinear predictive control. The client or server is freely available with interfaces in MATLAB, Python, or from a web browser.
GEKKO is a high-level abstraction of mathematical optimization problems. Values in the models are defined by Constants, Parameters, and Variables. The values are related to each other by Intermediates or Equations. Objective functions are defined to maximize or minimize certain values. Objects are built-in collections of values (constants, parameters, and variables) and relationships (intermediates, equations, and objective functions). Objects can build upon other objects with object-oriented relationships.
The APMonitor executable on the back-end compiles a model to byte-code and performs model reduction based on analysis of the sparsity structure (incidence of variables in equations or objective function) of the model. For differential and algebraic equation systems, orthogonal collocation on finite elements is used to transcribe the problem into a purely algebraic system of equations. APMonitor has several modes of operation, adjustable with the imode parameter. The core of all modes is the nonlinear model. Each mode interacts with the nonlinear model to receive or provide information. The 9 modes of operation are:
1. Steady-state simulation (SS)
2. Model parameter update (MPU)
3. Real-time optimization (RTO)
4. Dynamic simulation (SIM)
5. Moving horizon estimation (EST)
6. Nonlinear control / dynamic optimization (CTL)
7. Sequential dynamic simulation (SQS)
8. Sequential dynamic estimation (SQE)
9. Sequential dynamic optimization (SQO)
Modes 1-3 are steady state modes with all derivatives set equal to zero. Modes 4-6 are dynamic modes where the differential equations define how the variables change with time. Modes 7-9 are the same as 4-6 except the solution is performed with a sequential versus a simultaneous approach. Each mode for simulation, estimation, and optimization has a steady state and dynamic option.
APMonitor provides the following to a Nonlinear Programming Solver (APOPT, BPOPT, IPOPT, MINOS, SNOPT) in sparse form:
- Variables with default values and constraints
- Objective function
- Equations
- Evaluation of equation residuals
- Sparsity structure
- Gradients (1st derivatives)
- Gradient of the equations
- Gradient of the objective function
- Hessian of the Lagrangian (2nd derivatives)
- 2nd Derivative of the equations
- 2nd Derivative of the objective function
Once the solution is complete, APMonitor writes the results in results.json that is loaded back into the python variables by GEKKO
When the system of equations does not converge, APMonitor produces a convergence report in ‘infeasibilities.txt’. There are other levels of debugging that help expose the steps that APMonitor is taking to analyze or solve the problem. Setting DIAGLEVEL to higher levels (0-10) gives more output to the user. Setting COLDSTART to 2 decomposes the problem into irreducible sets of variables and equations to identify infeasible equations or properly initialize a model.".
Of course you can extend those capabilities with any number of other Python packages for Operation Research.
The pyFMI GUI is a fair bit more modern looking than Alteryx:
