8
$\begingroup$

I am trying to extract my variable values into unique lists so that I can pass them to a panadas dataframe and eventually export the dataframe to Excel after I solve my model. The idea is to have the variables of interest in a much more readable and familiar format.

My thought process right now looks like this:

  1. Create an empty list and add new lists.
  2. For each variable in the model, get the dictionary-key values and insert those in the list.
  3. The resultant list, value_list, will have each variables values stored in its own list.
value_list = []

for v in model.component_objects(Var):
    value_list.append( list(model.v.get_values().values()) )

I would think that the loop variable would replace the v component in the body of for loop and then run the operation as if it was the explicit variable name, but this is not the case.

Ultimately I am trying to produce a dataframe of my solved variables with the first series being the index of tuples (i,j,t) up to (I,J,T), but it's a bit cumbersome to manually create a series object for each of my variables. Note that all variables as shown below:

list_of_series = [  pd.Series(pyomo_index_list),
                    pd.Series(list(model.susceptible.get_values().values())),
                    pd.Series(list(model.inf_treated.get_values().values())),
                    pd.Series(list(model.inf_b4treat.get_values().values())),
                    pd.Series(list(model.level1.get_values().values())),
                    pd.Series(list(model.juvenilleTotal.get_values().values()))   ]

If there was a way to create a loop so that model.<var>.get_values().values() was not hard coded, then the code would be more reproducible for models that have 15, 30, or 50 variables.

The final output would look like this, but as a pandas dataframe:

          Susceptible inf_treated inf_b4treat level1 JuvTotal   
Key                                                             
(1, 1, 0)          50           7         NaN    NaN       21   
(1, 1, 1)        50.2     8.22148      16.443      1     13.8   
(1, 1, 2)     52.5085     10.1232     20.2464      1     3.27   
(1, 1, 3)     44.0863     26.5758     26.5758      0    1.569   
(1, 2, 0)          30           7         NaN    NaN        8   
(1, 2, 1)        25.7     4.93289     9.86577      1      5.3   
(1, 2, 2)     24.8171     2.51488     5.02976      1     1.25   
(1, 2, 3)     22.9502     1.47623     1.47623      0    0.602   
(2, 1, 0)          20           9         NaN    NaN       18   
(2, 1, 1)        17.3     4.47514     8.95028      1     11.7   
(2, 1, 2)     21.7349     1.44328     2.88656      1     2.79   
(2, 1, 3)     21.7496    0.443511    0.443511      0    1.332   
(2, 2, 0)          35           8         NaN    NaN       16   
(2, 2, 1)        35.1     6.82927     13.6585      1      7.9   
(2, 2, 2)     33.9407     5.58183     11.1637      1     2.23   
(2, 2, 3)     29.6549     8.06275     8.06275      0    0.934 
$\endgroup$
  • $\begingroup$ Do you want to have all the values for variables in each iteration of optimization? Or simply the optimal values(if any)? In second case you won't need a list for each variable. you can define an empty list and just append whatever the value of the variable is, to the end of that list. $\endgroup$ – Oguz Toragay Oct 1 '19 at 20:30
  • $\begingroup$ I think you can find answer here hselab.org/pyomo-get-variable-values.html $\endgroup$ – kur ag Oct 1 '19 at 21:51
  • $\begingroup$ @Oguz Toragay, my idea is that after the solve is done and optimal values are achieved for the variables that you could essentially loop through each variable and put it in it's own list like it's a container. For instance, I have some code right now that converts each variable into a pd.Series object. The problem is that I have to do that manually, and if my problem had 30 variables, it would be cumbersome. That's why, in the code above, I tried to use a loop-variable for model.var.get_values().values(). Is that more clear? $\endgroup$ – GrayLiterature Oct 2 '19 at 13:45
  • $\begingroup$ @D.Gray Now I understood what you need. You want to have a list of lists(like a matrix in which each row is a variable and each column in the indexed value of that variable). $\endgroup$ – Oguz Toragay Oct 2 '19 at 13:55
  • $\begingroup$ Essentially, but just the transpose of what you are suggesting - so that each column represents a variable and as you go down the dataframe you see different values of that variable. But the idea is to generate a dataframe that I can export to Excel for better readability. That is why I wanted to be able to loop over all of my variables in the manner mentioned above so that I can make the code much more reproducible. $\endgroup$ – GrayLiterature Oct 2 '19 at 14:01
4
$\begingroup$

v represent each of your variables. I assume that your model is called 'model':

from pyomo.environ import *
import pandas as pd 
# add the following to your python script
DF = pd.DataFrame()
    for v in model.component_objects(Var,active=True):
        for index in v:
            DF.at[index, v.name] = value(v[index])

This part of the code directly generates the pandas' data frame for you with columns named according to the name of your variables. I hope it works for you, I tried it in my own model and got the following data frame:

enter image description here

| improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ Very useful! I just need to figure out how to get rid of one variable that is in the model because it has 4 indicies (all others have 3). But this should be more than enough to get me started! $\endgroup$ – GrayLiterature Oct 2 '19 at 21:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.