variable use of nested blocks in pyomo

Question

I'm using pyomo to model a system consisting of subsystems which themselves consist of subsystems. Therefore, I use nested Blocks. Every block is described in a separate file. I want to be able to switch which models I use as subsystems. I'm wondering whether it's possible to give the outermost model the inner models as arguments. While this is straight forward for one level of hierarchy (a block in the outermost model), I can't seem to find a way to make two levels of hierarchy work (pass the innermost model to the 1st level). This would be very handy as I could change which inner models I load without changing the import model from file in the model file's headers all the time.

Here is an example code that shows my approach. For what I want to do I would need to be able to pass non-model related arguments to the pyo.Block function, which doesn't seem to work.

## model file
import pyomo.environ as pyo

def func_model(subblock,subsubblock):
    model = pyo.AbstractModel()
    model.idx = pyo.RangeSet()
    model.subblock = pyo.Block(model.idx, **subsubblock**, rule = subblock)
    return model

### script

def subblock(submodel,idx, **subsubblock**):
    submodel.jdx = pyo.RangeSet()
    submodel.subsubblock = pyo.Block(submodel.jdx, rule = subsubblock)
    return submodel

def subsubblock(subsubmodel):
    subsubmodel.kdx = pyo.RangeSet()
    return subsubmodel

Now I pass the data from a dict

data = {
    None: {
        "idx" : {None: [1,2]},
        "subblock": {
            "jdx": {None: [1]},
            "subsubblock": {
                "kdx": {None: [1,2,3]}
            }
        }
    }
}

and create the instance

model = func_model(subblock, subsubblock)

instance = model.create_instance(data)

Now checking the model.pprint() results in

2 Set Declarations
    subblock_index : Size=1, Index=None, Ordered=True
        Key  : Dimen : Domain               : Size : Members
        None :     2 : idx*subblock_index_1 :    0 :      {}
    subblock_index_1 : Size=1, Index=None, Ordered=Insertion
        Key  : Dimen : Domain : Size : Members
        None :     1 :    Any :    1 : {None,}

2 RangeSet Declarations
    idx : Dimen=1, Size=0, Bounds=(None, None)
        Key  : Finite : Members
        None :   True :      []
    kdx : Dimen=1, Size=0, Bounds=(None, None)
        Key  : Finite : Members
        None :   True :      []

1 Block Declarations
    subblock : Size=0, Index=subblock_index, Active=True

5 Declarations: idx subblock_index_1 subblock_index subblock kdx

So the nesting didn't work.

Edit:

As pointed out by jsiirola, nesting in combination with abstract models can get very cumbersome. So I tried to model it using concrete models and nest the blocks "by hand".

import pyomo.environ as pyo

def func_model():
    model = pyo.AbstractModel()
    model.idx_lb = pyo.Param()
    model.idx_ub = pyo.Param()
    model.idx = pyo.RangeSet(model.idx_lb, model.idx_ub)
    #model.subblock = pyo.Block(model.idx, rule = subblock)
    return model

def subblock():
    submodel = pyo.AbstractModel()
    submodel.jdx_ub = pyo.Param()
    submodel.jdx = pyo.RangeSet(submodel.jdx_ub)
    #submodel.subsubblock = pyo.Block(submodel.jdx, rule = subsubblock)
    return submodel

def subsubblock():
    subsubmodel = pyo.AbstractModel()
    subsubmodel.kdx_ub = pyo.Param()
    subsubmodel.kdx = pyo.RangeSet(subsubmodel.kdx_ub)
    return subsubmodel

Then, creating the subsubmodel

ssb_data = {None: {"kdx_ub" : {None: 3}}}
ss_model = subsubblock()
ss_instance = ss_model.create_instance(ssb_data)

and adding this into the submodel

import copy
sb_data = {None: {"jdx_ub": {None: 2}}}
s_model = subblock()
s_instance = s_model.create_instance(sb_data)

def copy_func(m,j):
    return copy.deepcopy(ss_instance)

s_instance.subsubblock = pyo.Block(s_instance.jdx, rule = copy_func)

I would also do the same for nesting the subblock into the model.

I think this is much better than the cumbersome abstract model instantiation I would normally do. However, in my real example I have a nested model that is interconnected. E.g., my (outermost) model puts constraints on the subblock's parameters. Is there any way to handle this with the upper approach?

Answer 1

This question is a partial duplicate of https://stackoverflow.com/questions/75136387/dependent-component-cant-be-instantiated-in-abstractmodel-with-pyomo-5-7-3.

First, I would generally recommend the use of ConcreteModel and not AbstractModel for structured (hierarchical) Pyomo models: As you will see, the data structure for the AbstractModel can get rather cumbersome. Further, it is frequently more natural (both for the modeler and for Pyomo) for these models to load / process the data first and then construct the (concrete) Pyomo model in a single pass. For your use case, it might be more natural for each subsystem to be constructed by a function, and you can then pass the constructor for the sub-sub system as a functor argument to the subsystem's construction function that returns the fully-constructed subsystem.

As for your example (based on abstract models) There are a couple issues here:

1. Current versions of Pyomo do not support initializing RangeSet objects from data. Instead, the RangeSet bounds / step should be defined using Param components (and then the Param components can be initialized from data).
2. The data structure for initializing abstract components is a dict that maps component indexed to the initialization data. For scalar components, the data structure should have a key "None" for the implicit index.

Putting that together, a working implementation of your model would be something like:

## model file
import pyomo.environ as pyo

def func_model(subblock, subsubblock):
    model = pyo.AbstractModel()
    model.idx_lb = pyo.Param()
    model.idx_ub = pyo.Param()
    model.idx = pyo.RangeSet(model.idx_lb, model.idx_ub)
    model.subblock = pyo.Block(model.idx, rule = subblock)
    return model

### script

def subblock(submodel, idx):
    submodel.jdx_ub = pyo.Param()
    submodel.jdx = pyo.RangeSet(submodel.jdx_ub)
    submodel.subsubblock = pyo.Block(submodel.jdx, rule = subsubblock)
    return submodel

def subsubblock(subsubmodel):
    subsubmodel.kdx_ub = pyo.Param()
    subsubmodel.kdx = pyo.RangeSet(subsubmodel.kdx_ub)
    return subsubmodel

with data

data = {
    None: {
        "idx_lb" : {None: 1},
        "idx_ub" : {None: 2},
        "subblock": {
            1: {
                "jdx_ub": {None: 1},
                "subsubblock": {
                    1: {
                        "kdx_ub": {None: 3},
                    },
                },
            },
            2: {
                "jdx_ub": {None: 1},
                "subsubblock": {
                    1: {
                        "kdx_ub": {None: 3},
                    },
                },
            },
        }
    }
}

that when run, would give:

>>> func_model(subblock, subsubblock).create_instance(data).pprint()
1 RangeSet Declarations
    idx : Dimen=1, Size=2, Bounds=(1, 2)
        Key  : Finite : Members
        None :   True :   [1:2]

2 Param Declarations
    idx_lb : Size=1, Index=None, Domain=Any, Default=None, Mutable=False
        Key  : Value
        None :     1
    idx_ub : Size=1, Index=None, Domain=Any, Default=None, Mutable=False
        Key  : Value
        None :     2

1 Block Declarations
    subblock : Size=2, Index=idx, Active=True
        subblock[1] : Active=True
            1 RangeSet Declarations
                jdx : Dimen=1, Size=1, Bounds=(1, 1)
                    Key  : Finite : Members
                    None :   True :     [1]

            1 Param Declarations
                jdx_ub : Size=1, Index=None, Domain=Any, Default=None, Mutable=False
                    Key  : Value
                    None :     1

            1 Block Declarations
                subsubblock : Size=1, Index=subblock[1].jdx, Active=True
                    subblock[1].subsubblock[1] : Active=True
                        1 RangeSet Declarations
                            kdx : Dimen=1, Size=3, Bounds=(1, 3)
                                Key  : Finite : Members
                                None :   True :   [1:3]

                        1 Param Declarations
                            kdx_ub : Size=1, Index=None, Domain=Any, Default=None, Mutable=False
                                Key  : Value
                                None :     3

                        2 Declarations: kdx_ub kdx

            3 Declarations: jdx_ub jdx subsubblock
        subblock[2] : Active=True
            1 RangeSet Declarations
                jdx : Dimen=1, Size=1, Bounds=(1, 1)
                    Key  : Finite : Members
                    None :   True :     [1]

            1 Param Declarations
                jdx_ub : Size=1, Index=None, Domain=Any, Default=None, Mutable=False
                    Key  : Value
                    None :     1

            1 Block Declarations
                subsubblock : Size=1, Index=subblock[2].jdx, Active=True
                    subblock[2].subsubblock[1] : Active=True
                        1 RangeSet Declarations
                            kdx : Dimen=1, Size=3, Bounds=(1, 3)
                                Key  : Finite : Members
                                None :   True :   [1:3]

                        1 Param Declarations
                            kdx_ub : Size=1, Index=None, Domain=Any, Default=None, Mutable=False
                                Key  : Value
                                None :     3

                        2 Declarations: kdx_ub kdx

            3 Declarations: jdx_ub jdx subsubblock

4 Declarations: idx_lb idx_ub idx subblock