Training ML models to be used as objectives in optimization problems

Question

Suppose that we have data (in my case, from a chemical process) which includes input data $X$ (characteristic of the material to be processed) and decision data $Y$ (decisions taken by operators to process this material), which produce an output value $z$, but there are no clear explanations on how these decisions impact the output.

To solve this, I can construct a machine learning model $f$ from the data, that represents how decisions $Y$ impact on the input data $X$ to produce this output. For example, I can construct a linear regression model $\hat\beta$ such that $f(X,Y) := \hat\beta_X X + \hat\beta_Y Y \approx z$. My final objective is to optimize the resulting model. That is, given a new input data $X'$, to solve $$ \min_Y f(X',Y) \text{ s.t. } Y\in\Omega$$ (for example, with a lineal regression mode, we can solve a linear problem subject to linear constraints easily)

My question is if this kind of ML models and this use of them has been studied formally. Because, in some sense, both types of data are quite different (one is an input, the other can be controlled) but a "classic" ML model will treat them similarly. It is very probable that strong correlations exist between the input data and the decision data, which should not be eliminated. For example, high levels of a compound (an input) can require high levels of a reactant (a decision), but a classic linear regression model can remove one of them from the model for being correlated, which is exactly the opposite that I need. Also, it is quite probable that very bad decisions are never taken (so there will be no data for these cases), so you don't want that the model overestimate these regions.

Probably this has been studied, and may even have a name, but I can't find it. I do found papers constructing ML models and then optimizing them, but most of them don't care about these issues between input and decision data on the construction of the ML model.

Any idea or pointer to the literature on this subject?

Answer 1

There is a recent paper by Bergman et al. [1] in the INFORMS Journal on Computing that integrates various types of predictive models with optimization models. It might be of interest.

[1] Bergman, D.; Huang, T.; Brooks, P.; Lodi, A. & Raghunathan, A. U. JANOS: An Integrated Predictive and Prescriptive Modeling Framework. INFORMS Journal on Computing, 2022, 34, 807.

Answer 2

This reminds me of two concepts:

1. The first is meta-modelling which is used in health economics (and probably also used in other disciplines). The way it is used there is that based on some decision X in a simulation model, we try to predict the resulting objective value of that decision. It is used to substitute computationally intensive simulations by regression models. See a paper about meta-modelling in health economics here. I am aware that it is still different from your problem, as you do not only have decision data but some other (external) input. But the concept of predicting the objective value is the same, so it might serve as a start point.

2. The second are specific types of value function approximation, such as Q-learning. In Q-learning (with function approximation), we try to approximate the value of a state action pair. In your case, a state could be described by input $X$ and your action is decision $Y$. Again, this is also not exactly suitable for your problem, because firstly, Q-learning is meant for sequential decision problems which I assume you do not deal with and secondly, the action (decision) in Q-learning is chosen by some strategy and not pre given. However, evaluating state action pairs is what you also want, so this might also serve as a starting point.

Additionally you should checkout inverse reinforcement learning where the goal is to learn an agent's objective by studying it's actions.

Answer 3

So you have historical observations of $(X, Y, z)$. I'll assume that you know how to construct a model $f$ such that $f(X, Y) \simeq z$, and that you have such a model ready by. In other words, you've done the training, and now you focus on optimizing its output, i.e., on solving a problem of the form \begin{align} \min_{Y} \quad & f(X, Y)\\ s.t. \quad & Y \in \Omega \end{align} for given $X$.

As you have already mentioned, depending on the nature of $f$, this may be more or less simple. If $f$ is linear, then your objective is linear. In addition to JANOS mentioned in another comment, you may find the OMLT software to be close to what you describe. That choice is in python, and it supports several classes of models out-of-the-box.