Although I only heard about Contextual Stochastic Optimization (CSO) a few months ago, I know now the excitement has been going on for a while. I'm not sure if the idea of CSO has been around for long, but at least this latest iteration of excitement is often linked to Bertsimas and Kallus "From predictive to prescriptive analytics" (2019). Along with those names, two other authors that have been working on the area and finding some successful results for a while are Elmachtoub and Grigas.
In CSO, we are concerned with a prediction problem (PP) and optimization problem (OP) that happen in tandem. The prediction problem can be cast as the minimization of a loss (or cost) function. Once solved, the results are used by the optimization problem (OP). From what I understand, CSO has become popular lately because it allows the use of contextual/side/covariate information that might otherwise go unused in different modelling frameworks. With the data-rich environments we have nowadays, it seemed a waste of useful data. Another reason CSO might have become so popular is that it does not focus on finding the best prediction error; rather, it focus on finding the best optimization error. Under this framework, the prediction is an aid to the OP.
But what happens when you have a bilevel contextual optimization problem (BCOP) and the prediction problem (PP) is part of the inner problem?
The way I know of solving bilevel problems is by re-writing the inner problem as constraints to the outer problem through the KKT conditions. If the PP is being used to estimate the parameters of the inner problem, then the PP itself is going to be re-written as an optimization problem, likely a loss-function minimization problem.
If, for instance, I assume my data follow an autoregressive model, I can estimate the parameters of the model by solving the Yule-Walker equations, which can be solved as a feasibility problem. Those are linear equations that can easily be "sent" to the outer problem through the KKT. Now, one thing that I can't seem to understand is how that approach will work if I chose to model the data using a ML approach, say a NN, for instance.