This post is not really about a specific question but rather a topic I am curious about to know more.
We know that when it comes to integrate machine/statistical learning with optimization for the purpose of replacing ideal equations with data-driven models, the choices are quite limited - mostly regression (linear, polynomial) and spline models or at best a shallow neural network that can be decoded back by hand as an analytical expression. Such limited choices naturally mean that generally speaking highly non-linear functions would be difficult to approximate well. Deep neural networks are known to do exceptionally well when it comes to model complex functions. However, the problem is that they are hard to represent then as an expression/formula (without needing a mile long paper), which however if becomes possible would indeed be great from optimization perspective.
I would like to know if there been some work done on this topic previously or is any active research going on that you guys may be aware of. Keen to hear about it. Thanks.
EDIT - My question is more from the side of using AML's (Pyomo, JuMP etc), where one has the luxury of directly writing the algebraic expressions.