I have seen this question How to avoid having your optimization models rusting?, which is kind of related, but I am more curious about the actual monitoring and the initial design of the software in order to be able to find bugs/problems in a production environment.
Assume you have a software running in production (so it is running without your interaction on a server/ or at the user), which is built on top of MIP solver (i.e. Gurobi,...). What kind of data do you collect and at what point in order to be able to:
a) identify critical instances (like numerical troubles) or when it is difficult to solve in general, or models become infeasible if they should not, or there is a bug in the solver itself?
b) debug your model afterwards?
It might be useful to always save the log file to a database and build statistics about the average runtime or grep for numerical troubles lines in the log and send like an alert or monitor the database from time to time. In order to debug the problem one needs in general at least an lp-file or better all the data that went into the model. This is kind of memory sensitive, because you can not write an lp file every time the application is running. If one wants the raw data that went into the model one needs to write an intermediate layer between the database (at the user) and your optimization model in order to be able to write down the data to the memory.