When designing optimization models for external organizations, I have witnessed the following:
We design an optimization model for a given problem.
We fine-tune it based on a portfolio of representative instances.
The model starts being used in production, it behaves well, providing good solutions in a reasonable time.
Over time, the business of the external organization changes, making the portfolio of instances that was used for the fine-tuning no longer representative. The results are now of much lower quality or the optimization takes much too long.
Silently the model stops being used.
Which approaches do you use to avoid having the model no longer fit the problem it is trying to solve?
I am mostly interested in approaches that can be applied both to exact solution methods as to heuristic solution methods.