We’re  in the thick of building out a testing suite for decision models, starting with a focus on route optimization. There are several angles we care about  and are considering: unit testing, scenario testing, historical testing, shadow testing, switchback testing, acceptance testing, and benchmarking.
I’ve seen some discussions here about different aspects of testing [3, 4]. My general impression: cohesive testing still has a ways to go, but it adds value if you can find the time to set it up and maintain it (after all that other work you did you simply build and deploy your model — DoorDash has written about this well ).
I’m curious to know: What do you like or dislike about the testing frameworks/tooling available today? What testing capabilities do you wish you could have?
 We’re a decision optimization company called Nextmv (https://www.nextmv.io)