How to compare models?

I have the following problem. I have a staff scheduling model that explicitly considers employee motivation in the modeling (this is modeled by additional constraints that sum up the number of preference violations at the current time), with the goal of minimizing the current understaffing. Once I take this fact into account implicitly in the modeling (by setting the factors 1) $$\gamma$$, which determines the motivation based on the number of preference violations, i.e. $$1-\gamma \times number$$, and 2) $$\omega$$, which defines how many preference violations are necessary for the motivation to decrease, greater than zero) and once I set $$\gamma$$ and $$\omega$$ to zero. I would now like to compare the first approach with the second (naive) approach. Of course, you could compare the degree of under-sampling of the models, but are there other ways to compare the models? Without looking at technical things like runtime or gap?

If you are interested, I would be happy to present my model.

You could plot the mean objective value (understaffing) v. the mean preference violations (or some other employee satisfaction metric, if you have other metrics) for various combinations of values of $$\gamma$$ and $$\omega$$ (including zero for both), where the mean is taken over a sample of test problems.
• Thanks for your answer. Would It also be a viable option (I read about it a few days ago) to use both metrics for different combinations of $\gamma$ and $\omega$ and then plot it for example in a pareto plot to determine the parrot frontier? What insights would I get by doing that? Commented May 30 at 9:08
• Assuming you have a single actual problem to solve (real data) and a decision maker looking for a solution, you could plot understaffing v. preference violations for that problem, with a point for each $(\gamma, \omega)$ combination tested, and then superimpose the Pareto frontier to point out the nondominated solutions. The information value would be giving the decision maker an idea of what tradeoffs are possible, which might inform their preferences. I'm not sure if this would be as accurate as minimizing a weighted combination of the two objectives and tweaking the weights.