# How to measure model quality in OR-Tools?

After creating my model for a problem, what steps should I take to test if a change in the model is actually helpful or not?

In Python, print(solver.ResponseStats()) returns:

CpSolverResponse:
status: FEASIBLE
objective: 4661700
best_bound: 9876900
booleans: 108186
conflicts: 6415
branches: 3173794
propagations: 7757468
integer_propagations: 24021224
walltime: 21.7395
usertime: 21.7395
deterministic_time: 6.71619
primal_integral: 4.72458e+06


Some ideas that I have are (with a small problem instance):

• Compare the time to solve it.
• Compare the number of conflicts.
• Compare the best objective bound given a time limit.
• Some suggestions: Time to first solution (possible with some measurement in gap for quality), number of feasible solutions, the area between lower and upper bound over time (small is good) – Simon Spoorendonk Feb 27 '20 at 13:12