I am interested in evaluating and demonstrating various aspects of the performance of general solvers for mathematical optimization problems (in particular MILPs). I assume that I have a good testset of instances. What "revealing" experiments should/could I perform, and more importantly, what graphical presentations can I provide to convey certain key aspects? These may refer to information on the testset, on the solver, on several different settings/versions of the solver, on particular aspects of the solver, etc. I am particularly interested in example figures from papers or talks.
Examples:
Performance variability displayed using the instance enlight13
: 100 runs of the same instance, each time with differently permuted input, and resulting CPU times; taken from the MIPLIB2010 paper.
Performace profiles for different solvers on the same test set, the further "top left", the better.
What other graphical aids do you find enlightening when evaluating solvers?