10
votes
How to (graphically) present computational results?
We've written a benchmark toolkit that spits out a html with a number of statistics we find useful. Of these, I'll highlight the ones that I use regularly in bold.
Over multiple datasets and multiple ...
- 4,685
9
votes
Accepted
How to (graphically) present computational results?
Some further plots that can be helpful are bar charts with the type of solver going along the horizontal axis and the number of problem instances on the vertical axis. However, each bar is colour-...
- 5,341
7
votes
How fair is limit solver threads when publishing?
The problem is, even if you use the same number of threads, a 'rigorous' comparison of different implementations in different programming languages - for example, in terms of running time or of the ...
- 1,637
6
votes
How to (graphically) present computational results?
There are a couple of situations where I would find box plots enlightening. For performance variability (either time to proven optimum or gap at a fixed time limit), box-and-whiskers plots with the ...
- 37.5k
6
votes
Accepted
Job Shop Scheduling: what objectives are harder?
Some observations why min sum objectives are computationally more difficult than the makespan objective:
For the decision problem, it can be shown that the total lateness objective is at least as ...
- 176
6
votes
How fair is limit solver threads when publishing?
Personal opinion: I think it is fair to use as many threads as makes sense in context. For instance, when I use CPLEX I believe it defaults to four threads (one per core on my PC). If I'm solving a ...
- 37.5k
6
votes
How to simulate computational execution time?
Short answer: you can't with any decent level of accuracy. The best you can do is ballpark comparisons.
There are so many factors that affect the outcome, that you can maybe get within 20% difference ...
- 11.9k
5
votes
How to simulate computational execution time?
In support of other answers and suggestions that you just run the other algorithm on your hardware, I would argue that failing to match the published results exactly is not necessarily cause for ...
- 37.5k
3
votes
Literature for building solver portfolios
For the automatic solver configuration, I know of this reference (there may be a journal version): A learning-based mathematical programming formulation for the automatic configuration of optimization ...
- 4,068
3
votes
confusing results of two models with different complexity
There are a number of possible explanations (not mutually exclusive). The larger model might have a tighter continuous relaxation. (You can test that by relaxing the integrality restrictions and ...
- 37.5k
2
votes
Job Shop Scheduling: what objectives are harder?
In addition to the above answer, the following might be worthwhile too.
Often, an algorithm for one scheduling problem can be applied to another scheduling problem as well. For example, $ 1|| \sum c_{...
- 8,197
1
vote
Literature for building solver portfolios
I'm not sure if this is exactly on point, but there is a paper by Carvajal et al. on the use of parallel trees (with communication between them) for solving integer programs. You might also be ...
- 37.5k
Only top scored, non community-wiki answers of a minimum length are eligible