10 votes

How to (graphically) present computational results?

We've written a benchmark toolkit that spits out a html with a number of statistics we find useful. Of these, I'll highlight the ones that I use regularly in bold. Over multiple datasets and multiple ...
Geoffrey De Smet's user avatar
9 votes
Accepted

How to (graphically) present computational results?

Some further plots that can be helpful are bar charts with the type of solver going along the horizontal axis and the number of problem instances on the vertical axis. However, each bar is colour-...
TheSimpliFire's user avatar
  • 5,382
7 votes

How fair is limit solver threads when publishing?

The problem is, even if you use the same number of threads, a 'rigorous' comparison of different implementations in different programming languages - for example, in terms of running time or of the ...
dhasson's user avatar
  • 1,667
6 votes

How to (graphically) present computational results?

There are a couple of situations where I would find box plots enlightening. For performance variability (either time to proven optimum or gap at a fixed time limit), box-and-whiskers plots with the ...
prubin's user avatar
  • 38.7k
6 votes
Accepted

Job Shop Scheduling: what objectives are harder?

Some observations why min sum objectives are computationally more difficult than the makespan objective: For the decision problem, it can be shown that the total lateness objective is at least as ...
SebastianK's user avatar
6 votes

How fair is limit solver threads when publishing?

Personal opinion: I think it is fair to use as many threads as makes sense in context. For instance, when I use CPLEX I believe it defaults to four threads (one per core on my PC). If I'm solving a ...
prubin's user avatar
  • 38.7k
6 votes

How to simulate computational execution time?

Short answer: you can't with any decent level of accuracy. The best you can do is ballpark comparisons. There are so many factors that affect the outcome, that you can maybe get within 20% difference ...
Nikos Kazazakis's user avatar
5 votes

How to simulate computational execution time?

In support of other answers and suggestions that you just run the other algorithm on your hardware, I would argue that failing to match the published results exactly is not necessarily cause for ...
prubin's user avatar
  • 38.7k
3 votes

Literature for building solver portfolios

For the automatic solver configuration, I know of this reference (there may be a journal version): A learning-based mathematical programming formulation for the automatic configuration of optimization ...
mtanneau's user avatar
  • 4,108
3 votes

confusing results of two models with different complexity

There are a number of possible explanations (not mutually exclusive). The larger model might have a tighter continuous relaxation. (You can test that by relaxing the integrality restrictions and ...
prubin's user avatar
  • 38.7k
2 votes

Job Shop Scheduling: what objectives are harder?

In addition to the above answer, the following might be worthwhile too. Often, an algorithm for one scheduling problem can be applied to another scheduling problem as well. For example, $ 1|| \sum c_{...
A.Omidi's user avatar
  • 8,642
1 vote

Literature for building solver portfolios

I'm not sure if this is exactly on point, but there is a paper by Carvajal et al. on the use of parallel trees (with communication between them) for solving integer programs. You might also be ...
prubin's user avatar
  • 38.7k

Only top scored, non community-wiki answers of a minimum length are eligible