Scheduling problem data generation

Question

I compare the strengths and weaknesses of paradigms and approaches within those paradigms for solving machine scheduling problems. The amount of real-world data I have is limited and (random) subsampling critically changes the characteristics of the sets so that they are no longer realistic. I do however feel it is necessary to provide a richer class of problem data as to see how different methods react to different intrinsics - as long as they are likely to be more than just theoretical. What is the most responsible and honest way to include this in my research?

There are papers mentioning typical industry ranges for certain characteristics, e.g. a rate for how tight deadlines are. Therefore, my best idea so far is to do carry out a (full) factorial analysis for those parameters. Other ratios also seem interesting, such as average transition vs. execution duration, but I don't exactly find resources mentioning them or generators for creating them.

I feel like I'm not being rigorous enough if I just disregard the part of research just mentioned, but at the same time, I don't want to muddy the waters by introducing irrelevant parameters. Preliminary results show that indeed, for the same problem formulation, the best result sometimes depends on the shape (and obviously, the size) of the data set.

Answer 1

It is not ideal, but sometimes I think the best you can do is utilize problem collections from published papers as benchmarks, even if their parameters are rather arbitrary (not necessarily based on industry experience). For example, there is a compendium by Oleg Shylo of experimental results for job shop problems at http://optimizizer.com/jobshop.php.

Answer 2

One type of data that is likely to exist in many real-world applications is the so-called "ordered" data. In an ordered setting, if the processing time of a job $j$ is larger than that of another job $j'$ on machine $r$, then it is the case on all machines, that is, the jobs can be ranked based on their processing times. Also, there is the same relationship between machines and they can be ranked as well. There is a publicly available "hard" benchmark for the ordered setting in the flow shop environment, that you can find it in Khatami et al. (2019)¹.

---

_Reference

_{[1] Khatami, M., Salehipour, A., Hwang, F. J. (2019). Makespan minimization for the m-machine ordered flow shop scheduling problem. Computers and Operations Research. 111:400-414.}

Answer 3

A generator of job-correlated, machine-correlated and mixed-correlated Permutation Flowshop instances is described in Watson et al. (2002)¹ and can be found here. From the abstract:

We introduce a method for generating structured flow-shop problems, which are modeled after features found in some real-world manufacturing environments.

Another list of instance benchmarks for various kinds of scheduling problems can be found here. For some of them, you can find there also the binary of the generator.

---

_Reference

_{[1] Watson, J. P. et al. (2002). Contrasting Structured and Random Permutation Flow-Shop Scheduling Problems: Search-Space Topology and Algorithm Performance. INFORMS Journal on Computing. 14(2):98-123.}