13
$\begingroup$

We often hear that in practice, not enough data of sufficient quality, consistency, recency, etc. is available for feeding into mathematical optimization models. Example: my university wanted to plan/optimize their weekly timetable using an integer program, but they did not know the number of rooms (let alone capacities, availabilities, equipment, location, etc.), they did not know the preferences of professors, nor which courses they actually taught (the system listed them as "responsible" for a course which did not imply that they were actually teaching that course!); they didn't know the number of students to expect in a course. I could contribute a lot of such stories.

Now, many companies (truthfully) claim that they collect data. E.g., sensor data from production, temperatures, filling rates, number of faulty products per hour, web clicks, customer orders, energy prices, etc., etc. I can't really grasp what makes we reject such data as "suitable" for optimization, and I am looking for a definition of what "different kind of data" needs to be collected in order to feed a typical mathematical program for e.g., timetabling, production planning, facility layout, or designing tariff zones. I thought for a while that the notion I am looking for is "actionable", but this doesn't capture it. Ideally, I would like to contrast this "optimization data" to data that is typically fed into machine learning algorithms (which extrapolate, cluster, predict, find trends, anomalies, patterns, etc.).

How would you call the number of students in a course, the availabilities of teachers, the capacities of rooms, the data that a course belongs to a certain curriculum?

$\endgroup$
  • $\begingroup$ Maybe "input data" but that's too general. How about "decision-making parameters"? In the sense that information about them (or their distribution) is taken as known/forecasted/input for the optimization/decision-making. And different scenarios will consist of different sets of parameters, that yield different decisions (be it for production planning, timetabling, etc) $\endgroup$ – dhasson Jun 30 at 16:16
  • $\begingroup$ yes, @dhasson, do you have words/attributes describing such data? "descriptive"? "instance-independent"? hmpf! :) $\endgroup$ – Marco Lübbecke Jun 30 at 16:19
7
$\begingroup$

I would call them decision-relevant data, because most optimization problems in practice help people do decisions better, which they already do in a heuristic fashion. This puts the focus on the decision and what is needed to effectively make this decision.

Alternatives would be system-describing data/system-boundary data, because the data defines the boundaries of the feasible states of the system/the boundaries of feasible decision.

On the other hand the data in machine learning i would call historical observational data, because you often have observable states of the system from the past.

I find it difficult to draw a line, that this data is for optimization and that data is for machine learning, because often data can be used for both. In your example with the timetable for the university courses you could for example not have the capacity of each room, but instead the average number of students per room for each day in the last year. This data would at a first glance be rather machine learning data, but you could use it to derive an estimate for the capacity to feed it to your optimization model.

I agree that pure observable data is often useless for optimization problems, because you only observe feasible states, but have no data on how much you can deviate from these and what are the effects on the deviation and optimizing is basically putting the system in an unseen state than before.

| improve this answer | |
$\endgroup$
  • $\begingroup$ thanks! this also sounds like "forward looking" and "backward looking" data (and yes, you are right, often no "pure" cases) $\endgroup$ – Marco Lübbecke Jun 30 at 18:27
7
$\begingroup$

I would just call it "planning data". I think it might be easier to convince an administrator that "planning data" needs to be recorded/captured than to sell them on "<insert techno-jargon phrase here> data". Administrators grasp what planning is (whether or not they are adept at doing it), and at some visceral level they probably realize that not planning is bad (which might make them a bit more inclined to make an effort to collect the data). If "planning data" does not do much to distinguish this sort of data from other sorts (salary data, student/faculty ratio, ...), perhaps that's a good thing. They already understand the relevance of the other data, and that it needs to be collected, so by association they might realize this data is also important.

| improve this answer | |
$\endgroup$
  • $\begingroup$ thanks, Paul, besides a name, I would also be interested in "charateristics", is planning data of a different "kind" than other data? I remember Jack Lewis who talked about "different" data that would need to be collected for prescriptive analytics than for, say, descriptive analytics. $\endgroup$ – Marco Lübbecke Jul 1 at 20:15
  • 1
    $\begingroup$ Marco: To me, "kind of data" could mean cross-sectional v. longitudinal (time series), measured (physically) v. opinion, actual v. fabricated (political). Depending on the application, an optimization model might require any mix of those (except, hopefully, fabricated). In terms of what you encountered with the academic timetable, the two most meaningful distinctions might be "currently collected (for other reasons)" v. "not currently collected" and "easy to (routinely) collect" v. "PITA to collect". $\endgroup$ – prubin Jul 2 at 21:22
5
$\begingroup$

Adhering to the rules of encapsulation, I would simply call it "parameters". If we're thinking of an optimisation model and, as you said, what changes is the number of things (number of students, number of classrooms, a table with the teachers' schedule, etc.), that's what we call usually parameters in optimisation modelling so I don't see a reason to use a different term.

If we wanted to make the name more descriptive, I would attach a problem-specific prefix there, e.g., "planning parameters".

I like this term because it indicates that the math would be the same (assuming that's the case here) even if those numbers change.

I would avoid the word "data" because it's too broad - we also use "data" to formulate the math.

| improve this answer | |
$\endgroup$
  • 2
    $\begingroup$ thanks! except that I am the mathematician, yes, but I talk to praftioners. And they tell me that they have a lot of sensor data and records which faulty parts they produced and the want me to do production optimization, and now what? they don't have the right data, the don't even have the right kind of data :) $\endgroup$ – Marco Lübbecke Jul 4 at 14:34
  • $\begingroup$ Haha well in that case I would just ask for "all relevant information" 😁 $\endgroup$ – Nikos Kazazakis Jul 4 at 17:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.