20
Generally speaking, verification refers to evaluating the conformation of a system, product, or service with its intended requirements and design specifications. On the other hand, validation refers to evaluate how much that system, product, or service is in accordance with the customers' or stakeholders' expectations.
AFAIK, there is no concrete literature ...
13
This is just to complement the answer that @Ehsan gave.
For both verification and validation, if possible, visualize your results. For example, if you are solving a routing problem, simply visualizing the resulting routes may provide great insights.
Does the solution (routes) make sense?
Are there constraints in the model that are violated and I had no ...
11
After further reading it seems that "surrogate models" do exactly this (aka. "meta-models"), a recent discussion of methods is here:
https://onlinelibrary.wiley.com/doi/full/10.1111/itor.12292
Also called "Surrogate model-based optimisation" (SMBO)
Survey of methods: https://martinzaefferer.de/wp-content/uploads/2016/10/survey-1.pdf
11
For models being deployed in the "real world" (something that, as an academic, I have heard rumors about), one element of validation is to describe both the objective criteria and the constraints to the decision makers (in their language, not in algebraic terms) and see if they agree with you. Another is to run the model on historical inputs and see if the ...
10
This link, provide 3 different logistic related games, TopShoe, JIT Flight and MyDC.
TopShoe: introduces the players to the logistical processes and the tactical decisions that must be made.
JIT Flight: is a multi-round bussiness game in which students optimize the process according to the JIT philosophy (minimum stocks, zero defects, minimum product ...
10
I agree with @QianZhang's answer (nice theoretical properties, easy to implement), and I would add that there is some theoretical justification too. If demands come from customer arrivals, then Poisson is a reasonable demand distribution since customer arrivals are well modeled by Poisson (often). And if the mean is large enough, then normal is a good ...
9
In my understanding, using normal/Poisson distribution for customer demand is mainly for two reasons.
These distributions have nice properties for theoretical analysis in supply chain models
These distributions are easier to implement or already been implemented for computational concerns
9
Let $a = \hat a$ and $b = \hat b$.
Denote $r = \frac{\hat b - \hat m}{\hat m - \hat a}$.
Then choose the shape parameters to be
$\alpha_1 = \frac{4 + 3r + r^2}{1 + r^2}$
and
$\alpha_2 = \frac{1 + 3r + 4r^2}{1 + r^2}$.
This will produce a Generalized Beta distribution with mode $m = \hat m$ and a variance $\text{Var}(X) = \left(\frac{\hat b - \hat a}{...
9
You have fallen victim to the renewal paradox, a.k.a. inspection paradox, a.k.a. length-biased sampling.
$F_{\Delta}$ is the distribution of service time for the kth customer, but it is NOT the distribution of service time for the customer being served at a preselected time $T$. The very manner of selecting the customer based on observing at a preselected ...
8
I don't know if a closed form solution is achievable. Assuming you can quantify how the robot selects its next direction when it hits a boundary (uniform over the entire circle, uniform over directions not within some angle of its last direction, some nonuniform distribution, ...), you could fairly easily build a simulation model (starting with an empty room ...
7
One simple game for educational purposes that doesn't use simulation is the "Slick Oil Distribution Game".
Disclaimer: I work at Opex Analytics!
7
Kleijnen (1995) analyses various methods of model verification and validation (V&V). Quoting the abstract (with slight changes to formatting):
For verification it discusses
general good programming practice (such as modular programming),
checking intermediate simulation outputs through tracing and statistical testing per module,
statistical testing ...
7
Not directly answering your question of how to code it manually but for discrete simulation of queues in R I would strongly recommend the simmer package. The minimal code for your example would look like this (adapted from the tutorial).
library(simmer)
library(simmer.plot)
lambda <- 2
queue <- trajectory() %>%
seize("server", amount=1) %>%
...
6
You might want to google "response surface methodology" and "simulation" together. There is a ton of literature on this, a mixture of how do to response surface modeling (and how to use it to optimize system parameters) and examples of its application.
6
If you want to find the number of workers that you need for the job to be done while the hiring cost and available hours differ from full-time work to part-time work and on the other hand you want to minimize the cost, that is exactly an optimization problem with the following specifications:
Objective function: will be the summation of all hiring costs. We ...
6
I'm in general agreement with Larry's answer, but with one qualification. If you are generating random demand quantities from the sample CDF for a year, your demands will not conform to any trends or seasonal patterns (or even just short-term autocorrelation) in the historical data. If you then generate forecasts from the randomly sampled observations, the ...
6
Unfortunately, much of the performance analysis and transient approximations for time-varying systems with non-Markovian (non-exponential) properties are not easily obtained in book form (see note at bottom).
This answer lists some books that don't require measure theory.
Some Queueing & Renewal theory books: (non-measure theoretic)
Probability, ...
6
I really like the paper by Coffin and Saltzman (INFORMS JOC, 2000), which argues that we should be using much more rigorous statistical tests when we compare the performance of one algorithm/heuristic against another.
This is not exactly about validation (which I interpret as "checking correctness") but rather about comparison, so feel free to tell me if ...
6
The company GameLab offers these types of games. There you will find a few more "serious games".
5
You could set your class the challenge of manually planning a set of delivery routes and seeing if they can find better routes (less vehicles used, less travel distance/time) than an optimisation algorithm can do?
We have an open source/free VRP solver desktop app called ODL Studio that would let you do this - see tutorial video here.
We also have a p-...
5
Why not build the forecasting directly into your simulation? So, in each period $t$, you generate a forecast $y_t$ using whatever method you want (moving average, exponential smoothing, etc.), and choose an order quantity based on the forecast and the current estimate of the standard deviation of the forecast error. Then generate the random demand, calculate ...
5
Although this question is subjective (as noted in the paper below), I believe that there is enough research done on this topic to make it more answerable.
Dias et al. (2016)
The paper is very much like a systematic review. However, the paper ranks commercial discrete-event simulation software, rather than the open-source SimPy your have mentioned. Results ...
4
I enjoyed Performance Modeling and Design of Computer Systems: Queueing
Theory in Action (Amazon link) by Mor Harchol-Balter, which sounds
like it fits your bill pretty well. I have it on my desk.
ISBN-13: 978-1107027503
4
I have used Stochastic Modeling: Analysis and Simulation by Barry Nelson and found it to be a pretty gentle introduction. It covers stochastic processes, queuing, and simulation.
4
Someone recently pointed me to a journal article about the movement of shoppers in a supermarket. A point of interest is that markets apparently use RFID technology to track shopper movements. If you could get a market (or market chain) to provide you with anonymized movement data and maybe store layouts, you might be able to have the students simulate ...
4
Question: Why do we normally assume normal distribution/Poisson distribution for customer demand in a supply chain?
Answer :
Based on my experience in the industry, I have seen that generally, business users use simple thumb rules-based methods for safety stock or inventory models. The next level of sophistication for these users is usage of Normal or ...
4
Note: This answer is intended to show what I have learned from the valuable answer provided by @Mark L.Stone. His post answered my question of why the simulation is biased. Hence, this post provides only additional insight. I chose to post it as an answer and not an edit due to the original question already being lengthy.
What has been learned comes from ...
3
Theoretically it is quite naturally convincing to assume that demand time points are independent. In other words knowing that an item was bought in t1 does not give one any good clue in understanding next sales time t2. It is like process renews/regenerate itself after each event and hence time between events are independent of each other. This particular ...
3
Many real-world datasets can be found using the google search engine dedicated to the dataset search in this web address. For example, you can use the datasets about the airports instead of restaurants, or you can search for 'Simulation Data' to find interesting datasets ready to be used in a simulation project.
3
The first step is to ascertain "reasonable" configurations of the facility being modeled. Configuration includes physical characteristics (types of machines, number of machines of each type, typical availability v. down time, etc.) as well as information about the demand patterns (types of jobs, arrival rates of jobs, typical due dates, etc.). For ...
Only top voted, non community-wiki answers of a minimum length are eligible