# Questions tagged [stochastic-process]

For questions related to processes that change randomly over time. Examples include Markov processes, random walks, and renewal processes.

19 questions
Filter by
Sorted by
Tagged with
73 views

### Resource allocation problem - RL or stochastic optimization?

I am currently working on a resource allocation problem and I am uncertain about which field of stochastic optimization and reinforcement learning encompasses this particular problem. The objective is ...
1 vote
23 views

### Understanding queueing theory used to model the vehicle routing problem

I am trying to understand some journal articles on the vehicle routing problem, specifically Vehicle routing with dynamic travel times: A queueing approach and An M/M/c queue model for vehicle routing ...
• 165
38 views

### m/m/1/k=1 variance

Is there a nice formula for the finite time variance of the m/m/1/k=1 queue in steady state? (Exponential arrivals $\lambda$, exponential service time $\mu$, 1 server and no queue, so if a job arrives ...
• 159
239 views

### Optimization for Stochastic Simulations

This questions stems from my original question here. I have a system, where each 'run' is a simulation of agents switching between tasks (or states) as in a Markov Chain. I would like to find ways to ...
2k views

### Are optimization results stochastic?

I'm asking myself for some time if the results of optimization are random. As search for stochastic and optimization gives results about stochastic optimization but does not help with the ...
• 481
64 views

### How to verify the correctness of forecast?

I would like to forecast the car rental (count time series). Given hourly integer valued car rentals for a month's period from 24th september to 24th October. I need to forecast car rental demand ...
1 vote
229 views

### The impact of utilization rate of a queueing system on its average queue time

For an $M/M/c$ queue where arrivals follow a Poisson distribution with rate $\lambda$ and are iid, service follows a Poisson process with rate $\mu$ and there are $c$ parallel servers, we can estimate ...
29 views

### How to establish the optimal value functions and optimal control policy for a controlled random walk problem?

Question: How to establish an explicit characterization of both the optimal value functions and the optimal control policy for a controlled random walk? Background: Assume our system is a perfectly-...
• 141
1 vote
138 views

### Simple inventory control with stochastic demand

There is a factory that produces one unit of stock uniformly so that $q$ units of stock are produced during a day. The warehouse near a factory has the maximal capacity of $q$ items, i.e. a daily ...
• 11
27 views

### What are some references for Fluid and Diffusion models in Queuing?

In queuing theory and stochastic process, there are many papers that have applied fluid and diffusion models to approximate the original models. However, the concepts and basics of these models are ...
• 2,150
54 views

### What kind of algorithm is this: water foraging

I am pondering if a machine learning problem might be better formulated as a reinforcement or control problem. I have no experience with the latter, so bear with me. Let's say I am organising a tour ...
• 249
62 views

### Mixing time exponent above threshold temperature for Glauber dynamics or annealing

It is well-known that the Glauber dynamics will converge in polynomial time to the Gibbs distribution for, say, the Ising model on a d-regular graph at high enough temperatures $T>T_c$. There are ...
79 views

### Best method to optimize the blending of different types of coal to ensure all quality parameters are met at the lowest possible price?

I am looking to optimize the blending of different types of coal for the coke making process of a steel plant. I want to take into account the statistical variation of each coal’s qualities, so for ...
77 views

### Optimize probability parameter in an optimal control problem

We have a game with infinite but countable rounds. We have one machine, that may either break down, or continue operating. For each round the machine operates, it gives cost $-1$ (so profit of $1$). ...
• 3,980
139 views

### Can demand be modeled by a limited amount of probability distributions?

In stochastic inventory control, it is very important to model the demand appropriate to set e.g. order levels. Demand can be fit to a certain probability distribution via many means (e.g. a goodness ...
• 1,043
135 views

### Model or State Uncertainty in Queueing Model due to uncertain arrival rate

Introduction I am currently modelling a scenario where two queues need to be served by a single server in a non preemptive discipline. I am quite sorted on generating the optimal policy via Value or ...
• 463
5k views

### Good textbook for queueing theory and performance modeling

Can someone recommend a good self-study textbook for queueing theory and performance modeling? My interest is in applying this to understanding the behavior of some real-world server networks, ...
259 views

### Stochastic VRP: Sources of uncertainty and modeling approaches

In one of the archived research topics of mine, I tried to model and solve SVRPs using Multiple Tandem Queues. I have two related questions: What are the sources for uncertainty in Stochastic VRPs? ...
• 8,622
Context: Given a counting process $\{N(t),\,t\ge 0\}$ which tracks the number of events (arrivals) by time $t$, the index of dispersion (for counts) is the variance-to-mean ratio of the cumulative ...