# Queuing Theoretic Model with Memory

Consider a telephone company which receives call request at some arrival rate and serves each request with some service rate. This can be modeled using a Poisson Process. However I wish to model the behaviour of individual calls and their individual service times, that is, the model should be able to tell whether the system has received a call request from user $$1$$, and if it indeed has, whether the request has been catered to with the requisite service rate. Note that the system is probabilistic. What would be an appropriate model to use to represent such a system? If I use a Markov Decision Process, how to characterize the arrival rates and the service rates?

What I have tried is the above Markov Process. I have kept states which keep track of call request from each user. Here, I have modeled for $$3$$ users. Once a call from a user is completed, it can move to any of the other states where a call from the other user is in progress or go back to the no call state. In addition, there would be states to keep track of calls from multiple users at the same time. I have not shown that for brevity.

In addition to this, what I would like to model is calls from user $$n$$ is arriving at rate $$\lambda_{n}$$ and its service rate is say $$s_{n}$$. Can this behavior be captured with the Markov Process? Or would a Birth-Death process be a better model of this?

• Can you provide some more information what you have tried, and what did/didn't work? That will make it easier for folks here to give you some help. – LarrySnyder610 Nov 13 '19 at 13:39
• Also, I think you mean Markov process, not Markov decision process. The two are different. Can you confirm? – LarrySnyder610 Nov 13 '19 at 13:40
• MDP will give you a policy which maximize the gains or minimizes the cost, not the type of customers in the system. If you give more details about your system, understanding of that will be easier. If you had more than one action to take for each service request in each epoch that would be a MDP. – Oguz Toragay Nov 13 '19 at 14:54
• @LarrySnyder610 I have added my attempted formulation. I thought of MDP since that would allow me to add non determinism on actions later for more complex formulations. – ephemeral Nov 14 '19 at 10:39
• @OguzToragay Can't an MDP capture the type of customers by representing the information via states, like I have done above? – ephemeral Nov 14 '19 at 10:41

Since you are assuming infinite capacity, this sounds like an $$M/M/\infty$$ queueing system.
• Actually, I should retract my answer; an $M/M/\infty$ model assumes a single arrival rate and a single service rate. – prubin Nov 24 '19 at 15:17