Questions tagged [dynamic-programming]
For questions about dynamic programming, a mathematical optimization technique where the optimal solution to the problem is found by breaking it down to simpler sub-problems and solved recursively.
44
questions
1
vote
0
answers
48
views
Subsequence Decision Optimization with Optional and Stopping Times
I have a problem that I haven't encountered before and would like to know if there is any literature on the problem - or maybe you can help me simplify my problem if you think I'm doing something ...
- 11
1
vote
1
answer
95
views
How is this function piece-wise linear?
I encountered this lemma in a research paper related to End-to-End inventory management model.
Please note that $d_{[t_1,t_2]} = \sum_{t=t_1}^{t_2} d_i$, where $d_t$ denotes demand at time instance t. ...
- 113
2
votes
2
answers
141
views
Can the following problem be solved recursively?
Consider the following problem
\begin{equation}
\begin{aligned}
\min_{x,y,z}
\quad & \sum_{i=0}^1 \sum_{j=0}^1 \sum_{k=0}^1 a_{ijk} \cdot f_{ijk}(x,y,z), \\
\textrm{s.t.} \quad &...
- 639
0
votes
1
answer
59
views
Successive approximation in negative dynamic programming
I am studying Stochastic Dynamic Programming using Sheldon Ross's book, "Introduction to Stochastic Dynamic Programming." In the book, Ross defines a dynamic programming algorithm to ...
1
vote
1
answer
129
views
Can dynamic programming find globally optimal solutions for scheduling problems
I want to know if dynamic programming can generally find globaly optimal solutions for scheduling problems? I think this might be difficult as dynamic makes one at a time decisions without calculating ...
- 1,682
2
votes
1
answer
58
views
Can I use continuous probability distributions when creating an SDDP.jl model?
I am using SDDP.jl for my research project and want to use continuous distribution, can I do so?
0
votes
1
answer
45
views
How can I write the output stream of SDDP.jl into an excel file?
I am using SDDP.jl for my research project in which I am developing a state-of-the-art actor critic algorithm which I am going to benchmark with SDDP but for it I need to plot graphs which require ...
1
vote
2
answers
130
views
Matching algorithm in an order batching problem
There is an order batching problem.
Given a set of orders, they need to be split into several batches, with a maximum order number of M per batch.
Each order needs to be picked from multiple storage ...
- 115
1
vote
1
answer
157
views
Is stochastic dual dynamic programming (SDDP) a deterministic solution algorithm or does it have a stochastic component to it?
I am currently working on a paper in which I am statistically comparing dynamic optimization algorithms like SDDP, Actor-Critics etc.
In this regard, should I be running SDDP algorithm for my ...
1
vote
0
answers
207
views
Wagner's-Whitin Algorithm
I'm having trouble in solving this problem dealing with the Wagner's-Whitin Algorithm, because the holding and ordering costs are not given, we only have the optimal costs from the beginning to each ...
- 21
1
vote
1
answer
81
views
Solving stochastic dynamic problem by space state MATLAB program
I have a maximization problem as shown in the picture below. The output has a common shock z that follows a two-state Markov chain with a transition matrix Π. Does anyone know how I would go about ...
- 11
1
vote
1
answer
114
views
Single item unconstrained lot-sizing with multiple suppliers and minimum order quantities
Variation of the traditional lot-sizing problem - with some additional complexities:
multiple suppliers (S1, S2, S3), with different procurement lead-time
Suppliers have to be allocated based on a ...
- 119
0
votes
2
answers
107
views
Bellman Equation for nonlinear model
Consider the following model:
\begin{align*}
max \quad Z &= 19x_1 - 3x_1^2 + 5x_2^2 - x_2^4 + 4x_3 \\
& s.t. \quad x_1 + 3x_2 + 3x_3 \leq 7 \\
& \quad \quad \quad x_1,x_2,x_3 \geq 0
\end{...
2
votes
0
answers
100
views
Optimal spending of cash problem
I have often wondered whether there is an optimal way to spend cash denominations. For example: Suppose that Bob needs to pay Jill \$5, Jane \$10, Billy \$3.50 and John \$45.75. Furthermore suppose ...
3
votes
0
answers
50
views
Control variables and cofounding effects in stochastic programming/,model predictive control/reinforcement learning
How can we be sure that confounding variables/control variables don’t pickup the effect our decisions w.r.t decision variables had on the actual control variable?
Since the term control variable ...
0
votes
1
answer
193
views
How to model history-dependent dynamic program?
Suppose there is a dynamic program that the state of the problem grows over time (more info is added to the state of the problem over time) and at each time, we need all historical data, full history, ...
- 2,160
5
votes
1
answer
242
views
Dynamic program for knapsack in $O(W)$ space?
A familiar dynamic programming algorithm for the binary knapsack problem
$$
\begin{align}
\text{maximize}\quad & v \cdot x \\
\text{subject to} \quad & w \cdot x \leq W \\
\quad&x_i \text{ ...
- 584
3
votes
0
answers
32
views
How to establish the optimal value functions and optimal control policy for a controlled random walk problem?
Question: How to establish an explicit characterization of both the optimal value functions and the optimal control policy for a controlled random walk?
Background:
Assume our system is a perfectly-...
- 141
5
votes
0
answers
77
views
Linear programming approach to dynamic programming - an initial pair of state-decisions
I aim to solve the following Bellman equation:
\begin{equation}
v(\vec{s}) = \min_{\vec{x} \in \Xi_{\vec{s}}} \big\{c(\vec{s}, \vec{x}) + \lambda \times \sum_{\vec{s}^{'}\in S} p(\vec{s}^{'} | \...
- 615
6
votes
2
answers
609
views
What kind of data structure should be used to store labels when implementing a labeling algorithm?
The shortest path problem with resource constraints is a common subproblem when doing column generation. It is often solved with a labeling algorithm. The procedure is very well explained here and ...
- 668
7
votes
1
answer
2k
views
Efficiency of Forward vs. Backward Recursion in Dynamic Programming
I have a question that has been bothering me for a while:
In our OR-introduction course, we introduce the concept of Dynamic Programming via backward recursion: Working backwards from a final state (...
- 121
1
vote
0
answers
144
views
Simple inventory control with stochastic demand
There is a factory that produces one unit of stock uniformly so that $q$ units of stock are produced during a day. The warehouse near a factory has the maximal capacity of $q$ items, i.e. a daily ...
- 11
3
votes
1
answer
153
views
Why are these constraint equations equal?
If we consider the dynamic lot sizing problem with:
$d_i$ as the demand per period $i$ and denote $\sum_{i=1}^t d_i$ being the total demand up to period $t$, where $t$ can take values $1, \dots, T$ ...
- 1,053
1
vote
0
answers
66
views
How to check if the state of a dynamic program is Markovian or not?
Based on Markov chain context, we say a stochastic process is Markovian if the state at time $t+1$, $S_{t+1}$ just depends on the state in the previous step, that is, $\Pr\left( S_{t+1}|S_1, S_2, \...
- 2,160
1
vote
0
answers
87
views
Bellman equation for darts that minimizes the number of turns
I want to formulate a game of darts as a dynamic program again. This question is closely related to my previous post where I wanted to minimize the number of throws to reach checkout.
Now the goal is ...
- 113
3
votes
1
answer
312
views
How to solve Stochastic Dynamic Program with huge state space?
I am modelling a stochastic dynamic program but because I need to store all information related to former sales, the state of the dynamic program increases and potentially it can growth so much which ...
- 2,160
5
votes
0
answers
62
views
Convergence of an approximate DP for a stochastic shortest path problem
I'm new to the field of sequential decision making. I got intrigued by a stochastic shortest path problem, described in Chapter 5 of this book by W. Powell.
Consider the following stochastic shortest ...
- 3,481
5
votes
1
answer
227
views
Circular reference in states of the Bellman equation
I want to formulate a game of darts as a dynamic program. The goal is to minimize the number of darts thrown while reaching checkout. A dart player has a score s. If his score is s = ...
- 113
2
votes
0
answers
163
views
Why is exact TDTSP solving much harder to do than TSP solving?
I was thinking about exact methods for solving the Time Dependent TSP (TDTSP). Clearly, it is at least as complex as TSP because it extends TSP, but why is it for exact approaches that difficult to ...
- 659
3
votes
1
answer
253
views
Is the Dynamic programming in Operations Research book the same dynamic programming in software industry?
Mostly when I search dynamic programming in google I get : dynamic programming (DP) in python, in C++, in java from web pages like geekforgeeks, litecode, codechef, and so on.
But in the operation ...
- 33
1
vote
1
answer
209
views
Dynamic Programming - Formulating recurrence relation
We wish to apply dynamic programming techniques to find the optimal betting strategy for a pool to wager on the outcome of the NCAA men's basketball tournament
64 teams compete in a single ...
- 271
2
votes
0
answers
84
views
Dynamic Programming problem of affecting equipment with budget constraint
I have a problem that I must formulate as a DP problem and solve.
A hospital is split up into 4 sections, each section has 1 or 2 or 3 backup generators. We have to maximize the likelihood that no ...
- 21
6
votes
1
answer
379
views
Multi-period linear dynamic programming with differing in-period dependencies and changes
I’m not sure if I’m wording this right but in a nutshell, my problem is:
I’m modelling potential actions a boat owner can do to their boat. Let’s say he wants to know over the 50 year lifespan of the ...
- 153
1
vote
2
answers
231
views
Formulate a problem as Mixed Linear Programming problem
I need to formulate the following problem as a Mixed Integer Linear Programming problem
A farmer needs to establish a 17-year business plan where he will decide when to sell or buy a new truck. The ...
- 13
7
votes
2
answers
3k
views
The general meaning of "constraint relaxation" in the context of the Shortest Path Problem
I've read that in the context of the Shortest Path Problem, the use of the term "relaxation" ("relaxing edges")
[...][the use of the term "relaxation"] is historical. The outcome of a relaxation ...
- 169
8
votes
1
answer
634
views
Dynamic programming example
I am going to buy a family car at the beginning of the New Year. I am going to stay in the UK
for the next 4 years. I am considering the possibility of being a customer of company A which
sells ...
- 107
5
votes
1
answer
484
views
Solving a variant of multiple knapsack problem/ generalized assignment problem
Consider $m$ knapsack and $n$ items. With each knapsack $j$ associated a capacity $c(j)$ and with each item $i$ associated a profit $p(i,j)$ (that depends on the knapsack, so it's not exactly the ...
- 973
20
votes
3
answers
5k
views
Solving a knapsack problem with a lot of items
I am wondering what are the fastest ways(faster than classical dynamic programming) to solve the knapsack problem (to optimality) with $n$ items when $n$ is nearly equal to $10000$ ?
Apart from ...
- 973
6
votes
0
answers
173
views
Minimization of car cost during 4 years problem
I am going to buy a family car at the beginning of the New Year. I am going to stay in the UK for the next 4 years. I am considering the possibility of being a customer of company A which sells BMW ...
- 107
-2
votes
1
answer
360
views
dynamic programming shortest path example
Could someone please show how he uses dynamic programming to solve for minimum cost of getting from 1 to 6? Is it recommended to use dynamic programming to solve this?
Edit: I know that dynamic ...
- 107
6
votes
1
answer
245
views
Dynamic programming problem with machines
A company will be using a new technology for 5 years. For this purpose a specialized machine is required. The company currently has one, which will be 2-year old at the beginning of next year. The ...
- 107
5
votes
1
answer
2k
views
Minimisation of shelving cost problem
A library must build shelving to shelve $200$ 4-inch high books, $100$ 8-inch high books, and $80$ 12-inch high books. Each book is 0.5-inch thick.
The library has several ways to store books. ...
- 107
10
votes
2
answers
194
views
Optimal set order to maximize stochastic reward
You have a ticket allowing you to visit up to $n$ of $M$ carnival booths offering games of chance. At each booth you have probability $p_{i}$ of winning a reward with average value $r_{i}$. Each booth ...
- 103
14
votes
2
answers
250
views
Suggestion of some courses in sequential decision making
I am studying about sequential decision making and I am willing to know if there is any course which is recorded and is publically available covering topics in dynamic programming (DP), reinforcement ...
- 2,160