Can Deep RL be used to find optimal division point in an application?

Question

I need to optimise the end-to-end latency for a multi-service application while distributing it on multiple devices. The application is a series of services interconnected to each other. The goal is to use some online optimisation algorithm to find the optimal cut point in a series of services connected of multi-service application, after the cut point the half part of the collection of services will run on one device and the intermediate result will be sent to another device considering bandwidth and resource capacity of the device, taking into account the modification of the initial assignment to deal with events such as load changes. The input to the algorithm is the latency to execute the services on two different devices, the bandwidth to send the intermediate the result, and application latency to optimise against. The output is the optimal point to divide the application into two different parts.

From reading different research papers and the majority of the papers have used deep Reinforcement learning (DRL), I could understand but I am unable to take a start point. What is the best way to solve the above problem other than using a linear search algorithm that would return the lowest end-to-end latency from the search space? Using linear search only returns the lowest end-to-end latency from the search space while not considering resource constraint or bandwidth constraint. Any help is highly appreciated.

Answer 1

This is a clustering problem, i.e., you want split your services into two clusters, each of which will run on a different device.

The way you would do this with RL is to use the goals an objective that includes weighted constraints.

This of course assumes you have the right data/setup to train the algorithm.

After enough training, your system will be able to predict the right clustering such that your constraints are obeyed, and your other goals are relatively minimal (assuming you're minimising).

It's important to be aware of the fact that this will not work properly 100% of the time, i.e., some of the time the predicted result is likely to violate your constraints. How often this happens and by how much, depends on how well you train the RL model.