9
$\begingroup$

The question I am about to ask is not a technical one but rather based on following a correct approach, which I am sure would be helpful to many.

I am currently working on a project which involves data driven optimization for a gas compressor network, where we read in plant's data for process variables such as pressure, temperature, flow etc. We in there, are building several statistical (regression) models that form part of our objective function and constraints. We are in the pilot phase, and our ultimate purpose is to deploy this tool real time in plant. At this moment. I am currently having with me a large data set that is reliable enough to build generalized models.

However, thinking it from the deployment perspective, we would not want to be in a position where we are doing an optimization based on data that is too old. In fact, my project adviser mentioned that I should have the facility to read in live data from the plant and update my model parameters based on the new data points.

Currently, what I can think of is to read in new data to new csv files via python and update my models. However, this requires me to manually run the code at fixed frequency (eg daily at 10:00 AM). I am wondering, if there is some more smarter way to approach this problem ? Ex. deploying my code on some cloud platform that is automatically able to stack new data to the existing database, run the code and update parameters, the latest values of which I can then retrieve to my local machine to run the optimization code. I would be keen to know if someone has worked on some similar problem before as I would like to learn more on this topic.

Thanks,

Improve this question
$\endgroup$
3
  • 3
    $\begingroup$ What is your data about? Inventory, resource consumption and like these? $\endgroup$
    – Oguz Toragay
    Commented Aug 6, 2019 at 5:55
  • 1
    $\begingroup$ My data is about several recorded process variables such as pressure, flow, temperature, composition etc of natural gas in a gas compressor network. $\endgroup$
    – chupa_kabra
    Commented Aug 6, 2019 at 6:05
  • 3
    $\begingroup$ I am old school, but the "classic" may be to run a "cron job" that updates the data and the models and periodically starts the optimization. I have deployed "solutions" like this and they have been productive for years. $\endgroup$
    – Marco Lübbecke
    Commented Aug 6, 2019 at 8:30

1 Answer 1

Reset to default
5
$\begingroup$

One of the PhD students in my office is working on a similar project that they called digitalization, in which they use raspberry pi and various sensors to collect the data from the machines in production line, send them to cloud and after extracting data-driven parameters, they use those parameters to estimate the down-time of the machine and control the inventory and plan the production.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.