I want to deploy my OR model (python, pyomo) as an Rest API (using Flask) that too on Azure Kubernet. All I have came across so far are ways to do the same for ML models, which can be uploaded as pickle file and used, but the same is not possible with my model.

I am looking for resources addressing OR models for the deployment but any help is appreciated, and a template will be a bonus.

  • $\begingroup$ As far as I know, some of the state-of-art solvers have a REST API that can be found on their host. For example this link. $\endgroup$
    – A.Omidi
    May 25, 2021 at 6:21
  • $\begingroup$ i dont want to access solver's api,but want to deploy mine as a service. $\endgroup$
    – vicious
    May 25, 2021 at 7:32

5 Answers 5


I have made OR models accessible through APIs on Kubernetes before. It isn't too hard, no pickling involved. In general, the steps aren't unique for an OR model API, but there might be a few things to consider (for example, an OR API might take more time than an average API to return a result).

  1. First of all, you need to decide on a data exchange format. The JSON format works really well for most applications. For example, let's assume your model is going to solve a TSP problem. Your model would expect a list of points and their coordinates and maybe a few parameters, such as a time limit or the optimality gap you are willing to accept. As a result, you would expect a JSON that contains the length of the solution and possibly the order in which the points are visited. A sample input could be as follows:

    {"points": [{"name": "Geneva", "long": 46.2044, "lat": 6.1432}, {"name": "Zurich", "long": 47.3769, "lat": 8.5417}, {"name": "Lucerne", "long": 47.0502 , "lat": 8.3093}, {"name": "Bern", "long": 46.9480, "lat": 7.4474}], "time_limit": 30}

    The expected result might be something like this:

    {"status": "optimal", "order": ["Zurich", "Bern", "Geneva", "Luzern"], "score": 598}
  2. You will now need to write a Flask app that can receive the JSON input, pass it on to your OR model and then return a result as a JSON. This app will need at least one route that you can send the data to. Here's what the basic structure could look like:

    from flask_api import FlaskAPI
    app = FlaskAPI(__name__)
    @app.route("/optimizer/", methods=['POST'])
    #@require_api_token # You can implement a wrapper method to authenticate requests to the API
    def optimizer():
        Route to start the optimizer. Expects JSON data sent as a POST request
        problem_data = request.data
        result = my_or_model(problem_data)
        if result: 
            return {'status': 'optimal', 'result': result}
        return {'status': 'infeasible'}
    def my_or_model(problem_data)
        # you can access the data (e.g. time_limit) as follows: problem_data['time_limit']
        # result = run your OR specific algorithm and return a result
        return 100
    if __name__ == "__main__":
        app.run(debug=False, port = 5000, host = "")

    Note that I left away authentication/ licensing, but this isn't too hard to add and you will find a lot of information about it if you search for Flask api authentication.

  3. Theoretically, this app is ready to be deployed, however, it is better to server Flask apps through Nginx or something similar. A very lightweight server is waitress-server, which is a python module that allows you to set up a very simple server:

    from waitress import serve
    import myflaskapp # assuming we have defined the flask app in the file myflaskapp.py
    serve(myflaskapp.app, host='', port=5000)
  4. You now have everything to run your API locally. If you run python waitress--server.py you will be able to access your API at http://localhost:5000/optimizer/ and send data to it.

  5. To get this deployed on Kubernetes, all that is left is to do is to dockerize your flask app and create a Kubernetes deployment. You will find more information on this elsewhere, but the most simple Dockerfile could look as follows:

    FROM python:3.7
    RUN mkdir /app
    WORKDIR /app/
    ADD . /app/
    RUN apt-get update
    RUN pip install flask-api
    CMD ["python", "/app/waitress-server.py"]
  6. After building the docker image (docker build . flask-image) and pushing it to a docker registry, e.g. docker hub or whatever Azure is providing, you need to create a Kubernetes deployment file. In this example I am creating a deployment that runs 4 instances of the app, but this isn't necessary:

    apiVersion: apps/v1
    kind: Deployment
      name: flask-deployment
          app: flask
      replicas: 4
            app: flask
          - name: flask
            image: path to your docker image on dockerhub
              - containerPort: 5000
            imagePullPolicy: Always
          - name: regcred
    apiVersion: v1
    kind: Service
      name: flask-service
      type: LoadBalancer
      - port: 5000
        targetPort: 5000
        name: http
        app: flask
7. That's pretty much it. You now just have to connect kubectl to Azure Kubernetes and you can create the deployment: `kubectl create -f deployment.yaml`

For further questions, I think that the Kubernetes Stackexchange would be the right place as this is really not specific to an OR model.

If your OR model is requiring more than just a couple of seconds you probably don't want to let the requester wait and risk the connection to time out. In this case it might be best if you can send back the result later (either to an API that is receiving the result or maybe as an email, similar to what happens when you solve a model on NEOS server.
  • $\begingroup$ Thanks a lot, I figured out that for an OR API I have to be my own, and steps u mentioned are ones I was tracing already(one way or another). Kubernetes bit is the one where I was in muddy waters, but this clears a lot of it. Appreciate it. $\endgroup$
    – vicious
    May 26, 2021 at 13:16
  • 1
    $\begingroup$ @vicious Don't hesitate to reach out if you have questions or get stuck. It sounds simple, but there is a lot of potential for mistakes. I have run into a lot of challenges myself, so I think I am qualified to help. $\endgroup$ May 26, 2021 at 13:23

I think in your case you have to separate between the data and the model. The model itself is part of the service in some way. Either you hard code it into the service or you have some sort of abstract description of the model (for example in some modeling language that the solver can read) so that you can upload new versions of the model to the service.

What the user then uploads is the data with which the model gets instantiated. This gives a problem instance that the service then solves and eventually returns a solution for.

So in your scheduling example, the user would only provide the jobs/machines/... and the service would construct a model from this based on the abstract problem representation.

  • $\begingroup$ That's exactly the way i have planned to do it. $\endgroup$
    – vicious
    May 26, 2021 at 13:55

I am unsure what exactly you want to pickle. Is the user posting data to your endpoint (the flask service)? In that case you probably need to re-run the algorithm.

In machine learning the model is usually pickled as you have already trained it. Thus the user in that case just need to provide the input for the model and it will immediately make a prediction on the input data.

As I see it you either want the user to call your model with new input, thus it needs to re-optimize. I don't think pickling anything here will help you particularly. If you just want the user to be able to access the model, then the question is why?

  • $\begingroup$ first of all i am not inclined to use pickle and yes i do need to re-optimize the way you described it and hence pickling is not of any help here.my only requirement is to provide an output(a schedule) by accessing my model(and solving it with new input every time). $\endgroup$
    – vicious
    May 25, 2021 at 7:40
  • $\begingroup$ Have you separated the model and the data so that the model can be called using different input data? In that case I am not really sure where you are in the process. $\endgroup$ May 25, 2021 at 12:43

As you have noticed there are ready-made services available for ML but not for OR. You will have to roll your own.

You can get a feel for the architecture requirements by looking at the code and REST APIs of open-source projects, e.g. graphhopper, vroom. It takes time to understand source code of others, and software architectures even more so. But the blue-prints are out there and readily available.

You will find that common components for a minimal software-as-a-service architecture include:

  • Data exchange format (probably JSON): separated between input/request file (input data and input parameters of your optimization problem) and output/response (solution of the optimization algorithm)
  • Server: backend that handles incoming API requests (mostly POST or PUT request), calls your optimization algorithm, and sends the solution back to the client
  • Client: executable programm code on the client side (downloadable programm, or for example a python package) that builds the data exchange format internally and sends it to the server. This is an optional convenience feature for REST services because the clients can also build the data exchange file themselves.
  • Licensing mechanism: checks against a database if a request has a valid API key
  • Database: holds a record of user data and valid licenses for the licensing mechanism
  • Website: user registration and API key purchase option for creating database entries
  • Documentation: can be part of the website or hosted by dedicated third party providers (there are plenty good options if your project is open-source)

Obviously licensing and database is optional if your service is free of charge and small-scale. Also, related.

  • $\begingroup$ That's true about rolling by myself, all the points u mentioned are actually taken care of, additionally there is queueing involved as well.thanks alot. Appreciate it. $\endgroup$
    – vicious
    May 26, 2021 at 13:17

Perhaps, you will find some software we built https://fuinn.github.io/mos-docs/ useful. Provides a REST API for interacting with optimization models.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.