Almost all convex optimization problems can be formulated as a conic optimization problem using only the cone types we can handle in practice. See the Mosek modeling cookbook for details. This often leads to the best solution time and you do not have to mess with derivatives. For instance Mosek can solve such conic optimization problems.
The upcoming Mosek version 10 (to be released spring 2022) will have a command called
that will take an array of optimization problems. Those optimization problems will be solved in parallel using a pool of threads. Moreover, the thread pool can also be used to parallelize the solution of each problem. This implies a good load balancing (=efficient usage of the threads) even if the optimization problems are of vastly different size.
It is not that hard to solve multiple optimization problems in parallel as suggested in another reply. However, you easy use too many threads if you try to solve each optimization problem in parallel too. This will most likely lead to a performance degradation.
PS. I work for Mosek but does a lot of our paralleization.