How to run MOSEK solver in CVXOPT

Question

I have written a small code to do a simple min variance optimisation using CVXOPT, you can see the whole code below

By using solvers.qp(P, q, G, h, A, b) in CVXOPT the code runs fine and it finds a solution

solvers.qp(P, q, G, h, A, b)

I wanted to try a different solver too, hence I used MOSEK by solving the same problem with the following parameters

solvers.qp(P, q, G, h, A, b, solver='mosek') 

When using solver='mosek' the code cannot run and it is giving me the following error

MOSEK error 1295: The quadratic coefficient matrix in the objective is not positive semidefinite as expected for a minimization problem

Can anyone explain why I got this error (have I coded something in the wrong way?) and if there is a workaround to solve the issue I am facing with MOSEK?

import numpy as np
import cvxopt as opt
import mosek
from cvxopt import matrix, solvers

def optimize_portfolio(n, Var_Cov):
        P = opt.matrix (Var_Cov)
        q = opt.matrix(np.matrix(np.zeros((n, 1))))
        G = opt.matrix(np.array(-np.identity(n)))
        h = opt.matrix(np.zeros((n,1)))             
        A = opt.matrix(1.0, (1,n))
        b = opt.matrix(1.0)

        # Finding a solution

        sol = solvers.qp(P, q, G, h, A, b, solver='mosek')
        return sol

### Parameters setup

Var_Cov = np.loadtxt('C:\VAR_COV.txt')
n = len (Var_Cov)

### solve
solution = optimize_portfolio(n, Var_Cov)

# Save Results
Port_Opt = np.matrix(solution['x'])

Answer 1

As determined in the comment exchange to the question, because the matrix P has minimum eigenvalue which is negative, it is not positive semidefinite, and therefore it is a non-convex problem. Therefore, Mosek won't accept it, and as you have seen, provided an error message explaining why.

Apparently, despite billing itself as a "software package for convex optimization", CVXOPT is not checking whether the submitted Quadratic Programming problem (QP) is convex, and passes non-convex QP to the specified solver. In the case of Mosek, the solver rejected it as being non-convex. I don't know what type of solution (e.g., local optimum?) solvers.qp provides when provided a non-convex problem. I wouldn't assume it is any kind of valid solution, and I suggest you carefully examine all solvers.qp solver output to see what it says, as well as examining the solution returned.

https://cvxopt.org/userguide/coneprog.html?highlight=solvers%20qp#quadratic-programming does not specifically address whether convexity is required for use of solvers.qp. It does mention Mosek as a solver option, but does not explicitly state, as it should, that Mosek requires the matrix to be positive semidefinite.

If you want to optimize non-convex QPs, there are a variety of options to solve to either local optimality, or to attempt to solve to global optimality. CPLEX, Gurobi 9.x, BARON, and some other global solvers can attempt to solve non-convex QPs to global optimality. CPLEX, MATLAB Optimization Toolbox's QUADPROG, and some other (but by no means all, as you have seen) QP solvers, as well as general non-convex nonlinear local solvers can solve for local minimum of non-convex QPs.

Edit: Looking at your code, the matrix P is labeled Var_Cov, presumably meaning Variance Covariance for your portfolio assets. I.e., a covariance matrix, which should ("must") be positive semidefinite. Given that it is not, I suggest you review the procedure used to generate the covariance matrix. it is either flat out wrong, is based on missing or inconsistent data, or has severe numerical difficulties. That might be the subject for another post at https://stats.stackexchange.com/ or https://scicomp.stackexchange.com/ , in which case please post a link here to any such new post. In such case, what you want to do is fix the procedure for generating the covariance matrix, or at least "repair" it to be positive semidefinite; and then use a convex QP solver, such as Mosek.