PuLp is ignoring all of the constraints given to it

Question

I am trying to solve a portfolio optimization problem using PuLP where given a dictionary of stock tickers and their returns for the day, returns a set of weights for each stock such that portfolio return is maximized. Both long and short positions are allowed. The weights must satisfy the following requirements

1. Full investment (the absolute value of the weights sum to 1)
2. Equity Market Neutral portfolio (long positions = short positions --> positive weights sum to 0.5 and negative weights sum to -0.5)

Here is an example list of assets (labeled assets in the code):

['META', 'GOOG', 'AAPL', 'BA', 'APO', 'FDX']

And a dictionary containing their returns (labeled returns_dict):

{'META': 0.005803442055417999, 'GOOG': -0.016066450130370023, 'AAPL': 0.0030249344157745966, 
'BA': -0.05464299504580779, 'APO': -0.02150687085217356, 'FDX': -0.01989503040490268}

I'm trying to solve this problem using the following code:

        lp_problem = pulp.LpProblem("Portfolio_Optimization", pulp.LpMaximize)
        weights = pulp.LpVariable.dicts("Weight", assets, lowBound=-1, upBound=1, cat='Continuous')

        lp_problem += pulp.lpSum([returns_dict[asset] * weights[asset] for asset in assets])
        
        # full investment constraint
        lp_problem += pulp.lpSum([(-1 if weights[asset] <= 0 else 1) * weights[asset] for asset in assets]) == 1
        #positive weights equal 0.5
        lp_problem  += pulp.lpSum([weights[asset] for asset in assets if weights[asset] >= 0]) - 0.5 <= 0.000001
        #negative weights equal - 0.5
        lp_problem  += pulp.lpSum([weights[asset] for asset in assets if weights[asset] <= 0]) + 0.5 <= 0.000001
        
        lp_problem.solve()

        # Check the status of the solution
        if pulp.LpStatus[lp_problem.status] == "Optimal":
            # Get and print the optimal portfolio weights
            optimal_weights = {asset: weights[asset].varValue for asset in assets}
            print("Optimal Portfolio Weights:")

However, PuLp is returning the following weights:

{'META': 1.0, 'GOOG': 0.0, 'AAPL': 1.0, 'BA': -1.0, 'APO': -1.0, 'FDX': -1.0}

which completely violate all of the constraints that I gave it. The absolute value of the weights don't equal 1, they equal 5. The sum of the positive weights equal 2 instead of 0.5 and the sum of the negative weights equal -3 instead of -0.5. Is there any changes that I can make to the constraints to prevent this from happening?

The reason that I'm not simply setting the stock with the max positive return to 0.5 and the stock with the max negative return to -0.5 is to stay within a 25% interday turnover constraint, which I have not yet implemented.

Thank you

Following sascha's advice, I attempted to preform comparisons on the returns instead

        #full investment
        lp_problem += pulp.lpSum([(-1 if returns_dict[asset] <= 0 else 1) * weights[asset] for asset in assets]) == 1
        #positive weights equal 0.5
        lp_problem  += pulp.lpSum([weights[asset] for asset in assets if returns_dict[asset] >= 0]) - 0.5 <= 0.000001
        #negative weights equal - 0.5
        lp_problem  += pulp.lpSum([weights[asset] for asset in assets if returns_dict[asset] <= 0]) + 0.5 <= 0.000001

but that didn't appear to fix my problem and I'm still getting bogus weight values:

{'GOOG': 0.0, 'META': -1.0, 'AAPL': -1.0, 'BA': -1.0, 'APO': -1.0, 'FDX': -1.0}

I am aware that this is a suboptimal solution and fails in certain scenarios where all the returns are negative/positive, but was hoping it would work as a proof of concept.

Answer 1

We can do this with fewer binary variables (and fewer constraints) than in the proposed PuLP model.

$$ \begin{aligned} \max & \sum_{i} \color{darkblue}r_i (\color{darkred}w_{i,long}-\color{darkred}w_{i,short}) \\ & \color{darkred}w_{i,long} \le \color{darkred}\delta_i \\ &\color{darkred}w_{i,short} \le 1-\color{darkred}\delta_i \\ &\sum_i \color{darkred}w_{i,p} = 0.5 && \forall p \in \{long,short\}\\ & \color{darkred}w_{i,p} \in [0,1] \\ & \color{darkred}\delta_{i} \in \{0,1\} \\ \end{aligned} $$

The solution would look like:

----     79 VARIABLE w.L  

            long       short

META       0.500
BA                     0.500

Not sure if it makes much of a difference in performance. The model is a bit more compact, and the difference between long and short is more explicit.

In general, I don't like to translate models line-by-line from one modeling tool to the next. I rather take a step back and translate concepts. This often leads to cleaner models.

Update: the linearized turnover constraint can look like:

$$\begin{aligned} & \color{darkred}t_i \ge \color{darkred}w_{i,long}-\color{darkred}w_{i,short} - \color{darkblue}w^0_i \\ & \color{darkred}t_i \ge -(\color{darkred}w_{i,long}-\color{darkred}w_{i,short} - \color{darkblue}w^0_i) \\ & \sum_i \color{darkred}t_i \le 0.25 \\ & \color{darkred}t_i \ge 0 \end{aligned} $$

The variable $\color{darkred}t_i$ needs to be interpreted with care. It is a bound on the turnover of asset $i$.

Answer 2

In case it helps, here's what it would look like in SAS:

data indata;
   input stock $ return;
   datalines;
META  0.005803442055417999
GOOG -0.016066450130370023
AAPL  0.0030249344157745966 
BA   -0.05464299504580779
APO  -0.02150687085217356
FDX  -0.01989503040490268
;

proc optmodel;
   set <str> STOCKS;
   num return {STOCKS};
   read data indata into STOCKS=[stock] return;

   var Weight {STOCKS} >= -1 <= 1;
   max TotalReturn = sum {i in STOCKS} return[i] * Weight[i];
   con SumPositive:
      sum {i in STOCKS} max(Weight[i],0) = 0.5;
   con SumNegative:
      sum {i in STOCKS} min(Weight[i],0) = -0.5;

   solve linearize;
   print Weight;
quit;

The resulting optimal solution is:

stock Weight 
AAPL   0.0 
APO    0.0 
BA    -0.5 
FDX    0.0 
GOOG   0.0 
META   0.5 

You could also impose a constraint as follows, but it is implied by the other two constraints:

   con SumAbs:
      sum {i in STOCKS} abs(Weight[i]) = 1;