I think its easiest to describe my goal first and continue with my implementation and the resulting problems!
My goal: Using Pyomo as interface and Gurobi as solver, if a variable $x_{i,t}$ falls below a certain value $b_{i,t}$ the deviation to given threshold shall be added to the objective, but only if it is below. $i$ denotes a state inside a state space representation and $t$ the current time step. So my objective should describe something like $$J=\sum_i^S\sum_t^T(\mathrm{if}\, x_{i,t}<b_{i,t}:b_{i,t}-x_{i,t}, \mathrm{else}\,0).$$ So it is comparable to a ReLU activation function
My Implementation/Problem: I tried implementing a binary variable $y_{i,t}\in\{0;1\}$ using BigM method $$b_{i,t}-x_{i,t}\le My_{i,t}$$ and describing $$J=\sum_i^S\sum_t^Ty_{i,t}\cdot(b_{i,t} - x_{i,j}).$$ The problem is if $x_{i,t}$ is above the threshold the optimizer also sets $y_{i,t}=1$ since this allows for negative objective terms, which minimizes the objective but is not what I intended.
Is there any way on how to implement this? In addition, later I want to implement the same just with an upper threshold. Both of these thresholds should be usable independently of each other (only the upper or only the lower), but also in combination with each other, creating a band where no objective is added (this is probably just an summation of both onbjectives).
Maybe as an additional note, I should mention that $x_{i,t+1}$ is calculated using Pyomo DAE based on its differential $\dot{x}_{i,t}$, the systems input $u_{j,t}$ and the internal integration scheme of Pyomo DAE (so basically a state stace). The first value $x_{i,0}$ is fixed via an equality constraint so the optimizer can only really manipulate $u$ since the differential at a given time step $\dot{x}_{i,t}$ is calculated via the current values of $x_{i,t}$ and $u_{j,t}$. Both $x_{i,t}$ and $u_{j,t}$ are bounded but not necessarily $\ge 0$.