Translate standard weighted least square regression to quadratic programming

Sorry if this is really easy for you gurus. I'm trying to derive the reformulation of a weighted least square regression to a quadratic programming form. I understand there is a closed form solution under some assumptions. Just to clarify, it's not a homework question, it's a reduced form of a bigger problem faced in work. Tried to google but didn't find detailed steps.

More specifically, the weighted least square regression is $$min_{f} ||W(Xf-r)||^2$$ and the QP form is just the standard form

I've made some derivation as shown at bottom. I think I got $$P$$ but I'm not sure about $$q$$ due to the $$f^T$$ in the second term. Can you suggest if my $$P$$ is right and also how to get $$q$$?

Yes, your matrix $$P$$ is right. Note that $$f^{\top} X^{\top} W^{\top} W r$$ is scalar and the transpose of a scalar is just the scalar itself, i.e.

$$f^{\top} X^{\top} W^{\top} W r = (f^{\top} X^{\top} W^{\top} W r)^{\top} = (X^{\top} W^{\top} W r)^{\top} f = r^{\top} W^{\top} W X f$$

and thus

$$q^{\top} f = -f^{\top} X^{\top} W^{\top} W r - r^{\top} W^{\top} W X f = -2r^{\top} W^{\top} W X f.$$

So we have $$q = -2(r^{\top} W^{\top} W X)^{\top} = -2X^{\top}W^{\top}W r$$.

• Ah, thank you sir! Forgot it's scalar. Thank you so much!
– inf
Jan 28 at 19:23
• BTW, I guess the last line is $q^T$ instead of $q$. So, $q = -2 X^TW^TWr$. But got your point. Thank you so much again!
– inf
Jan 28 at 19:37
• Yes, you're right. Good catch!
– joni
Jan 28 at 20:29