3
$\begingroup$

Sorry if this is really easy for you gurus. I'm trying to derive the reformulation of a weighted least square regression to a quadratic programming form. I understand there is a closed form solution under some assumptions. Just to clarify, it's not a homework question, it's a reduced form of a bigger problem faced in work. Tried to google but didn't find detailed steps.

More specifically, the weighted least square regression is $min_{f} ||W(Xf-r)||^2$ and the QP form is just the standard form enter image description here

I've made some derivation as shown at bottom. I think I got $P$ but I'm not sure about $q$ due to the $f^T$ in the second term. Can you suggest if my $P$ is right and also how to get $q$?

enter image description here

Improve this question
$\endgroup$

1 Answer 1

Reset to default
2
$\begingroup$

Yes, your matrix $P$ is right. Note that $f^{\top} X^{\top} W^{\top} W r$ is scalar and the transpose of a scalar is just the scalar itself, i.e.

$$ f^{\top} X^{\top} W^{\top} W r = (f^{\top} X^{\top} W^{\top} W r)^{\top} = (X^{\top} W^{\top} W r)^{\top} f = r^{\top} W^{\top} W X f $$

and thus

$$ q^{\top} f = -f^{\top} X^{\top} W^{\top} W r - r^{\top} W^{\top} W X f = -2r^{\top} W^{\top} W X f. $$

So we have $q = -2(r^{\top} W^{\top} W X)^{\top} = -2X^{\top}W^{\top}W r$.

Improve this answer
$\endgroup$
3
  • $\begingroup$ Ah, thank you sir! Forgot it's scalar. Thank you so much! $\endgroup$
    – inf
    Jan 28, 2022 at 19:23
  • $\begingroup$ BTW, I guess the last line is $q^T$ instead of $q$. So, $q = -2 X^TW^TWr$. But got your point. Thank you so much again! $\endgroup$
    – inf
    Jan 28, 2022 at 19:37
  • 1
    $\begingroup$ Yes, you're right. Good catch! $\endgroup$
    – joni
    Jan 28, 2022 at 20:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.