This article presents the following formulation of the best subset selection problem

$$\min_{\|\beta\|_0\leq k}\frac{1}{2}\|y-X\beta\|^2_2$$

I wonder where the $1/2$ comes from. Help appreciated.


1 Answer 1


The factor of $\frac{1}{2}$ is just for convenience, so that the Hessian of the objective does not have a factor of two.

The argmin (optimal value of $\beta$) is the same, whether or not the factor $\frac{1}{2}$ is included.

Note that rather than $\frac{1}{2}$, some "authors" use $\frac{1}{2n}$, This also does not affect the argmin.

  • 4
    $\begingroup$ Well, the square on the norm is also optional in same sense as the 0.5. Needed if you want a QP though. $\endgroup$ Jan 15, 2020 at 18:28
  • 1
    $\begingroup$ @ ErlingMOSEK Yes that is true. And as a true conic man, I know you appreciate the potential advantages, including not squaring the condition number. Nevertheless, least squares, and to a large extent, nonlinear least squares, is the genesis for the 1/2. $\endgroup$ Jan 15, 2020 at 18:49
  • $\begingroup$ Thanks @MarkL.Stone, could you please clarify in more details so that the Hessian of the objective does not have a factor of two (the rest is rather clear) $\endgroup$
    – k88074
    Jan 16, 2020 at 8:28
  • $\begingroup$ @k88074 Without the factor of 1/2 the Hessian of the objective would be $2X^TX$. With the factor of 1/2, the Hessian is $X^TX$. $\endgroup$ Jan 16, 2020 at 12:50
  • $\begingroup$ I the following reasoning correct? If want to minimize the objective (forget the constraint), I set the first derivative of the objective to zero, that is $-2Xy+2X^\top~X\beta=0$. Why does the $2X^\top~X$ bother us? We still get the same $\beta$ $\endgroup$
    – k88074
    Jan 18, 2020 at 8:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.