0
$\begingroup$

I have a pretty large data set with several independent variables and one variable that I would like to explain the behavior (dependent variable).

What I want to do is find the linear coefficient and the independent variables that would minimize the error factor.

In your opinion can I use an OR algorithm to setup the search of variables and coefficient until error is minimal? Have you ever tried this ?

Improve this question
$\endgroup$
6
  • $\begingroup$ It looks like you need a classical multiple linear regression ? Such a regression will give you the optimal coefficients that minimize the standard error (or the squared error). $\endgroup$
    – Kuifje
    Feb 3, 2021 at 8:32
  • $\begingroup$ My question is instead of doing try / error method can I use operational research algorithms to get optimal equation $\endgroup$
    – ooo
    Feb 3, 2021 at 9:02
  • 1
    $\begingroup$ If I understand correctly, you need a multiple linear regression. This is typically solved with an optimization algorithm (and not trial error). Any linear regression tool (such as the one in Excel) will give you the optimal coefficients. $\endgroup$
    – Kuifje
    Feb 3, 2021 at 9:16
  • $\begingroup$ Excel will give you the optimal coefficient for each variable.. he will not select the variables for you. I’m $\endgroup$
    – ooo
    Feb 3, 2021 at 9:20
  • 1
    $\begingroup$ You should probably include a summary of this paper in your question, and specify that you want an algorithm that also removes "useless" variables. But the answer to your initial question is in the link you provided : YES, OR can be used for this, and there is probably no better way. Excel does not remove variables, but gives you the information for you to do it yourself. And tools like RapidMiner do it automatically. Probably most modern tools do it as well. $\endgroup$
    – Kuifje
    Feb 3, 2021 at 9:57

3 Answers 3

Reset to default
4
$\begingroup$

As mentioned in other responses, mixed-integer optimization can be used to simultaneously fit a model and select variables. Other possibilities include best-subset regression, which many (most?) statistics packages include, and lasso regression, which uses optimization but does not involve integer programming.

Improve this answer
$\endgroup$
3
$\begingroup$

OR can help you in what you are doing. The article you mentioned can help you to set the coefficient and also to select the variables (best subset selection) -- among other things (e.g., sparsity, robustness). This other article is probably also worth looking at, since it focuses solely on features selection. It is a rather comprehensive approach. However, you should know that it results in a mixed-integer quadratic programming problem. With a too large dataset the resulting problem might become too difficult to solve.

The alternative is, probably, trial and error: you define different models with different subsets of variables and compare them. Libraries for this are widely available in, e.g., Python (scikit-learn), R, or even MS Excel. Compared with the above mentioned method, this is a heuristic approach: you may not find the best model. Nevertheless, it should be easier from a computational perspective, or at least scale better with the size of the dataset.

Improve this answer
$\endgroup$
2
$\begingroup$

By searching for "sparse linear regression" on the web, you will find a lot of interesting material on this topic. For example, have a look at this nice, comprehensive lecture.

As mentioned above, sparse linear regression can be modeled as a Mixed-Integer Quadratic Program and solved by using mathematical optimization solvers like LocalSolver. Nevertheless, it is generally tackled by heuristics in practice.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.