(Another OR noob question)
As I'm trying to learn about OR and Optimization methods for work, I'm having a hard time understanding the difference between "Optimization" and "Constrained Optimization": Isn't any real world optimization problem constrained one way or the other? Why do people talk about "Constrained Optimization" as a specific sub-topic?
You are right that most real-world problems are constrained, and therefore, for the most part, "optimization" and "constrained optimization" are synonymous.
However, some algorithms only apply to unconstrained problems: an easy example is bisection search. So when people say "constrained optimization," they are emphasizing that they're considering the general case, as opposed to the special case of unconstrained optimization.
Problems involving physical/tangible systems (such as production scheduling, crew assignment, ...) are inherently constrained, whether the modeler bothers to put all the constraints in or not. Any variable representing a physical process has a finite domain (presuming that the physicists are correct in saying there is a finite amount of matter in the known universe), whether the model author bounds them or not.
On the other hand, consider the problem of fitting a regression model (least squares) using gradient descent. The variables are the model coefficients, and while there may be ways to argue that this particular coefficient cannot be astronomically large in magnitude, it's unlikely there will be any known a priori bounds for the coefficients. (The computer's limited range of floating point values will inherently bound them, whether you like it or not.) So the problem is treated as unconstrained, with no loss of validity or functionality.
The way I think of it is in terms of budget. In general optimization, I could get the best solution given technology and all other CTQ's. However, i only can afford so much given my cash and budget, so I may find another solution within that subset that is best given what is possible, namely what is within my constraints.
While in general, these may be very similar, in application it is a totally different problem with a budget constraint, especially when you start adding other constraints like whole units (i cant buy 1/2 a bolt and still get the same use) and interdependencies...
