If you look at the Loss Function for a Neural Network:
Can we consider the above problem as a "Non-Linear Optimization Problem"?
The way I see it, the Loss Function of real world Neural Networks usually contains higher order terms that likely result in this function being Non-Linear. Seeing as the goal in Neural Networks is to effectively optimize this Non-Linear Loss Function, would it be technically incorrect to refer to this optimization problem as a "Non-Linear Optimization Problem" in the traditional sense?
For example, algorithms such as the "Interior Point Method" are often used to optimize typical "Non Linear Problems." Would the Loss Function of a Neural Network fall under a similar designation?