Here are two more "dimensions" to the question which have not yet been addressed in any of the other answers, but can be of great significance in practice.
Global optimum vs. local optimum: I will first assume that only globally optimal solutions are of interest.
Let us just consider feasible and globally optimal solutions to the problem. What does the solution consist of? It can be:
1) Optimal argument values, i.e., argopt. This is argmin for minimization and argmax for maximization
2) Optimal objective value
Even if there is a unique argopt, a complete description of the optimal solution consists of the argopt and the optimal objective value. However, there are some problems for which the "user" of the solution does not care about both.
For instance, in worst case engineering analysis, the user may only care about the worst case objective value (or a good enough bound for it), but not care at all the argopt achieving it. The user may choose to use a lower bound on the optimal objective value (for a minimization problem), obtained from convex relaxation in a global optimization algorithm, if the gap is below a specified tolerance; and not have, or care about, an argument value which achieves it. So that problem is "solved" without having an optimal argument value.
On the other hand, if the objective function is only a proxy for (or inaccurate approximation or statistical estimation of) the "true: objective function, then in some cases, only the argopt may be of interest. Furthermore, if the optimal argument value is not unique, there is more than one argopt. The user may or may not care about getting all argopts.
For users only interested in optimal objective value, closeness of an approximate solution to the exact optimal solution is based on closeness of objective values. For users only interested in optimal argument value, closeness of an approximate solution to the exact optimal solution is based on closeness of argument values between approximate and exactly optimal solutions.
As for globally vs. locally optimal solutions. Some users are only interested in globally optimal solutions. Other users consider any locally (or globally) optimal solution to be a "solution". Depending on the user, a solution might consist of a (any) single locally or globally optimal solution, or of all locally optimal solutions.