CPLEX solves non-convex quadratic problems to global optimality with a global optimality option (in version 12). The relevant pages are this and this. I benchmarked many solvers, and see that CPLEX is by far the best solver in this problem. I am curious about the algorithms they use. Are these generally hidden, or can I find more about this?
The best publicly available CPLEX global QP algorithm description I am aware of is the tutorial presentation by Ed Klotz of IBM at the March 2018 INFORMS Optimization conference.
MILP solvers have been improving for more than 40 years, and performance tuning tactics regarding both adjusting solver strategies and model formulations have evolved as well. State-of-the-art global nonconvex MIQP solvers have improved dramatically in recent years, but they lack the benefit of 40 years of evolution. Also, they use a broader notion of branching that can create different performance challenges. This talk will assess the effectiveness of existing MILP tuning tactics for solving nonconvex MIQPs, as well as consider more specific strategies for spatial branching. It will also examine in detail some tightening strategies specific to nonconvex MIQPs involving bilinear terms of binary variables and their associated linearizations.
Note: If you haven;t already done so, it might be worthwhile benchmarking BARON (which unlike CPLEX, only globally optimizes) on non-convex (possibly mixed-integer) QPs.
Update: As follow-up to the matter discussed in the comments to this answer of the different default values in BARON of epsR, the relative convergence tolerance: Going forward, the default value of epsR will be 1e-9 in all BARON interfaces.
Update 2: The newly released BARON 19.7.13 "Changed default value of epsr (relative optimality tolerance) to 1e-9 to guarantee better solutions"
In addition to the reference of Mark, you can have a look at his technical report:
Solving standard quadratic programming by cutting planes. by P. Bonami, A.Lodi, J. Schweiger, A. Tramontani
Since the authors are involved with the development of CPLEX, I guess this paper is relevant to your question.
Be aware that also Gurobi will soon have support for binary non convex quadratic problem. You can have a look at the slides of Tobias Achterberg at CPAIOR 2019 last June:
In both cases, it looks that a careful combination of cutting planes and spatial branch-and-bound is the winning approach.