The problem is, even if you use the same number of threads, a 'rigorous' comparison of different implementations in different programming languages - for example, in terms of running time or of the quality of the solution found in the same time limit - can be quite challenging or tricky. Citing Chapter 18 of Ahuja, Magnanti & Orlin¹:

"The existing literature on computational testing has a tendency to overrely on CPU time as the primary measure of performance. CPU time depends greatly on subtle details of the computational environment and the test problems such as (1) the chosen programming language, compiler, and computer; (2) the implementation style and skill of the programmer; (3) network generators used to generate the random test problems; (4) combinations of input size parameters; and (5) the particular programming environment (e.g., the use of the computer system by other users). Be-cause of the multiple sources of variabilities, CPU times are often difficult to replicate, which is contrary to the spirit of scientific investigation. Another drawback of the use of CPU time is that it is an aggregate measure of empirical performance and does not provide much insight about an algorithm's behavior. For example, an algorithm generally performs some fundamental operations repeatedly, and a typical CPU time analysis does not help us to identify these "bottleneck" operations. Iden-tifying the bottleneck operations of an algorithm can provide useful guidelines for where to direct future efforts to understand and subsequently improve an algorithm."

From the last citation, most likely points (1) and (2) - and probably (3) - will affect comparison of optimization algorithm performance and should be taken into account.

To the best of my knowledge - what I remember after being to talks of academics reading publications -, they tend to mention reducing the number of cores to 1 (in case of comparing to non paralellized heuristics) and more importantly turning off all kind of additional intelligence the commercial solver may include (e.g. presolve, automatic cutting plane generation), in order to the performance comparison to be as fair and unbiased as possible.

¹ Ahuja, R. K., Magnanti, T. L., & Orlin, J. B. (1988). Network flows.

The problem is, even if you use the same number of threads, a 'rigorous' comparison of different implementations in different programming languages - for example, in terms of running time or of the quality of the solution found in the same time limit - can be quite challenging or tricky. Citing Chapter 18 of Ahuja, Magnanti & Orlin¹:

"The existing literature on computational testing has a tendency to overrely on CPU time as the primary measure of performance. CPU time depends greatly on subtle details of the computational environment and the test problems such as (1) the chosen programming language, compiler, and computer; (2) the implementation style and skill of the programmer; (3) network generators used to generate the random test problems; (4) combinations of input size parameters; and (5) the particular programming environment (e.g., the use of the computer system by other users). Be-cause of the multiple sources of variabilities, CPU times are often difficult to replicate, which is contrary to the spirit of scientific investigation. Another drawback of the use of CPU time is that it is an aggregate measure of empirical performance and does not provide much insight about an algorithm's behavior. For example, an algorithm generally performs some fundamental operations repeatedly, and a typical CPU time analysis does not help us to identify these "bottleneck" operations. Identifying the bottleneck operations of an algorithm can provide useful guidelines for where to direct future efforts to understand and subsequently improve an algorithm."

From the last citation, most likely points (1) and (2) - and probably (3) - will affect comparison of optimization algorithm performance and should be taken into account.

To the best of my knowledge - what I remember after being to talks of academics reading publications -, they tend to mention reducing the number of cores to 1 (in case of comparing to non-parallelized heuristics) and more importantly turning off all kind of additional intelligence the commercial solver may include (e.g. presolve, automatic cutting plane generation), in order to the performance comparison to be as fair and unbiased as possible.

¹ Ahuja, R. K., Magnanti, T. L., & Orlin, J. B. (1988). Network flows.