In numerical tests of optimization algorithms, one often reads "We used a XY computer with X GB RAM for the experiments".
Usually, when I want to compare my results with theirs, I do not have an XY computer, but a YZ computer. And rerunning the tests of others is often difficult because I do not have the code.
I wonder whether one of the large cloud providers allows you to configure a virtual server in a way that others could easily do the same.
If the purpose of the comparison is to compare solution times (as opposed to validating the answers obtained), my feeling is that the XY v. YZ computer should not be an issue. Getting the source code (or binary) for the other algorithm is definitely a problem. Assuming you have their code (and that it runs on your computer, which may require getting the source and recompiling), and assuming you have enough memory for both methods, I think you can do a plausible comparison. If one algorithm has more stringent hardware requirements than the other (e.g., they need 16GB of RAM to solve problems you can solve in 4GB), that is worth noting. Memory aside, the primary difference between an XY and a YZ computer will be CPU speed and number of simultaneous threads it can handle (which relates to number of cores). Neither should dictate whether an algorithm can or cannot solve a problem (whereas memory might); they just affect solution speed. So running both methods on your computer v. theirs is like running a race on one track v. a different track: as long as both competitors are on the same track, it's a fair race.
For me, a trickier issue in algorithm races lies in comparability of the coding. If my code is in Python and theirs is compiled C code, they may have an inherent speed advantage unrelated to the wonderfulness of their algorithm. Even with the same language, differences in compilers could account for some speed differences. Similarly, if one method is coded by someone who is adept at optimizing for speed and the other is coded by someone who knows the math well but is a dabbler in coding, the "tighter" code might win even if the other algorithm is inherently better.
To get at the cloud part, I'm not aware of a cloud platform that lets you choose between different CPUs. (I think some of them let you configure the amount of RAM and possibly the number of cores, but not Intel XXX chip v. AMD yyy chip, say.) On the other hand, I'm no expert on cloud platforms.
I wonder whether one of the large cloud providers allows you to configure a virtual server in a way that others could easily do the same.
The NEOS Server is a free internet-based service for solving numerical optimization problems. Hosted by the Wisconsin Institute for Discovery at the University of Wisconsin in Madison, the NEOS Server provides access to more than 60 state-of-the-art solvers in more than a dozen optimization categories. There are 3rd party submission tools that allow you to write a simple program at home and submit your job to the server. See the NEOS FAQ for details.
OpenDreamKit offers a Jupyter notebook to: GAP (which supports OpenMath), PARI/GP (number theory) or try the simple web interface, SageMath (includes R, and a lot more), and Singular (a computer algebra system for polynomial computations with special emphasis on the needs of commutative and non-commutative algebra, algebraic geometry, and singularity theory).
Online Solver for the GNU Linear Programming Kit from the book: "Linear and Integer Optimization - Theory and Practice", by Gerard Sierksma and Yori Zwols.
You can install OR-Tools on Google Cloud (free year long trial) and Stack Overflow offers some support: "How to install ortools library on Google Cloud".
PHPSimplex is an online tool for solving linear programming problems.
OPTIL.io is the on-line judge system that receives algorithmic solutions of optimization problems in a form of source code from the crowd of developers, compiles it, executes in a homogeneous run-time environment and objectively evaluates using the set of test cases. Problems that are solved can be provided by external companies or scientists. Solutions can be submitted in almost any programming language.
