Right now, our submissions' runtime are based on elapsed time instead of CPU time. That means that the measurements are inconsistent even for the EXACT SAME PIECE OF CODE run multiple times. It also means that some of our solutions are unfairly timed, as many factors affect the runtime. Cache, server load, different CPUs, I/O operations etc.

The correct way to measure would be to measure CPU time, like how Kattis does it, which will give consistent measures every single time.

Quote from Google:
CPU time is the amount of time for which a central processing unit (CPU) was used for processing instructions of a computer program or operating system, as opposed to elapsed time, which includes for example, waiting for input/output (I/O) operations or entering low-power (idle) mode.

Example of the exact same piece of code with wildly varying runtimes. 400ms and 824ms are not even close.