Submission Runtimes are Terribly Inconsistent

I've been driving myself nuts since I joined LC trying to figure out why changes that I make from submission to submission affect runtime the way they do. Sometimes, I'll make a change that should clearly improve runtime and it somehow runs slower than the previous submission. Conversely, there have been times that I've made a change that should clearly slow down runtime and it somehow runs faster than the previous submission.

Today, I've finally confirmed that the runtimes reported for submissions are nonsense. Case in point, here are two C# submissions I made for a problem earlier today:

Literally the only difference between the two is that the second submission has comments. Yet, somehow, the second submission was reported as being twice as fast as the first.

I get that timing/benchmarking can be hard to keep completely consistent, but this margin of error is enough to make the reported runtimes basically useless. Next time, I guess I'll just keep clicking the submit button with the same code until it tells me that I have the fastest submission, because apparently that's possible.

Comments (2)