I've noticed that the "runtime performance" metric for successful solutions isn't very accurate, at least for Python 3. First, the metric is limited to increments of 4ms, so it's not terribly precise.

Second, if I test the same solution multiple times, each result will typically vary by as much as 12ms plus-or-minus from other test results (even under the best of circumstances). This can mean that an identical solution will sometimes test as "faster than 10%" while other runs are "faster than 90%." The "faster than" statement is largely meaningless.

This variance seems to be especially bad today (Sunday, April 5, 2020). As an example, here are six runs using the official solution for this problem:
https://leetcode.com/problems/valid-mountain-array/

Runtime: 256 ms, faster than 6.13% of Python3 online submissions for Valid Mountain Array.
Runtime: 208 ms, faster than 94.26% of Python3 online submissions for Valid Mountain Array.
Runtime: 212 ms, faster than 89.14% of Python3 online submissions for Valid Mountain Array.
Runtime: 316 ms, faster than 5.20% of Python3 online submissions for Valid Mountain Array.
Runtime: 212 ms, faster than 89.14% of Python3 online submissions for Valid Mountain Array.
Runtime: 388 ms, faster than 5.20% of Python3 online submissions for Valid Mountain Array.

To be clear, I am testing the exact same solution across multiple runs. All of the runs were conducted simultaneously, within a one-minute window. The worst result is nearly twice as bad as the best result.

This is really discouraging. I often want to test whether "solution A" is faster or slower than "solution B," but the results usually end up within the same margin of error (which is quite large). I have a pet peeve for people who post that their solution is "faster than 100%" or "faster than 90%" just because of a statistical dead-heat.

For the LeetCode administrators:

1. Is there something going wonky with your servers this weekend? My results seem to be especially unreliable today, moreso than normal.
2. As a general feature request, could you try to tighten-up the variance in performance tests? It would be great if you could improve the performance tests to 1ms increments (or less), with no more than a 1ms variance. This might require conducting multiple runs on your servers and then averaging the results.

Thanks.