Context

I will check the runtime distribution for multiple peaks representing solutions of different time complexities, however I often see the multiple peaks but find that the solutions at both peaks are the same. My guess for what is happening: the runtime of accepted solutions depends on the number of test cases. If there are new test cases, recent submissions will appear to be less time-efficient on average.
I've copied a sample submission from a distribution peak with low runtime and submitted it as my own to test this and the result had a runtime closer to a peak with higher runtime, which is consistent with my theory.

Example

Problem #1 is solved with a O(n) solution when the problem has 10 test cases. This results in a time distribution with a peak centered around n ms.
Then months later when Problem #1 has 40 test cases, more people solve Problem #1 with O(n) solutions. The new accepted solutions result in a time distribution with a peak centered aroun 4*n ms.
Viewing the time distribution now, there would appear to be two peaks (misleading be into believing there are two "classes" of solutions wrt time complexity), however both distributions are actually the result of the same class of solution.

Problems/Consequences

The percentile reported when submitting a solution may falsely indicate I have not found an optimal solution yet, so either:

• I am forced to view individual sample submissions to see if I was missing anything.
  • Inefficient use of my time to decipher others' uncommented code, as well as guess whether the differences in code alone would explain the difference in runtime
  • "spoilers" that I have to look at others' solutions in order to decide whether I should try to write a better solution
• or, getting a low percentile solution is not a strong indicator of a suboptimal solution, so I don't try to improve my solution
  • reduces the effectiveness of leetcode questions as a practice tool because I am practicing the wrong things!

Solutions

Potential changes that could fix this issue, in decreasing order of how well I personally think they solve the above issues:

• reevaluate old solutions against the new test cases whenever test cases are updated
• test new solutions against all test cases when deciding whether it is a valid solution, but only count the time spent on executing the original test cases for the purposes of the reported percentile and runtime distribution
• clear the solutions history for the purposes of runtime distribution when test cases are updated
• allow filtering the display of runtime distribution by the number of test cases the solution was run against
• display the number of test cases passed alongside the sample submission that is viewable by clicking on the distribution bar