When submitting a solution to a problem, I think most people get the general impression that once you pass 30-40 test cases, you are basically home free, i.e. you are highly unlikely to fail a later test case and can consider the problem solved. Despite this, many problems have upwards of 200+ test cases, which can mean several minutes of grading time in the worst case, not even taking into account time spent in the queue.

On a related note, the judging software seems to have been heavily overburdened recently, possibly linked to the coronavirus.

I think that a good feature to have would be to allow lots of test cases for problems which are relatively recently added, but once a problem has been solved by ~1000+ people, most of the test cases should be removed to allow submissions to go through quicker. In order to choose which submissions to remove, the system should keep track of how often each test case gets failed. Test cases which are almost never failed should be removed. I strongly suspect that over 50% of test cases on most problems would fall into this category, if not 80% to 90% for some problems.

Would this bring about too many false positives for submissions? (For older problems, do we even care?)