Reordering test cases while system testing

Revision en1, by phantom11, 2020-03-04 21:52:56

Most probably, smart people here might have already thought about this, but still..

During the system test phase of Codecraft' 20, I observed many submissions failing around the 100th test case or so. To me, this seems like waste of processing power, to let the codes run till 100 submission and then figure out its failing.

Isn't it possible to sort the test cases by: - decreasing order of number of non-ac submissions - in case of tie, increasing order of average time spent in judging it.

There could be an evaluator thread (for example), which does this after every 100 submissions or so (or something better)?

Is this something too difficult to achieve with the current codeforces infrastructure? One possible difficulty I can imagine is the scenario to show test cases to users. Are there more issues?

History

 
 
 
 
Revisions
 
 
  Rev. Lang. By When Δ Comment
en1 English phantom11 2020-03-04 21:52:56 866 Initial revision (published)