The purpose of pretests is so that the judge queue is not be overloaded during the contest. At first I thought this means that the judge never runs the systests until the end of the contest. However, after setting a few contests, I realize that the judge actually grades the systests during contest time, and as problemsetters we can even see in real time who will FST by the end of the contest. ↵
↵
I assume this means that the grader prioritizes grading pretests and when it has "free time" it grades the systests. Of course, this is a good system to prevent the judge queue from clogging if it ever becomes too long. However, for the four contests I've got the chance to observe the grading process, the systests does not seem to have any backlog, and almost all have been graded except those which just came in. This is probably because almost everyone uses multitests now and because of that we can usually fit the systest into the pretest.↵
↵
Which brings me to my suggestion: if the grader is free enough to grade systests, should it report the verdict to the contestants? If I were to imagine, it would go something like this. First the grader prioritizes grading pretests and reports the result to the user. If it is free it grades systests and reports the result to the user sometime afterwards. If the queue is bad, it could mean that you would get your systest verdict much later than your pretest verdict.↵
↵
I think the biggest benefit is preventing FST, which are not only very frustrating and to some extent luck based (whether the problemsetter decided to put a specific caseit in pretests or not). However, some problems I can think of:↵
↵
↵
↵
- There will be no guarantees how soon you might get your systest verdict, and is it fair that some will get know they FST after 1 minute and some will know they FST after 15 minutes?↵
- How do we handle hacks, do wereport the result real time or after contest? grade new testcases immediately? And do we FST all who fail?↵
- This probably is not good for those who enjoy hacking↵
↵
What are your thoughts on this? ↵
↵
↵
↵
↵
↵
I assume this means that the grader prioritizes grading pretests and when it has "free time" it grades the systests. Of course, this is a good system to prevent the judge queue from clogging if it ever becomes too long. However, for the four contests I've got the chance to observe the grading process, the systests does not seem to have any backlog, and almost all have been graded except those which just came in. This is probably because almost everyone uses multitests now and because of that we can usually fit the systest into the pretest.↵
↵
Which brings me to my suggestion: if the grader is free enough to grade systests, should it report the verdict to the contestants? If I were to imagine, it would go something like this. First the grader prioritizes grading pretests and reports the result to the user. If it is free it grades systests and reports the result to the user sometime afterwards. If the queue is bad, it could mean that you would get your systest verdict much later than your pretest verdict.↵
↵
I think the biggest benefit is preventing FST, which are not only very frustrating and to some extent luck based (whether the problemsetter decided to put a specific case
↵
↵
- How do we handle hacks, do we
- This probably is not good for those who enjoy hacking↵
↵
What are your thoughts on this? ↵
↵
↵
↵
↵
