BencilSharpeniro's blog

Opinion on USACO Scoring System
By BencilSharpeniro, history, 5 years ago, In English

Hi everyone!

I just wanted to get everyone's opinion on the USACO scoring system. For the uninformed, USACO uses a system where you are scored based on the number of test cases you get, rather than using subtasks and batch testing like other OIs.

One problem (that's recently been highlighted) is that it restricts problems that have yes/no output because just writing a program to print yes/no nets you at least 50% of cases. This is probably best shown through this problem: http://www.usaco.org/index.php?page=viewproblem2&cpid=941

So, is the grading system outdated? Should it be changed? Comment your opinion!

Complimentary Meme
  • Vote: I like it
  • +17
  • Vote: I do not like it

Comments Comments (7)
| Write comment?
»
5 years ago, # |
  Vote: I like it 0 Vote: I do not like it

As a sidenote: Sometimes USACO does have other constraints for a subset of the test cases (much like Hotels from POI). For example Stamp Painting has 75% of test cases under a different constraint.

→ Reply
  • »
    »
    5 years ago, # ^ |
    ← Rev. 2   Vote: I like it -10 Vote: I do not like it

    I hope USACO continues to TELL US more about the nature of their test cases (e.g. what percent have N <= 1000)...

    I think if one submits a simple O(N^2) then one can figure out themselves based on what TLE's, but that just seems... weird?

    → Reply
»
5 years ago, # |
← Rev. 2   Vote: I like it -28 Vote: I do not like it

From a selfish point of view, I got a 14/15 on one problem last February so it is more beneficial to contestants I think. However, it implicitly harms contestants when other people can cheese answers with some randomness but I think that's a rare scenario... and honestly that's lowkey intelligent.

Knock on wood, hope this doesn't bite me later in the butt — I'd prefer situations where people with an almost correct idea get a majority of points at the cost of what I think to be the rarer situation of other people getting points with mainly incorrect ideas.

(photo credits? ;) )

→ Reply
  • »
    »
    5 years ago, # ^ |
      Vote: I like it 0 Vote: I do not like it

    One thing to note is point inflation. If it helps a lot of people, they'll probably make the cutoff higher (as seen in the bronze contest I linked.)

    → Reply
    • »
      »
      »
      5 years ago, # ^ |
        Vote: I like it -13 Vote: I do not like it

      In my opinion, point inflation is not a bad thing since the organizers coordinate cutoffs anyway

      → Reply
»
5 years ago, # |
  Vote: I like it +22 Vote: I do not like it

That's why by now most Yes/No problems use a multitest system.

→ Reply
»
5 years ago, # |
  Vote: I like it -24 Vote: I do not like it

problems that have yes/no output

Are bad and should be avoided or modified to have more informative output. Exceptions exist, but they're rare.

→ Reply