Namnamseo's blog

IOI 2016 using PCMS2?
By Namnamseo, history, 8 years ago, In English

ITC has a CMS but Russia has its own for IOI 2016. (from Minutes of IC February meeting)

Thought of nothing but a private fork of CMS...

Hi everyone! IOI 2016 is just a few days left, hope everyone's working out all right.

IOI 2016 is going to use PCMS2, and I have no idea about it. The committee opened IOI 2015 virtual practice sessions (one for each day) and IOI 2016 online practice session. Testing through those, these are a few things that I noticed:

  • CMS uses Bootstrap to decorate and prettify elements.
  • PCMS2 doesn't — PCMS2's graphics looks similar to DOMjudge. It gives you just enough things for the contest, with a few background-colored texts. It does not auto-refresh (if I didn't miss something).
  • CMS gives you the full feedback (in most of the cases it is used, including IOI); it grades your solution against every test cases, and shows you the result for each test cases — including verdict(AC/WA/TLE/MLE/RE/CE/etc.) and time & memory spent.
  • PCMS2's feedback is less; its feedback for sample test cases is just like CMS, but for scored subtasks, it shows something like: 'Score = 0; Time limit exceeded, test 95'. And yet, for IOI 2015 contests, we need to 'Request feedback'. If I press the button, it asks me if I am 'sure that you want to request feedback'.
  • If I click on 'Monitor' button in IOI 2016 online practice session, I get logged out.

Sorry for my querulous tone. I personally like the interface and functions of CMS, and I keep finding myself comparing CMS and PCMS2.

I really don't mind if the buttons don't change color when hovered, or the corners of boxes are not round, or the site is not automatically refreshed. But I'm really doubtful of the feedback; it might sound obvious, but I find it much better to have a full feedback, especially when looking for the mistakes I made.

Also, the 'request feedback' feature: I can think of no other reasons for this but to use tokens, which limits the chances to 'request feedback'. But... they won't, will they?

I have no experience with PCMS2, but the system seems to have been used for quite a few years. At least for 6 years? Was it used (often) in other regular/large contests? If so, how was it? How was it compared to CMS?

Thank you :D see you in a few days!

  • Vote: I like it
  • +69
  • Vote: I do not like it

»
8 years ago, # |
← Rev. 5   Vote: I like it +34 Vote: I do not like it

I'm pretty sure the 'request feedback thing' won't be used, or so I hope.

Now about the feedback — I presume that it's your first participation because last year's CMS system was not giving the feedback you're talking about. They had edited it so it provided feedback very similar to the one from PCMS2. The CMS in Kazakhstan would tell you for each subtask only the amount of ACs,WAs,TLEs and RTEs, without giving you test-by-test information.

This is intentional. Test-by-test feedback allows for a lot of 'cheating' methods. For example, the simplest trick is to hash the input of every test, then by submitting codes that act differently depending on the hash (getting RTE,TLE,WA gives you information) you can find the hash of the input of each test and then know from within your program which test you are currently solving. This allows hardcoding answers and even finding test data if it's not too large.

P.S.

I don't really care about the graphics so I'm comfortable with the PCMS2. One thing you haven't mentioned which I absolutely love about PCMS2 is that it keeps the maximum score per subtask! So for example, getting the first 5 subtasks of Friend (IOI 2014) doesn't require you to try and find in which subtask you are so you can use one of 5 completely different solutions. You can just throw five short codes in there and you'll get the score for each subtask!

Also buggy solutions that catch the large subtask but fail on the smaller one (shouldn't happen, but does surprisingly often)! You can just submit the slow solution (that you probably have for testing anyway) to get the small ones, no need to mix codes.

→ Reply
  • »
    »
    8 years ago, # ^ |
      Vote: I like it +35 Vote: I do not like it

    I think the test data of the large subtask will contain the data of the small subtask..

    → Reply
  • »
    »
    8 years ago, # ^ |
      Vote: I like it +11 Vote: I do not like it

    Also buggy solutions that catch the large subtask but fail on the smaller one (shouldn't happen, but does surprisingly often)!

    This is actually an argument for not taking the maximum per subtask in normal problems, since a buggy solution should not pass. But it's good in problems with disjoint subtasks and anyway, I think has been as .o. said in past IOIs, which makes this not a problem.

    I don't really care about the graphics

    I actually prefer raw-HTML, no script sites because they actually load on shitty mobile connection deep in the mountains. Spending an hour just trying to load a contest dashboard is a nightmare.

    → Reply
    • »
      »
      »
      8 years ago, # ^ |
        Vote: I like it +23 Vote: I do not like it

      Thats not a problem. Subtask which have greater constraints contains all tests of smaller subtasks. In CMS it is done by adding them really, and caching judging results (at least in 2014 it was so). In PCMS on ROI there were dependencies on groups (if you didn't pass this group you are not tested on this).

      → Reply
»
8 years ago, # |
  Vote: I like it +56 Vote: I do not like it

This system is used in ROI, All-Russian olympiad for school teams, NEERC and many quarter-finals of NEERC subregion.

→ Reply
»
8 years ago, # |
  Vote: I like it +22 Vote: I do not like it

I'm curious if the system supports global live score? I know leaders will have login and passwords to look at the monitor option, but will random people at home be able to follow the results live?

→ Reply
  • »
    »
    8 years ago, # ^ |
    ← Rev. 2   Vote: I like it +8 Vote: I do not like it

    If it is, will there be a feature like in CMS, where you can choose some favorite participants and monitor their performance in parallel?

    → Reply
    • »
      »
      »
      8 years ago, # ^ |
        Vote: I like it +33 Vote: I do not like it

      As far, as i know, PCMS is now compatible with scoreboard used for last IOIs.

      → Reply
»
8 years ago, # |
  Vote: I like it 0 Vote: I do not like it

As it seems like some of you already managed to test the environnement via the online page, could someone tell me how to get my login information? (The only pages I can access are login and user guide.) ~ IOI participant from Switzerland

→ Reply
  • »
    »
    8 years ago, # ^ |
      Vote: I like it +22 Vote: I do not like it

    Your team's leader should have received an e-mail containing the credentials for each of the team member and leaders.

    → Reply
    • »
      »
      »
      8 years ago, # ^ |
        Vote: I like it +33 Vote: I do not like it

      Thanks for the quick answer. We found the email in the spam folder.

      → Reply