Rubanenko's blog

NBHEXT — Know Your Rating
By Rubanenko, 9 years ago, In English

Few weeks ago there was yet another discussion about "omg why we wait for the rating updates so long!?!?!" in Russian version of CF. MikeMirzayanov said that there are several reasons:

  1. cheaters are checked manually;
  2. rounds are usually held in the evening and Mike spends some time to get home from the University;
  3. sometimes his little daughter asks him to play with her and Mike can't refuse of course

After 1k_trash, Maxim and me read the third reason, we started to think what can be done about it. After a little thinking we came up with a solution that seemed to make contestants and Mike's daughter happy. The solution was a creation of browser extension that maintains rating changes of a current scoreboard whenever you refresh the result page. Something like this:

Luckily, CodeForces provided us with all the tools to make it happen: an API and a public implementation of the rating calculation method.

We had very little experience developing stuff like this, but we had strong desire to contribute to CodeForces. Eventually we had much fun and enjoyed the process a lot. At first, we couldn't make the extension work with large scoreboards (div2 or mixed rounds). The API couldn't serve too heavy requests about standings list. So we had to switch this process to the browser's background every time it's launched. So, actually, you have to wait around two minutes after you turned on the browser in order to let the extension download current ratings of all the users. Don't worry, it happens only once the browser has started. In other words, in the very worst situation the extension won't work for the first two minutes of a contest, in case you launched the browser just before the contest start.
When we were done with the routine described above, we noticed two things:

  • it takes about 30 seconds to compute the rating change of 5k participants round. It was written in JS :)
  • the rating changes it calculated were wrong!

Luckily, we could compare our implementation with the official one literally line-by-line. The issue was in type: official implementation used int, while we used...var. parseInt almost everywhere solved the problem, great :)
At this point we had right rating change calculation method, but it was very slow. Our team was quite upset, because we were sure that CF uses the fastest solution possible and we have no chance to speed it up. However, after a little thinking we noticed, that using int everywhere gives us a possibility to memorize some data. We ended up with O(min(N * K), N2 * log(N)) solution, with N being the number of contestants in a scoreboard, and K being the number of different integer rating values. In current situation K is about 3-4k, so our solution started to work about 20 times faster! Afterwards, we finished some small details and uploaded the extension to the Chrome Store.

Link to install

Everybody is welcome to contribute to our Github repository.
Hope you love it!

UPD

Thank you all guys for your feedback. After the extensions attracted so much attention we have no choice but to do our best in order to make it better. We will make a lot of changes this week: fix bugs you reported, add requested features, switch to backend in order to decrease pressure on CF servers during the contest.
A special shoutout goes to people, who contributed to our repository. We are very grateful and will look at every pool request these days.
A big thanks to Mike Mirzayanov for being friendly, ready to collaborate and giving few pieces of advice.

  • Vote: I like it
  • +647
  • Vote: I do not like it

Comments Comments (75)
| Write comment?
»
9 years ago, # |
← Rev. 2   Vote: I like it +6 Vote: I do not like it

seems awesome!!! I think everybody will love it :)

PS: It would be great if you develope the extension for other browsers such as Safari or FireFox :D

→ Reply
»
9 years ago, # |
  Vote: I like it +5 Vote: I do not like it

Awesome :)

Sorry I have a question ?! Why you don't calculate rating after contest just once and save it in a file or database, then extension must just download data from your host and use them without calculation ?

→ Reply
  • »
    »
    9 years ago, # ^ |
      Vote: I like it +6 Vote: I do not like it

    Thanks for your review! We thought it will be interesting to know how the contest is going right now. Sure, we will move our data to own host if current solution won't work well. Thank you again and have fun solving problems!

    → Reply
  • »
    »
    9 years ago, # ^ |
      Vote: I like it +8 Vote: I do not like it

    We are glad that you like it:)

    We think that seeing the rating change during a contest is pretty exciting :)

    → Reply
    • »
      »
      »
      9 years ago, # ^ |
      ← Rev. 2   Vote: I like it 0 Vote: I do not like it

      sorry, I didn't read it carefully. you're right , really exciting :)

      But how about server high pressure at middle of contest ? Maybe this extra requests (for downloading whole scoreboard) for server will decrease the speed of loading pages or ... ?

      → Reply
      • »
        »
        »
        »
        9 years ago, # ^ |
          Vote: I like it 0 Vote: I do not like it

        We don't think it will cause some visible problems. Anyway,will see tomorrow)

        → Reply
        • »
          »
          »
          »
          »
          9 years ago, # ^ |
            Vote: I like it 0 Vote: I do not like it

          I can't understand this statement clearly :

          So, actually, you have to wait around two minutes after you turned on the browser in order to let the extension download current ratings of all the users. Don't worry, it happens only once the browser has started.

          It means extension will download standing page just once and will calculate ratings!? or each time we open standing page must download it ?

          → Reply
          • »
            »
            »
            »
            »
            »
            9 years ago, # ^ |
              Vote: I like it +14 Vote: I do not like it

            Only once, when the browser is launched. All other requests work no more than a couple of seconds.

            → Reply
          • »
            »
            »
            »
            »
            »
            9 years ago, # ^ |
              Vote: I like it +3 Vote: I do not like it

            Extension will download all users ratings (with inactive users too) once when browser launched. It takes about two minutes. Actually, it reloads every 12 hours (in case if you keep your computer on and browser launched constantly). Standings data loading every time you open standings page.

            → Reply
»
9 years ago, # |
← Rev. 2   Vote: I like it +13 Vote: I do not like it

Thanks for the extension, it's awesome! There is a small bug. If you uncheck the show unofficial box, then an HTTP 403 error message is showed.

→ Reply
»
9 years ago, # |
  Vote: I like it +115 Vote: I do not like it

Looks like 1k_trash has an interesting pastime activity! :p

→ Reply
  • »
    »
    9 years ago, # ^ |
      Vote: I like it +26 Vote: I do not like it

    Dunno, somebody must have accepted a pull request with this change =___=

    → Reply
»
9 years ago, # |
  Vote: I like it +1 Vote: I do not like it

Someone converting to Safari?

I'd love to try my hand at it if no one else is already working on the same...

→ Reply
  • »
    »
    9 years ago, # ^ |
      Vote: I like it +1 Vote: I do not like it

    Oh sure, that would be great, thank you! And don't forget to make a pull request when you are done.

    → Reply
»
9 years ago, # |
  Vote: I like it 0 Vote: I do not like it

→ Reply
»
9 years ago, # |
← Rev. 2   Vote: I like it +6 Vote: I do not like it

After system test, the extension was showing that I will get -8, but instead I got +6. I think there is difference between your and Codeforces' algorithm (unless some people was banned from leaderboard because of cheating after systest; I'm not sure about that)

→ Reply
  • »
    »
    9 years ago, # ^ |
      Vote: I like it +8 Vote: I do not like it

    We will definitely investigate it.

    → Reply
    • »
      »
      »
      9 years ago, # ^ |
        Vote: I like it +5 Vote: I do not like it

      I am actually got +199 ratings, but your plugin showed that I should get +150. How about that? Any help? Thanks.

      → Reply
  • »
    »
    9 years ago, # ^ |
    ← Rev. 2   Vote: I like it +15 Vote: I do not like it

    Also, according to the extension, several unrated coders were supposed to get +800. Perhaps this bug is indirectly affecting rated coders?

    Edit: Btw, the fact that the extension is underestimating the rating change in most (all?) cases gives some supporting evidence towards this, as lowering the expected places of unrated coders would increase the expected places of rated ones and therefore cause them to be awarded less points.

    → Reply
    • »
      »
      »
      9 years ago, # ^ |
        Vote: I like it +8 Vote: I do not like it

      You are right. Rating for unrated users is 1500. We are changing rating from 0 to 1500 in our scripts. On testing data results were 100% correct. But accidently new users were not in rating list and their rating was undefined instead of 0. So their rating still were 0 instead 1500. Didn't catch it during our testing. With no doubt we will fix shortly.

      → Reply
      • »
        »
        »
        »
        9 years ago, # ^ |
          Vote: I like it +10 Vote: I do not like it

        Oh, thanks!

        By the way, thank you all for the extension! Knowing my expected rating change during the contest was very cool!

        → Reply
    • »
      »
      »
      9 years ago, # ^ |
        Vote: I like it +20 Vote: I do not like it

      Fixed and tested it locally. Results are absolutely correct.

      → Reply
      • »
        »
        »
        »
        9 years ago, # ^ |
          Vote: I like it +3 Vote: I do not like it

        Thank you for your effort. It's an amazing extension.

        → Reply
»
9 years ago, # |
  Vote: I like it +3 Vote: I do not like it

Hey, great work guys. Also, shouldn't the extension be under "productivity" in the Chrome store and not "accessibility"?

→ Reply
  • »
    »
    9 years ago, # ^ |
      Vote: I like it 0 Vote: I do not like it

    Hey, thank you! You are right, we're going to change the category soon.

    → Reply
»
9 years ago, # |
  Vote: I like it +13 Vote: I do not like it

Hello, I had a doubt. If I go to the standings of some past contest, the delta column I see there is completely different from the actual rating changes shown in the RATING CHANGES tab.

Is it because you calculate the rating changes of the past contests using the current rating of the participants and not the actual past ratings before the specific contest.

I know that this extension is actually meant for only live contests. But just a confusion why is it inconsistent with the past contests.

→ Reply
  • »
    »
    9 years ago, # ^ |
      Vote: I like it +3 Vote: I do not like it

    That's right, it always uses the current ratings.

    → Reply
  • »
    »
    9 years ago, # ^ |
      Vote: I like it +8 Vote: I do not like it

    Hello usaxena95, thanks for your reply. You are right, now the extension always uses current rating for calculation. We will fix it in the next version, since we will move everything to our own server. So deltas for past contests will be calculating correctly. And also virtual contests will be supported.

    Thank you for the interest, stay tuned!

    → Reply
»
9 years ago, # |
  Vote: I like it 0 Vote: I do not like it

It's really awesome !! Loving it !! :D

→ Reply
»
9 years ago, # |
  Vote: I like it 0 Vote: I do not like it

If i try to check the Show Unofficial box in the standing, it doesn't work and gives an error while keeping enabled the extension.

→ Reply
»
9 years ago, # |
  Vote: I like it -9 Vote: I do not like it

That doesn't look neither like changes that happened at time of the contest or like changes that should happen if this was current contest — deltas are way too high (like 2 times?). Screenshot was taken on #317 if that's of any relevance.

→ Reply
  • »
    »
    9 years ago, # ^ |
      Vote: I like it +5 Vote: I do not like it

    You know that rating formulas have been changed since then, right? :)

    → Reply
    • »
      »
      »
      9 years ago, # ^ |
        Vote: I like it -8 Vote: I do not like it

      Yes, I know, I have taken that into consideration — "That doesn't look (...) like changes that should happen if this was current contest" ;). How do you think that should explain such high deltas? Current changes seem to be smaller than previous ones.

      → Reply
»
9 years ago, # |
  Vote: I like it 0 Vote: I do not like it

it does not work for me. all the numbers are 0

→ Reply
»
9 years ago, # |
  Vote: I like it 0 Vote: I do not like it

Respect very nice idea

→ Reply
»
9 years ago, # |
  Vote: I like it 0 Vote: I do not like it

I think there might still be a minor bug in the extension, since I was supposed to get a +96 but got a +103 instead. I restarted my browser before the contest. Is there anything else that I should have done?

→ Reply
  • »
    »
    9 years ago, # ^ |
      Vote: I like it +9 Vote: I do not like it

    Yes CodeForces is more generous than this version :D

    → Reply
»
9 years ago, # |
  Vote: I like it +2 Vote: I do not like it

Not a correct output :).

However I guess that this one probably can be easily explained — I guess it didn't catch up after my submission of C? #bugreport

(I still wait for an explanation of my previous screen, which is still a mystery for me)

→ Reply
»
9 years ago, # |
← Rev. 2   Vote: I like it 0 Vote: I do not like it

I disabled the extension before contest and during the contest i enabled it. But, now it's not showing any rating change. Showing 0 for everyone !! :( what's wrong ??

UPD: Now it's working. :)

→ Reply
»
9 years ago, # |
  Vote: I like it +44 Vote: I do not like it

Is it only me or it isn't working?

→ Reply
»
9 years ago, # |
  Vote: I like it 0 Vote: I do not like it

Why does this show slightly different rating changes most of the time? And occasionally, it's completely off.

→ Reply
  • »
    »
    9 years ago, # ^ |
      Vote: I like it 0 Vote: I do not like it

    Looks like it only doesn't work for legendary grandmasters.

    → Reply
    • »
      »
      »
      9 years ago, # ^ |
        Vote: I like it 0 Vote: I do not like it

      That's strange, considering it was made after LGMs were introduced. And I think it worked for them before (can someone confirm/falsify this?).

      → Reply
      • »
        »
        »
        »
        9 years ago, # ^ |
          Vote: I like it 0 Vote: I do not like it

        It shows wrong results for unrated users sometimes. I once saw +1000 for an unrated user.
        It's usually wrong by 20-30, but in today's contest it was very accurate (at least for me and some of my friends)

        → Reply
      • »
        »
        »
        »
        9 years ago, # ^ |
          Vote: I like it 0 Vote: I do not like it

        Click.

        → Reply
        • »
          »
          »
          »
          »
          9 years ago, # ^ |
            Vote: I like it +3 Vote: I do not like it

          Ok, I didn't check the Russian comments.

          → Reply
  • »
    »
    9 years ago, # ^ |
      Vote: I like it +6 Vote: I do not like it

    I'm slightly saddened that you decided to screenshot my implementation failures for C and D for eternity :(

    I'm also happy to see I wasn't the only one to do that :)

    → Reply
»
9 years ago, # |
  Vote: I like it +13 Vote: I do not like it

Possible bug: on standings page, when i double click in any cell to view submission history, it doesn't get loaded (the 'loading' icon keeps on rotating). This doesn't happen when i disable the extension.

→ Reply
  • »
    »
    9 years ago, # ^ |
      Vote: I like it +5 Vote: I do not like it

    i have the same problem. but i forgot to write about it all time

    → Reply
»
9 years ago, # |
  Vote: I like it +83 Vote: I do not like it

Lately I'm just getting 0s for everyone all the time. Is it just me?

→ Reply
  • »
    »
    9 years ago, # ^ |
      Vote: I like it +18 Vote: I do not like it

    Seems like there are some shenanigans with the Codeforces API and calling the following method fails for some reason.

    → Reply
    • »
      »
      »
      9 years ago, # ^ |
        Vote: I like it +20 Vote: I do not like it

      So that means we should notify MikeMirzayanov to have it fixed, right?

      → Reply
      • »
        »
        »
        »
        9 years ago, # ^ |
          Vote: I like it 0 Vote: I do not like it

        yes, there seems to be some issue when activeOnly is set to false. activeOnly=true is working fine. Also, the extension starts showing non-zero changes (although incorrect, of course) if that parameter value is changed from false to true in the extension package.

        → Reply
        • »
          »
          »
          »
          »
          9 years ago, # ^ |
            Vote: I like it +10 Vote: I do not like it

          Thanks. Sent him a private message, let's hope it gets fixed soon. :)

          → Reply
»
8 years ago, # |
  Vote: I like it +43 Vote: I do not like it

Stopped working again, all 0s on today's contest :/

→ Reply
»
8 years ago, # |
  Vote: I like it +15 Vote: I do not like it

Can this extention be made for Firefox ???

:)

→ Reply
»
8 years ago, # |
  Vote: I like it +11 Vote: I do not like it

It doesn't work, shows 0s for everyone and when I double click on a cell to view the submissions it gets stuck on the loading page.

→ Reply
»
8 years ago, # |
  Vote: I like it +31 Vote: I do not like it

Seems to be a problem with the CodeForces API once again, the URL used by the extension (http://codeforces.net/api/user.ratedList?activeOnly=true) gives Error 503 Service Temporarily Unavailable. (This time also with activeOnly=false.) Messaging MikeMirzayanov to make sure he's aware...

→ Reply
»
8 years ago, # |
  Vote: I like it 0 Vote: I do not like it

Recently I noticed that the extension gives 0 for all handles :\

→ Reply
»
8 years ago, # |
  Vote: I like it +64 Vote: I do not like it

Nice app. But it usually doesn't work. Could you please fix it?

Rubanenko Maxim 1k_trash

→ Reply