the_midnight_sun's blog

Did Codeforces change the formula for rating system?
By the_midnight_sun, history, 5 years ago, In English

Hi everyone! In recent contests, my friends and I did not perform well. However, our ratings still increased somehow. In addition, CF predictor is no longer precise these days. Maybe I'm not the only one who feel strange. Then, did Codeforces change their formula for rating system?

  • Vote: I like it
  • +84
  • Vote: I do not like it

Comments Comments (30)
| Write comment?
»
5 years ago, # |
← Rev. 2   Vote: I like it +12 Vote: I do not like it

I only solved Div2 [A — B] and my rating increased to 1653! I thought I was going back to Specialist LOL

→ Reply
»
5 years ago, # |
  Vote: I like it +19 Vote: I do not like it

maybe it's because the number of participants has increased significantly due to lockdown

→ Reply
  • »
    »
    5 years ago, # ^ |
    ← Rev. 2   Vote: I like it 0 Vote: I do not like it

    The formula is stationary, right? So the number of participants should effect nothing.

    → Reply
    • »
      »
      »
      5 years ago, # ^ |
        Vote: I like it +2 Vote: I do not like it

      The predictions were good enough, in today's contest atleast

      → Reply
      • »
        »
        »
        »
        5 years ago, # ^ |
          Vote: I like it +5 Vote: I do not like it

        Well, -8 became +5 and +43 became +92. I dont think the predictions were good!

        → Reply
        • »
          »
          »
          »
          »
          5 years ago, # ^ |
            Vote: I like it -11 Vote: I do not like it

          How so? There's a huge difference when you get rank 500 with 1000 people vs 10000 people. Number of participants is probably there in their formula.

          Also most of the newcomers are beginners so they basically just increase the number of participants and since rating change is inversely proportional to participants you get a higher rating than expected

          → Reply
          • »
            »
            »
            »
            »
            »
            5 years ago, # ^ |
              Vote: I like it 0 Vote: I do not like it

            I understand what u said. My question is that CF predictor is also built based on CF formula in which it already considered your situation when the number of participants is large.

            → Reply
        • »
          »
          »
          »
          »
          5 years ago, # ^ |
            Vote: I like it +1 Vote: I do not like it

          +82 became +86....i think it's good enough

          → Reply
    • »
      »
      »
      5 years ago, # ^ |
      ← Rev. 2   Vote: I like it +17 Vote: I do not like it

      dude, do you know how does the formula work? it uses your expected place which is counted according to your rating and ratings of all other participants.

      → Reply
»
5 years ago, # |
  Vote: I like it +35 Vote: I do not like it

CF Predictor has never been more than several points off for me personally. It's easy to trick yourself into thinking that you didn't do as well as you did, given the large number of participants in the past few rounds. Your rating change doesn't take into account your absolute rank, but rather how you perform relative to the rest of the pool of participants.

→ Reply
»
5 years ago, # |
← Rev. 2   Vote: I like it +7 Vote: I do not like it

Probably because so many users participate in rounds nowadays, when more users with higher rating than you gets below you in standings your rating will increase more.

About the predictor It was 100% precise with me, maybe you didn't reload the page and standings have changed after you took a look at the predictor???

Edit: I just compared the rating change, you are right the predictor have some mistakes in Div. 2

→ Reply
»
5 years ago, # |
← Rev. 2   Vote: I like it +59 Vote: I do not like it

I believe today was the first when more than 10k contestants took part in a rated round. As I remember, I didn't considered such option when I was developing CF-Predictor.

This should be quite easy fix, I'll work on it on weekends.

P.S. As you can see prediction for the div. 1 is accurate, so formula hasn't changed.

→ Reply
  • »
    »
    5 years ago, # ^ |
      Vote: I like it +3 Vote: I do not like it

    Thank you for your work @@

    → Reply
  • »
    »
    4 years ago, # ^ |
      Vote: I like it 0 Vote: I do not like it

    BTW, during the last contest, CF Predictor showed me a prediction of ~40 points drop, while I ended up gaining 18 points in rating, much to my surprise. Of note, my last solution came in just a few minutes before the time ended, so not sure if ratings take time to propagate.

    It worked very accurately until this point, so not sure if something changed recently or if it was just coincidence?

    → Reply
    • »
      »
      »
      4 years ago, # ^ |
        Vote: I like it +14 Vote: I do not like it

      Well, recently CF Predictor has not been giving correct deltas, probably this is connected to the recent change in how new accounts are rated.

      → Reply
      • »
        »
        »
        »
        4 years ago, # ^ |
          Vote: I like it +8 Vote: I do not like it

        Thanks, so this was not an isolated case. I'm fairly new to the plugin (installed it a few weeks back and it's mostly been fairly accurate until the last contest).

        → Reply
        • »
          »
          »
          »
          »
          4 years ago, # ^ |
            Vote: I like it 0 Vote: I do not like it

          Probably we should ping WasylF and hope he can fix this.

          → Reply
          • »
            »
            »
            »
            »
            »
            4 years ago, # ^ |
              Vote: I like it 0 Vote: I do not like it

            Yup, I've sent him a message

            → Reply
          • »
            »
            »
            »
            »
            »
            4 years ago, # ^ |
              Vote: I like it +11 Vote: I do not like it

            I got a response from him. To summarize:

            • He doesn't think that the new changes in rules (ratings for new contestants) should impact the predictor in any major way
            • The issue that I faced could be due to a delayed propagation (according to him, it seems to be taking 5-7 minutes to reflect results now, while it used to take 1-3 minutes pre-Covid, when there were fewer participants)
            • He's hosting this on a free tier of a cloud platform so resources are quite limited. This may be the reason for the performance impact.
            • He's investigating a potential fix, though it isn't certain that this will resolve the issue.

            So let's wait and see. Until then, assume that ratings lag by 5-10 minutes when there are as many participants as there are these days (~20k+)

            → Reply
            • »
              »
              »
              »
              »
              »
              »
              4 years ago, # ^ |
                Vote: I like it +16 Vote: I do not like it

              Great, thanks! I think the lag might not be the reason, as for the last round I am still seeing +345 predicted for top 1, while he got +344. And the difference is more extreme for less-rated users.

              P.S. Actually I am surprised it is still not a default CF feature. And some $100/year from crowdfunding could definitely rent a top hosting for such project :)

              → Reply
              • »
                »
                »
                »
                »
                »
                »
                »
                4 years ago, # ^ |
                  Vote: I like it 0 Vote: I do not like it

                For sure — both are good ideas. A near real-time rating indicator is a valuable feature. WasylF has already done some of the work required. Would be great to see this as part of the core feature set.

                → Reply
              • »
                »
                »
                »
                »
                »
                »
                »
                4 years ago, # ^ |
                  Vote: I like it +8 Vote: I do not like it

                Thanks for using CF-Predictor :)

                Yes, there is an effect from new rating rules. It's quite small for existent contestants, but significant for new users. MikeMirzayanov has told, that he's going to add trueRating field to API, but this hasn't happened yet.

                So, I'm just waiting for it. After this field become available, prediction be accurate for all users. New users will see actual rating rather than displayed rating. Not sure which one is more interesting, but actual rating is easier to compute :D

                About update: I've published a new version (1.3.0) of the extension in Chrome today. I expect it to work a bit faster, Will see when more people update to the new version.

                As for Opera and Firefox: I submitted extension to Opera 2 days ago, it's still on review stage. I'm going to sent it for Firefox on weekends, because Firefox want me to change something in the extension :(

                → Reply
                • »
                  »
                  »
                  »
                  »
                  »
                  »
                  »
                  »
                  4 years ago, # ^ |
                  ← Rev. 2   Vote: I like it +16 Vote: I do not like it

                  Oh, and I temporarily added one more backend, so to use new version of the extension you'll need to accept new permission.

                  This fix is more about stability/reliability, rather then performance improvement. In a couple last rounds backends were down in the end of round, because of high QPS. I hope this update should fix that issue.

                  → Reply
                • »
                  »
                  »
                  »
                  »
                  »
                  »
                  »
                  »
                  4 years ago, # ^ |
                    Vote: I like it 0 Vote: I do not like it

                  Awesome work, thanks for doing this!

                  → Reply
»
5 years ago, # |
  Vote: I like it 0 Vote: I do not like it

It mainly happened because div2 C was solved by quite less number of participants. It is rated 1800 as you can see...So if a 1600 rated participant is not able to solve the 3rd problem and he solves the first two quickly, then his rating will definitely increase

→ Reply
»
5 years ago, # |
  Vote: I like it 0 Vote: I do not like it

I think because the number of participants during the quarantine time is high. So the more the under-rating-participants there are in the contest, the more rating you will increase after the contest

→ Reply
»
5 years ago, # |
  Vote: I like it +1 Vote: I do not like it

And I thought it is the power of FAN CLUB!!!

→ Reply
»
5 years ago, # |
  Vote: I like it 0 Vote: I do not like it

I think maybe it's base on your rank ratio. Ex: your rank is 1000 and we have 10000 participants so you have rank ratio 0.1, it's same as you archive rank 100 and only 1000 participants in the contest.

→ Reply