Hii
Recently I resumed giving contests on Codechef and I noticed something very strange. If you look closely at the standings and the delta that they get after getting that range, you will notice it. For example lets take the standings of yesterday's Starters83B (Starters 83B Standings).
- Rank 2: 3★jointquad (1797 --> 4★jointquad (1879) +82 Ratings
- Rank 5: 4★ryotori (1859)--> 6★ryotori (2262) +403 Ratings
- Rank 6: 3★ohameow (1723)--> 5★ohameow (2104) +381 Ratings
- Rank 9: 4★moscow_wale12 (1917)--> 4★moscow_wale12 (1988) +71 Ratings
Comparing this to CF's rating pattern, you will find these changes strange. Someone is getting a direct jump from lower 4star to mid 6star in just one contest and someone who scores even better doesn't even get a star change. It seems unfair and weird.
Though ratings doesn't matter much to many people but sometimes you feel bad when you get such a less delta compared to people below you getting direct upgrade to 6star in just 2 contests.
Can you explain these changes? I think earlier it used to be better...
Btw Starters 83 had some really great questions.
I'm not entirely sure of the exact formula but this is what I suspect:
Pretty much Codechef's rating system retains an uncertainty value with a user's first few contests: this is why accounts with under 5 contests completed will have a provisional rating. Second is that the initially displayed rating a new account starts at is a lower bound on what the rating system assumes is the user skill. Similar to what Microsoft's TrueSkill and CF's rating system does, new accounts visually display a rating lower than the actual internal rating used for calculations initially, and slowly bring the visual rating up to its actual value after a completing a few contests. This is so that newcomers normally feel a sense of progression in seeing their rating go up compared to being placed at the median and almost certainly losing rating on the first few contests. Codechef in particular likely uses 1000 as the initial displayed rating and 1500 as the internal rating for calculations. Furthermore, during the first few contests, I'm pretty sure Codechef will result in rating changes being amplified to more quickly bring new users ratings to match their perceived skill.
All of this means that a relatively strong coder who creates a new account on CodeChef can experience very large rating increases with even decent performances, whereas other users that did equally well with a similar visual rating may gain under 100 rating since with 10+ contests done, the system is more "confident" in their rating being accurate. The really wacky part of this is that the system's level of confidence in a rating is solely based off the number of contests completed, once you've done about 10 or so I'm pretty sure that's when the system reaches maximum confidence meaning that most contests your rating change is limited to about +-70 points. So in a technical sense, you could have an account with over 10 completed contests, with the last one being 18+ months ago, then spend 18 months learning various competitive programming techniques, then go back to CodeChef to do a contest for the sake of it, realize your account's rating hasn't updated to reflect your current skill level in the slightest, proceed to horribly outmatch almost everyone in a Division 3 contest due to you rating being inaccurate as all heck, and then only gain 62 ELO from placing 2nd out of 3353 contestants, all while 1st place was a new account that had only done 4 contests before hand and thus due to CodeChef Rating system being wack ends up gaining 609 rating.
That last point may or may not be relevant to myself getting trolled by CodeChef's rating system.
Most of your intuition is correct. But not "The really wacky part of this is that the system's level of confidence in a rating is solely based off the number of contests completed". It is instead based off consistency, as mentioned here. So in your case, if you give a couple more good contests, the rating will jump up faster. You can read more about the model, here.
Don't you think that's not fair?
In my opinion your current rating changes should not depend on your previous performances, rather It should be more intended towards the performance vs the current rating that you have (Like in case of codeforces).
I believe codeforces model is better, Because its frustrating to get stuck in the same range even after performing good in 2-3 codechef's contests. and I think that's the main reason why some people choose to use new Alts to give contests on codechef (which can also be clearly noticed in div4 standings lol. sometimes the most difficult question has same number of accepted in div1 and div4 and very less in div2 compared to both)
this is what happens when 3-star rated coders propose their ratings system
You can also become 7* (and 3rd best in the country) by giving a total of 6 contests, which are only Div 4, 3, and 2 (not a single Div1) (https://www.codechef.com/users/aayushjham). Note that it isn't provisional rating either, it's the confident one.
Or you can also get similar results by giving 2 Div4 and 1 Div3 contests as well (https://www.codechef.com/users/cthau).