Ahmed05's blog

By Ahmed05, history, 3 months ago, In English

When I was reading all the blogs about AI and saw many people doing a parallelism with chess, what firstly came to my mind was that in chess, cheaters do not impede you gaining rating normally. So isn't it possible to change codeforces rating calculation in such a way that the number of people who solved the problem doesnt affect it? This would lead to a big rating inflation but would also solve the cheating issue everyone is complaining about.

  • Vote: I like it
  • -25
  • Vote: I do not like it

»
3 months ago, # |
  Vote: I like it +1 Vote: I do not like it

Number of people who solved the problem is the best indicator of the rating of the problem.

»
3 months ago, # |
  Vote: I like it +1 Vote: I do not like it

The number of people who solved the problem already doesn't (directly) affect any rating. Your rating change is calculated based on (a) your rank in the contest and (b) the ratings of every participant before the contest.

What, specifically, is your suggestion? And why would it lead to big rating inflation?

In principle, it could be done. Give each problem a rating and calculate the rating change based only on which problems you solved and how fast you solved them.

In practice, this requires accurate estimates on how hard a given problem is. This is something we have historically not been very good at (some B-s in AGC turn out to be 3000+-rated). You can't use the ratings of the people who solved the problem, either (because relying on "who solved the problem" is something you are trying to get rid of).

  • »
    »
    3 months ago, # ^ |
      Vote: I like it 0 Vote: I do not like it

    My suggestion is exactly what you explained in your third paragraph.

    When I said "number of people who solved the problem" I refered to rank, that is directly related to how many solved more problems than you, or did it faster.

    I consider that if we could estimate precisely the rating of each problem before the round, and calculate rating change based not on your rank, but on how many problems you solved (as you explained), cheating would not be a huge problem anymore (or at least it wouldn't affect other participants).

    The rating inflation would surely come since many people are having a performance that would be good for their rating months ago, but now their rank is lower because of lot of cheaters. If we follow my suggestion they would also increase their rating, so gaining rating for low-rated participants would be easier.

    I don't have the knowledge to say if this suggestion is plausible or not, but at least should be considered I guess.

    • »
      »
      »
      3 months ago, # ^ |
        Vote: I like it 0 Vote: I do not like it

      It's plausible in principle, just very different, so I'm not sure how well this would be received. Estimating ratings is difficult as I said, so occasionally we will receive free rating boosts or undeserved rating drops.

      that would be good for their rating months ago, but now their rank is lower because of lot of cheaters

      Cheating is not a problem that appeared in recent months. I do not believe that it even got significantly worse in recent months. Unlike many people, I am not so sure that the rating drops are so large, either. I would estimate that at worst, your rating is reduced by -50. And that's only if your level is slightly below "typical cheater performance". If your rating is much lower or much higher, you shouldn't be affected at all.

      Unfortunately, cheating is one of these things where people write a lot, but no one comes presents actual data. So you see blogs like "lots of cheating with o1-mini today" that present a grand total of one example.

      • »
        »
        »
        »
        3 months ago, # ^ |
          Vote: I like it 0 Vote: I do not like it

        Probably the amount of cheaters has not increased too much, but the number of problems they solve did increase.

        When I look back at past contests (like 1 year ago) it's rare to see a Div2 D with more than 2k solves. Now practically every div2, problem D and even E are leaked. In Div 3 almost every problem is leaked during the contest, and the amount of them that get caught is low.

        As I am experiencing it personally on recent months, I can't agree that the cheating is not getting worse

        • »
          »
          »
          »
          »
          3 months ago, # ^ |
            Vote: I like it 0 Vote: I do not like it

          As I am experiencing it personally on recent months, I can't agree that the cheating is not getting worse

          It's kind of hard to take this statement seriously because you started actively participating 3 months ago. Of course you didn't personally experience it before, you didn't participate!

          In July and August of this year, there was only one Div. 2 round (not counting Educational) where problem D got more than 2000 solves among official participants: round 956.

          Data

          The numbers are a little lower in July and August of 2023, but the numbers vary so much even from one contest to the next that it's hard to really draw many statistical conclusions from it. The difficulty of problem D is not uniform across contests!

          • »
            »
            »
            »
            »
            »
            3 months ago, # ^ |
              Vote: I like it 0 Vote: I do not like it

            You are right, difficulty of problems is not uniform and discussing about that doesn't lead anywhere, but it is true that nowadays more problems get leaked than years ago, which leads to cheaters having better performances.