Rumination on AI and competitive programming

Revision en2, by ko_osaga, 2024-09-16 01:30:44

An OpenAI model can now achieve a rating of 1800 in Codeforces. That made me wonder about the future of competitive programming and why we do this. I came up with some opinions, which I'll be glad to share with you.

AI killed the competitive programming?

I don't think that's happening, because competitive programming is useless! Some people might think CP is less interesting because AI can solve CP problems — but come on, problemsetter can solve those problems, unless they are really bad. Many people still play Chess just fine, and I recently enjoyed participating in Sudoku competitions, although I was completely aware that computers would wreck me in a second. It didn't matter, and I didn't see how it could.

CP problems are not "open problems" — if they are, the problemsetters are to blame. So, you are not contributing anything to the world by solving CP problems; it all comes down to personal interest. As long as CP is fun for people, they will do it.

Flagship competitions such as IOI and ICPC will stay

They are onsite competition — they can stop people from using AI to solve problems, and according to their records (IOI banning all prewritten codes, ICPC only allowing teamnotes), they will. They will stay as is.

On online competitions

I want to note that the situation is already quite dire for those contests. GCJ is gone, MHC is still not recovering from virtual, Topcoder is gone, and AtCoder is less frequent; it's no secret we are going downhill.

I also think the contest integrity is already under attack, especially in the last five post-COVID years. No need to mention thousands of low-rated cheaters in CF. High-rated people also cheat (1 2).

Given all the records, a nonnegligible amount of people, regardless of their skill level, will try to cheat in various online contests with AI. That is not good news for CF and AtCoder, which partially or fully bans the use of AI.

Another argument is that it is still fair game if everyone uses it. For example, Internet search is allowed in CF, and notably, I took advantage of that. To whatever extent how it trivializes CP, everyone can use AI nowadays; what's the issue?

I think allowing AI is a fair game, but I wonder if it's a good game. Some issues I have with it:

  • CF skill might be very independent of one's skill for IOI / ICPC.
  • For most users (and probably later for much higher-rated users), CF will lose its status as a skill barometer, and this situation will quickly worsen.
  • The game might be unfun, which is a very serious issue.

One way to delay these impacts is to propose problems that AI can not solve by now. I think this is a bad trade.

  • It would be a matter of time before AI solves those problems anyway.
  • AI solves short, easy, and typical problems very well, such as problems on ABC. Those contests are crucial for beginners. If one bans such problems, it's impossible for beginners to understand what the game is like.
  • It will limit lots of fun problems. The one problem that AI did relatively terribly on IOI is Mosaic — but not a lot of people will consider that the best problem of this IOI, let alone good problem.

A problem should be judged independently over how well AI can solve it, and it's the right call to ban AI in online contests. Then, a CF rating will "really" be a number, similar to the online judge rankings. You can say it already is, but I bet I never saw a single high-rated user who literally considered it as just numbers without an ego attached, so get ready for some changes.

Do you think CP is useless?

If I'm being sincere, probably not.

One clear example is interviews. However, most corporate interviews don't ask about CP problems because they do CP at work. They will figure out some way if AI cheat is prevalent, so not our business.

Ok, those are boring talk. Why do people like competitive programming? I would imagine the following:

  • They think some ideas/implementations in CP are beautiful.
    • Under whatever definition of "beautiful" — for example, because the logic is not intuitive or has various applications, or it makes you question more, et cetera.
  • They think training in CP would have positive consequences for their interests.
    • Research or software engineering
    • Desire to be smarter
  • They just like it when they win.

The AI revolution will harm the third point as they will win less, but that's ok. That's not a healthy mindset, anyway.

For the other points, it even seems that competitive programming where we have AI assistance is a better idea, as long as you are using it for practice only — you can ask and learn cool new things from AI, as it will gather thousands of information and pick you the right one.

Should CP change?

Problem-wise, no, I think that's a bad idea, as I wrote earlier.

But we will see some changes. I predict that AI will contribute to the slow demise of online contests by harming their integrity even further — Many people, including me, will miss Codeforces, where the rating isn't a completely random number, and admire tourist for having such a high rating, so this is indeed a bad thing.

I also think the idea of "infinite resources" makes less sense. Before AI, I thought a contest allowing all possible human resources was an ideal model, and we should judge our skills based on that. Now, I feel that there should be a limitation to this, as what IOI and ICPC have been doing. I'm not entirely sure about this.

On the bright side, you should think of the rating as just a number, so this change might be a good thing, as it makes us keep the right mindset for competition.

Tags koosaga is crazy

History

 
 
 
 
Revisions
 
 
  Rev. Lang. By When Δ Comment
en2 English ko_osaga 2024-09-16 01:30:44 7 Tiny change: 'owed in CF anyway, and nota' -> 'owed in CF, and nota'
en1 English ko_osaga 2024-09-16 01:28:41 6001 Initial revision (published)