The problem of cheating has been addressed many times in blog entries, many cheaters being reported. However, it seemed to me that few solutions were actually given.
Therefore, I want to share my thoughts about how codeforces could prevent such behavior in the future: I think that, besides automated check (which also probably could be improved), there should be some sort of report mechanism, through which anyone could report a user for cheating — this could be implemented as a simple report button beside a standings line. Adding a short text or an image/link to an image to the report could also be an option.
These reports could be handled by some trustworthy members of the community (high rated/contributing users). Even some sort of two-step verification could be implemented: after the first filtering of the trustworthy users, there could be a final check by someone from the codeforces staff (The first filtering phase will most probably reduce the quantity of reports). Some automation to this process could also be implemented: If some user receives a large number of reports, then he will be automatically considered cheater.
Finally, after a user's participation in a contest has been decided as cheating, I think some action must be taken (beside just eliminating him from that contest). One example could be automatically decreasing his rating (or contribution) by some value, or having a "1 rated contest ban", meaning that his next contest will not affect his rating, or implementing a new thing just like contribution and rating, measuring the honesty of the user.(Personally, I tend to be against the latter, unless it provides some sort of mechanism for "rehabilitation").
I know that this system has also some flaws, like: It is hard to remove cheaters from past contest (if the rating was already calculated, if you remove a single user it affects the rating changes of all other users by some small value). Also, for new contests, it will require some delay between the testing phase and the rating changes, a phase in which the system is expecting the reports from regular users (as the ones that are written in blogs after contests).
What are your thoughts about the solutions that can actually be implemented in order to find and discourage cheaters?
PS. I know there are a lot of blogs exposing cheaters, but in a quick search I could not find a blog talking about solutions. There were several solutions proposed in comments, but I don't remember in which blogs they were posted. Also, from what I remember, most of the solutions were about some automated check of copied solutions. Probably some similar solution to the one posted above was also posted before, but I cannot remember.
"Some automation to this process could also be implemented: If some user receives a large number of reports, then he will be automatically considered cheater". Please don't do this, alt accounts still are a thing y'know....
when I say large number of reports, I mean large, like 100 or so. I doubt someone has so many alts.
They may have so many bots though
Or you just say something that a lot of people find sufficiently offensive, and they all report you.
100 ppl can report tourist just for lulz and he will get banned lol
This is true, however, if a user submits many unfounded reports, they could lose this privilege
codeforces has over 0.2e6 users as far as i know, and banning all the clowns would take too much resources
Instead of automatically considering them cheater, any submission that gets enough reports should be reviewed to just make sure those reports were genuine.
My favorite idea was from SHZhang involving running some kind of sting operation against cheating groups, and posting fake solutions for them to get penalties on.
This probably causes its own complications (such as the sting group leaking problem hints itself) so I'm not sure if it's actually a good thing to do or not, but it'd definitely be fun :D
I think it is the kind of solution that is really cool but not actually useful.
hmmm, why not? I don't think the penalties would actually be the thing for cheaters to worry about, but if solutions in a cheating group were "diluted" enough, the main damage would be that people wouldn't be able to copy and paste solutions without thinking about them.
this assumes that the large cheating groups are the primary issue though, as opposed to small rings of cheaters.
A cheater could just recode the solution to his own style. This is enough reason to not do that.
I think this could actually be useful in the following scenario: Every time a new contest is proposed, the preparators could add a special test (which is not pretest), and then submit fake solutions to cheating groups which fail only on that specific test. Then, every submission that fails that specific test on systest will be considered cheat.
Creating such a test and (slightly but not obviously wrong) source code depends from problem to problem, but in general I think it is achievable with minimal effort.
I really think that a permanent ban for a user who cheated is a good option. After a submission is diagnosed as plagiarism, the author should recieve a e-mail, which has 2 our 3 days to answer. If he prooves he is not cheating (if he is using 3rd party code for example) the contest gets rated for him, otherwise he gets permanetly banned, and possibly a "mark of shame" on his profile name.
There is no excuse for cheating. You are making others expirience worse while you are just lying to yourself. Creating a second account would still be an option to the cheater, but all his work will be lost. This may actually replace the need of a phone number check or something.
Well, thats mine opinion.
''Mark of shame'' sounds funny
Yes I agree to the "mark of shame" idea. Maybe a RED dot (in the contest they have cheated) in their rating graph. Then they will not be able to show off their ratings to their recruiters. That's a very good idea in my opinion.
I don't think bans matter since it's so easy to create a Codeforces account. We would need to make it harder to create accounts (tie it to govt id, or a phone number), which isn't a popular idea afaik.
Actually, they matter. Imagine you've been cheating to reach CM. Now, imagine you got caught. All the trouble you got to have a fake CM rank got discarted. This will certainly discourage you to create a new account and cheat again.
A lot of cheaters are low ranked, though. So resetting their cf rating doesn't hurt them that much. Rather, we could just lower their rating without banning their account. I think perma ban is an idea that seems good in theory but is terrible in practice — sites that implement this end up with some people who consistently create new accounts and have no attachment to them. They may like cheating for the thrill, or they think they can cheat smarter, or something else, but they keep doing it and since they don't have any attachment to their account, it becomes impossible to monitor them and make their antics less successful unless you increase surveillance on ALL new accounts. We should strive to ensure that users have an attachment to their accounts so that a punishment or a normal ban or rating means something — perma banning can cause real problems because it makes accounts meaningless for the banned.
Let's do it in the university way: If a user is found cheating, just put a "cheated" tag on their profile page, then ban that account. Consider the fact that they need to put their real names and address on the account in order to show them to employers, having a "cheated" tag basically renders those accounts useless. Also, let say they even if they get away with cheating in some rounds, later on they can still be caught and have that "cheated" tag on their accounts (which is even worse since they have already applied to companies using these accounts).
To make the above work even better, Codeforces should also have a system to report cheating, and probably high-rated users can approve those reports (let's say >= 3 or 5 approval) and after that those submissions will automatically be considered as cheated.
P/S: Not sure if I'm gonna get downvoted by cheaters for this. ~~ Praying for my contribution ~~
P/S 2: Very recently, there was a case of cheating that I know. The guy had his face as his avatar, had his name, address, gmail, school everything on the account and then he was reported cheating in a blog. Mike then disabled the account, which? Which effectively made his cheating record permanently exist on Codeforces, and what's worse? It can be googled so easily, even worse that Academic record where you only hand it to your employers.
Adsitionally, name in the profile should be required and not changable. And also birthday, to avoid collisions (there are too many cheaters from one specific country, only names are not enough).
A big issue is then using somebody else's credentials as a joke.
Agree. Yet nowadays defamation, if someone really wants to do so, is always possible by setting up a new account, get it to Orange++ then cheat / post some stupid things then get banned. But anyway one can always say it’s a defamation case, as long as he hasn’t submitted that account to employers.
I think publishing the cheater's names and handles in a high viewed blog post or a separate "leaderboard" is the way to go. No better deterrence than public humiliation.
"wall of shame"
As a high rated user, I simply don’t care about cheaters. They don’t affect my rank and cheating related blogs pollute the blogs area from discussing algorithmic problems. I just don’t want to see and care any of these.
They rate at which the number of cheaters are growing, you will have to care about them one day.
Yes, the cheating discussion concerns me right now.
As a community member, I simply don’t care about your egotistical opinion. It doesn’t contribute anything but pollutes the comment section from meaningful discussions.
Once you grow up, you'll learn that the world doesn't revolve around you.
I am contributing by give reasons why high rated users might not care, which is a part of the discussion.
A separate place for cheating discussion is good, and my suggestion is to make it hideable for ones who are not interested.
If you could phrase your original comment like this, it would have been completely fine. Codeforces blogging system is obviously far from ideal and we have no choice but to ignore what we're not interested in.
I could be terribly wrong here. But I think we need to address the root cause.
People cheat because high ratings and rankings have some incentive associated with them, I think that should be changed.
In principle, one way to do it might be — On all rating/ranking pages, all OJs like Codeforces, Codechef, Atcoder etc, should write in bold caps that THESE RATINGS DO NOT INDICATE ACTUAL SKILL BECAUSE ONLINE COMPETITIONS ARE PRONE TO CHEATING
While this seems like it won't change anything at all, I hope, it would cause companies to consider the fact they shouldn't be hiring/offering interviews based on OJ ratings.
They have certificates of onsite competitions.
I think you have been placed already. Devoted u successfully. Btw Happy New Year demoralizer
I am in 3rd year, so not placed yet.
:(
But getting good ranks inn onsite competitions is like really hard. So basically, you're saying that only super skilled people's achievements will be recognized? What about someone who is at 2000 rating or so? Even that takes effort to reach, but usually isn't enough for good performance at on-sites.
If they stop considering ratings, they'll come up with something better, that's hopefully less prone to cheating. The current system benefits cheaters by a lot and encourages literally everyone to cheat, which makes it not about skill at all.
In the current system, non-cheaters are actually just less clever.
Sad but true, it seems
Though, I don't think you should call non — cheaters "less clever", thats sad.
If the system continually keeps on hiring cheaters, obviously it's encouraging people to cheat.
It kind of sends out the message that cheating is an essential life skill and people who can't cheat aren't fit to work in their company. (Because companies also know that there's a lot of cheating in online Competitions, they aren't blind.)
So basically, I'm not calling them "less clever", companies that hire based on cheating prone competitions are calling them "less clever".
Sad and true.
What way would you suggest, if you were the CEO etc of some company, BTW, to curb cheating?
If someone is caught cheating, say more than once (it is hardly a coincidence if someone is caught unintentionally plagiarizing twice or more) then maybe their accounts can be disabled/closed with a "mark" on their profile which states the reason why their account was closed. (It is similar to what the online chess website chess.com does.)
Maybe making a compulsion for the users to mention their name and country and having a verified email address (even when someone changes it) would help a bit too (since it may make it a little less easier to create alts).
Here is a simple algorithm.
if(user == cheater) delete(user);
I don't trust people who write normal sentences in pseudocode.
You shouldn't trust random strangers on the internet. Whether they write in pseudocode or not.
Fun idea: Take inspiration from other area, such as fighting cheaters in online gaming.
Fall Guys secretly adds "Cheater Island" where hackers are forced to play against each other.What if we add cheaters to a "Cheater list", and whenever they compete, their ratings only change compared to others from the same list?
In Crysis Warhead, the cheater's guns shoot chicken instead of bullets and that person can't kill any enemies.
Analogous to this, the judge shows "WA" or "In queue" on submission.
All that matters to cheaters is the rating that they can show and gloat about. CF already has a feature that for the initial contests, the user does not get to see his true ratings (for 6 rounds). If we do make the next few rounds of cheaters unrated, they won't be able to gloat about it and it'll have a direct cost attached to them since they won't be able to gloat about their ratings for the next month or so. And for a cheater, that is a long span of time since we've seen that they are rarely persistent enough
why not directly IP ban, if done consecutively,
also lets say X amount of contest happen on codeforces in one month, randomly on one contest there should be rigorous and manual checking with IP BAN,
also there should be incentives like CODEFORCES COINS( like if you collect 1000 coins, you will get 10 coins, but both should be from different ip address).
IP bans don't do anything. Often IP addresses are so dynamic that you can circumvent the ban without doing anything. And on the other hand you may accidentally ban everyone from some dorm.
Also, no "Codeforces coins" please (whatever they do).
Duplicating my suggestion from another thread:
What about developing automated checks that, instead of looking for similarities between submissions for different users, look for discrepancies and anomalies between submissions for the same user? Sure, code style changes, it changes especially often for beginners, sometimes you copy-paste code from a prewritten library with a different style etc. — but still, in a lot of cases there will be stuff that is very suspicious, and I believe modern ML magic is strong enough to highlight a lot of that. To give some random examples, I don't need to find where the code was copied from in order to be suspicious about a user who suddenly decides to use 3 different languages in the same contest. Or, for example, I'd be interested to know motivation of a contestant from, let's say, Egypt, who suddenly decides to use Polish words for variable names in one of their solutions.
I am not seriously involved in chess, so I don't know how well that stuff actually works there, but I've heard that modern online chess are already relatively good at detecting suspicious users (e.g. the ones who may be using chess engines).
Yes, I saw some users' submission times. 0:59 (A) , 2:30 (B), 2:36 (C), 2:41 (D), 2:50 (E). I mean how come someone solve B,C,D,E suddenly if it took them 1 hour to solve A, and that too newbies. ML algorithms can surely detect these absurdities
However, someone accustomed to CNOI-mode contests tend to work on many probs at once and submit them in the end.
This is a talk by the creator of Lichess, he talks about how they handle cheating around 17:50. He claims they have it largely under control, but a very large amount of effort seems to be needed from the side of Codeforces to do this. Does Codeforces have the manpower or the know-how to do all this? Is cheating generally enough of a problem to warrant it? I've only been doing CP with some seriousness since August, but it's the first time I've heard of such a widespread occurrence of cheating. I'm inclined to think it's just because people have holidays and nothing better to do.
A really nice idea but the use of code libraries may complicate this. Specially if the library they use is 80-90% of the submitted code, the model may detect anomalies.
[DELETED]
By this logic we should not take any action and just let many people cheat.
I agree that there always be people who will try to cheat, and try to find more complicated solutions to cheat that will not be detected, but I think most persons will be discouraged from cheating if some minimal anti-cheat system will be implemented, and also some sanctions will be applied to cheaters.
3 skips on an account would result in perma ban.
one can only make new accounts upto their tolerance.
Skip submission appeals to be allowed for people who are at least experts.
( Even if a wrongly accused <= cyan was to get banned due to this.. it won't take them long to make it back up provided they are not cheating )
my opinions on this topic.. people can feel free to disagree.
My suggestion would be implementing a credibility rating . It would be displayed on the user page and would be a score between 1 and 0 . A high rating would mean that the user has not been caught or suspected of cheating. On the other hand a 0 would mean that the user has been caught cheating . Coming to the implementation, The score would be implemented using the seeds decided before the round. After the round based on the user's rank , his score would change . I there is a large positive variation between the seed and rank , his score would decrease. For example, if a user is seeded 6000 but attains a 3000 rank his score would take a hit . On the other hand if his rank is lower than his seed his score would be unchanged . The proper way on deciding the amount of change in credibility score would be decided by the admins. Another way a user's score would decrease would be if his/her answer is plagiarized and is caught by MOSS . Now coming on ways to improver one's score, There would be none . Instead of a user improving his/her score , the score would heal automatically over time i.e. after a period of time( 1 year or 6 months) the score change due to a contest would be rolled back. In this way the user for that period would be disabled from using his account publicly for that period. I think this method will work because a low credibility score would stop anyone from adding there codeforces ID in their resume during job application . This means that an employer would not hire a user with a high rating but low credibility score because he/she knows that the rating s false and was probably gained by cheating. Another bonus would be reporting accounts . Currently codeforces is unable to do any sort of report mechanics because of the presence of the alt accounts by many user . However by using credibility score as a sort of check , a feature to report user can also be added. Anyways there are my thoughts and would love to know what the rest of community would think about this . Tagging Errichto , Monogon and MikeMirzayanov to know what they think about this.
P.S> Please overlook any mistakes in the grammar.
don't tag me if it's not related to me -_-
I think codeforces should try to cut the motivation for cheating i.e rating (show off) . Make rating private only the user will able to see it and not able to share it. I know that this solution would demotivate many people to do CP. but those who do cp because they love it will not be demotivated.
It's probably impossible to have perfect solution for catching cheaters.
Almost every cheater is an Indian who think just having high Codeforces/Codechef rating will give them high paying job.
May be alerting participants at time of registering that "no one give you job just because of your rating can reduce number of such cheating cases".
If there are already enough people who got this belief from somewhere, to me it sounds very optimistic to hope that such alerts will make people change their mind.
I don't think Codeforces should alert people about such things, especially if it is not a problem in most parts of the world.
More than half of participants are Indian now-a-days and more than 90% of them do it only for job. It may not a problem in most parts of the world, but it is problem with around half of CF users.
AFAIK there are 2 ways cheating happens. There maybe more.
Someone solves the problem and shares his code with his friends/telegram group.
Someone innocent uses Ideone where by default his submissions is public. Cheaters scrape recent ideone submissions, finds the AC solution for the contest and shares his solution with his friends/telegram groups.
I can't suggest anything for first type since we already have too many suggestions for this — Better detection, banning groups etc. But I have few suggestions for second type.
Make Ideone submissions private by default. I know this is far fetched since codeforces isn't related to Ideone but I think Codechef and SPOJ are in some way related to ideone. And since plagiarism is also a very big issue in Codechef long challenges maybe they will consider it.
Block recent submissions API & page on Ideone so that scraping becomes hard. Allow sharing by link as default option. Once again far fetched solution.
Codeforces feature request 1: A way to test given testcases on problems page so that less users are forced to use online compilers. Just like hackerearth, topcoder has their own compiler and run button for given testcases.
Codeforces feature request 2: Report button for every submission like this blog suggested. Similar to what leetcode has.
Does Codeforces have a feature that allows you to report solutions?
There is a feature to hack solutions (produce a test case which makes the code fail), but I am not aware of a feature to report plagiarism in contest.
It's not there as of now. That's why I said feature request.
Compile and test would make queues longer.
Report a cheater
1.Just give option to report a cheater. The community is enough to tackle these cheaters.
2. Mike should assign some people who can give judgment of these cheating claims.
3. Give them some punishment to discourage them. (ex ban them from participating for 3 months or 6 months.)
set the rules or this will keep happening.
I think that we should focus our efforts on what concerns the Codeforces platform itself.
First, let's notice that "cheating on Codeforces with the intent of having a better chance at being hired" is the equivalent of "lying on your curriculum". There are no solutions to catching curriculum liars, beside somehow making them go through some hiring test. That's what the companies should focus on, and it's outside the scope of Codeforces.
What concerns Codeforces is that the cheaters' actions are hurting the community, by making it an unfair competition. One may argue that I shouldn't care about my positioning, and that I should care only about my learning and self-development. I agree, however we all know that this is a competition, and the feeling of getting more points than my competitors make the whole experience more fun.
I think reports could be implemented similiar to Recent actions tab. When a submission is reported, it would be displayed publicly in the Recent reports tab (sorted by the number of reports) and anyone will be able to check the reported submission. Similiar to contribution, a submission reported by higher rated users should also receive higher number of reports. Other features could be added to further improve the accuracy of reported submissions, e.g. 10 contest participations.
One thing to consider is the people who cheat mostly have low ratings. So, who is the one providing them solutions?. Probably some high rated people. A newbie or pupil can't solve E or F on his own within minutes. Probably a CM or Master user is providing them solutions. We need to track the peoples whose providing solutions in telegram groups or elsewhere.
Many people in the above comments have suggested some awesome short term solutions.
Most cheaters want high ratings to show off to recruiters/ put on their CV/university applications. It's about survival of the fittest, getting ahead of the competition in most cases. They want to increase their chances of getting into a place which has a high rejection ratio.
This is a very long term approach but leaders of the world must really think about keeping the population under control. There is simply no way jobs created are going to be able to match up with the rising population. Even if enough jobs are going to be created, a lot of them will be about scamming other people ("teaching skills to kids") in areas with high population density, rather than adding actual value to the country (let's say building a search engine optimised for local communities or some tech that revolutionises healthcare, etc).
Bill Gates in one of his AMAs had said something similar too (I don't hold this opinion just because he said it though :P)
Sure, when there is increased competition, it is better for everyone since competition helps people improve. But when there is a lot of competition for something, the people at the bottom start feeling hopeless about doing well and the folks who are hiring need an easy and cheap filter to select a small number from the large pool of applicants.
Just think about the scenario when there would be way fewer applicants: if the hiring guys want, they can still keep a test to assess problem solving skills, but it would just be a part of the judging. Each candidate could be judged by skills required for that specific job by asking to do a small project, conducting face to face interviews, etc. The University guys, could also conduct their own tests/interviews with applicants.
Downvote if you find this absolutely irrelevant, but I wanted to speak about this since a long time and did that today.
I couldn't find an actual suggestion in your spoiler — just a few would-bes.
How about asking every user to add a phone number to their CF account? People will think twice before cheating and whenever a cheater gets caught, he will then have to use a new phone number when creating a new account and will eventually have to stop, and this will also limit the number of alt account (like this one lol)
When a cheater gets caught, Mike will text them that he is disappointed.
I thought we were going to discuss this.
Obvious attempted fraud should result in the account being blocked.
However, what is "obvious" would be needed to be carefully defined, and there should be some human involved if the blocked account has a more or less long history. But I do not see why blocking/removal of fresh cheater accounts could be a bad thing.
Link govt. Id to account. This way alts will not be created and cheaters would have fear
WTF
What? I believe it is the best solution. There should be no rating system for the accounts that are not linked. In most countries, govt issue id card and for countries which don't they should at least be able to link phone numbers via otp.
please do something about cheating otherwise there won't be any meaning of us giving contest on cf
Fundamentally, we need to find a solution to reconcile two hard (economical) facts:
#1 It's labor intensive to identify a cheater while minimizing false positive.
#2 It's hard to deter a cheater if s/he can always create a new handle and cheat to gain 200-300 points more than s/he deserves in just one contest.
A good solution won't be perfect, but should significantly reduce the cost of enforcement (#1) and increase the cost of violation (#2).
Here are some additional ideas of finding the balance:
#1. Reduce the cost of enforcement:
Introduce automated check, but add additional filters to reduce false positive, including: 1) only applying automated check on harder problems (for example, after Div1C/Div2D), 2) only flag when the solution suspected is way out of league of a user's rating (a 1500-rated user solving a 2000-rated problem)
Once flagged, the burden of proof shifts to the flagged user to explain why his/her solution is materially similar to another (earlier) solution within 3 days. This step can itself weed out a lot of real cheaters since they will be more reluctant to respond and they will have very little to explain. Conversely, in false positives, an innocent user can find ways to defend him/herself.
if a suspected user responds, then a panel of trustworthy members can vote based on evidence. (Note the first two steps can be done automatically so the burden on human inspection will be greatly reduced.)
#2. Increase the cost of violation:
Also, we don't need to sacrifice the timely post-test rating calculation and change (which is liked by many). We can still keep the current practice but only change the offender's rating as an offline penalty. Yes, it's not perfect and may cause a little rating inflation, but we shouldn't pretend we can flag 100% of cheaters with 0% false positive, so any imperfection already exists today. But if we can have light-touch enforcement and rigid penalty in place, we will most likely live with much less cheating happening without detection.
That should at least be raised to 2400 because it is possible for a 1500-rated person to solve a 2000 problem.
I have solved 1454E - Number of Simple Paths, a 2000 problem by myself when I was rated 1481.