Hello Codeforces.
I'm writing this post to make comments about a situation on the last Codeforces Round and tell the bad news. We did many mistakes and due to them, the round caused much dissatisfaction from the Codeforces community.
Some of our mistakes:
Some stupid mistakes in the statements.
In the problem D2E/D1C the numbers $$$d_i$$$ were not necessarily sorted, but there were no pretests for that.
The checker of the problem D2F/D1D didn't check one of the requirements and due to that $$$8$$$ solutions failed on pretests during the system testing.
After the editorial was posted the mistake was found in the main solution of the problem D1E (more details here). Now we don't know the correct solution.
So, the round was very unsuccessful, I'm very sorry about this. I will make conclusions and will try to avoid such situations later. Please, treat with respect to the authors, even in such a bad situation, a big work was done and I hope you enjoyed the problems.
Now about the bad news: due to the wrong checker and the wrong solution, we decided to make Div1 round unrated, Div2 round is still rated, Sorry for all, who had a big positive rating change.
Also, we are making a D1E problem-solving challenge: if you have an algorithm, that can be proven, please share your approach.
So, that was all news, sorry again,
Ivan.
P.S.
If you are angry now, you can use this post to set a dislike, please don't dislike an announcement/editorial.
RIP
Oof
It's a terrible day for rain
Really RIP
we go agane
RIP->really "impressive" problem
RIP GM
Same here man :(
When arranging contests, there's a "strategic" point of view. There were many contests arranged on Codeforces, and hopefully many more will come.
Humans make mistakes, programmers make plenty of mistakes. Try as we might, we can't fix every single mistake, ever. What we can do, however, is to set up processes to minimize the chance of them happening during preparation, and processes to uncover them and minimize their impact when they happen during a contest. Polygon, Codeforces, their authors, and community as a whole have gone a long way in that regard. Here's a "thank you!" to the people involved, might sound trivial, but I mean it. And each new mistake is an opportunity to improve these processes for the future.
In short, ponder on how to avoid similar mistakes next time. Then cheer up, and move on.
Sorry, Ivan Belonogov!
One more round for that ?
Thanks for this write-up!
We all make mistakes (my contests haven't been flawless either). The best we can do is learn from them and try to find ways to prevent them from happening again in the future.
Side note: Event though div1E didn't work out, I still really liked the concept of the problem and it gave rise to some quite interesting discussions on whether this is possible at all ;)
Can't just fix the rating of those negatively affected? Me and many others didn't suffer from those mistakes...
Yes, it's the first time I see such an outcome to unrate the round after official results are final... I am ok with keeping accepted solutions to E. If you would like to remove this task, remove it and unrate for those who got this accepted and will have a negative change after the removal.
You don't know what you're talking about. Keeping the round rated without any kind of solution to one of the tasks is just unfair to those who tried to solve it (which there were many in Div. 1).
You don't know what you're talking about. There were many contests where either the judging queue was extremely long or the checker to some task was incorrect, or the tests were wrong, yet the rounds were rated, even though the impact was obvious for the much bigger number of participants.
There were not too many participants, who tried to solve it as it was one of the last tasks in D1 (it is a reasonable assumption, everyone could tell now that they always start solving from the last task :) ). I did not even read it.
Edit. See some examples below, there were tons of such rounds. Cf should be consistent. Looks like it tries to care for the community instead. In this case, after the results are final and the expected number of affected participants is low, this is a bad decision.
Well, I only saw wrong author's solution two times and both rounds were unrated. There is a huge difference between wrong checker or bad tests and wrong proof. As Xellos says below, D1E is the real reason, otherwise the round would probably be left rated. I'm sorry, I didn't mean to offend you in any way and I am sorry for your loss. But it's not the end of the world, so please accept it as it is.
If there wasn't that div1E, I'd expect that to happen, but "wrong problem" is the reason to unrate. You can't check who was affected by trying to solve it and it should serve as a warning to contest organisers everywhere to really make sure it doesn't happen.
This. Unrating the round is the only right thing to do when a problem is just outright wrong, but people with positive deltas are (predictably) going to complain about it.
This should probably be linked on the official announcement, too.
In div1C, there was no maxtest in pretests, not even something stupid like islands at 0,1,...,10000 with $$$G=R=1000$$$. I got completely unnecessary MLE because I allocated 2e7 vectors worst-case. There's a simple workaround in implementation with which it passes — use an initially empty
vector<vector<int>> Vand allocate 2e7 ints where the $$$i$$$-th of them is:V[k]Since the solution is just Dijkstra on small distances and the $$$i$$$-th vector contains states which had distance $$$i$$$, we're only appending, which is easy and fast. If the answer is small, few vectors are used, and if it's large, it's very likely that most vectors are 1-element vectors with no overhead. The same argument works for efficiency since there's one less memory access for vectors with size at most 1 and if there are few vectors to access, it's more cache-efficient, but that's probably not important, I expect the number of distance updates (costly due to
push_back) to be much smaller than the number of states.Still, this is unnecessarily convoluted, implementation tricks to remove a constant factor from time/memory shouldn't be the main point of failure in this problem, which they definitely were regardless of the intended solution.
Even if you can't prove it — maybe someone else will.
Tbh it makes more sense to downvote a mistake and then upvote effort to admit/fix it. Sure you can ask people to downvote you instead, but most downvotes from non-smurfs have been cast on the announcement already...
I had the same problem with ML in div1C. The easiest way to fix this is use
vector<forward_list<int>>. Empty forward list uses three times less memory than empty vector.I have a better workaround for this problem, since R,G≤1000 the maximum edge weight can't exceed 2000 and at any time the maximum and minimum distance in the priority queue won't differ more than 2000 (easy to prove it by mathematical induction).
Now we don't 2e7 vectors,we just need 2005 vectors. push_back in v[d%2005] and clear the vector when you visit it.
Here's my accepted solution: https://codeforces.net/contest/1340/submission/77908138
Yeah, good idea, although a lot harder to realise. It also avoids push_back reallocation overhead since clear keeps the allocated space. I just did the first quick dumb thing that came to mind.
After doing this I thought I had invented a way to implement Dijkstra in linear time for small weights, but then a friend told me it's called Dial's algorithm and it's pretty famous :(
But still the problems are fairly nice.
NOOOOO, RIP GM(((((((((((((
Why not make it semi-rated? I remember a similar situation happened in round 611 and problem Friends and Gifts had something wrong in the checker source after that round became semi-rated as the number of affected participants was small.
so If the number of participants that got affected in this round small, I think it should be semi-rated
Semi-rating or rejudging after taking out who had something wrong sounds like a better ideia than rooling back all the ratings. Only a small amount got stuck in div1E I suppose.
Username checks out
I remember also, on round 601, they had an appeal form for people who were affected by an issue with Div. 2 B (which thousands of people would have attempted!), but left it rated. Maybe this could be reproduced here?
I don't like Mike (or some other coordinators?) trying their best to make rounds with wrong model solutions to be rated.
For that particular round 601, there indeed was a correct solution, so it should have been:
But absurdly someone just decided to add a constraint $$$m \le n$$$ (meh, seriously), and give participants with wrong solutions a REWARD of 1000 points JUST BECAUSE THEY WERE CARELESS AS THE AUTHORS WERE.
This could give participants a negative signal: if a problem is on the easy side but you don't know the solution, just submit a stupidly simple solution. Either it is actually correct, or authors will add additional constraints to make my solution correct. lol
Hi guys, why the downvotes?
A fake account doesn't have rights to give a comment.
Irrespective of all the other issues involved, deciding to undo rating changes over a full day after the contest is honestly a disgrace to the platform. There was no advance warning that the round might be unrated, and rating changes were rolled out over a day ago. If there was any consideration at all of unrating the round, rating updates should have been postponed until a final decision was made. Moreover, most of the issues in this round, most notably the grammar errors and the weak pre-tests, should have been very easy to identify and fix. From a coordinator’s perspective, this round never should have been approved in its current state, and after the round, the numerous issues should have received a much quicker response.
The linked comment about div1E was posted 13 hours ago, long after rating changes. Then it definitely took some time to come to the conclusion that it really is wrong.
I guess it's just because D1E's jury solution has just been hacked which is a must-fix to me. The other issues are there but probably not enough to get the round unrated.
For what it’s worth, I’d support a solution along the lines of what some others have suggested so far—allow participants to submit a form if the issues with the round, possibly excluding the weak pre-tests on C (as weak pre-tests are unfortunately not overly uncommon), affected their performance. This has been implemented in the past on a much larger scale than would be required here. Moreover, since the biggest issue with this set was the incorrect solution to E, this would allow for the preservation of rating changes for the vast majority of participants, which seems preferable at this point for a number of reasons.
Why is reverting rating changes a disgrace? I think an actual disgrace would be to call a round with wrong author's solution rated.
To be clear, my problem is not (strictly) with unrating the round. My problem is with the way the situation was handled--all but one of the issues with the round were clear well before rating changes were applied (it was also known that most solutions that passed E were not fully correct, I believe, though the error in the author's solution was not yet discovered), but rating changes went through and we never heard anything about the possibility of unrating the round until a sudden announcement a day longer.
In general, once ratings update (and certainly after ratings have updated and a brief time to check for errors in the calculation has passed), they should be final. If this means that rating updates need to take longer so that an informed decision about whether to keep each contest rated can be made, then so be it. (This also would not be difficult to implement or create significant delays if editorials were released immediately after each round, which doesn't seem implausible--in fact, creating the infrastructure to automatically publish editorials after the round seems quite plausible. This round's problems could have been dealt with far more efficiently if the editorial was published when the contest ended, rather than 12 hours after the fact.)
Why are you so attached to these ratings? It's a very rare case that model solution gets hacked, you can't predict that and you can't measure how much it affected contestant who have tried it and it is a disgrace to keep a round with unsolvable problem rated.
Again, I don't understand why. I'd rather have my rating recalculated or reverted once in a few months, than wait for a whole day or more for it to change after each contest.
I agree with you regarding editorials. In a perfect world editorials should not only be finished before the round but also early enough for testers to read and check them.
... and none of them could be a reason to make the round unrated.
Imagine that John is accused of insulting his neighbor multiple times. There is a case and a judge just gives John a warning but basically John is decided innocent (no penalty). A few hours later it turns out that John actually killed his neighbor a day before. Will you say that the verdict should hold and John is still innocent?
'In general, once ratings update, they should be final'
Yep, and generally they are. This is a rare exception to the rule, which doesn't 'disgrace' this platform at all.
Hmmm, goodbye, IGM.
RIP, ratings :(
I get to be 10th again! :D
I really hope Belonogov will get another round in his name.
Codeforces Round #XYZ — Sorry, Ivan Belonogov!
If the current rating must be rolled back, can you just keep the max-ever? I don't know if I'll be able to reach that high in future :(
I also thought that when I reached 2580
Thanks for your encouragement, but I'm afraid that there's no time for me after August this year…… If I don't perform well in the CNOI, I'll have to say goodbye to programming and of course, Codeforces, for years. Well, thanks anyway.
How was your stay in India?
Are you in Wuhan?
Div2 should be unrated also
You are right! Let's downvote this blog!
Keep positive deltas and undo for the rest. Seems good to everyone xD
Good but almost impossible xD
good bye, orange!
Good news for me!!!!!!!!
N.O!
I think you can solve this problem by the same way as this: https://codeforces.ml/blog/entry/71534
Revive
lose rating, get contribution on making a meme about your loss of rating.
I think life is fair(at least for you)
Rating over contribution any day.
Internet points over internet points any day.
The usual term is e-penis length.
Lol even the title of this blog has a grammatical mistake.
For my opinion it's not correct make Div1 round unrated. For example there are contestants, who did not read the problem div1E (for example me). You can make round unrated for people, whose rating was downed.))
Ever heard of inflation?
Yes, but I remember round, in which codeforces used this method).
A round where every rating change was maximized with 0? Show me.
Read the last sentences.
Nope, your argument isn't valid, at all.
In that educational round, it was absolutely possible to identify "the participants affected by it", by rejudging all previously accepted solutions and see which becomes WA. Also, that problem had a provable 100% correct solution.
In this round, both of the points above are not true.
Hello guys, mind if you could share why you gave downvotes in this comment?
Cuz most people don't care about what a fake account says
You just showed a round where some people were affected and they could appeal. It's completely different from maximizing rating change with 0 for everybody. And how do you want to check if somebody was affected by an unsolvable problem? I can find an example of such issue making a round unrated.
Some people are removed and rating recalculated for the rest.
Grandmaster Hyperbolic (04.24.2020 ~ 04.25.2020)
It happens sometimes, no problem, and we hope for the best in the next rounds
Was this the reason that codeforces.com was down after the contest?
I decided to backup all the submissions from the contest in the state they were before system testing. Currently, it is a manual operation: just one SQL-statement to be executed. I executed something like
CREATE TABLE Submission_1340_1341 SELECT * FROM Submission WHERE NOT contestId IN (1340, 1341);. And I don't know why, but I addedNOTinto it. It starts process to backup all the submissions EXCEPT the current contest. Unfortunately for MariaDB/MySQL it is difficult to abort/rollback such statements because of ACID. TLDR; the database hangs in the state of rolling the transaction, blocked the table with submissions, and so on. After some time I decided to kill the database process and re-run it (it is a little bit dangerous, but it seemed that rollback could take hours). After restarting the database again tried to rollback it. So I restarted it again with a special option to disable incomplete transactions recovery. That's the whole story. I think I turned a little gray in the process of all this.Is there also some reason why http://codeforces.net/ is unavailable for several days?
codeforces.ru is not supported, please always use codeforces.com There was a permanent redirect from codeforces.ru to codeforces.com for years.
Oh No, my +1 is gone!
People who demand the round to be rated just because they had positive change — are you insane? People who are doing it while have solved E — I don't know what is wrong with you.
If the only way for you to get high rating is to get wrong solution accepted — you don't deserve that rating.
But why are those people insane? Isn't it natural to feel pity when you even didn't touch that problem and you lose your positive change because of an issue with that problem?
Well, I think it's at least selfish to demand a round to be rated because it is convenient to you, despite there being a serious problem.
It may feel like the problem does not affect you because you didn't touch it, but indirectly it does. For example, maybe someone below you wasted a lot of time on E and such people are partly the reason you have positive rating delta.
About rating — I try to not think about it that way. It's supposed to be an estimate of skill. Yeah, it's really shaky at times but I think if you deserved that positive delta, you'll get that positive delta in the next few rounds. If you did not deserve that positive delta, then you'd likely lose it soon in the future anyway. I can sorta understand that if your rating is far below your real skill then such rounds will slow "progress". But then again probably a lot of people just trick themselves into thinking that their rating is far below their real skill.
Regarding the last part, I totally agree.
What about demand, well , let's be realistic. Assume we somehow have all the participants who struggled on E. Let's remove them from rankings. Will the rating changes of remaining participants have massive changes? I don't think so.
That's not very realistic.
I don't think so either (and since it was on a hard problem anyway, my opinion is maybe not so strong). Still, the fact that a problem is wrong is serious enough for me to unrate the round on principle.
Bruh you haven't seen insane until you read some forums with discussions that go in circles for tens of pages.
Why didn't you submit the counterexample during the contest if that issue affected you so badly?
There is inconsistency in cf making the rounds rated or unrated. I would understand that every time some shit happens (long judging queue, unresponsive server, wrong pretests, etc.), the round gets unrated. This is fair. For some strange reason, cf tries to determine the impact on the community before making the round unrated. In this particular case, the impact was negligible. The fact that the ratings were final and nobody noticed the issue just confirms that. Yet, they decided that the final results will be reverted — which I haven't seen since the beginning of this platform.
It's not like everybody who can't solve a problem during a round already has a counterexample to an unknown to him faulty model solution xD
I actually had a counterexample to this particular solution during the round, but I didn't know that it was model :)
The impact of problem E's model solution being incorrect is absolutely unquantifiable. Do you think people begin ascertaining the correctness of the model solution if they're unable to solve a problem ?
"Hey, I can't solve a problem, the only solution I have came up with is wrong, here is countertest to this solution. Isn't it the model solution by any chance?" — great clarification, I would kill a person who sent it to me if I were the authors.
Hey I have a counterexample to outputting
15on every testcase. Why would you set a problem with a wrong model solution?!?As an author, I would definitely give attention to such clarification if it came from a veteran like you. When such concerns are indeed voiced by very experienced contestants, they have a chance to be right. But still, much more often it's not the right use of the contestant's time.
Isn't that the ArnoldC language? It represents a parse error
I feel only those people should conduct Div1 Rounds who have conducted few Div2 Rounds earlier. Div 1 Rounds require more effort and more experienced problem setters. I know anybody can make a mistake but the chances of such errors will be less in case of an experienced problem setter.
It is the first time that I solved div1D in one contest... RIP IM
Nice to see my -136 is gone xD (but tbh I did deserve some rating loss since my poor performance had nothing to do with those inadequacies)
Why keep Div2 rated but not Div1?
Yes, I had a big negative rating change(-49) after this Div.2 contest as my programme to problem C failed when system testing. The pretests are't strong!
If anyone is angry about this contest, let your friends on codeforces downvote this blog!!!!!!!!!!
Everyone makes mistakes, don't care)))
jenish9599 Lucky XD
Codeforces Round #637 — Sorry, Ivan Belonogov!
Also, there are weak tests on problem D2D/D1B (you can see a lot of uphacks).
Yes, that sentence actually made things worse :D
*Sounds of crying*
Participants just move their up-votes from the announcement to this blog:|
The most terrible way to thank Ivan Belonogov!
Good that they came to know this was the worst way to thank Ivan Belonogov! as they are no longer using it in the title.
What would happen if an onsite ICPC contest has a problem with no solution? I suppose they would not make another contest, just remove all wrong ACs and move on. I guess the same could be done here, whoever wasted the time in the problem with no solution is kind of their fault too (but of course its not ideal situation).
I dont really care that much honestly, but in my opinion the correct thing to do would Be to keep this round rated. Keeping this an official thing would be cool. If one problem has no solution the round is still rated, 2 or more unrated.
Why is only Div.2 rated? I had a big negative rating change. Can anyone help me downvote this blog?
So what's gonna happen to this div1E? Is there still no solution? Can the problem be modified to something easier that allows the old solution? Or will it be deleted?