//This is a short post motivated by the comment section of the last Div2 round (and of several rounds before)
Every problemsetter tries to make his contest as good as possible, and is upset when something goes wrong. Most problemsetters don't set contests for money, the payments are not worth the time and efforts spent on coming up with problems, preparing them, and hosting the contest. The most rewarding thing in this process in not the money, it's to see contestants enjoying your problems.
That's why feedback is so important. If the comment section is filled with comments praising the contest, setter will most likely want to come back with another contest, and if it's filled with hate and offenses, it's possible that setter will just give up on problemsetting and won't want to host another round.
This means two things.
First, if you liked the round, or some particular problem, why not to write a comment about it? I am sure that most setters would really appreciate it, and feel more motivated to come up with problems like the one you liked. Positive comments from high rated contestants, or from experienced problemsetters matter especially, but I rarely see positive comments from such users.
Secondly, we should try to avoid hateful comments, or unconstructive criticism. This doesn't mean that there shouldn't be any criticism, it is necessary to help setters to improve, but not all criticism is good. Below I will give several comments, which, in my opinion, aren't the best way to express your opinion about the contests. I won't mentioning authors of these comments, as I don't want to call anyone out, and I myself sometimes left not very cool comments.
What's the point of leaving such comments? How will this help setters to improve in problemsetting? These comments are not even telling what's exactly is wrong with the contest — it's just hate and a complete lack of respect to the setters (though I would consider Codechef comment as a compliment).
This is another type of comments — just saying that some particular problem was bad/shitty. However, this isn't much better, as this still doesn't help setter to improve much, it just discourages him from writing any problems in the future. If you are criticizing some problem, tell what exactly was wrong about it: was it well-known, standard, was it too caseworky, or too implementation heavy, maybe the constraints were too strict?
Again, I am not saying that criticism of contests in comments is bad. Here are some good examples (in my opinion).
They aren't disrespectful to the setters, and the authors of these comments are saying what exactly they didn't like about the contest, so that setter can avoid such mistakes in the future.
People may have different opinions about what's a good problem, or what makes a contest good, and setters won't always take all the criticizing comments into account when hosting their next round, but the feedback is still very important for improvement. So, when you leave negative comments, please try to not use them as an opportunity to call the author a stupid moron. It's better to tell what exactly went wrong, politely and respectfully, without any hate. There aren't that many problemsetters out there, so instead of telling people to stop writing problems, it's better to support them.
:)
wow nice will keep in mind this advice before criticising contest
why downvotes , I just appreciated this blog
They will downvote you for no reason. Don't take this seriously.
Anton, thank you. You wrote everything correctly. I actually think we should think about censorship for such unfounded criticism. What do you think about it?
If you were to explore censorship, I do think clear rules need to be established. Although the examples antontrygubO_o mentioned above definitely feel too extreme; it's hard to decide whether things like "I didn't enjoy the contest" or "The contest wasn't that interesting." should be censored IMO. I agree that such comments aren't helpful either, but I don't think they deserve censorship. Because of this, it's hard to decide whether some specific example is "taking it too far" per se. In my opinion, the comments that should be deleted are the ones that personally attack the author, such as
(Both of these are also from round 737's announcement blog).
Hm...isn't there an indirect censorship method already? I mean the feature that hides a comment when it has many downvotes.
Unfortunately, if many people dislike a contest, hateful and offensive comments can end up with upvotes instead of downvotes
Maybe then the censorship mechanism can be just to hide a comment manually even if it doesn't have sufficient downvotes. This way the comment is still there (hasn't been completely censored). This also means that admins won't have to go around fully censoring comments which are already hidden due to downvotes.
But if the community is giving likes to something offensive, then it can be actually seen as a constructive criticism all together that it was not met to expectations by the target audience and that's just part of the game.
A very relevant example to this can be found here
What’s constructive about that? “Your problem sucks” without any additional comments on how it could be better isn’t constructive.
cool idea maybe disable comments or make them to go through moderation before posting? xd
btw better idea is to add option to filter "hateful" content in profile settings or add warning on such comments, like on ones that have negative rating.
I agree that these comments are not helpful at all, but censorship is really bad as well.
As a problem setter, you are exposing your work to the public and even if your problems are really good, you will still face criticism from some people (as with many other things in your career) and you should learn how to distinguish constructive feedback from unconstructive feedback.
Censorship is against the freedom of speech. We already can downvote comments and blogs. If we will follow this path, then who will decide what is constructive and what is not?
I agree that the comment section to should have less comments that simply just attack the author or some specific problems. However, I believe that it would be too much to apply censorship such as deleting those comments. We already had the function of downvote to hide comments. A possible solution is to give the function to hide comments to Masters or Grandmasters, since most attacking comments are from lower ratings.
or you could give this function to top X contributors.
Codeforces censorship is already a thing and in my opinion there already is too much of it. I'm talking about Eva's blogs and comments having been deleted, sometimes despite raising serious topics and being constructive. I also remember comments from other people which were suspiciously hidden (not deleted), while I couldn't imagine why.
Please don't develop these methods, it doesn't make codeforces look good and is completely unnecessary as I see it.
Never thought that I'd downvote Mike's comment. But here I am. I'm already so tired of all this censorship that exists in our lives, some are official censorships and some are unofficially followed by the society.I think one should have the right to show their views, even if it's a wrong one. Because that's how I've learnt a lot of things. I corrected a lot of my behaviour only because I expressed it and someone else pointed it out. If there's censorship, I doubt if that would've happened. Not all censorships are necessary.
I do not think censorship is the best way. Do you want codeforces become Twitch?
Actually kinda funny ngl.
+1
And to contest authors, don't take every negative comment seriously. Sometimes it isn't that a problem was bad but the person just performed poorly and decided to take out their anger on the authors, it's normal human behavior.
Just read constructive criticisms, learn, improve and be happy!
"(though I would consider Codechef comment as a compliment)."
Nice
Do the pictures just have problem for me ?
I can't see them.
An actual solution to problem like this might be adding functionality to give a feedback about rounds (or personal rating like 1-5 stars). This will encourage more people to give constructive criticism and avoid public abuse.
Also rounds with very good feedback should receive "good contest" badge to encourage problemsetters to aim to higher quality.
Well, there is such a mechasim. It is comments in the announcement
Yeah, but Anton and Mike are whining about the current mechanism. People need a voting system so they can vote "1 out of 5 stars" anonymously instead of saying "contest is shit" in the anouncement.
Also, it doesn't help that currently there will always be testers telling how wonderful the problemset is, regardless of quality. The testers that do dislike the problems just stay shut. If we could see the actual testers' average rating of a contest before it takes place, it would help tremendously everyone's expectations and we'd see less of this offensive criticism (these folks wouldn't even participate in that case).
I doubt this would make any difference because:
It wouldn't be perfect, and it doesn't have to be. But if we had like 10 or so high rated testers, I'm sure we'd get something meaningful.
That's fine? If a contest is only interesting to LGMs because of a few problems at the end, then most participants would find it a bad contest, because it actually is a bad contest for them.
My point is not to have less people participating, but having more accurate expectations before a round.
Interesting or not is largely subjective but I trust high rated users to have a better judgment on the quality of easy problems as well as the overall balance of a problemset compared to the reverse.
It will be good if a comment if deemed hateful by community (or a select wide committee of trusted members>) removes commenting privileges for some time (like twitter does with hate speech). The same committee can also assign ban cheaters who game plaigiarism check
After reading this blog and comments, especially about censorship, I think I have to say this. Don't turn codeforces into another fucking liberalistic crap like twitter or facebook. Never take away freedom of speech. NEVER. If you don't like comment, downvote it, so it will be hidden and it would be clear that this opinion is unpopular. If (some "geniuses" above mentioned) "hateful" comment will end up with upvotes, isn't that mean that this comment is approved by community and contest is really as bad as described in comment? If you remove such comment, you steal the right to opinion. And yes, I don't want to see contests on codeforces that such comments have upvotes more than downvotes. Understand, that bad contest is first of all setter's fault, feedback is just a consequence. Work on your contests quality instead of blaming community for feedback that "hurts you"
I'll probably get heavily downvoted for this but there you go.
My experience from the few contests I have tested was that setters work extremely hard for setting a good round. The money they get from setting a contest is not worth the effort the setters put in, and most of it is largely self-motivated, since they want the community to enjoy solving their problems. People who have other intentions would probably not end up getting a chance to set a contest, since they might withdraw given how long the queue really is (~6 months at any given point for Div 2 rounds).
You don't just need to prepare problem statements, you need to prepare tests and pretests, and keep in mind the server load, all possible edge cases, and adversarial strategies for interactive problems. This is only what I saw as a tester, a lot of it goes behind the scenes. Making tests is hard enough already, and handling all these other things is much harder. I remember people discussing whether to keep some test as a pretest just because a randomized solution would FST eventually. So, setters try to keep the whole contest experience as watertight as possible, and do not derive sadistic pleasure from seeing people FST.
The same goes for problem statements, every tester is asked for detailed feedback on problem statements, and the setter takes into account every piece of feedback while making statements. Anyway, they have to deal with issues in the round as well if the statements are not crystal clear (due to being bombarded by clarifications). Most coordinators are legendary grandmasters, and they do a very thorough work of scrutinizing the contest too and aim to make the contest as high quality as possible.
Please stop blaming the setter for all the faults of the contest that are not in their hands. An interesting idea might be really hard to check (for instance, think of how you would make tests for a problem which needs Frievald's algorithm), and an interesting game problem might be almost impossible to check. The system is far from perfect (and I doubt it will ever be), but it's still pretty good at what it does, and the problemsetters deserve more sympathy than what they get when their round has some issues. After all, it's more of "their" round than your round, and no one likes to mess up on things that will be attached to their name/handle for quite a long time.
One more thing to note -- this really negative feedback affects new problemsetters a lot, and stops them from creating contests in the future. People are so accustomed to near-perfect rounds that any sense of discomfort directly leads to a huge backlash, and it's impossible to control the amount of negativity in the comments.
If you look at the round announcements/editorials from a few years ago, you'd observe a much more civilized comments section. I wonder what went wrong.
Until this moment I referred to the word "setter" as whole team that prepared or tested problem/contest. If you understand word setter as a person who purposed problem, then I'm answering to this:
If you read comments quoted in this blog again, you will see that nobody criticizes exactly author. "problem X is bad" does not mean that author of the problem X is stupid or how do you understand it. It means that author purposed a bad problem, then coordinator didn't reject it, testers didn't tell to coordinator that the problem is bad etc. Moreover, I can even say that author is not guilty at all, author's job is to purpose anything he can, and coordinator's job is to either accept or reject it. Nobody blames exactly author, please stop manipulating facts.
I agree, problemsetting is not easy. But if you decided to set contest, please, prepare it as it has to be, in case you want to receive positive feedback. (Again, these words, as long as contest feedbacks, appeals to whole preparation team, not exactly author!!) And if your contest doesn't satisfy community requirements and criticism "hurts" you, then move to the codechef, it accepts shitty contests, it pays out more and there is no community feedback at all.
You're indian, you know the reason better
It's not just the author who prepares the problems, true. The rounds I tested had the author as well as their friends helping in setting the problems.
However, the claim that "nobody blames exactly the author" seems misinformed, most people identify the problem with the author (and that is evident in comments like "please stop writing contests"). Even then, the "team" puts in a proportionate amount of effort, criticizing which to no extent is quite bad. And if you are writing a round, and get criticism on it, even if the feedback is meant for the team, it demotivates you too.
I understand where you're coming from, of course we don't want setters to slack and reduce the contest quality. But the amount of criticism contest authors get is a bit too much to be honest.
"Community requirements" is quite a vague term (and getting contests to participate in is a privilege, and not a right; remember paying for a Codeforces account?), and I believe that coordinators do quite a good job, leading to higher contest quality, which makes it easier to spot issues in contests. It's easier to spot a mistake on an almost-flawless thing as compared to something that already has gone bad.
If you're implying that the fact that I'm Indian means I have some more insights into reality, well, I do not.
If you're implying Indians are the people who criticize the contest the most loudly, I can't do anything about that but to ask you notice how most of the negative comments are from non-Indian high-rated people. I don't think most low-rated programmers (which corresponds to the majority of Indian competitive programmers) even care about contest quality, since a taste in problems is acquired with quite some practice, and most of them care about getting better for solving interview problems to get a job, where they don't have the luxury of solving well-posed and elegant problems. Just my point of view.
I mostly agree with the spirit of this comment, but also I feel authors aren't exactly sacrosanct either. In some of the few CF rounds that I have been able to test, I remember saying "problem X is shit, you can easily come up with something better," and getting a response along the lines of "agreed, but we have already spent so much time on this round and this feels passable."
I guess this is especially common in non-Anton rounds, and with less (or extremely) experienced setters. So, anyway I think if authors are fine with including problems they know are shitty/standard, then they should be fine with people calling them out on it too.
That explains my bias, I suppose, since all the rounds I've tested were Anton rounds and I remember stuff going pretty well there.
Of course, it's fine to call out setters for their own mistakes, but still, the criticism shouldn't be as harsh as it tends to be. I believe that the authors would appreciate it even more if there was specific and constructive feedback as to what went wrong.
As cathartic the negative (but noisy and unspecific) feedback might be, it doesn't align with the best interests of the community and does nothing more than elicit a chain reaction of such reactions.
You completely missed the point but I agree with not censoring comments.
I would feel bad whining about round for which authors were paid <5$/hour. Just post a valid criticism without insults, cp community should have mental intelligence higher than that of child
I didn't miss any point. Half comments offer censorship as solution to all problems related with bad rounds.
You're right, the payment is little. But as long as there are people who still interested in purposing problems, there will be more rounds. And again, if you want money, move to codechef.
If you don't like comment, there's downvote tool. I think nobody will be offended by a comment with -420 score. And comment's author will understand what people actually think about his opinion.
You read comments only but said "after reading this blog and comments".
I wanted to say that authors set problems because they want to give back to community, enjoy coming up with new ideas, etc. I wouldn't say "eat shit" to those people.
If there's a problem — ignore it, solve it somehow. Maybe you just don't create problems? Censorship is not a solution, so I hope after reading this blog people will think twice before insulting authors.
Speaking of censorship, I think one of the rare things that should be censored is a post about one of the highest-rated users (I think there's not a real point to conceal the identity, but to stay true to principle) wedding. It reveals a piece of private information only he and his wife should be allowed to share (even naming your spouse is a bit iffy, but if bank accounts are joint I guess privacy could be as well). Unconstructive comments are just rude and if they are not in private messages shouldn't be censored, although I agree people who give them are usually jerks (not just in CP, but life in general).
Bruh, watch you private information and who do you trust it to. Codeforces as any other social network is not obliged to maintain your information spreading if you screwed up to keep it a secret.
Here are my thoughts on the situation for the 3 people who see it Disclosure — I have never written a contest nor tested in cf or any platform, so my opinion might be discredited to some however I believe that these values don't just apply to problem setting in specific; anything that one person takes their own time and volunteers(or gets payed a small amount idk what cf does) for others to enjoy shouldn't be given trash critisim. If you geniunly want them to improve, then leave constructive comments about why their problems where "trash" (although a problem can be bad, it can never be trash). Anton talked about this a lot but I want to emphasise the respect portion for the setters. They are college/highschool students or even out of school who take their time to write these contests. PurpleCrayon, who set some problems once, said that it is very hard and stressful to meet the quality that cf boasts. Please respect the setters even if the problems aren't as good or easy as they should be they at least had effort put into them. Thats all I have to say and I feel compelled to say this since I have seen many horrible comments on blogs.
As an author of a few contests and an uneloquent person with an opinion on the internet, I generally agree with this blog. That being said, on the topic of moderating negative feedback, I think there are two different issues at hand:
For me personally, I can handle some people telling me to commit ritual suicide for setting a problem they didn't like, so I see A as a much bigger issue than B. Given the extreme lack of useful feedback, for me in particular having stupid negative feedback is actually somewhat useful, since it at least allows you to gauge which problems were poorly received, which is better than nothing.
If moderation for negative comments is introduced, my opinion is that it should be accompanied by some system that allows for that basic feedback to still exist, for instance a system for users to rate how much they liked a problem or a form to submit comments about the round in general. Otherwise the only remaining way to gauge how well received a round is would be to count upvotes/downvotes on the announcement or something like that, which is a total pain in the ass and hard to interpret.
So tl;dr in the absence of a system to provide feedback to authors, or a general culture of providing useful feedback I am against the idea of moderating negative comments, since I feel that they're basically the only feedback that exists at all right now.
It feels weird writing some critique in blog/DMs. Maybe some special form for this so I won't feel coming out of nowhere.
Perhaps introduce an option to rate the problems (1-5 stars). Some possibilities: A. rating enabled only for people who solved it during the original contest; B. separate ratings: 1. from people who solved it during the contest, 2. from people who participated in the context, 3. from all CF users
Only people who solved the problems should be able to rate, because they are the only ones who have fully experienced the problem and writing a working solution. There is no need to limit it to the original contest either. This way we only have one rating system to maintain.
There is already an upvote/downvote functionality for blogs and comments; it can be adapted for problems (only allowing solvers to vote) as well.
First of all, I don't care about negative voting and if some contests problems are tough that doesn't mean the problems are worst. The only thing I didn't like about the last contest was the statements of problem B. But that was okay (constraints on k) if you solve some uva or problems set by old setters. But I was literally sad after watching comments of those ill-minded people, though interlude and dqxym(sorry if I miss-typed) were still trying to help people that make them such two good guys. I don't know why you criticize people. DOn't break the peace of this community.
Moreover, I find condolence thinking there are bad and good people in the world. Try to follow the good ones.
I agree
me too babe