Preparing a contest is not as popular a topic as the perennial question of "how to become red in three months", but still it stirs some interest in Codeforces community now and then. Earlier I've written about preparing Surprise Language Round and about emotional aspect of problemsetting; now it's time to share some facts about running regular contests.

Problems

How much time does it take to prepare, select, recall or find problem ideas?

How much experience solving competitive programming problems is necessary to be confident about inventing problems of your own?

Idea generation is a long-term, nearly continuous process. A couple of years ago when I was still an active problemsetter I used to create ideas from literally everything (a squirrel running... hey, this could be a great problem! — seriously, the problem is still there in my drafts, and a pretty complicated one) and to write them down. Then when I had time and inspiration to run a contest, I went through the drafts looking for ideas which would be 1) solvable and 2) nice and unusual enough, and combined them in a problem set. The drawback of this approach is that I can't invent problems based on "we need a problem on DP" here, I can only start with the legend and solve it, but not the other way around.

Inventing complicated problem ideas totally requires lots of experience solving them during the contests — at least if you aim to invent a good idea. You can always take a standard algorithm with a long dreary implementation, add some geometry, a couple of corner cases to taste, shake and polish with a not-so-clear statement. But creating a problem that nobody can solve is hardly a problemsetter's virtue. A virtue is creating a problem with a cool idea, hard but possible to solve, and last but not least — a joyful problem which would give a "spend an hour thinking, couldn't figure it out, read the editorial, what an amazing idea!" feeling, not a "spent an hour coding, failed test 89, urgh!" one. The more experience one has competing, the more interesting and unusual their problems are.

First 4 of my contests (TC High School — problems for high school students, a bit easier than Div2) 1000pt problems were almost the same — breadth-first search, with a twist of geometry or a non-square grid. It was fine for a high-school contest in 2007, but it's certainly too boring for a Div2 contest in 2014 :-)

And if we talk about Div1 problems, in all my years as a problemsetter I've created only 4 problems which were harder than what I could typically solve: two Div1 1000pts on TC (1, 2) and Div1 132D - Constants in the language of Shakespeare and 132E - Bits of merry old England on CF. One of them was geometry + graph theory (i.e. cruel implementation), and for the other three I used some red people's help.

There also were easier problems in which testers challenged my author's solution (and sometimes with tests I didn't quite understand), so we had to switch to their solutions as reference ones; for example, in 345E - Black Cat Rush my pure geometry solution was wrong, and we had to use a ternary search instead. A lot of problems were ruled out by more experienced problem coordinators/coauthors for being boring or standard, though they felt like new and complex to me. And even more problems are still in drafts, because I still have no idea how to solve them. I can't speak for all yellow/violet members, but personally I need help of redder members to create a Div1 round.

How much time does it take you to prepare a round?

A lot. It is known that a problem = idea + statement + reference solution + infrastructure (checker, test validator, test generator, tests for special cases etc.). And while it might take 60 seconds to get an idea, everything else takes time, a lot of time, and the harder the problem is, the more time implementation takes. Even entertaining problems for special rounds with trivial solutions couple lines long and standard tests (several integers from certain range or a string of certain characters — no fancy restrictions) take at least an hour from creating the problem in Polygon to first version check-in. Infrastructure reuse helps a bit — you can borrow validators and generators from old problems with similar test structure, and sometimes you can even borrow the problems themselves (for SLR, for example). All in all contest preparation can take anything from 3-4 days (if it's a weekend and the contest is simple) to several weeks. Last couple of days before the contest I often have to stay up well after midnight — for some reason preparing problems always takes longer than I've planned.

Tests are a separate nightmare. I've vowed against writing problems with graphs and mazes a long time ago, because the fun of solving them is trumped by the pain of creating a strict validator (can take several days). Besides, the more complicated the input data is, the easier it is to make a mistake generating or validating it. My model bug in this area is problem TrianglesBoard; you can try to find it if you wish before reading on.

Hint: TC Arena prohibits entering lines over 50 characters long in test.

Answer: "board will contain between 1 and 50 elements, inclusive." + "Element i of board will contain exactly 2*i+1 characters." means that max test has 50 lines, and last line has 99 characters — and such test will be impossible to enter during Challenge phase. Alas, that contest had crippled Challenge phase, and that was really embarrassing.

People

How do you communicate with CF admins? How fast is the process?

How many iterations can it take for everyone to agree on the problem ideas before starting to prepare the contest?

We communicate via email and google talk. For the last 18 months the time zone difference between me and CF headquarters is around 11 hours, so the process is not exactly fast :-) For regular contests (not the ones "I want to write an SLR, please set up COBOL compiler and give me a date, and I'll do everything else myself") problem discussion can take a week or more, especially for harder problems in which I have only a vague idea and no certainty about how to solve it. On the other hand, approximately 24-48 hours before the contest communication becomes swift and intense — unlike TC, which has a lockdown 24 hours prior to the SRM, and you can modify the problems in lockdown only in case of a great need.

Who can help if I don't have enough problems or their quality is not as high as I'd like? Do CF headquarters help here?

How are testers chosen and what is their contribution to the process?

TC appoints testers officially from a very limited pool, and the problem writer doesn't have a say in the matter. On CF testers are whoever the writer invited to help (well, except for Gerald). Same goes for missing problems — they will be contributed by whoever you can persuade to contribute. I think CF has a concept of "coauthor" rather than "tester" — a coauthor can help compose the problem set, criticize some ideas and offer others, they can even provide reference solution and infrastructure for a problem if the writer doesn't have time for this (although this is a very optimistic scenario), and everything else as agreed. There are two extremities here — I write Surprise Language Rounds pretty much alone, and April Fools contest 2012 was prepared by maksay (who btw is a total saint!), I just invented the contest concept and problem ideas and then looked through problems when they were ready. Most cases are something in between.

Is author's presence and participation necessary during the contest itself?

Yes, sure. Each contest has admin interface with participants' questions which need answering. This is a kind of automated feedback — if you wrote a bad statement, please answer a hundred questions with clarifications; if you have written an SLR, please explain that you really can submit in only one language, and no, C++ is not available. And if all is well and quiet — well, enjoy and writing the editorial to publish it as soon as possible (I believe I've never yet managed to write an editorial beforehand).

And that's how a contest looks from inside.