Yes, I am agree with you. (●'◡'●)

Or until the end of the year (maybe year 2023)

You can turn this to a book

Well , I will try to reach it ,not only have the purple color while magic is on

You can change the rank at https://codeforces.net/settings/rank.

I think the codeforces platform is already capable of handling a heuristic contest.

For final standing I highly recommend system testing (running the solutions on a different set of testcases after the end of the contest). Unfortunately we saw how not to do it in the Huawei contest in September, where the final testcases were too different from the initial ones and some solutions failed with TLE or MLE.

That leads me to the second point: provide is a local tester (with testcase generator) so participants can test offline. Topcoder gives you a Java application with a visualizer, atCoder has a webpage from where you can download testcases and upload your solution to analyze step by step.

The alternative (no system testing) will result in a lot of submits with almost identical code and some testcase detection to decide which features or random seed to use to make the submission score higher without actually having a better solution.

Another point is relative scoring (relative to other contestants or to some upper bound). If the possible score per testcase differs too much, some testcases would just become irrelevant. Topcoder does relative scoring to the highest scoring submission on a testcase. While I think that's the best way to handle it in order to evaluate the strength of different solutions, it might confuse newcomers (your own score can drop, when someone else submits a better solution). AtCoder uses some upper bound and normalizes to it (every testcase could give 100k points in theory, although not always possible in reality — just a rough estimation).

I'd like to have Deadline format because I show good results in it:

• you calculate all results locally
• relatively small number of tests (usually 10), each test must be represented as a problem in terms of Codeforces
• you submit only one archive with answers for all tests, and it's counted as you sent solutions for all problems
• all tests are random so we don't have to find their structure as is HashCode

It's not that difficult and as eulerscheZahl mentioned, most of the infrastructure is already here. Honestly, I agree with everything that he said and I have no clue why he's being downvoted. Most of the things I'm going to say are more or less the same, just using different words.

In general, I'd say there are only two good ways of designing heuristic contests:

1) Same as Topcoder & AtCoder does: every problem needs an official test generator that is available for download (with source code included). You need a "system phase" as well, where you run all of the final solutions against a large hidden test set. This solves the overfitting problem and makes the contest fair at the same time.

2) Alternative is to give away all of the testcases for download and contestants only submit the answers. There's a version of submitting code instead of output only, but this usually devolves into a contest of compressing your output into the solution. It's fun to do that every once in a while, but it gets boring very quickly.

I would generally discourage from (2) as I don't think such design is good for building community and it's generally not very educational as there's no mandatory code sharing. There's also no incentive for submitting your solutions, which makes the contest way less exciting. It also gives major advantage to people with access to a lot of computing power. The main upside is that, it's easier to setup such contests and that you don't need to care about support different language etc. Since CF already has all of the important components, I really don't see any reason to do such contests.

As for (1), you already had a contest meeting all of those criteria: https://codeforces.net/contest/636

I remember that after the contest I gave feedback to Gassa (who coordinated the contest) about things that could be improved. I can check them later and give a tl;dr. But based on what I remember, most of them were small issues.

That being said, in order to have good quality heuristic contests you need:

• Good problem editor & problem testers. Designing good heuristic (and bot) contests is very similar to designing games. The goal is to create something that has very simple rules, but it has a lot of depth. Recent Huawei matches were complete opposite of this. Bonus if it looks cool and you can make a nice visualizer. Topcoder has dimkadimon, AtCoder has wata and both of them are doing great job. Note that I'd be mostly concerned with the editor/coordinator role. From my experience, a lot of people have great ideas for contests, but it takes experience to turn great idea into a great problem.

• Problem visualizer. Not a strict requirement, but this is what makes those contest fun. Not every contest needs a visualizer though. In theory, community can build visualizers as well and share them. The downside (other than having an official visualizer) is that in such situations it's tricky to design exact rules for competitors. Topcoder (and I think Atcoder as well) have very strict rules about no possible cooperation, code sharing or even talking about results on specific testcases while the contest is active.

I'm happy to talk more about this if you're interested. After I'm done with tips that is.

What a nonsense take.

I meant in terms of optimization. Heuristics are like cherry on top of the cake.

From rocket landing algorithm to image classification via Neural Networks, everything uses optimization of some sort. I am freaking amazed by those algorithms.

Optimization + Heuristic FTW

Bot programming competitions are theoretically closer, but I still never heard that anybody used sophisticated ML or NNs and got a good result in such a contest

In this contest reCurse managed to train a neural network. And he won by a massive lead. CodinGame doesn't really support it, so he had to code his own NN framework and fit it in a 100k character submit limit. See his forum post for more details. Mad Pod Racing (formerly known as Coders Strike Back) has NNs at the top4, but only some years after the contest they got good enough to beat more standard techniques. Most simple board games on CG are dominated by machine learning approaches now. For more complicated ones you still see some other search (like GA or MCTS) at the top.

.

Cool.Thanks!