MikeMirzayanov's blog

Codeforces: Uphacking
By MikeMirzayanov, 5 years ago, translation, In English

Hi Codeforces.

Recently there have been several improvements on the website. A little later, I will write a short note about them. Here I want to tell about a noticeable new functionality that has recently been implemented by kuviman.

Sometimes after the end of a round there are requests in comments to add a particular test. And problem writers or coordinators add it manually. Now this process is automated.

Meet the uphacking phase!

Now, after the end of almost any contest, participants from Div.1 have the opportunity to hack any solution in this contest during the week. Yes, including official solutions from rounds. In case of successful hacking:

  • Your test will be automatically added to the problem and will be used in the future when testing new solutions for this problem;
  • If a hacked solution is a practice (i.e. upsolving), then its verdict will change to “hacked” otherwise its verdict will remain unchanged;
  • The participant whose solution has been hacked will be notified via system private message;
  • Each solution can be successfully hacked only once.

In order to add only adequate and reasonable tests, it is forbidden to hack solutions that intentionally contain a mistake. Please strictly follow this rule. It is because of the possibility of inadequate behavior that we cannot open uphacking for everyone.

I hope that this innovation will be useful and tests for problems will become even better!

  • MikeMirzayanov

UPD: Probably, later we will extend uphacking phase and reduce the rating bound.

  • Vote: I like it
  • +784
  • Vote: I do not like it

Comments Comments (51)
| Write comment?
»
5 years ago, # |
  Vote: I like it +81 Vote: I do not like it

What about some limit? You should be blocked for an hour after ten unsuccessful hacking attempts. There are some randomized solutions that will break after 1000 random tests. Otherwise, that's a nice feature. By "including official solutions from rounds" do you also mean author's solution?

→ Reply
  • »
    »
    5 years ago, # ^ |
      Vote: I like it +18 Vote: I do not like it

    Later we will introduce some limits if needed.

    Here, official means contest solutions of official participants.

    → Reply
    • »
      »
      »
      5 years ago, # ^ |
        Vote: I like it +103 Vote: I do not like it

      Let us have some fun. Publish authors' and testers' solutions too :>

      → Reply
      • »
        »
        »
        »
        5 years ago, # ^ |
          Vote: I like it -9 Vote: I do not like it

        You are also one of them xD

        → Reply
      • »
        »
        »
        »
        5 years ago, # ^ |
          Vote: I like it +43 Vote: I do not like it

        How would you hack an author's solution? Hacks run on their solution to determine expected output. If your solutions conflict, how will the system tell who is right?

        → Reply
        • »
          »
          »
          »
          »
          5 years ago, # ^ |
          ← Rev. 2   Vote: I like it +64 Vote: I do not like it

          What about the author's solution gets TL/RE instead of WA?

          Or the problems need to output a plan which has multiple answers, the author's solution can't pass the checker.

          You can even hack the validator or checker.

          → Reply
          • »
            »
            »
            »
            »
            »
            5 years ago, # ^ |
              Vote: I like it +14 Vote: I do not like it

            Uphacking works basically same as usual hacking during the contest.

            We still assume that authors' solutions are correct, you would get system failure in case they are not.

            → Reply
            • »
              »
              »
              »
              »
              »
              »
              5 years ago, # ^ |
                Vote: I like it 0 Vote: I do not like it

              well, it's still possible to show that author solutions disagree (or too show that output is certainly incorrect in case of "smart" checker)

              → Reply
              • »
                »
                »
                »
                »
                »
                »
                »
                5 years ago, # ^ |
                  Vote: I like it +12 Vote: I do not like it

                Yes, if you get failed verdict, that's most likely this.

                But I don't think there can be done anything automatically here, this would require authors to check, hence it is a failure

                Uphacking can help with fixing the tests, not authors solutions, unfortunately :)

                → Reply
                • »
                  »
                  »
                  »
                  »
                  »
                  »
                  »
                  »
                  5 years ago, # ^ |
                    Vote: I like it 0 Vote: I do not like it

                  Well, if you'd show author solutions, it would help with authors solutions too :)

                  → Reply
            • »
              »
              »
              »
              »
              »
              »
              5 years ago, # ^ |
                Vote: I like it +5 Vote: I do not like it

              There are always a bunch of solutions made by authors and testers. It's enough to find a test for which some of them will disagree.

              → Reply
  • »
    »
    5 years ago, # ^ |
      Vote: I like it +8 Vote: I do not like it

    I agree with you that may lead to DOS attack if someone automated the test so limits will avoid DOS attacks

    → Reply
»
5 years ago, # |
  Vote: I like it +7 Vote: I do not like it

If a solution is hacked after rating change of a rated contest, will it affect the result of the contest? Like will the standings update or the rating change?

Thanks a lot for the great feature! I remember there were rounds that got pretty weak tests.

→ Reply
  • »
    »
    5 years ago, # ^ |
      Vote: I like it +2 Vote: I do not like it

    If a hacked solution is a practice (i.e. upsolving), then its verdict will change to “hacked” otherwise its verdict will remain unchanged;

    → Reply
    • »
      »
      »
      5 years ago, # ^ |
        Vote: I like it 0 Vote: I do not like it

      Thanks! That rule make sense for me now :)

      → Reply
»
5 years ago, # |
← Rev. 3   Vote: I like it 0 Vote: I do not like it

Hello MikeMirzayanov , There is a bug,when we open a solution to hack, I am unable to read the complete message about uphacking ( if I try to scroll down then the hacking window also scrolls down). To reproduce the bug, open any solution and scroll down, the complete hacking window does not fit on screen.

UPD: Initially I was getting option to hack but now this option has been removed, I think it was a mistake !!

→ Reply
»
5 years ago, # |
  Vote: I like it +53 Vote: I do not like it

Will there be separate hackers ranking? (Similar to Contributors)

→ Reply
»
5 years ago, # |
  Vote: I like it +2 Vote: I do not like it

should add hacking points on profile for incentive

→ Reply
  • »
    »
    5 years ago, # ^ |
      Vote: I like it 0 Vote: I do not like it

    But this will also give incentive to hacking solutions that were created by mistake, hence adding way too many tests to the problem and slowing down codeforces significantly. Better to leave it as is

    → Reply
»
5 years ago, # |
← Rev. 2   Vote: I like it 0 Vote: I do not like it

There might be more features that should be available only for experienced users, so that would be great if we have some sort of "experience point" system. Users participated in enough contests and achieved enough rating (therefore increase their XP) will gain privileges to up-hacking, proposing a problem/contest, faster feedback, faster reporting of disrespect behaviours/cheating, etc...

→ Reply
»
5 years ago, # |
  Vote: I like it +31 Vote: I do not like it

What about constructing solution-specific countertests for randomised solutions with otherwise very low probability of failure? Should that be allowed?

→ Reply
»
5 years ago, # |
  Vote: I like it +2 Vote: I do not like it

after that does there a new counter for successful and unsuccessful hack in profiles (like contribution)?

→ Reply
»
5 years ago, # |
  Vote: I like it +4 Vote: I do not like it

Good quality update. Hope to see many more in the future.

→ Reply
»
5 years ago, # |
  Vote: I like it +7 Vote: I do not like it

it is forbidden to hack solutions that intentionally contain a mistake

I think implementing a check mark for intentionally mistaken solutions is good.

→ Reply
»
5 years ago, # |
  Vote: I like it +14 Vote: I do not like it

Great, now I can't even practice without getting hacked

→ Reply
  • »
    »
    5 years ago, # ^ |
      Vote: I like it +13 Vote: I do not like it

    Now you can practice without considering your wrong solution as a correct one.

    → Reply
»
5 years ago, # |
  Vote: I like it -21 Vote: I do not like it

In case of a successful hack on an official rated solution, would the rating change for those whose solutions are hacked? Or, will the ratings updated only after a week when the uphacking phase is over?

→ Reply
»
5 years ago, # |
  Vote: I like it 0 Vote: I do not like it

Things are getting more complicated now....

→ Reply
»
5 years ago, # |
  Vote: I like it -6 Vote: I do not like it

So what submission can people hack.

→ Reply
»
5 years ago, # |
  Vote: I like it +1 Vote: I do not like it

Seems like one can't hack submissions from virtual participation?

→ Reply
»
5 years ago, # |
← Rev. 3   Vote: I like it 0 Vote: I do not like it

"participants from Div.1 have the opportunity to hack any solution in this contest during the week."

As of now, we can't hack everyone's solution but only div 1 participants, being precise then we can hack solution of participants who currently (after the contest) has rating greater 1900. Is it a bug or I misinterpreted it??

→ Reply
»
5 years ago, # |
  Vote: I like it 0 Vote: I do not like it

When is this feature possible?

→ Reply
»
5 years ago, # |
← Rev. 2   Vote: I like it +53 Vote: I do not like it

Uphacked submissions in the standings should have another color. Now both failed and uphacked submissions are red. I suggest brown (as the color of what has passed system tests).

ping MikeMirzayanov

→ Reply
»
5 years ago, # |
  Vote: I like it +4 Vote: I do not like it

Surprised this wasn't mentioned when this post first appeared

Your test will be automatically added to the problem and will be used in the future when testing new solutions for this problem

This holds true even in virtual contests — 57403374 was an in-contest AC that when I resubmit in a virtual contest I get WA — 57458328. Why??? How is that a virtual experience of a contest? Please MikeMirzayanov just let us have i) option to test solutions against pretests only during a VC and ii) option to test solutions against just the system tests, without uphacks after VCs, so we can simulate the contest experience more realistically.

→ Reply
  • »
    »
    5 years ago, # ^ |
      Vote: I like it 0 Vote: I do not like it

    Judging just on pretests is more important.

    → Reply
    • »
      »
      »
      5 years ago, # ^ |
        Vote: I like it 0 Vote: I do not like it

      What was that?

      → Reply
    • »
      »
      »
      5 years ago, # ^ |
        Vote: I like it 0 Vote: I do not like it

      I agree ... ?

      → Reply
      • »
        »
        »
        »
        5 years ago, # ^ |
          Vote: I like it +5 Vote: I do not like it

        I just mean that we wait for years for the feature of judging only on pretests in VC, while behavior of uphacks in VC is far less important. (It doesn't mean that your proposal shouldn't be implemented.)

        → Reply
»
5 years ago, # |
  Vote: I like it +31 Vote: I do not like it

Uphacking has been a really nice feature to make tests even stronger for practice. Wouldn't it be better if hacking people outside your room was allowed in contest time, with no positive point for successful hack, maybe negative point for unsuccessful hack? Maybe total number of hacking attempt outside room can be limited per user.

I think this will strengthen the systests, specially for hard problems.

→ Reply
»
5 years ago, # |
  Vote: I like it +16 Vote: I do not like it

MikeMirzayanov I've noticed a bug with uphacking. Uphacked contest submissions don't show that they've been hacked if you are logged out, see for example https://codeforces.net/contest/1263/submission/65981072.

Also the following is just a small suggestion. I get that it is a feature that uphacked contest submissions still have status "accepted". The issue is that I've been uphacking a lot of submissions on yesterdays E, and now I have a hard time to go over those submissions that haven't been hacked (only way to check if a contest submission is uphacked is to open up the submission). Would it be possible to add something that distinguishes true contest AC and uphacked contest AC?

→ Reply
»
5 years ago, # |
  Vote: I like it 0 Vote: I do not like it

Could you extend the scope of UPHACK, maybe to 1600+. It's a really nice feature but few people could use it.

→ Reply
»
21 month(s) ago, # |
  Vote: I like it +11 Vote: I do not like it

I am confused about rules of up-hacking and open hacking (in educational, Div.3 or Div.4 rounds).

In Codeforces Round 855 (Div. 3), I noticed that tests of successful hacking during open hacking phase weren't added to future tests (first and second submission), while tests of up-hacking were added (second and third submission).

Later in Codeforces Round 859 (Div. 4), I noticed that tests of up-hacking were not added this time (fourth and fifth submission), either.

So what is the difference between them? Is it because anyone (maybe the writer?) set constraints like "no new tests are allowed", or the hack behavior violated some rules (or probably something else)?

→ Reply