MikeMirzayanov's blog

Codeforces: new computers for testing and testing compatibility mode
By MikeMirzayanov, 11 years ago, translation, In English

Rah-rah!

We are moving forward to accelerate testing. In fact, when our rounds attract 3500+ registrations, it is hard to test all the submissions on nine computers "Core 2 Duo E6750, 2.66 Ghz, 3Gb".

Computer Science Department of Saratov State U has recently got 20 new computers i5-3470, 8Gb. After a little research it was found that if boost CPU to 3.5Ghz (of course, turning off all sorts of turbo boost and after many hours of stability check), the speed is approximately equal to twice the speed of the old testing machines. This is very convenient as it will in future only to divide all the time constraints in the problems by 2.

We have moved to testing on new computers and we do it in compatibility mode. This means that all the time constraints before the start of the program are divided into 2, the program is executed, and at the end a running time is multiplied by 2. In fact, if you do not use the cut-off by time or some other unsavory stunts, you will not detect any changes.

Old testing machines are switched off now.

To be very precise, the new computers are not so fast as 2x old. That is in compatibility mode it is normal to notice a slight degradation in speed.

  • Vote: I like it
  • +104
  • Vote: I do not like it

»
11 years ago, # |
  Vote: I like it +5 Vote: I do not like it

but in very few problems like Round#192-E.Graph Reconstruction we should use clock() or something else that tell us the current running time, so what should we do if we want determine the running time? divide the time of the problem by 2?

→ Reply
»
11 years ago, # |
  Vote: I like it +8 Vote: I do not like it

hope this solves the problem of long queues during contests! :)

→ Reply
»
11 years ago, # |
  Vote: I like it +9 Vote: I do not like it

Great !!! Hope there will be no "long time in queue" problem now.... :D

→ Reply
»
11 years ago, # |
← Rev. 2   Vote: I like it +72 Vote: I do not like it

In fact the new computers are not as fast as 2x old on MOST tasks and that is very obvious on some data structure tasks.

I just resubmited 3 DS problems and got very different results: 1308176 4174661 || 1821573 4174655 || 3870771 4174638

It is really worth mentioning this annoying difference.

→ Reply
  • »
    »
    11 years ago, # ^ |
      Vote: I like it +22 Vote: I do not like it

    We've improved some settings. Now the results are much closer. I've rejudged your older solutions, you may check it.

    → Reply
    • »
      »
      »
      11 years ago, # ^ |
        Vote: I like it +16 Vote: I do not like it

      Now my D from CR #194 pass in time...

      → Reply
      • »
        »
        »
        »
        11 years ago, # ^ |
          Vote: I like it 0 Vote: I do not like it

        OMG, I submitted this during the contest with two stupid bugs and was hacked. I thought it was TL, but now I'm sure that without those bugs it should be AC, even with old time settings.
        Why this problem was D? Just because you should be too brave to submit such solution?

        → Reply
        • »
          »
          »
          »
          »
          11 years ago, # ^ |
            Vote: I like it 0 Vote: I do not like it

          I believe for both D and E jury had not came up with n^3 / 32 solutions

          → Reply
        • »
          »
          »
          »
          »
          11 years ago, # ^ |
            Vote: I like it 0 Vote: I do not like it

          O(N^3) was not expected for the authors

          → Reply
»
11 years ago, # |
  Vote: I like it +2 Vote: I do not like it

Wouldn't it be wiser to run the complete tests on background while the contest is still on and publish the result afterwards? You can do this while computers are idle because not that many people are submitting code so we don't have to wait 1 hour to get the final results, instead of having solutions like dividing the time constraints by two because the speed of the new computers are "approximately" equal to twice.

→ Reply
  • »
    »
    11 years ago, # ^ |
      Vote: I like it +16 Vote: I do not like it

    We do it ~2 years.

    → Reply
    • »
      »
      »
      11 years ago, # ^ |
        Vote: I like it 0 Vote: I do not like it

      Is there some particular algorithm for the layout (not sure if it's the good word) of complete testing? I mean: a program could memorize statistics about users submissions during contest, and a submission coming from an user who make often resubmissions should have less priority than others for system testing. The complexity will be not very important, and it could decrease system testing time.

      → Reply
»
11 years ago, # |
  Vote: I like it +5 Vote: I do not like it

I have noticed something strange:

  • 4185223 (submitted during the contest, 592 ms)
  • 4188247 (submitted after the contest, 1248 ms)

but these two codes are completely the same. Is this behavior (System test is twice as fast as usual judging) intended?

→ Reply
»
11 years ago, # |
  Vote: I like it +9 Vote: I do not like it

2X is too much. My simplest codes in Java 7 used to run in 62 or 93 ms, now simple codes run in 218 ms.

→ Reply
»
11 years ago, # |
  Vote: I like it -28 Vote: I do not like it

Wouldn't it be wiser to run several solutions on the same machine (assuming the problems that most likely don't need big time) and in case it gets TLE then it gets tested on a machine w/o other solutions?

Well, yes, the time wouldn't be precise then but u know queue problem is so dreadful it is worth considering.

→ Reply
»
11 years ago, # |
  Vote: I like it -8 Vote: I do not like it

Sorry I have a problem , when I run my code on my PC pn Dev-C++ its answer and tasks answer is the same but when I send it to your system its different , and why ?

→ Reply
»
3 years ago, # |
  Vote: I like it +8 Vote: I do not like it

Is the information on the CPU used still up to date?

→ Reply