ADJA's blog

[Tutorial] Stress testing
By ADJA, history, 5 years ago, In English

Hello, CF community.

I am writing about algorithms for fun and to spread the knowledge.

Please read my new article about stress testing – a useful technique to automatically find bugs in your solution: http://www.algos.school/stress-testing

Would you be interested in the future algorithm articles like this? Please follow algos.school on facebook, twitter, or telegram.

Thanks!

  • Vote: I like it
  • +61
  • Vote: I do not like it

»
5 years ago, # |
← Rev. 2   Vote: I like it +27 Vote: I do not like it

I don't like your generator and script. Using time as seed gives you the same test within a second, so you will check only 60 tests per minute. And your script is quite long + you don't see that tests are indeed being checked.

Instead, use argument argv[1] as seed and consider the following script:

for((i=1;;++i)); do
  echo $i
  ./gen $i > int
  diff <(./solution < int) <(./stupid < int) || break
done

I appreciate the effort and more educational blogs is great, but do some research first and see how others do the same thing.

→ Reply
  • »
    »
    5 years ago, # ^ |
    ← Rev. 2   Vote: I like it 0 Vote: I do not like it

    Hey, thanks for checking this out!

    Using time as a seed is a nice catch. I copied some old code of mine (2013?), and didn't pay enough attention to that. I will fix this in the article.

    Regarding the length of the script, I believe it's not that much longer than yours – your version omits all comments and compilation part. When writing the article, I was trying to make script as readable as possible, not short. That said, I can definitely tweak my script a bit more, thanks for an example!

    And about research: I read everything I could on the topic, but there are not many good sources that I found – that was one of the reasons to write my article as well :)

    → Reply
    • »
      »
      »
      5 years ago, # ^ |
        Vote: I like it +8 Vote: I do not like it

      There are no ifs in my script and I don't create output files, those are bigger differences. By the way, it's easy for a mistake in generator if you write 2 + rand() % 999, I recommend a function instead: int r(int a, int b) { return a + rand() % (b - a + 1); }. I made a Youtube video with detailed explanation what to do and why, https://www.youtube.com/watch?v=JXTVOyQpSGM. Codes: https://github.com/Errichto/youtube/tree/master/testing

      → Reply
      • »
        »
        »
        »
        5 years ago, # ^ |
          Vote: I like it +22 Vote: I do not like it

        Or you could just use uniform_int_distribution from C++11 header <random>

        → Reply
      • »
        »
        »
        »
        5 years ago, # ^ |
          Vote: I like it 0 Vote: I do not like it

        Nice, thanks for pointers to your videos – I didn't see them before.

        As for 2 + rand() % 999 and script differences, these are mostly stylistic differences, and I will keep my version for now.

        I updated an article with random seed part though, again thanks for that!

        → Reply
        • »
          »
          »
          »
          »
          5 years ago, # ^ |
            Vote: I like it 0 Vote: I do not like it

          Ok, that's fine.

          Windows script is still the old version, and I think it compiles everything for each test again, right?

          → Reply
          • »
            »
            »
            »
            »
            »
            5 years ago, # ^ |
              Vote: I like it 0 Vote: I do not like it

            Correct, just updated.

            P.S. I don't have a Windows system right now, so let me know if there are still problems with it.

            → Reply
»
5 years ago, # |
  Vote: I like it 0 Vote: I do not like it

Very well! I liked this topic and the project Algos.school as a whole, although I hear about it now! My question is: will you promote this site in the future with more resources?

→ Reply
  • »
    »
    5 years ago, # ^ |
      Vote: I like it 0 Vote: I do not like it

    Thanks! I will try to post to the site more frequently

    → Reply