LuchoBazz's blog

Quick Test CLI | Stress Testing | CP Tool
By LuchoBazz, history, 2 years ago, In English

Hi Codeforces!

I want to share with the entire competitive programming community my latest project: Quick Test CLI, which is a multiplatform open source tool to perform stress testing in competitive programming and that was inspired after watching the following screencasts: 1v1 Coding | 2020 Lockout Championship and How to test your solution in Competitive Programming, on Linux? Made by Errichto some time ago.

✍ Source Code: https://github.com/LuchoBazz/quicktest

❑ Docs: https://luchobazz.github.io/quicktest

⚡ Installation: https://luchobazz.github.io/quicktest/docs/getting-started/installation

What Quick Test CLI provides vs. the traditional way of performing stress testing?

  • Multi-platform
  • Supports several languages (C++, Java, Python, Rust Lang, Go Lang, C, Kotlin)
  • Friendly user interface
  • You don't have to create or use tedious bash scripts
  • It can be a standard for the community
  • It is a collaborative project that can continue to grow for the benefit of the competitive programming community.

Quick Test CLI supports several types of tests:

main gif

quicktest cmp | qt cmp

It checks the correctness of the algorithm we want to verify by comparing it with a brute force solution which is usually very slow, but is 100% sure to provide the correct solution.

quicktest cmp --target-file=main.cpp --correct-file=correct.cpp --gen-file=gen.cpp
# Or shorter:
qt cmp -t main.cpp -c correct.cpp -g gen.cpp

quicktest check | qt check

In some problems more than one answer is accepted, so the quicktest cmp command would not work correctly, in this case a script checker is used to verify the correctness of the algorithm.

quicktest check --target-file=main.cpp --checker-file=correct.cpp --gen-file=gen.cpp
# Or shorter:
qt check -t main.cpp -c check.cpp -g gen.cpp

quicktest stress | qt stress

Verify that the code execution time does not exceed what is allowed, using a random generator for multiple test cases.

Note: In this scenario there is no slower solution with which to compare the correction.

quicktest stress --target-file=main.cpp --gen-file=gen.cpp
# Or shorter:
qt stress -t main.cpp -g gen.cpp

Future Updates

Feature for testing interactive problems is planned to be implemented in future versions

I will be grateful to receive feedback, requests for new features and requests for improvements

Don't forget to give it ⭐in the Github repository

Thanks!

  • Vote: I like it
  • +21
  • Vote: I do not like it

Comments Comments (4)
| Write comment?
»
21 month(s) ago, # |
  Vote: I like it +4 Vote: I do not like it

I see your program requires a generator, a file in C++ that will generate the random test cases.

I suggest the following feature: make a standard for a file that will contain the "template" for the testcases, and then your program will transform that template into the testcases.

For example:

template.txt:

n[1-100]
n {
  a[2-100] b[2-100][>=a]
}

When your program interprets this file, it should generate something like:

4
42 58
2 94
63 75
26 98

The syntax I chose was a super quick and dumb example, but you can come up with some idea.

There will be problems though, you will need to figure out how to deal with uniqueness (sometimes needed), or with other conditions, like generating graphs that are a single connected component, etc.

I'm not aware if a tool like this exists already.

→ Reply
  • »
    »
    21 month(s) ago, # ^ |
    ← Rev. 2   Vote: I like it 0 Vote: I do not like it

    I am trying to formalize your idea and it would be something like this, however I still have some doubts, to generate samples for more complex problems, I still do not know how it would be.

    However here are some notes on the formalization of your idea:

    To generate a single random number, you could use the following format:

    n[min-max]

    Where n is an identifier that you can use to refer to the generated number, and min and max are the lower and upper limits of the range in which the number will be generated. For example, n[1-100] would generate a random number between 1 and 100, both inclusive.

    To generate a series of random numbers with a specific format, you could use the following syntax:

    n[min-max] {
  a[min1-max1] b[min2-max2][condition] ... z[minN-maxN][condition]
}

    Where n is an identifier that you can use to refer to the series of generated numbers, and a, b, ..., z are identifiers that refer to the individual numbers within the series. Each identifier has associated with it a minimum and maximum range (min and max) in which the random number will be generated.

    In addition, each identifier can have an optional condition that must be met for the generated number to be valid. The condition is specified in square brackets and uses the logical operators >, <, >=, <=, == and !=, as well as the Boolean operators && and ||.

    For example, the following specification would generate a series of 5 numbers a, b, c, d and e, where a and b are numbers between 2 and 100, and c, d and e are numbers between 2 and 100, with the additional condition that d must be greater than or equal to c:

    n[2-100] {
  a[2-100] b[2-100] c[2-100] d[2-100] [>=d] e[2-100]
}
    → Reply
»
21 month(s) ago, # |
  Vote: I like it +1 Vote: I do not like it

how can i use it for my fast olympic coding setup in sublime text

→ Reply