Hello, Codeforces!
Two months ago, I created a certain utility that I'd like to share with you. As noted in the title of the topic, its main purpose is to evaluate your solution on a set of tests. It is likely (and I'm really sure of it) that there exist similar utilities that are capable of doing the same thing or even more, but my primary intent was to implement something lightweight and cross-platform.
Here is the utility (Github link).
Some kind of a backstory:
One day, I was getting rid of unnecessary files that had been accumulating on my hard drive. I stumbled upon an archive of an old contest that I seemingly had missed and hadn't had an opportunity to participate in. I thought that the tasks from this contest were interesting enough, so I went to Codeforces and tried to find it in “Gym” section. To my dismay, there was no such contest there.
I was not attracted by the prospect of writing the contest without being able to test my solution, so I decided to write an utility that would help me to run my program on the system tests from archive. It took me only a few hours before the first working version was ready.
Days in the future, but not many, another idea struck me: to evolve the utility into a full-featured testing system that would still remain lightweight. As a proof of concept I did even write a simple working web application for submitting solutions, but that was it. The script, however, eventually proved itself to be pretty useful in everyday life.
Notes:
The source code is work in progress, but the utility itself is usable. The script was primarily designed for use in conjunction with checkers that utilize testlib, but I think that every other checker that confines to the same input and output requirements would suffice.
The utility requires Python 3 to run (you can learn more about the dependencies in the readme).
The script have been tested on OS X and Linux, so I'd like you to report if something is the case. Please also note the compilers.json file that describes compilers and interpreters: it's very likely that the default provided commands may not work for you.
UPD: Demo.
why not break the judge.py in several other files? one file with 400+ lines may become un-maintainable.
As I have already mentioned in the post, the script was created with the intention of being lightweight (and easily integrable into other scripts and projects) so you don't have to carry around a whole bunch of files (consider testlib.h, which was designed with the same purpose in mind and is considerably longer).
In its current state (448 lines), it's still smaller than the invokers used in full-featured testing systems. I'd surely have considered breaking it into small parts if writing such a system was my primary goal from the beginning, but that wasn't the case.