Inspired by the other test case parsers users have been making, I made a version that suits my needs better. If anyone wants to use it, feel free to take a look at the repo.
In short, it parses a contest and puts all sample tests into separate files A.txt, B.txt, A.out, B.out etc. with all inputs and correct outputs, and there's also a script that'll run your executable on all the test cases and check whether output is correct.
Hope you enjoy, if you're interested.