Hi! In this post I would like to introduce you to an application called Hightail.
Hightail is an automatic tester for programming contests such as CodeForces rounds. It will parse the problem statement, extract sample test cases (inputs and outputs) from it, and verify the correctness of your program against them. It is built to provide maximum automation and to relieve the contestant as much as possible.
A list of Hightail's features:
- parsing problem statements
- parsing entire contests (autoloading all problems)
- scheduling contest parsing ahead of time
- ability to comfortably view, edit and add test cases
- handling of all verdicts: WA, TLE, RE, even AC
- customizable time limits
- ability to create in/out files in your working directory
- detection of floating point values (comparing 0.1 vs. 0.10 does not give WA)
- easy-to-use UI, keyboard shortcuts
- resilient: it is multi-threaded so it will not let your program hang it; it will withstand large amounts of output from your program
- support for all contest languages (C++, Java, Python, ...)
- written in Java and should run on any OS
Some screenshots to give you an idea:
You will find more information on its homepage (scroll down to see the readme). And here's a direct link to downloads.
Hightail started out as a simple program that I wrote for myself quite some time ago, then it became a small open source project when some other people contributed code to it. I think that it is now sufficiently developed that it can be released to the public, i.e. announced publicly here, rather than being hidden somewhere on the net, possible to find only if you already know the name. (It's probably long overdue, actually.)
If you see bugs, have feature suggestions etc., please use GitHub to its fullest extent :) And look at the readme — it has a tutorial on how to contribute to Hightail.
I hope it will be useful to you :)
I well tested "HighTail" for several times and it worked perfectly
Thanks for adding floating point detection ... one needed property ;)
Great work I saw it several times on your perfect screencast
Thanks also for your reply on 16 month ago post I don't know how you remember to reply on a post written from more than year ago.
I just hate unfinished business. :)
Legends say you still hate unfinished business, do you?
Do I?
Do you still?
Always
I've tried testing my last contest's problem and it works nicely. I've been hoping a long time for a tool like this! I really look forward to using it on my next contest :)
Congratulations for the great work!
First,thank you for the awesome tool. Second, one small advise, you may add this feature in the future version. Some problems just allow multiple correct answers, with some words like "if multiple answer exists, print any of them". If this tool can sense this and remind me "Though your answer differs with the standard answer, but it might be correct.",this tool will be more lovely. Thanks again.
Added an issue: https://github.com/dj3500/hightail/issues/58 Thanks.
Cool!
There was a bug where the Russian version of CodeForces did not work. It's corrected now.
Doesn't work on MacOS :( It waits on running..
Maybe you specify the wrong executable file? Because, I thought that there must be cpp file and "Hightail" was waiting on running, too. After I fixed this, everything worked properly.
Which file do you use as a executable file?
File, File.exe, File.exec?
First contest with Hightail, +135 in rating... I think I will stick with it! :)
Jokes apart, though I had some problems with the scheduled contest, it was very helpful!
Nice rating increase :) Thanks!
Could you say what problems you had? (Can be in PM, or as an issue on GitHub, if you like.)
Deleted; I sent you a PM
Oh, I have some problems with the scheduled contest.
Ten minutes before a CF contest, I was unable to parse a contest. (Is that OK?)
So I set up a scheduled contest, but when the contest starts, it just come up with some words like "Scheduled failed". I have to parse the contest manually. Though it only takes several seconds.
Any help? Thanks.
Okay, I'll investigate this further. Thanks.
What do u mean by...manual parsing...? It is parsing and saying 'parsing failed' for a new problem
FOr a new contest...it parses forever... What to do?
What do u mean by...manual parsing...? It is parsing and saying 'parsing failed' for a new problem
FOr a new contest...it parses forever... What to do?
I earned +156 rating first time :)
and last too
How to run it in Windows?
To quote from the readme:
To get a fairly up-to-date version — in the form of a .jar file which is ready to run if you have Java (JRE) installed — visit Downloads. Just save it somewhere on your computer and run it from there. It will create a config file in its directory.
Is this work on another Web that doing contest? such as TopCoder, or CodeChef.
CodeChef somewhat works (i.e. it will parse most problems, but not all), I used it successfully on the last CookOff. TopCoder is not supported.
Wow! I just wanted to code such a thing, quite useful
What a great program!! :) What language did you make this program?
Java.
Amazing software! Let me have a use~ Thank you very much for introduce this tester~!
for problems having multiple solution, it gives w/a for the alternate answer..this case will have to handle manually ..
I've been using Hightail for a long time now. Awesome stuff! :)
Wow this is great ... Hope from next time we don't have to use online compilers.... :)
How can I run jar extension? Which program do I need?
Java JRE. If it doesn't work by just doubleclicking, you can run
java -jar Hightail.jarin the console.It gives an error-unable to access jarfile Hightail. Can You please give a detailed procedure to use hightail after downloading and extracting the zip file. It would be highly appreciated if you make a video. Thank You
Did you install Hightail using snap? I think that is currently not working, so try instead to just download the .jar file from the Releases page and run it by double-clicking.
i have a problem ( my OS is ubuntu)
screen shot : http://www.gulfup.com/?ipfwjE
Well, I think it's right — the executable does not exist (unless you like to name your executables like
A.exeeven under Linux :) ). Point it to the compiled executable file (if you compile it with g++ with no options, it is usually calleda.out).Thank you :) , it was a stupid comment due to lack of knowledge !
How exciting the news is!We heard of this software long before,but i never found it bofore this.It was a generous gift.
I can't wait to use this feature . But can you explain how does this help us in contests ?
Well, you don't have to copy-paste the inputs. I thought this would be clear :)
Ok . Thanks . But now my question is does this feature really help us in contests to for example increase our rating 156 ?
This is more of a philosophical question :) I think that testing on the samples is a nuisance and a distraction which is better eliminated so that you can focus on the problems. It can actually save a lot of time sometimes when you don't get the program right from the start, but every time you correct it, there is one testcase (not always the same one) which doesn't pass... and so on. The time you gain will not only give you slightly more points, but sometimes you might be able to solve one more whole problem in the last minute ;)
Of course it's just that. It won't actually make you a better coder in any way.
can it run in Macbook?
It should be able to. However, one person above reported that it does not work. You should just try.
You can, i'm using it on Mac Os with Jar Launcher.
First of all i'm sorry for my bad English .
My question is that why this application use executable file to test our code ? Why not main source code ? Coz ,when we debug our code, we need to update our executable file by compilation . But if this app uses our source file directly, we just need to save our new code only.
Btw, Thanx for this awesome application :)
It's a design decision which might change in later versions (but probably not anytime soon). The downside is that the configuration (and the entire application) would be made more complicated. And when there are compilation errors, we would have to display them somewhere, and we couldn't do it as nicely as an IDE would (pointing the user to the line with the error etc.). Also, should we display warnings or not? The assumption is now just that the user will compile the source code themselves.
Hi Jakub, I tried using Hightail with java language, but it's not working. I think, I have made some path configuration or setting mistake. For your info, I am adding link of screenshot of my setting. Can you please tell me what's that mistake ? Link of screenshot
Thanks a lot for this comment, I see that there indeed was a problem there. I have put up a new version now, where it should be possible to run Java programs (see notes on the config panel for more info). Please let me know if it works for you. :)
It's still not working for me. I don't know where I have made the mistake. Did it work for you when you tried ? If yes, then please share screenshot of your settings. By the way thanks for giving your time. :)
Did you download the new version and configure it as described in the long note which is now on the config panel?
yes
For whoever is interested, it works now (one should just follow the new instructions).
This is amazing!
I've been a waiting for a program like this that works with C++ for ages. I've tested it and it works beautifully so far.
Thanks a lot!
Thank you for sharing this tool to the rest of us, i'll try to contribute to this project as soon as i can. Though i did find a bug, if the file path has any spaces in it, the program doesn't run at all. It keeps showing "running..." in the last column. Just letting you know, so you can add a issue on gihub.
Thanks, I'll check this out.
Did anybody get it to work behind a proxy?? If yes, then please guide me. I tried to parse a previous contest but was unable to do so.
Another person (BLANKRK) also told me it doesn't work behind a proxy. It seems that it really doesn't. Of course, you're more than welcome to implement proxy handling :)
Great!! I would be glad to work on that part, but I would need a little bit of mentoring. Would it be ok with you? I am really eager to work and get involved in this project. :D
Sure. For starters, read the readme, it has a section on how to work on Hightail.
I made an issue for you. https://github.com/dj3500/hightail/issues/60
does Hightail also work well with virtual contests? since i haven't used it before, i would like to test it on a virtual contest (or maybe even practice) and check its performance, before deciding whether if i'll use it in tomorrow's contest.
Hightail accesses CodeForces as a logged-out user, so it should normally see all tasks in any past contest.
Great Job! But unfortunately I always get the message "runtime error", I am working in Ubuntu with Netbeans and openjdk 7. These are my Configuration Settings:
My configurations Setting image link...
And this is the error Message:
My error message image link...
It looks great and I would like to automtize my training and contests, I heat to copy and paste test cases :) thanks in advance
Follow the instructions on the config screen (read the long text).
Great!!!! Thank you for your time, it worked! :) I was misstaking the configuration parameters. Hightail works for java without any problem.
My new settings:
http://i879.photobucket.com/albums/ab358/marcelodata/highlight00.png
And this is Hightail working:
http://i879.photobucket.com/albums/ab358/marcelodata/hightail.png
Hightail is a wonderful tool, thank you very much, it will help me a lot automatizing my submissions and training. Thank you and apologize my mistake configuring the settings, I read the instructions on the config screen, it's correct :)
I'm glad it works for you. :)
I am using proxy in my college. So Hightail doesnt work for me ?
hi, Path from working directory to executable : is this simply not "a.out" (in ubuntu)?
My a.out file is in the same directory as the jar file. I am using a proxy to access internet in my college. So Hightail doesnt work for me. As of now, it is not parsing anything...
No, sorry, Hightail does not currently work behind a proxy. See here and here.
unfortunate again due to this proxy thing.... I thought this proxy issue is only on the topcoder arena...but here it shows up again :(
but, you have done a wonderful job....for all the contestants, thank you :)
Does'nt seem to be working with java
You should configure it as per the instructions in the config dialog. Also try a path with no spaces in it, as there may be problems with that.
Can the program parse problem from Uva online judge? I try — but it can't parse
No. Please see the readme to see what online judges are supported.
I just had some trouble picking contest time at 00.00 AM (MemSQL Round2). It said "Wrong time — Chosen time is too early".
Oh, right. Thanks for the report. In Europe contests are never near midnight, so we didn't think to support next-day scheduling :) I made an issue: https://github.com/dj3500/hightail/issues/66
Support for Jutge.org online judge is now added, courtesy of Sergio Rodriguez Guasch (sergioRG).
Could you add support for acm.sgu and usaco? :)
If you write a parser, I would be ecstatic to add it :)
Hey!
As you are developing this in Java it would probably be good idea to have common API for parsers for Hightail and CHelper so that if someone write parser it could be used by both
Sorry for not replying for so long, but I didn't get a notification email from CodeForces...
I agree this would be a good thing. Are you suggesting to unify the interfaces of the parsers, or also the libraries that they use (Hightail parsers now use a HTML parser library that I don't enjoy too much, but it gets the job done)? Do you have a more concrete proposal of what this should look like?
Is there a way to see "cerr<<" on Hightail's output? It would very useful for me since I use it to debug my codes.
Thanks!
My thoughts exactly :) https://github.com/dj3500/hightail/issues/29
I'm afraid it would be hard to display both in the same textarea (at least it would be hard to maintain the relative order in which lines are output by the program). But something might be possible.
Although the Windows/unix console actually does accomplish this somehow, so... I don't know. :)
print functions usually flush on newline if they detect that program is run from tty, you could try to make it think that it's run from terminal when it's run by you.
Some relevant page
Though the idea of converting lines that start with "DBG:" to green doesn't entirely solve the problem, it would be very useful; adding "DBG:" to my template before each round is a very low price for the great benefits of being able to use Hightail both debugging and testing.
Thanks for the quick answer :)
Unfortunately, nobody (i.e. me) has much time to work on Hightail. If you really want this feature, probably you would have to implement it. But, if you did, that would be really great. :)
The latest version gives support for proxy.But I m not able to use it even now.There are Parsing failed again.Pl tell me if i hav to do something else except setting system proxy.
Let's move the discussion here: https://github.com/dj3500/hightail/issues/60#issuecomment-72460851
Try configuring the proxy settings in Internet Explorer again.
It seems that hightail leaves the tests hanging on round #288 B with the code I submitted on contest. It tests the first case and sometimes returns the result but the rest seems to be untested. It shows as I could abort the testing but nothing happens. It works fine on the other problems from that round though.
Thanks, I made an issue: https://github.com/dj3500/hightail/issues/74
I pull requested a fix to this bug ( https://github.com/dj3500/hightail/pull/78 ).
First of all thanks for making this software, it seems very good. However I do have an issue right now. I would like to use Hightail to run python codes. This is all well and good, and when I put python in the "Command to prepend executable with" the hightail software successfully runs the Python 2.7 I have installed on my computer. However I would like to use Python 3 that I have. Now in the terminal I can run a python file as python 3 by just typing python3 and the name of the python file. However when I put python3 rather than python in the dialog box for hightail it does not work, as when I run the tests it simply says running... for infinite time. Do you have any advice for how I can get hightail to run my Python codes as Python 3? Many thanks in advance. Also, I am on mac if this makes a difference.
Hmm. If you can run
python3from the console, then it should be on the system PATH, but maybe it's somewhat different under Mac... Maybe you can put the full path to thepython3script in Hightail?Thank you for the suggestion. Everything works now, I should have just tried the full path earlier myself, but did not think of it. Again, thanks for this tool!
Hi...
What's wrong with hightail?! It fails repeatedly on parsing contests!!!
I couldn't use it for recently contests at all because of "Parsing Failed"!
Hm have not had this experience. Worked for me for the contest today. The only time recently it has not is when Codeforces itself was down. Do you have any more details of when it has failed?
I checked it in different time spans of the contest, up to 45 minutes after the starting of contest; but it failed on parsing continusly!
Do you mean the Rockethon? It didn't work because one of the tasks (D1) was unparseable (for some reason it displayed nothing, and the problem statement was only shown on the D2 tab).
In general, if it can't load all problems (because of CodeForces load or unavailability), it won't parse a contest. The only solution now is to retry until it does.
No, it was recent codeforces round #291. I tried other contests, but it failed too!
I can open codeforces on contests time, but can't parse problems by hightails.
I tried using Hightails it works fine, but when I close it and afterwards reopen, my problem that I was trying to test is gone. Am I doing something wrong or Hightails does not save problems on exit?
It does not save problems on exit.
Sorry for bothering. But it seems Hightail no longer works for Codeforces round. It keeps giving out parsing failed. Anyone else?
And help? Thanks in advance.
Works for me ;) Can you give an example of a round which it does not parse?
Sorry, I should've try more before asking. It seems I've changed my network proxy settings. And it caused this event. After I changed it to the default setting, it works fine. Thanks anyway.
New release v0.9.5, with new features:
Get it here :)
I use hightail a lot but not able to use hightail for atcoder.
how to add the problem from atcoder send me demo picture
Hi. Apparently, my program is getting TLE for large test cases when using hightail (using a time limit of 1.0s) but when I manually input the test case file in terminal, my code runs in 0.09s.
Also, is it possible to request to add "load test cases" feature so that we don't have to manually re-enter all our test cases every time we load hightail?
Could you raise this issue on GitHub? Try to give an example of an input and a program for which this behaviour appears.
For the second one, please also open an issue. Although the chances that someone will implement this are not so high :) I always envisioned Hightail rather as a tool for situations where you solve a problem in one shot (without restarting your computer etc.).
Great tool!, thank you,
i was wondering, is there some way to get the problem's name when you parse the problem statement through the url, and put that name as a default name for searching the executable?
This please!
And it's done by fabianun :)
Well, we already do that, except we treat "A" as the name. This would not be too hard to implement, so feel free to open an issue and maybe someone (or you) will do it :)
Does it support interactive tasks?
I tried the example problem [A] given by mike. And it didn't work. Because in interactive tasks the input is coming after you output a number but hightail takes the example input and output which are just random.
No.
New release v0.9.6, with new features:
Get it here :)
WOOOWW!!! I was just watching Petr's screencast for February lunchtime on codechef to know how they code fast ,, what kind of ide they use..and I saw him testing problems on the editor itself...i wondered if there is something like this for c++ and this 3 year old post came alive....just WOWW
I think there is an issue with Hightail on Mac OS. Hightail keeps showing 'Running' for files which are located in folders whose names contain spaces.
Take a look at this screenshot. In this picture there are no spaces in any of the folders.
Now take a look at this picture. In this screenshot the directory "ds" has been changed to "d s" and Hightail shows 'Running'.
Is there any fix for this problem?
how to use contest time system ? . Before 5 minute of the contest should i add the url on the url box ? And what is parsing ?
Or paste the contest link before the contest (you can also use the link to the list of registrants), set the time and press Schedule.
I have just install "HighTail" but i don't know how to use it. I can't make file .exe in codeblocks. Someone can help me?
Sorry for my poor English.
You'll find the .exe file in \projectname\bin\Debug after you build the project(ctrl+F9) on codeblocks
Hi. I didn't find the license file, but I'm interested if it's OK to use parts of the code for the alternative application?
If you trying please make like chelper for c++ :)
https://github.com/AlexeyDmitriev/JHelper
cf blog if you are interested for some reason
Yes. What is your application, by the way?
Well, I'm only thinking about it yet. Basically, I'd like to use most of the interface. What I want to change:
Use CHelper parsers, because they support more platforms, GCJ and FHC being the most important for me.
Generate a project using CMake and launch tests from IDE instead of the app. This way I don't need to switch between windows at all and debugging will be much more natural.
Use Testlib checkers instead of internal one to be consistent with online judge environment and enable custom checker support.
Integrate library code inlining somewhere in the pipe.
Cool! Points 1 and 2 sound exciting to me. Especially 1, I guess. As far as I know, this implies having a browser extension or something like that?
Just saw this tool today via the "autopromotion" message :-) It works perfectly and is going to be incredibly helpful. Thanks!!
Wonderful tool. Just wanted to know how schedule contest work? Whenever I am scheduling a contest it is saying "Schedule successful".But after the scheduled time nothing is happening.
Well, at most ~30 seconds after the scheduled time something should happen :) If it doesn't work for you, could you make a bug report (issue) on GitHub (with details, like which contest and what you are entering in the fields)?
Thanks for such a quick reply. Does Schedue contest feature works for live contest only or for any contests(like virtual contests)?
Should work for both, I would say. Whatever a logged-out user can see. (In case of AtCoder live contests, also what a logged-in user can see, if you give your login credentials.)
Hightail Gives Runtime Error When i use it with codeblocks ( which creates both a .exe and .o file) but works perfectly fine with devc++ ( which creates only .exe file) can you look into this as codeblocks is my primary ide :)
Well, both work for me... Does the CodeBlocks .exe work when executed from outside of CodeBlocks, like from a console?
Works Fine Now , There Were Some Problems With My Codeblocks Linker :)
Is There Any Way I Can Use Hightail If I'm Using a proxy ? ( because i can access internet only using a proxy in my college )
Should work behind non-authenticated proxies. See: https://github.com/dj3500/hightail/issues/60
It is only for Windows right? It's asking .exe and all
No, it's cross-platform.
So, there is no .exe files in Ubuntu. Do you have any tutorial how to use it on Ubuntu? It would be of great help to many people.
Just point it to your executable file, whatever its name is. It has some comments in the configuration dialog which should make it clear.
Is there anyone here working with javaFX? I want to ask him a simple question.
If anyone is still visiting this site, I've added hightail to the Ubuntu snap store, so that any Linux users can just do
sudo snap install hightail.Currently it is still undergoing review so if the command above doesn't work, try
sudo snap install hightail --channel=candidateIs there any way to show error stream (cerr in C++ instead of cout) in Hightail?
Thanks for this awesome program btw!
EDIT: Say a problem has x time limit, when a user creates a new test, (imho) it's better to set default time limit of the new test to x instead of default 3 seconds. So we don't need to manually set the time limit for each new test. Thanks for hearing my opinion :)
See here and here. Thanks for your feedback!
I see, thank you!
WOW, it's awesome
Hi,
I recently started using it and I love it. I use Sublime to code in C++ on my Mac, and I was trying to use the prepend command feature to compile the file. Eg. I have in the path A.cpp, and a command "g14 name" that compiles name.cpp into name.exe I tried prepend command = g14 %P; But this doesn't work because %P does not evaluate to "A" in the prepend command. Is there any way to do this?
You would need to make a script that both compiles and runs your file (like
make a && ./aon unix). Then have the prepend command beg14(with a space after) and the path from ... be%P. That should work...Thanks for your help. I did this but it says "running..." I am not sure why it didn't work :(
I have g14 to compile and r14 to run.
g14() { g++ -Wall -std=c++14 -Wno-unused-const-variable -o $1.exe $1.cpp }
r14() { g14 $1 && ./$1.exe }
And my settings:
Note that it runs like:
r14 /Users/.../arather than/Users/.../r14 a. So you should make sure thatr14is in your PATH, for example. Try runningr14 /Users/.../afrom the console and see if that works.New release v0.9.7
Big change: now supports Competitive Companion (can parse tasks/contests from browser, supports Google Code Jam, Facebook Hacker Cup, etc.). See video.
Also:
Get it here :)
Thanks for sharing this fantastic plugin.
There's a small bug running on my win10. When I use cmd to compile my C++ code, it works perfectly. However, when I use bash on win10 to compile my code, it just keep running. I just downloaded Hightail-v0.9.7.jar here. Is there anything I missed?
When you compile with bash on win10, the executable's target is Linux, so it can't be runned on Windows. However you can run Hightail on Linux (with bash on w10) by installing an X server like Xming. Here is a screenshot
The instructions posted in the screenshot won't work with WSL2.
See this for WSL2: https://github.com/microsoft/WSL/issues/4106
Hightail-v0.9.7 is pretty awesome.
One suggestion: If competitive companion directly opens hightail when hightail not open in the system.
Feature request: Can you please get it to work for authenticated proxies? Many college students are not able to use it because of that.
Thank you :)
Hightail now has over 6000 downloads. Thanks everyone!
Many people use file input-output for debugging and testing during the contest using freopen(c++). So the executable file expects the input from the input file and writes to the output file. Can you add functionality that makes hightail work from the specified input method for the executable? Basically, before running the executable, write the input to the file and read the output from the file as well.
But what's the point of doing both? In my head, Hightail replaces freopen...
Still some OJs (USACO) use file I/O and there this feature would help :)
I got run time error again and again in cpp in codeblocks so any one can tell me issue ??
make sure that your code doesn't include "return x " for ex : for(int i = 0 ; i < n ; ++i) if(i == 5) return i; this process with hightail will get you a RTE
I am not able to run it on Linux. It's showing:_The file '/home/roshan/Desktop/coding/Hightail/Hightail-v0.9.7.jar' is not marked as executable. If this was downloaded or copied from an untrusted source, it may be dangerous to run. For more details, read about the executable bit._
Do This —
1- Go to the location of file and select it.
2- click ctrl + I and go to Permissions tab.
3- there's a row named execute, tick the "Allow executing file as program" option.
And you're probably done, now double clicking would work, if still there's an error, do this and you're done.
honestly, didn't get it what does it do
UPD: Alright, sorry, i get it now
it gives runtime error