Typically upon reading a problem, I either see the solution immediately, or get stuck with no idea how to proceed. Getting stuck usually means spending hours just thinking about it slowly, getting distracted, and failing to make any progress.
What is the optimal amount of time to keep thinking about a tough problem before reading the editorial solution?
For example, if I spend 5-6 hours in "stuck" mode, not working on anything else, and often still failing to solve the problem in the end — it seems like an inefficient waste of valuable practice time.
Of course, just giving up and solving easy problems doesn't improve my skills either.
My answer will be relevant to people who have relatively less experience with competitive programming or are stuck in lower ratings.
There is no 'optimal' time to give to a problem before seeking help, but maybe I can help you decide your own optimal time and effort.
When you're relatively less experienced, there's a lot to learn, and most of the problems don't contain very 'new' data structures or algorithms. Most of the problems use known and regularly used data structures, algorithms or simply tricks/techniques.
So, let's say you try a problem for 30 mins and you can't come up with an insight on how to even begin thinking or roughly which direction you should think in, chances are the solution requires something that you don't know yet. At that point, you can look at the editorial, confirm the case, and google for tutorials and easier problems having similar solutions (similar data structures or techniques). Read 2-3 different tutorials on the same, solve the easier problems, then come back to the original problem and try it again. So now you'll know which direction to think in when you encounter a similar problem in future.
On the other hand, you might make some progress in those 30 mins, and feel like you have proceeded in the right direction a bit, but can't get to the full solution, yet. In such a case, give more time to the problem. If you solve the problem after a good amount of thought, great! If not, have a sneak peek into the editorial. Don't read the full editorial, look for keywords and terms associated with commonly used data structures and tricks. Try the problem again, now with the knowledge of what could be used in the solution. Again, if you feel like you've made more progress, keep trying. Otherwise look into the editorial.
All in all, I would not recommend trying a problem for more than 1-1.5 hrs at your level.
One more important thing, avoid getting stuck in the loop of solving easy problems and then wondering why you can't improve. If you can solve a problem in say 15 mins(logically, excluding coding time), then probably that problem wasn't good enough to teach you something new. On the other hand, don't go trying Div 1 E problems and then feeling bad about not being able to solve anything. As you gain more and more experience through practice and contests, you'll roughly find that sweet spot of difficulty that'll force you out of your comfort zone, teaching you something new. Good luck and have fun learning!