So I installed the free version of PyCharm professional last week and have been encountering the problem where I am debugging code on a remote server and when I try to display variables, it simply says "collecting data" and then if I try to continue the debugging process PyCharm breaks.
I have been researching solutions and I have Gevent compatible enabled as well as tried all 3 variable settings; Synchronously, Asynchronously, and On Demand.
I should also note that I am running into a problem where the debugger is skipping all my break points as well and I have to restart my server connection in order to get the break points to hit (and sometimes it takes a couple of tries)
I know that it is entirely possible to see the variables that are collecting data as one of my co-workers who recommended PyCharm has no problem, and there was 1 run where I was able to see the variables, but when I re-ran the commands (with absolutely no change), I was back at square one.
I've been going through PyCharm forums and it seems as if this has been a reoccurring issue for a handful of years now, but knowing that it worked once for me, and it works for my coworker, am I simply missing something?
Just recently my PyCharm has started to behave this way as well. I researched and tried the same solutions you did, to no avail. On certain projects it simply hangs forever on "collecting data" - projects that used to work - where code hasn't changed. Pls let me know if you find anything else; I will keep researching and testing as well
EDIT: FWIW. In my particular case I isolated the cause of this (at least I think). I had a very large dataframe in memory and if this DF is not in memory, the debugger does not hang. None of my watches were explicitly on this DF, but I guess the debugger needed to inspect it upon break and the object was just too big somehow. Note that it hung even if "variables loading policy" was set to "on demand", so the debugger still must automatically investigate all variables somehow.
Related
I apologize for not having a reproducible example. My problem is with a large base of proprietary code and I don't have an extract that shows the same behavior. Even better, it isn't my software and I know about 2% of how it works.
Simply, this Python program I'm dealing with takes about 80 seconds to complete its entire setup and get to the point where all its flask code is running and the webserver being created is up and able to respond to requests. BUT -- that's only the first time I run it on Windows after rebooting. On subsequent times starting the python script in question, it takes more like 10 seconds.
And the nutty part is, in a workgroup of 10 people, mine is the only computer that has the problem.
Things I can say:
Python 2.7.11, Windows 7, git bash version 2.9.0.windows.1.
It doesn't appear to matter whether I invoke my python program from the git bash command line or the Windows command line.
However, in git bash, saying "python" gets no response forever until I hit Ctrl-C, but saying "winpty python" opens an interactive python session as it should. I mention this because for a while I thought my main problem was related to the git bash shell bug (https://stackoverflow.com/a/32599341/5593532). But point 2 above would seem to contradict this. No such weirdness occurs in invoking a bare python interactive session from the Windows command line.
I've had trouble getting meaningful profiling output, partly because of multi-threading or child processes or something. And the web server doesn't have an exit event per se, thus I can only stop it by smacking it with a Ctrl-C in the command-line window where I ran the script, which seems to kill the part of the process that would save the profiling data.
From the fragmentary profiling info I was able to produce (with gratitude to https://ymichael.com/2014/03/08/profiling-python-with-cprofile.html), I am suspicious that something weird is happening in loading a large number of imported packages, and perhaps especially the alembic and/or werkzeug packages. (And maybe even sqlalchemy.) The profiling output didn't have much tottime in those packages, but it did have rather a lot of cumtime there.
My sys.path inside Python doesn't seem meaningfully different from anyone else's nearby. I might have one or two different items in the list, or three .egg files on the path when they've only got one, but it's mostly the same list in the same order. So much for the idea that it's taking a long time to hunt and learn where packages are and then re-using the information later.
I've got PyCharm Community Edition able to run the script and its associated junk in IDE mode, set breakpoints, and all that jazz, so I can set breakpoints and follow execution to a degree, in case that would answer a noteworthy question you could raise for me.
Anyone got a wild notion what's up? (he asked quite unreasonably)
I am currently getting an issue with an external executable crashing when it is launched from a Python script. So far I have tried using various subprocess calls. As well as the more redundant methods such as os.system and os.startfile.
Now the exe doesn't have this issue when I call it normally from the command line or by double-clicking on it from the explorer window. I've looked around to see if other people have had a similar problem too. As far as I can tell the closest possible cause of this issue is that the child process unnecessarily hangs due to the I/O exceeding 65K. So I've tried using Popen without PIPES and I have also changed the stdout and stdin to write to temporary files to try and alleviate my problem. But unfortunately none of this has worked.
What I eventually want to do is be able to autorun this executable several times with various outputs provided by xmls. Everything else is pretty much in place, including the xml modifications which the executable requires. I have also tested the xml modification portion of the code as a standalone script to make sure that this isn't the issue.
Due to the nature of script I am a bit reluctant to put up any actual code up on the net as the company I work for is a bit strict when it comes to showing code. I would ask my colleagues if I could but unfortunately I'm the only one here who actually has used python.
Any help would be much appreciated.
Thanks.
As I've not had any response I've kind of gone down a different route with this. Rather than relying on the subprocess module to call the exe I have moved that logic out into a batch file. The xmls are still modified by the python script and most of the logic is still handled in script. It's not what ideally would have liked from the program but it will have to do.
Thanks to anybody who gave this some thought and tried to at least look for an alternative. Even if nobody answered.
I am running a Python script using Spyder 2.3.9. I have a fairly large script and when running it through with (300x600) iterations (a loop inside another loop), everything appears to be working fine and takes approximately 40 minutes. But when I increase the number to (500x600) iterations, after 2 hours, the output yields:
It seems the kernel died unexpectedly. Use 'Restart kernel' to continue using this console.
I've been trying to go through the code but don't see anything that might be causing this in particular. I am using Python 2.7.12 64bits, Qt 4.8.7, PyQt4 (API v2) 4.11.4. (Anaconda2-4.0.0-MacOSX-x86_64)
I'm not entirely sure what additional information is pertinent, but if you have any suggestions or questions, I'd be happy to read them.
https://github.com/spyder-ide/spyder/issues/3114
It seems this issue has been opened on their GitHub profile, should be addressed soon given the repo record.
Some possible solutions:
It may be helpful, if possible, to modify your script for faster convergence. Very often, for most practical purposes, the incremental value of iterations after a certain point is negligible.
An upgrade or downgrade of the Spyder environment may help.
Check your local firewall for blocked connections to 127.0.0.1 from pythonw.exe.
If nothing works, try using Spyder on Ubuntu.
I've got a problem with a python script which is responsible for syncing changes from a VCS (among other things) which may include changes to itself / libraries it depends on. In such cases, I would like to be able to detect if the sync touched anything I depend on, and restart if that was the case.
On POSIX platforms this is easy. exec(), done. On Windows it's incredibly annoying. I can Popen or exec*() and then die, but the problem is my user will see the cmd prompt show a new line, which messes up the flow of output, not to mention completely breaks any future input on stdin, allows them to run commands while my script is still running, which is undesirable.
I've searched high and low without finding anything approaching an answer to this (and yes, I know there are alternative ways to accomplish the same goal, but it'd be nice to have this work). Seemingly the problem is telling the console that the new process is now the "owner" of the console, but I can't find a way to do that. I tried calling https://msdn.microsoft.com/en-us/library/windows/desktop/ms681952(v=vs.85).aspx through ctypes, and that didn't work either.
Any help much appreciated. Thanks!
I've found that under some circumstances the Eclipse python debugger can be unreliable. For example, when stepping through a memory-hungry Python program I've found that after a certain point the debugger fails to respond. The entire process hangs with 100% cpu load.
I've heard (unconfirmed) reports from developers that when this occurs it might actually not be a hang but very slow, for example a step which usually takes a micro-second might be taking five minutes.
I've observed this with both the regular eclipse debugger (e.g. when you do debug as->python run) and the pydev network debugger. The point at which things stop working seems quite arbitrary but re-producable. I'm debugging purely numerical functions so there's no obvious socket or tcp activity which could disrupt a network debugger. Nevertheless, the point at which the failure occurs seems to be quite consistent for a given workstation & task.
I get almost exactly the same effect whether I debug using debug as->python run as I might if I use the pydev's network debugger ('import pydevd;pydevd.settrace()') - where one fails the other is also likely to fail.
If I were to run the script normally (no debugger) it executes without any hangs. Furthermore, if I replace the eclipse debugger with the old-skool inline break-point ('import pdb;pdb.set_trace()') everything works just fine. Pdb almost never goes wrong, however there are times I simply cannot use it, for example when I am debugging a python function invoked from Excel - as there is no console present.
I've noticed that this failure never seems to occur with trivial programs, hence I've tentatively concluded that the fault may be memory related.
So - is anybody aware of what might be causing this unreliability. Are there any tweaks I can make to the debugger or environment that might make the debugger work more reliably?
FYI, running Python 2.4.4 on Windows XP 32bit with the latest Eclipse & Pydev. Psyco (JIT Compiler) is not available.
There are known issues with Python 2.4 (so, if possible, get a newer version), still, if you're unable to get a newer version, at least use threadframe extension (http://pypi.python.org/pypi/threadframe). If None of that's possible, it's really expected that the debugger doesn't work as well as it could (as the debugger needs features that are not available in that version).
Not sure if its related, but I had a scenario where too many debug messages will cause my PC to become very slow. You can try go to Window -> Preferences -> Run/Debug -> Console... tick Limit console output, and reduce Console buffer size (mine is set to 40000).