I'm just learning Python and I have tried this simple loop based on Learn Python The Hard Way. With my basic understanding, this should keep printing "Hello", one letter at a time at the same position. This seems to be the case, but the print is not fluid, it doesn't spend the same amount of time on each character; some go very fast, and then it seems to get stuck for one or two seconds on one.
Can you explain why?
while True:
for i in ["H","e","l","l","o"]:
print "%s\r" % i,
you are running an infinite loop with very little work done in it, and most of it being printing. The bottleneck of such an application is how fast your output can be integrated in your running environment (you console).
There are various buffers involved, and the system can also schedule other processes and therefore pause your app for a few cycles.
Related
I have a python program which features a C++ core python wrapped. It's written in parallel as it is computationally very expensive and I'm currently making it run on a server in remote on a Ubuntu 16.04 platform.
The problem I'm experiencing is that, at a certain number of cycles (let's say 2000) for my test case, it freezes abruptly without giving error messages or anything. I detected the part of the code where it stops and is a python function which doesn't feature any for cycle (so I assume it's not stuck into a loop). I tried to simply comment the function where it gets stuck out of the code, as it does minor calculations and now, at the exact same number of cycles, it gets stuck a little bit ahead, this time inside the C++ written part. I'm starting to assume that a possibility is some memory problem related to the server.
Doing htop from terminal when the code is stuck I can see the units involved in the computation are fully loaded as they are currently involved in some unknown calculations. Moreover, the memory involved in the process (at least when the process is already stuck) is not fully occupied so it may not be a RAM problem neither.
I also tried to reduce drastically the number of output written at every cycle (which, I admit, where consistent in size) but nothing. With the optimum number of processors it takes like 20 minutes to get to the critical point of 2000 cycles so the problem is not easy reproducible.
It is the first time I'm experiencing these sort of problems, Is there anything else I can do to highlight the issue?
Thanks for the answer
Here is something you could try.
Write a code which checks which iterations is taking place and store all the variables at the start of the 2000th iteration.
Then use the same variable set to run the iteration again.
It won't solve your problem, but it will help in reducing the testing time, and consequently the time it takes for you to find the problem.
If it is definitely a memory issue, the code will not get stuck at 2000 (That's where you start) but will get stuck at 4000.
Then you can monitor memory at 2000th iteration and replicate that.
Does python automatically terminate or generate an error for an infinite "for" loop?
for example, would
list = [1]
for number in list:
print (number + 1)
list.append(number + 1)
cause the program to self-terminate or error?
I have tested it a bit, going as far as around 10,000.
No, the program will run forever. It may crash because your call stack is to big (if you use recursion somewhere) or you used to much memory though. Your example will eventually crash.
while True:
pass
Will run "forever" with no issues however.
That program will end, but it will be a very long time until it does. Eventually, you'll run the system out of memory for your list and the program will crash with an error.
It won't unless you specify a range or a limit. The way you currently have it it'll keep going until you get a memory error.
Also, don't use list as a variable since it's a reserved function by python.
So I am working on a Matlab application that has to do some communication with a Python script. The script that is called is a simple client software. As a side note, if it would be possible to have a Matlab client and a Python server communicating this would solve this issue completely but I haven't found a way to work that out.
Anyhow, after searching the web I have found two ways to call Python scripts, either by the system() command or editing the perl.m file to call Python scripts instead. Both ways are too slow though (tic tocing them to > 20ms and must run faster than 6ms) as this call will be in a loop that is very time sensitive.
As a solution I figured I could instead save a file at a certain location and have my Python script continuously check for this file and when finding it executing the command I want it to. Now after timing each of these steps and summing them up I found it to be incredibly much faster (almost 100x so for sure fast enough) and I cant really believe that, or rather I cant understand why calling python scripts is so slow (not that I have more than a superficial knowledge of the subject). I also found this solution to be really messy and ugly so just wanted to check that, first, is it a good idea and second, is there a better one?
Finally, I realize that the Python time.time() and Matlab tic, toc might not be precise enough to measure time correctly on that scale so also a reason why I ask.
Spinning up new instances of the Python interpreter takes a while. If you spin up the interpreter once, and reuse it, this cost is paid only once, rather than for every run.
This is normal (expected) behaviour, since startup includes large numbers of allocations and imports. For example, on my machine, the startup time is:
$ time python -c 'import sys'
real 0m0.034s
user 0m0.022s
sys 0m0.011s
In Python I am doing a number of different string processing functions in a program. The user enters a term in a form and the term is processed through different functions. These include, stemming, stop word removal, punctuation removal, spell checking and getting synonyms.
Stemming is done using the stemming package,
stop word & punctuation removal using string.replace() and REGEX,
spell checking using pyEnchant
getting synonyms using the Big Huge Thesaurus API.
The term is sent to an API. The results are returned and put through a hard-coded sorting process. After all that the results are output to the user. The whole process takes over 10 seconds which is too long. I'm wondering if the fact that I am using many extensions, thereby importing them, causing the long delays.
Hope this isn't against the stackoverflow rules but I'm new to python and this is the kind of thing that I need to know.
I'm wondering if the fact that I am using many extensions, thereby importing them, causing the long delays.
Very unlikely. If you just import once, then call in a loop, the loop should take most of the time. (Or are firing up a Python process per word/sentence?)
As a rule of thumb, computer programs tend to spend 90% of their time executing 10% of the code. That part is worth optimizing. Things like import statements are usually not. To find out where your program is spending its time, use a profiler.
Time how long each of the individual checks take. Then compare the results to see what is actually taking the most time.
import time
start = time.time()
#after the individual piece has completed
end = time.time()
print (end - start, "seconds")
It'd be interesting to actually know how long each component of the string processing is taking.
Im sure there is a better way to do this, but I am quite the newbie so I did it the only way I could figure it out. The thing is, I have a script that updates a textfile with the newest posts from an RSS feed (I got some help from you guys to figure it out). But I want this script to be automated, so I made this:
import time
import os
seconds = 3600
kjor = 'python vg.py'
time.sleep(seconds)
os.system(kjor)
time.sleep(seconds)
os.system(kjor)
time.sleep(seconds)
os.system(kjor)
I continued with copying those 24x downwards. I know this problably can be done alot better with some loop (while?), but Im afraid I dont have alot of knowledge in that field (yet).
My question, however, is as following: Can the system be damaged in any way if I let this run over a longer period of time?
To answer your question, no, this won't hurt anything. While the time.sleeps are sleeping, the program will take very little processing power and the rest of the system can run normally.
Now, as for your looping issue. If you want the code run forever (or until you stop the program) the code you want is
while True:
os.system(kjor)
time.sleep(seconds)
This is, literally, and infinite loop, but in this case that (is probably) what you want.
If you are attached to having a particular number of iterations, then you could do something like sunqiang's answer (repeated here)
for loop in xrange(240):
os.system(kjor)
time.sleep(seconds)
Finally, if you are on a Unix platform (such as Linux or Mac) you should take a look at cron, as it is designed to set up recurring programs to run and particular time periods. You could set it up to run your script every minute and it will happily do so until the end of time (or you remove it, whichever comes first).
Use xrange please, don't copying your code 24x times.
for loop in xrange(240):
time.sleep(seconds)
os.system(kjor)
It will not damage your system, as far as I know.
It does not damage any system and it is pretty common as well.
Just create a loop so as your application will gently stop running after some time;
Or better yet, make it check for a condition, maybe listen to a tcp port waiting for someone to ask it to quit (then you'll need to create a second application to send this quit message).