I have the following script 186.py:
S=[]
study=set([524287])
tmax=10**7
D={}
DF={}
dudcount=0
callcount=0
def matchval(t1,t2):
if t1==t2:
global dudcount
dudcount+=1
else:
global callcount
callcount+=1
D.setdefault(t1,set([]))
D.setdefault(t2,set([]))
D[t1].add(t2)
if t1 in D[t2]:
DF.setdefault(t1,set([]))
DF[t1].add(t2)
DF.setdefault(t2,set([]))
DF[t2].add(t1)
for k in xrange(27):
t1=(100003 - 200003*(2*k+1) + 300007*(2*k+1)**3)%(1000000)
S.append(t1)
t2=(100003 - 200003*(2*k+2) + 300007*(2*k+2)**3)%(1000000)
S.append(t2)
matchval(t1,t2)
t1=(100003 - 200003*(55) + 300007*(55)**3)%(1000000)
S.append(t1)
t2=(S[31]+S.pop(0))%(1000000)
S.append(t2)
matchval(t1,t2)
for k in xrange(29,tmax+1):
t1=(S[31]+S.pop(0))%(1000000)
S.append(t1)
t2=(S[31]+S.pop(0))%(1000000)
S.append(t2)
matchval(t1,t2)
D.setdefault(524287,set([]))
DF.setdefault(524287,set([]))
print D[524287]
print DF[524287]
print dudcount,callcount
print "Done"
The last line prints "Done" but python doesn't exit when this happens. I type the following command:
$ time python 186.py
And get the results:
set([810528L, 582178L, 49419L, 214483L, 974071L, 651738L, 199163L, 193791L])
set([])
11 9999989
Done
But I have to ctrl+C to get the time:
real 34m18.642s
user 2m26.465s
sys 0m11.645s
After the program outputs "Done" python CPU usage is very little... but the memory usage continues to grow... I used ctrl+C once it got to 80% of my system memory (its an old system).
What is going on here? What is the program doing after Done is printed? Shouldn't it be done?
Thanks,
Dan
I ran the same code on my 2 GHz dual-core laptop with 2GB RAM and it took about 1 1/2 minutes in Cygwin. The memory usage got up over 600 MB before the program quit and it took about 2-4 seconds after Done appeared for the prompt to come up and the memory to be released. However, I didn't see any memory increase after the Done appeared.
My guess is it has to do with memory management. After the Done appears, Python is working on freeing all of the memory which might take quite a while on an older machine with less RAM. I'm not sure why the memory actually increases unless there is just a delay in whatever is telling you how much memory is being used.
There are no indications in what you've posted that match the described symptoms. Perhaps the indentation in the executed code is stuffed and you are really about to run another loop around the whole pile. Note that only brave people are going to bother trying to reproduce your problem when it takes 34 minutes to run. Can you reproduce this problem in a shorter time?
When you did Control-C, what did the traceback say?
In any case, I'd strongly advise not having hard-coded constants(?) all over the place e.g. 524287 .... give it a meaningful name and do meaningful_name = 524287 at the start. Or if it's really a variable, feed it in via sys.argv.
Related
I was practicing writing python code, and I was trying to generate numbers of the Fibonacci series from 1,1,...
I wrote it and it was quite good, but then something odd happened which I write it down here:
# Generating Fibonacci Series in a List:
N = 1
M = 1
Fibonacci = []
ChoiceMethod = 0
ChoiceNum = 0
BelowNum = 0
ChoiceMethod = input('Choose Your Method of Work (1/2/3): ')
if int(ChoiceMethod) == 1:
ChoiceNum = input('Give your Choice Number: ')
while len(Fibonacci) != int(ChoiceNum):
Fibonacci.append(N)
Fibonacci.append(M)
N += M
M += N
print(Fibonacci)
You know it's the first part of the code, if you Run the code and enter 1 in the input (1st method) and then when it say Give your Choice Number, for example give it an even number like 2, it will work just fine, like any even numbers. as soon as you give it an odd number like 5, it will crash the PC. and I can't figure out why.
Found a Fix for it too, if you change while "!=" in "len(Fibonacci) != int(ChoiceNum)" to "<" it will work fine but idk what's wrong overally.
I'M TELLING YOU, IT WILL MOST PROBABLY CRASH YOUR PC TOO, Be Careful with it.
You are causing an infinite loop.
If you introduce an odd number like 5, see that you are adding two numbers at each iteration which means that the length of your Fibonacci list is never going to be odd, it will always be even. For that, you should check the number that you have to introduce and run the loop while checking that max number for your sequence and probably, breaking the loop or adding just 1 element.
As other answers explain, there is a bug in your program that means that it is an infinite loop. So what it actually does is that it builds an ever growing list of ever larger fibonacci numbers.
An infinitely large list requires an infinite amount of memory. And since the numbers themselves don't have an upper bound, that means even more memory.
But why does it crash your PC?
Well, Python doesn't have any built-in limit on the amount of memory it uses. So, a pathological Python script like your one will just keep asking the OS for more and more memory. This is detrimental to the stability of your PC. A couple of things can happen:
When the OS runs out of physical RAM, it will start allocating virtual memory. But virtual memory works by swapping virtual memory pages between physical RAM and your hard disk or SSD. And the OS typically will grab memory pages from other applications.
If there is too much swapping going, the paging rate exceeds your system's ability to keep up. Eventually some vital system service is impacted and ... the system crashes.
Some operating systems have defenses against programs that generate too much paging activity. For example, on Linux there is a kernel process called the OOM Killer which tries to identify processes that are creating excessive paging. When it finds one, it kills it.
The problem is that the OOM Killer can identify the wrong "culprit". But if the victim is a vital system service, and the OOM Killer kills it .... crash.
Fortunately, this kind of crash doesn't usually do any long term damage.
Try this:
Fibonacci = [1, 1]
...
while len(Fibonacci) != int(ChoiceNum):
Fibonacci.append(Fibonacci[-2] + Fibonacci[-1])
With this loop you will add the sum of last two numbers of your Fibonacci list to the list itself. This way your ChoiceNum can be uneven. !However!, do an extra check to see IF ChoiseNum == 1 return the first value of your Fibonacci list, else do the while loop.
I am trying to make a simple program in komodo edit using python that when it runs will print out 10 seconds worth of time in the command output.
The code is as follows:
import time
seconds = 0
while seconds != 10:
time.sleep(1)
seconds += 1
print(">", seconds)
When I run this in komodo edit, it doesn't print out the numbers as wanted.
I want the number 1 to be printed after one second of time, the number 2 to be printed after two seconds of time, and etc.
Instead, it prints out all the numbers (1-10) after 10 seconds of time.
I have run this exact same program in the python IDLE and it works as it should, printing one number per second.
What am I doing wrong or what do I not understand/know about?
The program is likely running in an environment where it does not believe its output is connected to a terminal, so stdout defaults to block buffering, not line buffering. Since you're outputting very little data, the buffer never fills, and is only flushed just before the program exits.
The simplest fix is to add the flush=True argument to your print call, so the buffer is explicitly flushed after each print:
print(">", seconds, flush=True)
I have a simple program that does the following:
1) User points a mouse somewhere,
2) then user presses Space,
3) and computer does certain amount of left-botton-mouse-clicks at that point.
The program works fine, there is only one problem - it eats 30-50% of processor time on a 4-core processor. Where is the problem?
import pyautogui
import ctypes
pyautogui.FAILSAFE = True
def get_space_state():
hllDll = ctypes.WinDLL ("User32.dll")
VK_SPACE = 0x20
return hllDll.GetKeyState(VK_SPACE)
while True:
if get_space_state() == -127 or get_space_state() == -128:
print ("yes")
pyautogui.click(clicks=40 , interval=0.01)
Thanks a lot.
Correct answer: I suspect a constant polling because of while True:. Insert sleep or pyautogui.PAUSE there (inside while loop, before if), if process sleeps for a while (even less then a second) it frees a lot of CPU cycles
Minor optimizations:
Also you're initializing entire User32.dll in every loop... twice (because of or), it seems.
And User32 is HUGE
Hints and notes:
If I remember python rules right, you just can move hllDll to a module level (above function defintion), get_space_state() will find it anyway. Or you can pass it as a parameter. And you don't need to redefine VK_SPACE every function call - though this is a micro-optimization
If all these fixes won't work, you should use debuggers, to find a true source of slowdowns
If you happen to have problems like this in the future, use something like Immunity or WinDbg to attach to process and see what's going on there
Firstly, very much Py Newby!
I have written a program to import data from a file and display it as an image using tkinter. The loop that is misbehaving runs thus:
Get data and plot
for x in xrange(WIDE):
for y in xrange(HIGH):
dataPointLo = inFile.read(1)
dataPointHi = inFile.read(1)
pixelValue = ((ord(dataPointLo) + 256*(ord(dataPointHi)))-31500)
colour = rgb[pixelValue]
#print below makes prog run!
print pixelValue
img.put(colour, to=(x,y))
As suggested by the comment, leaving out the print stops it working, but it locks one core of the processor at 100% for as long as you leave it (well at least 20 mins!). This effect occurs both in IDLE and from the command line (Ubuntu 12.04). Of course, the print to the IDLE window slows the program down, so I would like to remove it! Any thoughts?
it sounds like the process you are running takes a long time to complete, i would suggest that the reason you think it stops is because the window doesn't update while the process is busy unless you tell it to. i suggest you add a function like the following to your code and call it once before you enter your loop:
def keep_alive(self):
self.update()
self.after(100, self.keep_alive)
this way you are adding an event to update the window every 100ms(ish) to the event loop, which will keep the program responsive. you can adjust the timing to suit you, too often will slow your loop down, too far apart and the program will feel sluggish.
Short version: I have a program that prints to the terminal successive integer numbers in an infinite loop. At some point, the terminal became black, and no output is visible, but I can still execute commands.
Details:
I read this answer in PCG and wanted to try it in Python. Here it is:
#!/bin/python2
class Dier(object):
def __init__(self):
global ct
ct += 1
print ct
def __del__(self):
Dier()
ct = 0
Dier()
This program loops indefinitely, printing the number of iterations on each step. Left overnight (we get to the tenths of millions in a matter of minutes) executed from an Ubuntu gnome terminal, the terminall just shows black. The program is still running, and new lines appear, but nothing is visible. I killed the program, but the terminal, including the command prompt, is black. I can input commands, and they work, but no output is visible.
Why is this happening?
Some information I provided in the comments:
The memory used by the program (reported by top) remains constant and low.
I can print absurdly large numbers to the terminal without getting this behaviour. The largest number I have printed has 10^10^6 digits (of course, the terminal has forgotten the beginning of it, and only shows the last digits, but its log10 is 1246124). My program couldn't have gone that far, that would take millions of times the age of the universe.
On an added note, if I try to print something ever bigger than that, it just seems to freeze, using the CPU but without any output (while the original output was there, but invisible).
Seems like the output is being buffered. You should try and change the print to
print ct, "\n"
or
print ct
sys.stdout.flush()
alternatively you can tell python to not buffer the output by calling it with -u parameter (see https://docs.python.org/2/using/cmdline.html#cmdoption-u) or setting PYTHONUNBUFFERED=1 env variable (see https://docs.python.org/2/using/cmdline.html#envvar-PYTHONUNBUFFERED)