Develop Reference

Instance variable of multiprocessing Process in Python3 not being written on terminate

A surprising thing which I came across while writing a logic of saving some value during process termination was a bit strange for me. Writing a toy example program to show the problem.
import multiprocessing
import time
class A(multiprocessing.Process):
def __init__(self):
self.till = 0
super(A, self).__init__()
def run(self):
i = 0
while True:
print(i)
i += 1
self.till = i
time.sleep(1)
def terminate(self):
print("Terminating : {}".format(self.till))
super(A, self).terminate()
if __name__ == "__main__":
obj = A()
obj.start()
time.sleep(5)
obj.terminate()
The output for the above program is -
0
1
2
3
4
Terminating : 0
Why is terminate() not printing out 4? Anything I am missing?

What you are doing is actually running terminate on the main process, look at this code:
class A(multiprocessing.Process):
def __init__(self):
self.till = 0
super(A, self).__init__()
def run(self):
i = 0
print(os.getpid())
while True:
print(i)
i += 1
self.till = i
time.sleep(1)
def terminate(self):
print("Terminating : {}".format(self.till))
print(os.getpid())
super(A, self).terminate()
if __name__ == "__main__":
print("parent pid:")
print(os.getpid())
print("child pid:")
obj = A()
obj.start()
time.sleep(3)
obj.terminate()
Will lead to the output:
parent pid:
12111
child pid:
12222
0
1
2
Terminating : 0
12111
At terminate, you are actually sending SIGTERM to the child process, it is done from the parent process, thus the memory is of the parent, where there was no increments to self.till

init and terminate methods run on the main process hence the sub-process prints 0 for your terminate function. Run method only increments in the sub process. You can confirm this by using os.getpid() method in python.
Edit: This problem probably only occurs in Windows since it does not have a fork() system call like in Linux/Unix systems. Windows starts the whole module from the beginning to achieve the effect.

Python - Exiting while loop externally

I am writing a web server that will log temperatures. The user clicks "collect data" on the web interface, that then triggers a flask function to run a "collect temperature" function which just collects temperature data indefinitely. I then want to be able for the user to hit a "stop data collection" button that would stop the collect temperature function while loop.
The problem (my understanding at least) boils down to something like the following code:
class myClass:
counterOn = 0
num = 0
def __init__(self):
self.num = 0
def setCounterOn(self, value):
self.counterOn = value
def printCounterOn(self):
print self.counterOn
def count(self):
while True:
if self.counterOn == 1:
self.num += 1
print self.num
time.sleep(1)
then the server file:
myCounter = myClass.myClass()
myCounter.setCounterOn(1)
myCounter.count()
time.sleep(5)
myCounter.setCounterOn(0)
Ideally I would like the server file to create a counter object, then turn on and off the counter function externally. As it functions now, it is stuck in the while loop. I tried threading only to discover you can't pause or stop a thread. Am I looking at this completely wrong, or is it as simple as a try/except?
Edit:
The external file idea is great. I was having some trouble parsing the text file consistantly across my functions and wound up stumbleing across ConfigParsers to read .ini files. I think I'm going to go that way since eventually I want to have a PID controller controlling the temperature and it will be great to be able to store configurations externally.
I implemented just a while loop that looped forever and only recorded if it saw the config file configured to collect. The problem was that, in my flask file, i would run
#app.route('/startCollection', methods=['POST'])
def startCollectData():
print "collectPressed"
config.read('congif.ini')
config.set('main', 'counterOn', '1')
with open('config.ini', 'w') as f:
config.write(f)
C.count()
return "collect data pressed"
#app.route('/stopCollection', methods=['POST'])
def stopCollectData():
print "stop hit"
config.read('config.ini')
config.set('main', 'counterOn', '0')
with open('config.ini', 'w') as f:
config.write(f)
C.count()
return "stop pressed"
def count(self):
while True:
self.config.read('config.ini')
print self.num
time.sleep(1)
if self.config.get('main', 'counterOn') == '1':
self.num += 1
From my observation, the startDataCollection was getting stuck on count(). It would never return data, so then when i would try to stop data collection, the flask script wouldn't be there to interpret the stop command.
So i moved on to the mutex. That is exactly the functionality i thought would come out of the box with threads. It seems to be working fine, other than there is usually a really long delay in the 2nd time i stop collection.
#app.route('/')
def main():
print "MYLOG - asdf"
cls.start()
cls.pause()
return render_template('index.html')
#app.route('/startCollection', methods=['POST'])
def startCollectData():
print "collectPressed"
cls.unpause()
return "collect data pressed"
#app.route('/stopCollection', methods=['POST'])
def stopCollectData():
print "stop hit"
cls.pause()
return "collect data pressed"
results in the following output if i click start, stop, start, then stop:
collectPressed
1
10.240.0.75 - - [22/Apr/2016 15:58:42] "POST /startCollection HTTP/1.1" 200 -
2
3
4
5
6
7
8
9
stop hit
10.240.0.207 - - [22/Apr/2016 15:58:51] "POST /stopCollection HTTP/1.1" 200 -
collectPressed
10
10.240.0.166 - - [22/Apr/2016 15:58:57] "POST /startCollection HTTP/1.1" 200 -
11
12
13
14
15
16
stop hit
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
10.240.0.75 - - [22/Apr/2016 15:59:24] "POST /stopCollection HTTP/1.1" 200 -
So i hit stop, then it collects for 20 seconds, and then it finally stops. My collection points are going to be 5 minutes apart, so its not a big deal, but just curious.
import threading
import time
class myThread(threading.Thread):
num = 0
def __init__(self, threadID, name, counter):
threading.Thread.__init__(self)
self.threadID = threadID
self.name = name
self.counter = counter
self.mutex = threading.Lock()
self.paused = False
def pause(self):
if(not self.paused):
self.mutex.acquire()
self.paused = True
def unpause(self):
self.mutex.release()
self.paused = False
def run(self):
print "starting" + self.name
while True:
self.mutex.acquire()
self.num += 1
print self.num
time.sleep(1)
self.mutex.release()
Anyways, thanks for the help. I've been stuck on how to handle this for about 4 months and its great to finally make some progress on it!
Edit 2
Actually, just ran it again and it took 100 seconds for it to actually stop counting. Thats not going to cut it. Any idea whats going on?

I would try using threads again. The fact of the matter is that you have a computation that needs to run, while another instruction sequence (namely the GUI logic) also needs to execute.
I would approach the problem with mutex's (a standard concurrency control technique) which should be able to supply a pause/unpause functionality:
import time
import threading
class myClass(threading.Thread):
num = 0
def __init__(self):
super(myClass, self).__init__()
self.num = 0
self.mutex = threading.Lock()
self.paused = False
def pause(self):
if(not self.paused):
self.mutex.acquire()
self.paused = True
def unpause(self):
self.mutex.release()
self.paused = False
def run(self):
while True:
self.mutex.acquire()
self.num += 1
print self.num
time.sleep(1)
self.mutex.release()
cls = myClass()
cls.start()
time.sleep(10)
cls.pause()
time.sleep(2)
cls.unpause()
time.sleep(2)
And this should output: (or something similar)
1
2
3
4
5
6
7
8
9
10
(wait)
11
12

Threading does not thread in my Python code

First of all, I learned Python on my own from online tutorials and (mostly) learning by doing, so I might did some strange things in my code. :)
So, I am working om my first bigger project with Raspberry Pi, and for that I need codes running parallel. I wrote this part of my code for managing a simple D-pad:
Problematic code
import threading
import time
import pigpio
# input from the D-pad goes to these pins
BUTT_UP = 12
BUTT_LEFT = 16
BUTT_CENTER = 25
BUTT_RIGHT = 20
BUTT_DOWN = 21
class dpad_monitoring(threading.Thread):
'''thread for monitoring the D-Pad'''
def __init__(self, thread_ID, butt_up, butt_left, butt_center, butt_right, butt_down, res = 10.00):
threading.Thread.__init__(self)
self.running = True
self.thread_ID = thread_ID
# number of checks per sec
self.res = res
# key pins
self._pins = [butt_up, butt_left, butt_center, butt_right, butt_down]
#key monitor
self.last_pressed = 0
'''key numbers:
UP LEFT CENTER RIGHT DOWN
1 2 3 4 5 '''
# setting up GPIO
self.pi = pigpio.pi()
for i in range(0, len(self._pins)):
self.pi.set_mode(self._pins[i], pigpio.INPUT)
self.pi.set_pull_up_down(self._pins[i], pigpio.PUD_UP)
def stop(self):
'''stopping the thread cleanly'''
self.pi.stop()
self.running = False
def run(self):
'''checks which button is pressed as many times per sec as specified
in the res variable in init. If any of them is pressed, it suspends itself
until self.last_pressed is set to 0 again by main()'''
while self.running:
states = []
for i in range(0, len(self._pins)):
state = not self.pi.read(self._pins[i]) # pi.read returns 1L if the pin is high,
states.append(state) # what means the button is not pressed, 0L when pressed
for i in range(0, len(states)):
if states[i]:
self.last_pressed = i+1
'''UGLY AS SHIT but works now, will change to locks after the code works'''
if self.last_pressed != 0 :
while self.last_pressed != 0:
pass
else:
time.sleep(1/self.res)
print 'im groot' # for debugging
def main():
print 'ok' #debug
dpad = dpad_monitoring(0, BUTT_UP, BUTT_LEFT, BUTT_CENTER, BUTT_RIGHT, BUTT_DOWN)
dpad.run()
print 'okey' #debug again
while i != 3:
i = dpad.last_pressed
if i == 1:
print 'UP'
dpad.last_pressed = 0
if i == 2:
print 'LEFT'
dpad.last_pressed = 0
if i == 4:
print 'RIGHT'
dpad.last_pressed = 0
if i == 5:
print 'DOWN'
dpad.last_pressed = 0
print 'CENTER, stopping'
time.sleep(0.5)
dpad.stop()
if __name__ == '__main__':
main()
The problem is when I run the code, I get this:
ok
im groot
im groot
im groot
im groot
im groot
im groot
... (endless groot)
So it seems the code gets stuck at dpad.run(). Now AFAIK, the main point of threading is that the code continues after calling the run() function and can interact with the threading object, so I don't know what the he'll is going on. Could you, all experts, help me out please?
(Since the code after dpad.run() never ran, I don't know if it works, it may be all garbage. :P
The strange thing is that this simple test code works with no problem:
Cool code:
import threading
import time
class thread1(threading.Thread):
def __init__(self, threadID, start_from):
threading.Thread.__init__(self)
self.threadID = threadID
self.i = start_from
self.running = True
def run(self):
while self.running:
time.sleep(1)
self.i = self.i+1
def stop(self):
self.running = False
class thread2(threading.Thread):
def __init__(self, threadID, start_from):
threading.Thread.__init__(self)
self.threadID = threadID
self.i = start_from
self.running = True
def run(self):
while self.running:
time.sleep(0.5)
self.i = self.i+10
def stop(self):
self.running = False
thread1 = thread1(1, 10)
thread2 = thread2(2, 1)
thread1.start()
thread2.start()
for j in range(30):
print thread1.i, thread2.i
time.sleep(0.3)
thread1.stop()
thread2.stop()
The output is
10 1
10 1
10 11
10 11
11 21
11 31
11 31
12 41
12 41
12 51
13 61
13 61
13 71
13 71
14 81
14 91
14 91
15 101
15 101
15 111
16 121
16 121
16 131
16 131
17 141
17 151
17 151
18 161
18 161
18 171
------------------
(program exited with code: 0)
Press return to continue
So there I got the main thread plus the two other run parallel, unlikely to the previous code. What the he'll is going on?

Instead of
dpad.run()
do
dpad.start()
When calling run() directly you are skipping the whole threading functionality and using it as a regular class.

multithreading not working

#KSFT
currently frustrated by my inability to either decipher the formatting on stackoverflow OR make a simple python script work...
whats wrong here?
the program prompts for input to determine the value of dwell but it does not result in an led turning on.
import threading
import RPi.GPIO as GPIO
import time
GPIO.setmode(GPIO.BCM)
GPIO.setup(7, GPIO.OUT)
frequency = 0.05
dwell = 0.01
def get_input():
while True:
dwell=raw_input("Brightness: ")
input_thread=threading.Thread(target=get_input())
input_thread.start()
while True:
time.sleep(frequency)
GPIO.output(7, 1)
time.sleep(dwell)
GPIO.output(7, 0)

input_thread=threading.Thread(target=get_input())
is wrong!
input_thread=threading.Thread(target=get_input)
is right!
threading
class threading.Thread(group=None, target=None, name=None, args=(), kwargs={})
If you want to give arg to get_input ,you need to give it throw args and kwargs.
Example:
1 #!/usr/bin/python
2 #current's number of threads
3 import threading
4 import time
5
6 def worker():
7 print "test"
8 time.sleep(1)
9
10 for i in xrange(5):
11 t = threading.Thread(target=worker)
12 t.start()
13
14 print "current has %d threads" % (threading.activeCount() - 1)
target=worker().
target=worker.

Multi threading in python using parallel threads

I created two threads each running different functions.
What i tryed to achieve is if first thread ends then the second should also end ( i tryed achieving it using global variable)
Once both the threads end the same procedure should continue.
The script is not working as expected.
I am using Linux - Centos and python 2.7
#!/usr/bin/python
import threading
import time
import subprocess
import datetime
import os
import thread
command= "strace -o /root/Desktop/a.txt -c ./server"
final_dir = "/root/Desktop/"
exitflag = 0
# Define a function for the thread
def print_time(*args):
os.chdir(final_dir)
print "IN first thread"
proc = subprocess.Popen(command,shell=True,stdout=subprocess.PIPE, stderr=subprocess.PIPE)
proc.wait(70)
exitflag=1
def print_time1(*args):
print "In second thread"
global exitflag
while exitflag:
thread.exit()
#proc = subprocess.Popen(command1,shell=True,stdout=subprocess.PIPE, sterr=subprocess.PIPE)
# Create two threads as follows
while (1):
t1=threading.Thread(target=print_time)
t1.start()
t2=threading.Thread(target=print_time1)
t2=start()
time.sleep(80)
z = t1.isAlive()
z1 = t2.isAlive()
if z:
z.exit()
if z1:
z1.exit()
threading.Thread(target=print_time1).start()
threading.Thread(target=print_time1).start()
print "In try"
Where am i going wrong?

You could create an object to share state, and have the dependent thread check that state. Something like:
import threading
import time
import datetime
class Worker1( threading.Thread ):
def __init__(self, state):
super(Worker1, self).__init__()
self.state = state
def run(self):
print_time_helper("Worker1 Start")
time.sleep(4)
print_time_helper("Worker1 End")
self.state.keepOnRunning = False
class Worker2( threading.Thread ):
def __init__(self, state):
super(Worker2, self).__init__()
self.state = state
def run(self):
while self.state.keepOnRunning:
print_time_helper("Worker2")
time.sleep(1)
class State( object ):
def __init__(self):
self.keepOnRunning = True
def main():
state = State()
thread1 = Worker1(state)
thread2 = Worker2(state)
thread1.start()
thread2.start()
thread1.join()
thread2.join()
def print_time_helper(name):
print "{0}: {1}".format(name, datetime.datetime.now().time().strftime("%S"))
which will output something like this (numbers show current time seconds):
Worker1 Start: 39
Worker2: 39
Worker2: 40
Worker2: 41
Worker2: 42
Worker1 End: 43
However, this is a bit simplistic for most situations. You might be better off using message queues - this is a good intro.

Use a threading.Event instead of an int and wait for it to be set.
Also your logic appears to be wrong in print_time1 because your while loop will never run since exitflag is initially 0, but even if it was 1 it would still just exit immediately. It's not actually waiting on anything.