Python multi processing start the script over again? - python

I am using multi processing in Python. The following is the demo of my code:
In function main:
from multiprocessing import Process
def __name__ == "__main__":
print "Main program starts here."
SOME CODE....
process_1 = Process(target=proc1, args = (arg1, arg2))
process_2 = Process(target=proc2, args = (arg3, arg4))
process_1.start()
process_2.start()
process_1.join()
process_2.join()
And in function proc1 and proc2:
def proc1(arg1, arg2):
print "Proc1 starts from here."
SOME CODE....
So what I expect to see as output is:
Main program starts here.
Proc1 starts from here.
Proc2 starts from here.
However, what I got is:
Main program starts here.
Main program starts here.
Main program starts here.
It seems that both the proc1 and proc2 start the main rather than the procs.
May I know what is wrong with my code?
Many thanks.

def __name__ == "__main__":
Should be:
if __name__ == "__main__":
But if I execute your code with python 2.7 I get:
def __name__ == "__main__":
^
SyntaxError: invalid syntax
If I rename def to if:
Main program starts here.
proc2
proc1
So I wonder how you get your code working (you probably modified it before pasting it here)

Related

Python: how to terminate a function of a script from another script

I have a script main.py which called a function fun from a library.
I want to exit only from fun continuing the script main.py, using for this purpose another script kill_fun.py.
I tried to use different bash commands (using os.system) with ps, but the pid it gives me is referred only to main.py.
Example:
-main.py
from lib import fun
if __name__ == '__main__':
try:
fun()
except:
do_something
do_something_else
-lib.py
def fun():
do_something_of_long_time
-kill_fun.py
if __name__ == '__main__':
kill_only_fun
You can do so by run fun in a different process.
from time import sleep
from multiprocessing import Process
from lib import fun
def my_fun():
tmp = 0
for i in range(1000000):
sleep(1)
tmp += 1
print('fun')
return tmp
def should_i_kill_fun():
try:
with open('./kill.txt','r') as f:
read = f.readline().strip()
#print(read)
return read == 'Y'
except Exception as e:
return False
if __name__ == '__main__':
try:
p = Process(target=my_fun, args=())
p.start()
while p.is_alive():
sleep(1)
if should_i_kill_fun():
p.terminate()
except Exception as e:
print("do sth",e)
print("do sth other thing")
to kill fun, simply echo 'Y' > kill.txt
or you can write a python script to write the file as well.
Explain
The idea is to start fun in a different process. p is a process handler that you can control. And then, we put a loop to check file kill.txt to see if kill command 'Y' is in there. If yes, then it call p.terminate(). The process will then get killed and continue to do next things.
Hope this helps.

Python multiprocessing creates sub-process using wrong function

I'm trying to write code that create sub-process using another module(demo_2.py),
and exit program if i get wanted value on sub-processes.
But result looks like this.
It seems that demo_1 makes two sub-process that run demo_1 and load demo_2.
I want to make sub-process only runs demo_2.
What did i missed?
demo_1.py
#!/usr/bin/env python
# -*- coding: utf-8 -*-
from multiprocessing import Process,Queue
import sys
import demo_2 as A
def multi_process():
print ("Function multi_process called!")
process_status_A = Queue()
process_status_B = Queue()
A_Process = Process(target = A.process_A, args = (process_status_A,))
B_Process = Process(target = A.process_A, args = (process_status_B,))
A_Process.start()
B_Process.start()
while True:
process_status_output_A = process_status_A.get()
process_status_output_B = process_status_B.get()
if process_status_output_A == 'exit' and process_status_output_B == 'exit':
print ("Success!")
break
process_status_A.close()
process_status_B.close()
A_Process.join()
B_Process.join()
sys.exit()
print ("demo_1 started")
if __name__ == "__main__":
multi_process()
demo_2.py
class process_A(object):
def __init__(self, process_status):
print ("demo_2 called!")
process_status.put('exit')
def call_exit(self):
pass
if process_status_A == 'exit' and process_status_B == 'exit':
should be
if process_status_A_output == 'exit' and process_status_B_output == 'exit':
Conclusion: The naming of variables is important.
Avoid long variable names which are almost the same (such as process_status_A and process_status_A_output).
Placing the distinguishing part of the variable name first helps clarify the meaning of the variable.
So instead of
process_status_A_output
process_status_B_output
perhaps use
output_A
output_B
Because Windows lacks os.fork,
on Windows every time a new subprocess is spawned, a new Python interpreter is started and the calling module is imported.
Therefore, code that you do not wish to be run in the spawned subprocess must be "protected" inside the if-statement (see in particular the section entitled "Safe importing of main module"):
Thus use
if __name__ == "__main__":
print ("demo_1 started")
multi_process()
to avoid printing the extra "demo_1 started" messages.

Killing a process launched from a process that has ended - Python

I am trying to kill a process in Python, that is being launched from another process and I am unable to find the correct place to place my ".terminate()".
To explain myself better I will post some example code:
from multiprocessing import Process
import time
def function():
print "Here is where I am creating the function I need to kill"
ProcessToKill = Process(target = killMe)
ProcessToKill.start()
def killMe():
while True:
print "kill me"
time.sleep(0.5)
if __name__ == '__main__':
Process1 = Process(target = function)
Process1.start()
My question is, where can I place ProcessToKill.terminate(), ideally without having to change the overall structure of the code?
You can hold onto the ProcessToKill object so that you can kill it later:
from multiprocessing import Process
import time
def function():
print "Here is where I am creating the function I need to kill"
ProcessToKill = Process(target = killMe)
ProcessToKill.start()
return ProcessToKill
def killMe():
while True:
print "kill me"
time.sleep(0.5)
if __name__ == '__main__':
Process1 = function()
time.sleep(5)
Process1.terminate()
Here, I've removed your wrapping of function in another Process object, because for the example it seems redundant, but you should be able to do the same thing with a Process that runs another Process.

Need help on tornado coroutines

I'm new to python and tornado. I was trying some stuff with coroutines.
def doStuff(callback):
def task():
callback("One Second Later")
Timer(1,task).start()
#gen.coroutine
def routine1():
ans = yield gen.Task(doStuff)
raise gen.Return(ans)
if __name__ == "__main__":
print routine1()
I'm trying to get the result of doStuff() function, which I expect to be "One Second Later". But it's not working. Any help would be appreciated. Thank you
What's probably happening is, you haven't started the IOLoop, nor are you waiting for your coroutine to complete before your script exits. You'll probably notice your script runs in a couple milliseconds, rather than pausing for a second as it ought. Do this:
if __name__ == "__main__":
from tornado.ioloop import IOLoop
print IOLoop.instance().run_sync(routine1)

Multiprocessing beside a main loop

I'm struggling with a issue for some time now.
I'm building a little script which uses a main loop. This is a process that needs some attention from the users. The user responds on the steps and than some magic happens with use of some functions
Beside this I want to spawn another process which monitors the computer system for some specific events like pressing specif keys. If these events occur then it will launch the same functions as when the user gives in the right values.
So I need to make two processes:
-The main loop (which allows user interaction)
-The background "event scanner", which searches for specific events and then reacts on it.
I try this by launching a main loop and a daemon multiprocessing process. The problem is that when I launch the background process it starts, but after that I does not launch the main loop.
I simplified everything a little to make it more clear:
import multiprocessing, sys, time
def main_loop():
while 1:
input = input('What kind of food do you like?')
print(input)
def test():
while 1:
time.sleep(1)
print('this should run in the background')
if __name__ == '__main__':
try:
print('hello!')
mProcess = multiprocessing.Process(target=test())
mProcess.daemon = True
mProcess.start()
#after starting main loop does not start while it prints out the test loop fine.
main_loop()
except:
sys.exit(0)
You should do
mProcess = multiprocessing.Process(target=test)
instead of
mProcess = multiprocessing.Process(target=test())
Your code actually calls test in the parent process, and that call never returns.
You can use the locking synchronization to have a better control over your program's flow. Curiously, the input function raise an EOF error, but I'm sure you can find a workaround.
import multiprocessing, sys, time
def main_loop(l):
time.sleep(4)
l.acquire()
# raise an EOFError, I don't know why .
#_input = input('What kind of food do you like?')
print(" raw input at 4 sec ")
l.release()
return
def test(l):
i=0
while i<8:
time.sleep(1)
l.acquire()
print('this should run in the background : ', i+1, 'sec')
l.release()
i+=1
return
if __name__ == '__main__':
lock = multiprocessing.Lock()
#try:
print('hello!')
mProcess = multiprocessing.Process(target=test, args = (lock, ) ).start()
inputProcess = multiprocessing.Process(target=main_loop, args = (lock,)).start()
#except:
#sys.exit(0)

Categories

Resources