I am using two loops and cannot figure out how to properly update the dictionary in one loop and use it in the other loop.
In the first loop I am adding a new pair to the dictionary, but in the second loop I don't see any changes, how do I do it correctly?
import time
from multiprocessing import Process
dict_1 = {1:'1'}
def func_1():
while True:
dict_1.update({2:'2'})
print('Result func_1-',dict_1)
time.sleep(5)
def func_2():
while True:
print('Result func_2-',dict_1)
time.sleep(5)
if __name__ == '__main__':
p1 = Process(target=func_1)
p2 = Process(target=func_2)
p1.start()
p2.start()
p1.join()
p2.join()
Result func_1- {1: '1', 2: '2'} Result func_2- {1: '1'}
In the first cycle I see a new pair, but I do not see it in the second cycle.
You can solve this by using multiprocessing.Manager to create a managed dictionary for your purpose this way:
import time
from multiprocessing import Process, Manager
manager = Manager()
dict_1 = manager.dict({1:'1'})
def func_1():
while True:
dict_1.update({2:'2'})
print('Result func_1-',dict_1)
time.sleep(5)
def func_2():
while True:
print('Result func_2-',dict_1)
time.sleep(5)
if __name__ == '__main__':
p1 = Process(target=func_1)
p2 = Process(target=func_2)
p1.start()
p2.start()
p1.join()
p2.join()
Related
hellow,
please some help.
i want to take variables when using repeating statement.
Actually in my code, there are so many variables and function to handle variables.
so i have to use multiprocess for some reason, but it's doesn't work for what i want.
below is simple code,
please help me.
from multiprocessing import Process, Manager
import time
def a(final_list):
c=0
while True:
c += 1
final_list.append(c)
time.sleep(1)
print(final_list)
def b(final_list):
while True:
print(final_list[-1])
time.sleep(1)
if __name__ == '__main__':
manager = Manager()
final_list = []
final_list = manager.list()
#print(a)
p1 = Process(target=a, args=(final_list,))
p2 = Process(target=b, args=(final_list,))
p1.start()
time.sleep(3)
p2.start()
I think you forgot to use join() for the processes. try this:
from multiprocessing import Process, Manager
import time
def a(final_list):
c=0
while True:
c += 1
final_list.append(c)
time.sleep(1)
print(final_list)
def b(final_list):
while True:
print(final_list[-1])
time.sleep(1)
if __name__ == '__main__':
with Manager() as manager:
final_list = manager.list()
p1 = Process(target=a, args=(final_list,))
p2 = Process(target=b, args=(final_list,))
p1.start()
time.sleep(3)
p2.start()
p1.join()
p2.join()
So I want to start a nested while loop in one multiprocessing function from another multiprocessing function. In one function, I'm changing a variable (action) to "fn2", and in the other function there is a nested while loop whose condition is while action == "fn2":.
See code:
from multiprocessing import Process
running = True
action = None
def func1():
global action
if 1+1 == 2:
action = "fn2"
print(action)
def func2():
while running:
while action == "fn2":
print("fn2")
if __name__ == '__main__':
p1 = Process(target=func1)
p1.start()
p2 = Process(target=func2)
p2.start()
p1.join()
p2.join()
However, when I run it, the code just prints "fn2" once (confirming that action is equal to "fn2"). But the nested loop inside func2() does not execute. Sorry if the answer is obvious, I'm new to multiprocessing.
i added two comments (with print statements) to highlight the error.
basically action=None in func2() so that is why...
from multiprocessing import Process
running = True
action = None
def func1():
global action
if 1+1 == 2:
action = "fn2"
print(action)
def func2():
while running:
print('got here') # <--- loops infinitly here
print(action) # <--- this is none
while action == "fn2":
print("fn2")
if __name__ == '__main__':
p1 = Process(target=func1)
p1.start()
p2 = Process(target=func2)
p2.start()
p1.join()
p2.join()
In order to share values when multiprocessing, which is called Sharing state between processes you need to use value or array for a single device shared memory or alternatively, Manager for networks of servers.
Here is a link:
https://docs.python.org/3/library/multiprocessing.html
The basic format looks like this:
from multiprocessing import Process, Value, Array
def f(n, a):
n.value = 3.1415927
for i in range(len(a)):
a[i] = -a[i]
if __name__ == '__main__':
num = Value('d', 0.0)
arr = Array('i', range(10))
p = Process(target=f, args=(num, arr))
p.start()
p.join()
print(num.value)
print(arr[:])
So in the case of the question what the variable action is equivalent to n (variable) or a (list) etc.. and this can be shares across functions.
Also note that one can parse arguments into multiprocess functions with the args keyword: args=(num, arr ...).
I have two functions and needed the return values to proceed with the further part of the script...but currently my code giving only the output of the first function...
import multiprocessing
def gm(name):
h = "Good Morning"+str(name)
qout.put(h)
def sal(name):
k="Hi "+str(name)
qout.put(k)
if __name__ == '__main__':
qout = multiprocessing.Queue()
p1 = multiprocessing.Process(target=gm, args=("ashin",))
p2 = multiprocessing.Process(target=sal, args=("ashin",))
p1.start()
p2.start()
p1.join()
p2.join()
result = qout.get()
#output - "Good Morning ashin"
#required output - "Good Morning ashin" & "Hi ashin"
Appreciate your help......
qout.get() gets you the first element from queue. I do not know the bigger picture of what you're are trying to achieve, but you can get all elements from queue like in the following.
from multiprocessing import Process, Queue
def gm(name):
h = "Good Morning"+str(name)
qout.put(h)
def sal(name):
k="Hi "+str(name)
qout.put(k)
if __name__ == '__main__':
qout = Queue()
p1 = Process(target=gm, args=("ashin",))
p2 = Process(target=sal, args=("ashin",))
p1.start()
p2.start()
p1.join()
p2.join()
list1 = []
while not qout.empty():
list1.append(qout.get())
temp = list(map(str, list1))
print(" & ".join(temp))
output
Hi ashin & Good Morningashin
Instead of managing your own output queue, just use the latest Python 3 concurrency features:
from concurrent.futures import as_completed, ProcessPoolExecutor
def gm(name):
return f'Good Morning {name}'
def sal(name):
return f'Hi {name}'
if __name__ == '__main__':
with ProcessPoolExecutor() as exe:
futures = [exe.submit(x, 'ashin') for x in (gm, sal)]
for future in as_completed(futures):
print(future.result())
Why while loop is ignored in work1? I would like to update value from string to another value in loop and output this value in process work2. Also already tried with Queue, but problem is I have only one variable which I would like to update in work1 and access to it at work2.
from multiprocessing import Process, Manager, Value
from ctypes import c_char_p
import time
def work1(string):
i = 2
string.value = i
# while True:
# print("work1")
# string.value = i + 1
# time.sleep(2)
def work2(string):
while True:
print("Value set in work1 " + str(string.value))
time.sleep(2)
if __name__ == '__main__':
manager = Manager()
string = manager.Value(int, 0);
p1=Process(target=work1, args=(string,))
p1.start()
p1.join()
p2=Process(target=work2, args=(string,))
p2.start()
p2.join()
That is because you didn't make your program parallel with two processes, but instead, two processes run in tandem. What you need to do is to start both process before any join. Like my modification below:
from multiprocessing import Process, Manager, Value
from ctypes import c_char_p
import time
def work1(string):
i = 2
string.value = i
while True:
i = i+1
string.value = i
print("work1 set value to "+str(string.value))
time.sleep(2)
def work2(string):
while True:
print("Value set in work1 " + str(string.value))
time.sleep(2)
if __name__ == '__main__':
manager = Manager()
string = manager.Value(int, 0, lock=False);
p1=Process(target=work1, args=(string,))
p2=Process(target=work2, args=(string,))
p1.start()
p2.start()
p2.join()
p1.join()
Indeed, if you write the code in this way, the join never happened due to the infinite while loop.
I am trying to use python multiprocessing to fill a queue with strings and then print them back out but am having trouble. Could someone point out what I am doing wrong?
import multiprocessing
my_q = multiprocessing.Queue()
my_list =[i for i in range(0,100)]
def enqueue(q):
for data in my_list:
q.put(data)
def get_it(q):
while not q.empty():
item = q.get()
print(item)
p1 = multiprocessing.Process(target=enqueue, args=(my_q,))
p2 = multiprocessing.Process(target=get_it, args=(my_q,))
p1.start()
p2.start()
p1.join()
p2.join()
This program executes without printing anything.
If get_it is executed before the queue is populated, it will return immediately, printing nothing.
You need to make sure the queue is populated before the get_it is called.
For example, wait until enqueue is called until all items are enqueued:
...
p1 = multiprocessing.Process(target=enqueue, args=(my_q,))
p1.start()
p1.join()
p2 = multiprocessing.Process(target=get_it, args=(my_q,))
p2.start()
p2.join()
Or modifying get_it like below not to end too early:
...
def get_it(q):
while True:
item = q.get()
if item is None: # loop until sentinel value (None) appear.
break
print(item)
my_list.append(None) # sentinel value to denote end of input value
p1 = multiprocessing.Process(target=enqueue, args=(my_q,))
p2 = multiprocessing.Process(target=get_it, args=(my_q,))
p1.start()
p2.start()
p1.join()
p2.join()
or use multiprocess.pool.Pool.map instead:
import multiprocessing.pool
def get_it(item):
print(item)
pool = multiprocessing.pool.Pool()
pool.map(get_it, range(100))