Could you please explain me these basics.
server_init() -> start_db() -> it makes variable state = 5 -> server starts start_client() in 2 seconds -> client calls get_state() and gets value = 5
Is it right that client sees 5? How long will module variables live?
Thanks.
server.py:
def t3():
time.sleep(2)
start_client()
def server_init():
start_db()
thread = threading.Thread(target=t3)
thread.start()
server_init()
db.py
state = 0
q = queue.Queue()
def t2():
global state
while state < 5:
state += 1
global q
q.put(state)
def start_db():
thread = threading.Thread(target=t2)
thread.start()
def get_state():
print('q.get(): %s' % q.get())
print('state: %s' % state)
client.py
def t1():
get_state() # prints 5
def start_client():
print('start_client')
thread = threading.Thread(target=t1)
thread.start()
Modules are stored in the global mapping sys.modules. As long as there is any reference available to a module object, it'll remain alive. Typically, modules are not removed from sys.modules until the Python interpreter shuts down.
Related
I defined a class Node, which defines a listener service to constantly communicate and update local variables. The listener is started using multiprocessing. The class looks like:
# Pseudo-code
import multiprocessing
class Node(object):
def __init__(self, x):
self.variables = x
def listener(self):
while(True):
COMMUNICATE WITH OTHERS # Pseudo-code
UPDATE self.variable # Pseudo-code
print(self.variable) # local printer
def run(self):
p = multiprocessing.Process(target=self.listener)
p.start()
In the main process, I created two nodes a = Node(x1), b = Node(x2), and let them run
if __name__ == "__main__":
x1 = 1 # for example
x2 = 1000 # for example
a = Node(x1)
b = Node(x2)
a.run()
b.run()
while(True):
print(a.variable) # global printer
print(b.variable) # global printer
In this way, Node-a communicates with Node-b and updates its variables, and so does Node-b.
Now I come with a problem: Local printers output updated variable values correctly, but global printers do not. Actually, the global printers always output unchanged values (x1, x2, same as initial).
What's wrong with the code Or how to see the child process variables?
You won't be able to do that unless you use any mechanism to communicate with the parent. I recommend you use Manager dicts.
import random
import time
import multiprocessing as mp
class Child(mp.Process):
def __init__(self, shared_variables):
super(Child, self).__init__()
self.shared_variables = shared_variables
def run(self):
for _ in range(5): # Change shared variable value 5 times
self.shared_variables['var'] = random.randint(0, 10)
self.shared_variables['var1'] = random.randint(0, 10)
time.sleep(3)
if __name__ == "__main__":
shared_variables = mp.Manager().dict()
child = Child(shared_variables)
child.start()
while True:
print('Form parent')
for k, v in shared_variables.items():
print(f'-> {k}: {v}')
print('***********')
time.sleep(3)
And the output would look like this:
Form parent
-> var: 8
-> var1: 6
***********
Form parent
-> var: 7
-> var1: 7
***********
....
I found this code in online and while running this code I found print("This is awful {}".format(self)) this portion is not being executed. But if I
run without if like then both function are working (self.connecting()). I do not why this is happening. Can you please describe.
class MyThread(Thread):
def __init__(self, val):
''' Constructor. '''
Thread.__init__(self)
self.val = val
def run(self):
for i in range(1, self.val):
print('Value %d in thread %s' % (i, self.getName()))
self.printing_fun()
# Sleep for random time between 1 ~ 3 second
#secondsToSleep = randint(1, 5)
#time.sleep(secondsToSleep)
def connecting(self):
print "Establishing connection right now........."
def printing_fun(self):
# if i run like self.connecting() without previous if then all are
working fine.
if self.connecting():
print("This is awefull {}".format(self))
# Run following code when the program starts
if __name__ == '__main__':
# Declare objects of MyThread class
myThreadOb1 = MyThread(4)
myThreadOb1.setName('Thread 1')
myThreadOb2 = MyThread(4)
myThreadOb2.setName('Thread 2')
# Start running the threads!
myThreadOb1.start()
myThreadOb2.start()
# Wait for the thre`enter code here`ads to finish...
myThreadOb1.join()
myThreadOb2.join()
print('Main Terminating...')
Result:
Value 1 in thread Thread 1
Establishing connection right now.........
Value 2 in thread Thread 1
Establishing connection right now.........
Value 3 in thread Thread 1
Establishing connection right now.........
Value 1 in thread Thread 2
Establishing connection right now.........
Value 2 in thread Thread 2
Establishing connection right now.........
Value 3 in thread Thread 2
Establishing connection right now.........
Main Terminating...
nothing to do with threading. Look at this code:
def connecting(self):
print "Establishing connection right now........."
def printing_fun(self):
# if i run like self.connecting() without previous if then all are
# working fine.
if self.connecting():
print("This is awefull {}".format(self))
self.connecting() has no return statement, so python makes it return None.
And if None: condition is never met: it never enters the if
connecting is a stub of some connection procedure, but it's incorrectly implemented. To stub it properly you should just make it return something truthy:
def connecting(self):
print("Establishing connection right now.........")
return True
how do I update a variable inside a running thread, which is an infinite loop based on such variable?
a simplified version of what I tried is what follows, to no results of course, and I can't find any pointer.
import some_module as mod
import threading
class thr (threading.Thread):
NUM = 5 # set a default value to start the script
def run (self):
mod.NUM = NUM
mod.main_loop()
try:
thr().start()
time.sleep(1)
thr().NUM = 2
time.sleep(1)
thr().NUM = 6
time.sleep(1)
thr().NUM = 8
The problem is that you're creating a new thread each time you "call" (i.e. instantiate) thr. Change your code to
t = thr()
t.start()
time.sleep(1)
t.NUM = 2
time.sleep(1)
t.NUM = 6
time.sleep(1)
t.NUM = 8
time.sleep(1)
Maybe try use queue for change NUM variable.
https://docs.python.org/2/library/queue.html
Check examples here :
https://pymotw.com/2/Queue/
Generally speakinig the queue allows You send data between threads. Use get() for getting data from queue and put() for put data to queue.
I have a function to update a global/class variable.
So, What should care after regularly invoke such function as subthread?(in asynchronous way)
Or, any suggestions to avoid using this pattern? (the pathonic way)
import time
import threading
# through global variable or class variable
_a = 123
def update_a(): # may be called more than once
"slow updating process"
time.sleep(3)
global _a
_a += 10
return
if __name__ == '__main__':
print(_a)
th = threading.Thread(target=update_a)
th.setDaemon(True)
th.start()
print(_a)
# updating aynchrounously
time.sleep(5)
print(_a)
First of all, threads are a thing to avoid in Python altogether, but if you really want to, I'd do it like this. First, create a thread-safe object with a lock:
class ThreadSafeValue(object):
def __init__(self, init):
self._value = init
self._lock = threading.Lock()
def atomic_update(self, func):
with self._lock:
self._value = func(self._value)
#property
def value(self):
return self._value
then I'd pass that to the thread target function:
def update(val):
time.sleep(3)
val.atomic_update(lambda v: v + 10)
def main():
a = ThreadSaveValue(123)
print a.value
th = threading.Thread(target=update, args=(a,))
th.daemon = True
th.start()
print a.value
th.join()
print a.value
if __name__ == '__main__':
main()
That way you will avoid global variables and ensure the thread-safety.
This demonstrates that addition is not threadsafe (See Josiah Carlson' comment. effbot.org seems to be down right now; you can check out an archived version of the page through the wayback machine here.):
import threading
x = 0
def foo():
global x
for i in xrange(1000000):
x += 1
threads = [threading.Thread(target=foo), threading.Thread(target=foo)]
for t in threads:
t.daemon = True
t.start()
for t in threads:
t.join()
print(x)
yields some number less than 2000000. This shows that some calls to x += 1 did not properly update the variable.
The solution is to protect assignment to your global variable with a lock:
lock = threading.Lock()
def safe_foo():
global x
for i in xrange(1000000):
with lock:
x += 1
x = 0
threads = [threading.Thread(target=safe_foo), threading.Thread(target=safe_foo)]
for t in threads:
t.daemon = True
t.start()
for t in threads:
t.join()
print(x)
yields 2000000.
I'm trying to understand the basics of threading and concurrency. I want a simple case where two threads repeatedly try to access one shared resource.
The code:
import threading
class Thread(threading.Thread):
def __init__(self, t, *args):
threading.Thread.__init__(self, target=t, args=args)
self.start()
count = 0
lock = threading.Lock()
def increment():
global count
lock.acquire()
try:
count += 1
finally:
lock.release()
def bye():
while True:
increment()
def hello_there():
while True:
increment()
def main():
hello = Thread(hello_there)
goodbye = Thread(bye)
while True:
print count
if __name__ == '__main__':
main()
So, I have two threads, both trying to increment the counter. I thought that if thread 'A' called increment(), the lock would be established, preventing 'B' from accessing until 'A' has released.
Running the makes it clear that this is not the case. You get all of the random data race-ish increments.
How exactly is the lock object used?
Additionally, I've tried putting the locks inside of the thread functions, but still no luck.
You can see that your locks are pretty much working as you are using them, if you slow down the process and make them block a bit more. You had the right idea, where you surround critical pieces of code with the lock. Here is a small adjustment to your example to show you how each waits on the other to release the lock.
import threading
import time
import inspect
class Thread(threading.Thread):
def __init__(self, t, *args):
threading.Thread.__init__(self, target=t, args=args)
self.start()
count = 0
lock = threading.Lock()
def incre():
global count
caller = inspect.getouterframes(inspect.currentframe())[1][3]
print "Inside %s()" % caller
print "Acquiring lock"
with lock:
print "Lock Acquired"
count += 1
time.sleep(2)
def bye():
while count < 5:
incre()
def hello_there():
while count < 5:
incre()
def main():
hello = Thread(hello_there)
goodbye = Thread(bye)
if __name__ == '__main__':
main()
Sample output:
...
Inside hello_there()
Acquiring lock
Lock Acquired
Inside bye()
Acquiring lock
Lock Acquired
...
import threading
# global variable x
x = 0
def increment():
"""
function to increment global variable x
"""
global x
x += 1
def thread_task():
"""
task for thread
calls increment function 100000 times.
"""
for _ in range(100000):
increment()
def main_task():
global x
# setting global variable x as 0
x = 0
# creating threads
t1 = threading.Thread(target=thread_task)
t2 = threading.Thread(target=thread_task)
# start threads
t1.start()
t2.start()
# wait until threads finish their job
t1.join()
t2.join()
if __name__ == "__main__":
for i in range(10):
main_task()
print("Iteration {0}: x = {1}".format(i,x))