Develop Reference

Queues or Dictionaries for shared ressources in threads

I have two threads with while loops in them. The first process data that the second needs to elaborate in parallel. I need to share a variable.
let's introduce dummy input:
data = iter([1,2,3,4,5,6,7,8,9])
My first class of Thread:
import threading
from queue import Queue
import time
class Thread1(threading.Thread):
def __init__(self, queue):
threading.Thread.__init__(self)
self.queue = queue
_download = {}
def run(self):
i = 0
while True:
_download[i] = next(data)
self.queue.put(next(data))
time.sleep(1)
i += 1
My second class of Thread:
class Thread2(threading.Thread):
def __init__(self, queue):
threading.Thread.__init__(self)
self.queue = queue
def run(self):
while True:
self.queue.get()
time.sleep(3)
with the main method:
q = Queue(maxsize=10)
t = Thread1(q)
s = Thread2(q)
t.start()
s.start()
I illustratedthe two alternatives for the case. I can access queue variable from the second Thread but I also want that the second Thread access the dictionary.
what can I do to access also the dictionary from Thread2?
for which choice should I opt?

Send data to Python thread, then read a response using Queue

It's quite easy to send or receive data through threads using Queue's module when doing each thing at a time, but I didn't figure out how to send something to a thread, then expect for a return properly.
In the below example, I was expecting to send something to thread in order to be processed, then harvest the result, but the t.queue.get() in the main function receives what what just sent above instead of waiting for the thread to return. How can I get around it?
#!/usr/bin/env python3
from threading import Thread
from queue import Queue
class MyThread(Thread):
queue:Queue
def __init__(self, *args, **kwargs):
super().__init__()
self.queue = Queue()
self.daemon = True
# receives a name, then prints "Hello, name!"
def run(self):
while True:
val = self.queue.get()
if not val:
break
self.queue.put(f'Hello, {val}!')
def main():
t = MyThread()
t.start()
# sends string to thread
t.queue.put('Jurandir')
# expects to receive "Hello, Jurandir!",
# but "Jurandir" is immediately returned
ret = t.queue.get()
print(ret)
if __name__ == '__main__':
main()

Thing is that you are getting the alleged result immediately from the queue, and the worker has still not added the result. You can split into an "input queue" and a "results queue". And then wait in the main thread until there's some output in the queue.
#!/usr/bin/env python3
from threading import Thread, Lock
from queue import Queue
class MyThread(Thread):
def __init__(self, *args, **kwargs):
super().__init__()
self.input_queue = Queue()
self.results_queue = Queue()
self.daemon = True
# receives a name, then prints "Hello, name!"
def run(self):
while True:
val = self.input_queue.get()
if not val:
break
self.results_queue.put(f'Hello, {val}!')
def main():
t = MyThread()
t.start()
# sends string to thread
t.input_queue.put('Jurandir')
ret = t.results_queue.get()
while ret is None:
ret = t.results_queue.get()
print(ret)
if __name__ == '__main__':
main()

Worker process is not being called in simple example?

Here's my simple example code:
# -*- coding: utf-8 -*-
import multiprocessing
import time
def some_function():
return "hello!"
def get_address_type(address):
return 'tcp'
class PseudoThing():
def __init__(self, address):
self.address = address
self.server = None
def attach(self):
if get_address_type(self.address) == 'tcp':
ip, port = self.address.split(':', 1)
self.server = some_function()
else:
self.server = some_function()
def worker(self):
print("in worker!")
for i in range(10):
print("I'm a worker doing worker things...")
self.attach()
if self.server:
return True
time.sleep(0.2)
def parent(self):
print("I'm a parent doing parent things!")
def start_process(self):
p = multiprocessing.Process(target=self.worker)
print("starting process")
p.start()
return p
def main():
nvr = PseudoThing("kitty")
p = nvr.start_process()
p.join()
print("__name__ = {}".format(__name__))
if __name__ == "__main__":
print("doing main!")
main()
However, nothing is getting printed out from worker, as I would expect. Instead, this is the output of the program:
__name__ = __main__
doing main!
starting process
I am suspecting this has something to do with the class structure (I recall having issues with this on Python 2), but I am not entirely sure. Where can I add more print statements to figure what's going on? Where are things going wrong?

How to open file for each Python thread?

I have six threads
class ConsumerThread(Thread):
def __init__(self,queue):
super(ConsumerThread, self).__init__()
self.queue = queue
def run(self):
item = queue.get()
parser = MyHTMLParser()
new_con = parser.feed(item)
print(new_con)
queue.task_done()
I want to put new_con into file, with single file for one thread.

class ConsumerThread(Thread):
def __init__(self,queue):
super(ConsumerThread, self).__init__()
self.queue = queue
def run(self):
item = queue.get()
parser = MyHTMLParser()
new_con = parser.feed(item)
with open('file_name-%s' % self.getName(),
mode='w', encoding='utf8') as f:
print(new_con, file=f)
queue.task_done()
This will produce files with names like file_name-Thread-1 etc. You might want to use something nicer than this, but as a first shot, this solves your issues.

Regularly check whether a webserver is up with a Thread

I wrote a Threading class which tests whether a webserver is up or not.
import urllib
import threading
import time
import Queue
class Thread_CheckDeviceState(threading.Thread):
def __init__(self, device_ip, queue, inter=0.1):
self._run = True
self._codes = {}
self._queue = queue
self._device_ip = device_ip
self._inter = inter
self._elapsed = 0
threading.Thread.__init__(self)
def stop(self):
self._run = False
def run(self):
start = time.time()
while self._run:
try:
code = urllib.urlopen(self._device_ip).getcode()
except Exception:
code = "nope"
finally:
measure = time.time()
self._elapsed += measure-start
print self._elapsed, code
self._codes.update(
{self._elapsed:code}
)
time.sleep(self._inter)
self._queue.put(self._codes)
q = Queue.Queue()
thread = Thread_CheckDeviceState("http://192.168.1.3", q)
thread.start()
time.sleep(10)
thread.stop()
print q.get()
It works fine - until I disconnect my pc from the network. From that moment on the thread just does nothing until it is stopped. I would expect it to just continue and set the code to "nope", like I wrote it in the exception handler. Why doesn't it work

You need to use urllib2 instead, and specify a timeout parameter when you call urlopen().