When we launch threads, is it known for SURE which thread will be executed first or is it something not predictable ?
I say this because square is always called first and then cube.
import threading
def print_cube(num):
# function to print cube of given num
print("Cube: {}".format(num * num * num))
def print_square(num):
# function to print square of given num
print("Square: {}".format(num * num))
if __name__ == "__main__":
# creating thread
cuadrado = threading.Thread(target=print_square, args=(10,))
cubo = threading.Thread(target=print_cube, args=(10,))
# starting thread 1
cuadrado.start()
# starting thread 2
cubo.start()
print("Done!")
I would like to understand the method Threading.start()
Does the order of calling Threading start() matter?
But, if I sleep the yarns with the same time, then it is random order
import threading
import time
def print_cube(num):
# function to print cube of given num
time.sleep(3)
print("Cube: {}".format(num * num * num))
def print_square(num):
# function to print square of given num
time.sleep(3)
print("Square: {}".format(num * num))
if __name__ == "__main__":
# creating thread
cuadrado = threading.Thread(target=print_square, args=(10,))
cubo = threading.Thread(target=print_cube, args=(10,))
# starting thread 1
cuadrado.start()
# starting thread 2
cubo.start()
# both threads completely executed
print("Done!")
I would like to understand the method Threading.start()
A Python threading.Thread object is not the same thing as a thread. A thread is an object in the operating system—separate from your code. I like to think of a thread as an agent who executes your target function.
The purpose of the Python Thread class is, to provide a platform-independent interface to the various different thread APIs of various different operating systems. One peculiarity of Python's Thread is that it does not actually create the operating system thread until you call its start() method. That's what start() does: It creates the underlying OS thread.
Does the order of calling Threading start() matter?
Depends what you mean. Your program definitely always starts the cuadrado thread before it starts the cubo thread, but the whole point of threads is to provide a means to achieve concurrency in your program; and what "concurrency" means is that the things happening in different threads are not required to happen in any definite order. By calling print_cube() and print_square() in different threads, you effectively are telling Python (and the OS) that you don't care which one prints first.
Maybe print_square() will always be called first on your computer. Maybe print_cube() will always be called first on somebody else's computer. Maybe it will be unpredictable which one goes first on a third computer.
Sounds a little chaotic, but the reason why we like concurrency is that it gives the OS and the Python system more freedom to get things done in the most efficient order. E.g., if one thread is waiting for some network packet to arrive, some other thread can be allowed to do some useful work. So long as the "useful work" doesn't need the packet that the other thread was waiting for, that's a Good Thing.
Related
I am using threading python module. I want to execute a function which runs an expression entered by a user. I want to wait for it to finish execution or until a timeout period is reached. The following code should timeout after 5 second, but it never times out.
def longFunc():
# this expression could be entered by the user
return 45 ** 10 ** 1000
thread = threading.Thread(target=longFunc, args=(), daemon=True)
thread.start()
thread.join(5.0)
print("end") # never reaches this point :(
Why is this and how can I fix this behaviour? Should I try to use multiprocessing instead?
I suspect in this case you're hitting an issue where the join can't execute while the global interpreter lock is held by the very long-running calculation, which I believe would happen as a single atomic operation. If you change longFunc to something that happens over multiple instructions, such as a busy loop, e.g.
def longFunc():
while True:
pass
then it works as expected. Is the single expensive calculation realistic for your situation or did the example just happen to hit a very bad case?
Using the multiprocessing module does appear to resolve this issue:
from multiprocessing import Process
def longFunc():
# this expression could be entered by the user
return 45 ** 10 ** 1000
if __name__ == "__main__":
thread = Process(target=longFunc, args=(), daemon=True)
thread.start()
thread.join(5.0)
print("end")
This prints "end" as expected.
I am trying to run a code, with multiple threads, the user can decide how many threads he wants to run. I tried doing it with the threading module in Python 3.7
My code is shown below, but my problem is, instead of running all the threads together, it runs one after the other...
import threading
x=int(input("Enter number of threads: "))
def main():
print("My main function")
print("Does some stuff...")
while x > 0:
print("Starting Threads.")
x=x-1 #At every time the while loops gets passed, x gets decremented, so once it hits 0 it stops
t1=threading.Thread(target=main) #for every time the loop passes, a new thread gets created
t1.start() #and the thread starts here
Now I need to find out, how can I make it, that they run at the same time, not one after the other. Thanks
your code runs in parallel (note: just on one single core though; that is a python limitation due the global interpreter lock).
to make it more obvious change your main function a little; the way it is right now it just finishes too quickly. i suggest:
from time import sleep
from random import random
def main():
print("main starting")
sleep(random())
print("main done")
this will output something like
Enter number of threads: 4
Starting Threads.
main starting
Starting Threads.
main starting
Starting Threads.
main starting
Starting Threads.
main starting
main done
main done
main done
main done
I want to learn how to run a function in multithreading using python. In other words, I have a long list of arguments that I want to send to a function that might take time to finish. I want my program to loop over the arguments and call the functions on parallel (no need to wait until the function finishes fromt he forst argument).
I found this example code from here:
import Queue
import threading
import urllib2
# called by each thread
def get_url(q, url):
q.put(urllib2.urlopen(url).read())
theurls = ["http://google.com", "http://yahoo.com"]
q = Queue.Queue()
for u in theurls:
t = threading.Thread(target=get_url, args = (q,u))
t.daemon = True
t.start()
s = q.get()
print s
My question are:
1) I normally know that I have to specify a number of threads that I want my program to run in parallel. There is no specific number of threads in the code above.
2) The number of threads is something that varies from device to device (depends on the processor, memory, etc.). Since this code does not specify any number of threads, how the program knows the right number of threads to run concurrently?
The threads are being created in the for loop. The for loop gets executed twice since there are two elements in theurls . This also answers your other two questions. Thus you end up with two threads in the program
plus main loop thread
Total 3
I have two different functions f, and g that compute the same result with different algorithms. Sometimes one or the other takes a long time while the other terminates quickly. I want to create a new function that runs each simultaneously and then returns the result from the first that finishes.
I want to create that function with a higher order function
h = firstresult(f, g)
What is the best way to accomplish this in Python?
I suspect that the solution involves threading. I'd like to avoid discussion of the GIL.
I would simply use a Queue for this. Start the threads and the first one which has a result ready writes to the queue.
Code
from threading import Thread
from time import sleep
from Queue import Queue
def firstresult(*functions):
queue = Queue()
threads = []
for f in functions:
def thread_main():
queue.put(f())
thread = Thread(target=thread_main)
threads.append(thread)
thread.start()
result = queue.get()
return result
def slow():
sleep(1)
return 42
def fast():
return 0
if __name__ == '__main__':
print firstresult(slow, fast)
Live demo
http://ideone.com/jzzZX2
Notes
Stopping the threads is an entirely different topic. For this you need to add some state variable to the threads which needs to be checked in regular intervals. As I want to keep this example short I simply assumed that part and assumed that all workers get the time to finish their work even though the result is never read.
Skipping the discussion about the Gil as requested by the questioner. ;-)
Now - unlike my suggestion on the other answer, this piece of code does exactly what you are requesting:
from multiprocessing import Process, Queue
import random
import time
def firstresult(func1, func2):
queue = Queue()
proc1 = Process(target=func1,args=(queue,))
proc2 = Process(target=func2, args=(queue,))
proc1.start();proc2.start()
result = queue.get()
proc1.terminate(); proc2.terminate()
return result
def algo1(queue):
time.sleep(random.uniform(0,1))
queue.put("algo 1")
def algo2(queue):
time.sleep(random.uniform(0,1))
queue.put("algo 2")
print firstresult(algo1, algo2)
Run each function in a new worker thread, the 2 worker threads send the result back to the main thread in a 1 item queue or something similar. When the main thread receives the result from the winner, it kills (do python threads support kill yet? lol.) both worker threads to avoid wasting time (one function may take hours while the other only takes a second).
Replace the word thread with process if you want.
You will need to run each function in another process (with multiprocessing) or in a different thread.
If both are CPU bound, multithread won help much - exactly due to the GIL -
so multiprocessing is the way.
If the return value is a pickleable (serializable) object, I have this decorator I created that simply runs the function in background, in another process:
https://bitbucket.org/jsbueno/lelo/src
It is not exactly what you want - as both are non-blocking and start executing right away. The tirck with this decorator is that it blocks (and waits for the function to complete) as when you try to use the return value.
But on the other hand - it is just a decorator that does all the work.
I have two functions, draw_ascii_spinner and findCluster(companyid).
I would like to:
Run findCluster(companyid) in the backround and while its processing....
Run draw_ascii_spinner until findCluster(companyid) finishes
How do I begin to try to solve for this (Python 2.7)?
Use threads:
import threading, time
def wrapper(func, args, res):
res.append(func(*args))
res = []
t = threading.Thread(target=wrapper, args=(findcluster, (companyid,), res))
t.start()
while t.is_alive():
# print next iteration of ASCII spinner
t.join(0.2)
print res[0]
You can use multiprocessing. Or, if findCluster(companyid) has sensible stopping points, you can turn it into a generator along with draw_ascii_spinner, to do something like this:
for tick in findCluster(companyid):
ascii_spinner.next()
Generally, you will use Threads. Here is a simplistic approach which assumes, that there are only two threads: 1) the main thread executing a task, 2) the spinner thread:
#!/usr/bin/env python
import time
import thread
def spinner():
while True:
print '.'
time.sleep(1)
def task():
time.sleep(5)
if __name__ == '__main__':
thread.start_new_thread(spinner, ())
# as soon as task finishes (and so the program)
# spinner will be gone as well
task()
This can be done with threads. FindCluster runs in a separate thread and when done, it can simply signal another thread that is polling for a reply.
You'll want to do some research on threading, the general form is going to be this
Create a new thread for findCluster and create some way for the program to know the method is running - simplest in Python is just a global boolean
Run draw_ascii_spinner in a while loop conditioned on whether it is still running, you'll probably want to have this thread sleep for a short period of time between iterations
Here's a short tutorial in Python - http://linuxgazette.net/107/pai.html
Run findCluster() in a thread (the Threading module makes this very easy), and then draw_ascii_spinner until some condition is met.
Instead of using sleep() to set the pace of the spinner, you can wait on the thread's wait() with a timeout.
It is possible to have a working example? I am new in Python. I have 6 tasks to run in one python program. These 6 tasks should work in coordinations, meaning that one should start when another finishes. I saw the answers , but I couldn't adopted the codes you shared to my program.
I used "time.sleep" but I know that it is not good because I cannot know how much time it takes each time.
# Sending commands
for i in range(0,len(cmdList)): # port Sending commands
cmd = cmdList[i]
cmdFull = convert(cmd)
port.write(cmd.encode('ascii'))
# s = port.read(10)
print(cmd)
# Terminate the command + close serial port
port.write(cmdFull.encode('ascii'))
print('Termination')
port.close()
# time.sleep(1*60)