Python multithreading/subprocess with functions - python

I want to execute two commands 1st in background and second on top of it.
import time
loop =[ 1,100]
start_time_loop = time.time()
for in loop:
print i
end_time_loop = time.time()
multi()
start_time_func = time.time()
c= 5*2
end_time_func = time.time()
Loop should run in background while multiplication is done.
I want to prove:
start_time_loop < start_time_func
end_time_func << end_time_loop
Any pointers will be helpful.

You need to use multiprocessing to do what you want. It basically starts a new (cloned) copy of python in the background.
import time
from multiprocessing import Process
def thing_1():
"""We'll run this in a subprocess."""
for i in range(10):
print('thing_1: {}'.format(i))
# let's make it take a bit longer
time.sleep(1)
def thing_2():
"""We'll run this as a normal function in the current python process."""
time.sleep(1)
c = 5 * 2
print('thing_2: {}'.format(c))
# let's make this take a bit longer too
time.sleep(1)
if __name__ == '__main__':
# start the first thing running "in the background"
p = Process(target=thing_1)
p.start()
# keep a record of when we started it running
start_thing_1 = time.time()
# let's run the other thing
start_thing_2 = time.time()
thing_2()
end_thing_2 = time.time()
# this will wait for the first thing to finish
p.join()
end_thing_1 = time.time()
print('thing 1 took {}'.format(end_thing_1 - start_thing_1))
print('thing 2 took {}'.format(end_thing_2 - start_thing_2))
At the end you'll see:
thing 1 took 10.020239114761353
thing 2 took 2.003588914871216
So, while thing_1 is running in the background your local python can carry on doing other things.
You need to use special mechanisms to transfer any information between the two copies of python. And printing will always be a bit weird because you don't really know which copy of python is going to print next.

Related

How to sleep python script for xx minutes after every hour execution?

I am trying to make a python script that works in a loop mode with iteration through a text file to run for periods of one hour and make 30minute pauses between each hour loop .
After some searching I found this piece of code :
import datetime
import time
delta_hour = 0
while:
now_hour = datetime.datetime.now().hour
if delta_hour != now_hour:
# run your code
delta_hour = now_hour
time.sleep(1800) # 1800 seconds sleep
# add some way to exit the infinite loop
This code has a few issues though :
It does not consider one hour periods since the script starts running
It does not seem to work continuously for periods over one hour
Considering what I am trying to achieve (running script 1hour before each time it pauses for 30mins) what is the best approach to this ? Cron is not an option here .
For clarification :
1hour run -- 30min pause -- repeat
Thanks
Here is a so simple code, I have written for teaching purposes, which is very clear
from datetime import datetime
class control_process():
def __init__(self, woking_period, sleeping_period):
self.woking_period = woking_period # working period in minutes
self.sleeping_period = sleeping_period # sleeping period in minutes
self.reset()
def reset(self):
self.start_time = datetime.utcnow() # set starting point
def manage(self):
m = (datetime.utcnow() - self.start_time).seconds / 60 # how long since starting point
if m >= self.woking_period: # if exceeded the working period
time.sleep(self.sleeping_period * 60) # time to sleep in seconds
self.reset() # then reset time again
return # go to continue working
cp = control_process(60, 30) # release for 60 minutes and sleep for 30 minutes
while True: # you code loop
cp.manage()
'''
your code
'''
in which 'control_processobject - I calledcp- callscp.manage()` inside your executing loop.
you reset time via cp.reset() before going in the loop or whenever you want
Based on Comments
The simplicity I mean is to add this class to your general library so you can use it whenever you want by instantiation of cp then one or two controlling functions 'cp.manage()` which control the working cycles, and cp.reset() if you want to use it in another location of the code. I believe that use a function is better than a long condition statement.
Using the default library you could do something like call the script itself using subprocess. By checking whether conditions are met the process could do a task and call itself. Extending the logic with a kill pill would make it stop (I leave that up to you).
import argparse, time
from subprocess import call
DELAY = 60 * 30 # minutes
WORK_TIME = 60 * 60 # minutes
parser = argparse.ArgumentParser()
parser.add_argument("-s",
help = "interval start time",
type = float,
default = time.time())
parser.add_argument("-t",
help = "interval stop time",
type = float,
default = time.time() + WORK_TIME)
def do_task():
# implement task
print("working..")
return
if __name__ == "__main__":
args = parser.parse_args()
start = args.s
stop = args.t
# work
if start < time.time() < stop:
do_task()
# shift target
else:
start = time.time() + DELAY
stop = start + WORK_TIME
call(f"python test.py -t {stop} -s {start}".split())
The simplest solution I could come up with was the following piece of code, which I added inside my main thread :
start_time = int(time())
... #main thread code
#main thread code end
if int(time() - start_time >= 60 * 60):
print("pausing time")
sleep(30 * 60)
start_time = int(time())
From the moment the script starts this will pause every hour for 30mins and resume afterwards .
Simple yet effective !

How to create a timer function that can be called multiple times before it ends?

I want to set multiple timers (same function) at the same time, but with different ending times. Coding in python 3.
My code currently is:
import time
def timer(t):
start = time.time()
stop = False
while not stop:
if time.time()> start+t:
print("I'm done counting to %d" % t)
stop = True
timer(4)
timer(1)
timer(5)
Now I would like it would first print 1, then 4 and finally 5, but instead it runs completely timer(4) and only after that it continues to the next timer.
I've heard a bit about multi-threading, but couldn't find a good example how to implement it in my code.
Eventually, I would also like to add an option to delay the start of the timer with n seconds.
Thanks a lot!
If it's just about timers, you can use directly timers, without more complicated multi-threading:
https://docs.python.org/3.8/library/threading.html
https://docs.python.org/3.8/library/threading.html#timer-objects
import threading
def hello():
print("hello, world")
t = Timer(30.0, hello)
t.start() # after 30 seconds, "hello, world" will be printed
Try this one: (Tested on my machine, Python 3.8.2)
from threading import Timer
def hello(t):
print("Counted to", t)
t1 = Timer(4, hello, [4])
t1.start()
t2 = Timer(1, hello, [1])
t2.start()
t3 = Timer(3, hello, [3])
t3.start()
In order to delay the start, add other timers which call a function that does nothing.
These type of timers are called only once though at the ending time.

Weird problem getting a variable out of a function in Python

What my code does is generate an X amount of random lines, each is the length of X
My program is almost finished but I couldn't get the time to work, more specifically, get it out of the function, I know it calculates the time correctly because if I put a print in the break section it actually prints the time, but when I want to use it outside the function it never is defined
I tried tons of stuff, like globalizing it in the function, outside function (it doesn't change from 0, which means the function is not changing the variable for some reason) , tried float, tried adding the line elapsed = MainCode() , but all never working, I spent like 6 hours with this single issue, 3 of those hours with other programmers from Python's official discord server for extra help, all were also confused as to why is it still not considered defined
here's my full code, any help is appreciated
import random
import string
import multiprocessing
import os
import sys
import time
if __name__ == "__main__":
lines_wanted = input("input how many lines: ") #user can input how many lines he wants
length = input("How long: ") #user can enter how long each line should be
def MainCode(lines_wanted, length):
global elapsed
elapsed = 0.0
starttime = time.time() # start the time
file = open("blablabla.txt", "a",) # open/create the file
i = 0
while 1 == 1:
file.write(''.join(random.choice(string.ascii_letters + string.digits + ".") for i in range(int(length))) + "\n") # write the randomly generated lowercase string
i += 1 * multiprocessing.cpu_count() # count how many lines the loop is on
if i >= int(lines_wanted): # If it reaches the lines it would stop writing
endtime = time.time() # stop the time
elapsed = endtime - starttime #calculate the time
return elapsed
break
processes = []
if __name__ == '__main__':
for _ in range(multiprocessing.cpu_count()):
p = multiprocessing.Process(target=MainCode, args=(lines_wanted, length))
p.start()
processes.append(p)
for process in processes:
process.join()
elapsee = MainCode()
if __name__ == "__main__": # Prints this after finshing the func above
print(f"Finished Writting {lines_wanted} lines, took {elapsee:.1f} to generate")
elapsed is only written by MainCode, which is only called in subprocesses started by multiprocessing.Process. Being in a different process, your main program has no opportunity to observe those variables; unlike threads, subprocesses do not share your namespace. As such, you want to communicate the value somehow. This is conveniently handled by multiprocessing.Pool:
#! /usr/bin/env python3
import multiprocessing
import os
import time
def worker(arg):
starttime = time.time()
time.sleep(3)
endtime = time.time()
return endtime - starttime
if __name__ == '__main__':
wallstart = time.time()
with multiprocessing.Pool() as pool:
times = pool.map(worker, [3]*os.cpu_count())
wallstop = time.time()
print(f'Wall time: {wallstop-wallstart}, worker time: {sum(times)}')
Example run result:
$ python3 multiprocess.py
Wall time: 4.356384515762329, worker time: 24.002115488052368
Of course, the typos others pointed out would keep you from finding the intended variable as well.
One way to know this wouldn't do what you intended is the fact that __name__ must differ between the subprocess and main program. That means it's in a different module, even if they were sharing a process, so they don't share a global namespace.
First, you're trying to print "elapsee" instead of "elapsed".
Second, you probably want to put all the time code outside MainCode completely, instead starting before the process start and the joins, if you want an accurate total elapsed time.

Set a timer for running a process, pause the timer under certain conditions

I've got this program:
import multiprocessing
import time
def timer(sleepTime):
time.sleep(sleepTime)
fooProcess.terminate()
fooProcess.join() #line said to "cleanup", not sure if it is required, refer to goo.gl/Qes6KX
def foo():
i=0
while 1
print i
time.sleep(1)
i
if i==4:
#pause timerProcess for X seconds
fooProcess = multiprocessing.Process(target=foo, name="Foo", args=())
timer()
fooProcess.start()
And as you can see in the comment, under certain conditions (in this example i has to be 4) the timer has to stop for a certain X time, while foo() keeps working.
Now, how do I implement this?
N.B.: this code is just an example, the point is that I want to pause a process under certain conditions for a certain amount of time.
I am think you're going about this wrong for game design. Games always (no exceptions come to mind) use a primary event loop controlled in software.
Each time through the loop you check the time and fire off all the necessary events based on how much time has elapsed. At the end of the loop you sleep only as long as necessary before you got the next timer or event or refresh or ai check or other state change.
This gives you the best performance regarding lag, consistency, predictability, and other timing features that matter in games.
roughly:
get the current timestamp at the time start time (time.time(), I presume)
sleep with Event.wait(timeout=...)
wake up on an Event or timeout.
if on Event: get timestamp, subtract initial on, subtract result from timer; wait until foo() stops; repeat Event.wait(timeout=[result from 4.])
if on timeout: exit.
Here is an example, how I understand, what your Programm should do:
import threading, time, datetime
ACTIVE = True
def main():
while ACTIVE:
print "im working"
time.sleep(.3)
def run(thread, timeout):
global ACTIVE
thread.start()
time.sleep(timeout)
ACTIVE = False
thread.join()
proc = threading.Thread(target = main)
print datetime.datetime.now()
run(proc, 2) # run for 2 seconds
print datetime.datetime.now()
In main() it does a periodic task, here printing something. In the run() method you can say, how long main should do the task.
This code producess following output:
2014-05-25 17:10:54.390000
im working
im working
im working
im working
im working
im working
im working
2014-05-25 17:10:56.495000
please correct me, if I've understood you wrong.
I would use multiprocessing.Pipe for signaling, combined with select for timing:
#!/usr/bin/env python
import multiprocessing
import select
import time
def timer(sleeptime,pipe):
start = time.time()
while time.time() < start + sleeptime:
n = select.select([pipe],[],[],1) # sleep in 1s intervals
for conn in n[0]:
val = conn.recv()
print 'got',val
start += float(val)
def foo(pipe):
i = 0
while True:
print i
i += 1
time.sleep(1)
if i%7 == 0:
pipe.send(5)
if __name__ == '__main__':
mainpipe,foopipe = multiprocessing.Pipe()
fooProcess = multiprocessing.Process(target=foo,name="Foo",args=(foopipe,))
fooProcess.start()
timer(10,mainpipe)
fooProcess.terminate()
# since we terminated, mainpipe and foopipe are corrupt
del mainpipe, foopipe
# ...
print 'Done'
I'm assuming that you want some condition in the foo process to extend the timer. In the sample I have set up, every time foo hits a multiple of 7 it extends the timer by 5 seconds while the timer initially counts down 10 seconds. At the end of the timer we terminate the process - foo won't finish nicely at all, and the pipes will get corrupted, but you can be certain that it'll die. Otherwise you can send a signal back along mainpipe that foo can listen for and exit nicely while you join.

Time measure script in python

I received the following script from a fellow programmer:
from time import *
start = strptime(asctime())
end = strptime(asctime())
print 'you took %i minutes' % (end[4] - start[4])
As you might already know, the script measures the time between the third and fourth line. However, it seems to measure it in minutes. How can I go about to measure it in seconds, with at least one decimal place (ex. 7.4 seconds).
Also, I would like to add one extra piece of functionality. Say the script runs, and I am asked by the program to type any word. At the end of my typing, I have to press the enter key to exit the program and measure the time from the first keystroke to the moment I press enter. How can I go about measuring this?
First, I would avoid using import * as it's considered bad practice. You can use time.time() to get more precision:
>>> import time
>>> start = time.time()
>>> end = time.time()
>>> end - start
5.504057168960571
You could also use datetime.datetime.now().
#source: http://docs.python.org/library/timeit.html
def test():
"""Stupid test function"""
L = []
for i in range(100):
L.append(i)
if __name__ == '__main__':
from timeit import Timer
t = Timer("test()", "from __main__ import test")
print t.timeit()
If you are trying to optimize a python web service call, you can do the following.
import time
In the beginning of the function, right
start = time.time()
in the line put (540 is the line number),
l540 = time.time()
print("--------l541 ------------")
print(l540 - start)
in the next line put (608 is the line number),
l608 = time.time()
print("-------- 609 ------------")
print(l608 - l540)
You can add as many as you want and it will tell you where exactly the program is taking time.

Categories

Resources