I have a python script where a certain job needs to be done at say 8 AM everyday. To do this what i was thinking was have a while loop to keep the program running all the time and inside the while loop use scheduler type package to specify a time where a specific subroutine needs to start. So if there are other routines which run at different times of the day this would work.
def job(t):
print "I'm working...", t
return
schedule.every().day.at("08:00").do(job,'It is 08:00')
Then let windows scheduler run this program and done. But I was wondering if this is terribly inefficient since the while loop is waste of cpu cycles and plus could freeze the computer as the program gets larger in future. Could you please advise if there is a more efficient way to schedule tasks which needs to executed down to the second at the same time not having to run a while loop?
I noted that you have a hard time requirement for executing your script. Just set your Windows Scheduler to start the script a few minutes before 8am. Once the script starts it will start running your schedule code. When your task is done exit the script. This entire process will start again the next day.
and here is the correct way to use the Python module schedule
from time import sleep
import schedule
def schedule_actions():
# Every Day task() is called at 08:00
schedule.every().day.at('08:00').do(job, variable="It is 08:00")
# Checks whether a scheduled task is pending to run or not
while True:
schedule.run_pending()
# set the sleep time to fit your needs
sleep(1)
def job(variable):
print(f"I'm working...{variable}")
return
schedule_actions()
Here are other answers of mine on this topic:
How schedule a job run at 8 PM CET using schedule package in python
How can I run task every 10 minutes on the 5s, using BlockingScheduler?
Execute logic every X minutes (without cron)?
Why a while loop ? Why not just let your Windows Scheduler or on Linux cron job run your simple python script to do whatever, then stop ?
Maintenance tends to become a big problem over time, so try to keep things as lightweight as possible.
Related
I want to be able to run my python script which scans for something related to cryptocurrencies 1 minute after the same script has just been complete. Is this possible? Or could I maybe set a limit before the loop repeats itself or something?
The code I have isn't something I am willing to share due to its sensitive nature. It is a trading bot.
You have a few solutions:
Use a cron job if you are on a unix like platform (you can use this for the syntax, or man cron in the terminal to learn more about it)
Create a long running python script that sleeps for one minute before executing your logic again. Something like this:
import time
while True:
# execute code here
time.sleep(60)
If you are running it on Windows platform ,
You can create a batch file to run your script in cmd using the following command:
start "" "path_to_python.exe" "path_to_python_script"
then create a task in windows task scheduler .
You can refer
https://dev.to/tharindadilshan/running-a-python-script-every-x-minutes-in-windows-10-3nm9 . It might helps.
I have a script of a bot deployed on azure that has to be always running. it's a python bot that tracks Twitter mentions in real time by opening a stream listener.
The script fails every once in a while for reasons not directly related to the script (timeouts, connection errors, etc). After searching for answers around here I found this piece of code as the best workaround for restarting the script every time it fails.
#!/usr/bin/env python3.7
import os
def run_bot():
while True:
try:
os.system("test_bot.py start")
except:
pass
if __name__ == "__main__":
run_bot()
I am logging all error messages to learn the reasons why it fails but I think there just be a better way to achieve the same, I would very much appreciate some hints.
So this is wrong way to run a script, you are running it in while loop forever.
A better way is either to schedule your main script in a cron job: Execute Python script via crontab
You can schedule this job to run every min, or hour or a specific time of the day, up to you.
If you wish to run something always, like a system monitor. Then you can run that part inside a while True loop that is fine. Like a loop which checks temperature every 5 secs writes to a file and sleep for 5 secs.
sample sudo code for the script: prog.py
while True:
log_temp()
time.sleep(5secs)
But if the script fails then schedule something to restart the script. Dont start the script inside another while loop.
Something like this: https://unix.stackexchange.com/questions/107939/how-to-restart-the-python-script-automatically-if-it-is-killed-or-dies
I currently have a big long file that queries a webpage to build a dictionary. I'd like this file to restart at 4am every day as the webpage will have updated with fresh info. What do I need to put inside my while True: loop?
Current status:
##Various Imports
##Selenium code to get details
##Dictionary Compile
while True:
now = datetime.datetime.now()
current_time = now.strftime("%H:%M:%S")
if current_time == ("04:00:00"):
##The Code to Restart the process goes here
else:
#Other Stuff happens with the dictionary
Building and testing on windows but will ultimately run on a Raspberry Pi
I would suggest you make the script simply get the information from the web page then do whatever it needs to do with your dictionary and end. Only one time. Then you can schedule this script to run at 4:00 AM every day with Windows Task Scheduler or with a Cron Job on linux. Here is a link on how to set up a cron job to run a python script.
If you want the functionality of a cron job in Python you could use the schedule library. It looks like it has support to keep the script running and then restart at 4AM. More can be read on this StackOverflow Question.
I am working on a Python program. It needs to run every 15 minutes. It currently waits 870 seconds (14.5 mins) before running again, but as the time it takes to complete the action varies, sometimes it runs before it has been 15 minutes since it last run, sometimes after 15 minutes.
My code for this part currently looks like this:
print(colour.BOLD, colour.PURPLE, "Finished", colour.END)
print(colour.BOLD, colour.BLUE, 'WAITING 15 MINUTES (900 SECONDS)', colour.END)
time.sleep(870)
Is there a way I can get it to run at xx:15, xx:30, xx:45, xx:00 where xx is every hour from 00 to 23?
Sorry if I'm being confusing here. Thanks for any help in advance.
I would use the schedule module: https://pypi.org/project/schedule/
you would run:
schedule.every().minute.at(":00").do(job)
schedule.every().minute.at(":15").do(job)
schedule.every().minute.at(":30").do(job)
schedule.every().minute.at(":45").do(job)
Use your OS tools to achieve similar results.
They are very reliable and, in case of your script failure, it will run anyway next time.
Linux
Use crontab.
How to set it will slightly change depending on your Distribution.
As a general idea:
sudo crontab -e
Inside the crontab write (be sure to customize the python executable and script path):
*/15 * * * * /usr/bin/python /path/to/your/script.py
This will make sure that your script is executed every 15 minutes.
Windows
How to schedule a task on Windows is more dependent on the Windows version you are using and it is a very visual task.
Googling "How to schedule a task in Windows" will return way better / more specific / updated results than the one I could clumsily explain here.
Here's a nice one I have found for you.
Mac
Read the amazing answer by Meki here on StackOverflow.
Having a script that does a thing at discrete intervals taking control of its own destiny like that makes me squirrely. I would use an external scheduling framework to run this job at discrete intervals. In Linux, this can be done with cronjobs; in Windows, it can be done with Task Scheduler.
Linux:
In terminal, type
crontab -e
to edit the cron schedule for the current user context. Docs on editing cron can be found all over the internet - here's one: https://www.raspberrypi.org/documentation/linux/usage/cron.md
Windows:
You can schedule a python script to run on that schedule in Windows Task Scheduler. here's a link walking through that: https://www.esri.com/arcgis-blog/products/product/analytics/scheduling-a-python-script-or-model-to-run-at-a-prescribed-time/
be certain to utilize the "if the task is already running" and "run task as soon as possible after a scheduled start is missed" options if you do this method to control appropriate behavior:
could someone help me with the following problem. I would appreciate it.
I am trying to run several processes in the same time. When I do it for the first time it works fine. However when I run it for the second time, it does not work properly. It waits some time and finishes in 5 minutes when it took 30 second to finish at the first time. When I look to resource monitoring I see that after the first run there are several processes "python 2.7" with zero to ten values (0 or 10) in CPU column. If i restart python and run code it works fine for the first time. After it again stops properly working. How could I set sustainable work?
import nltk
import concurrent.futures
def try_my_operation(k):
sentences= nltk.sent_tokenize(k)
return sentences
executor=concurrent.futures.ProcessPoolExecutor()
futures = [executor.submit(try_my_operation, data['text'][i]) for i in range(0,data.shape[0])]
concurrent.futures.wait(futures)
executor=concurrent.futures.ProcessPoolExecutor(1)
executor.shutdown(wait=False)
gc.collect()