so I have a python script that make a POST call with some arguments (ID and Date), that means that at a specific date time makes a POST passing and ID.
Now what I want to achieve is having a script that puts jobs in the queue and another one that execute those script at that specific date time.
The script stays the same every time but it's date and ID that changes.
I can of course run multiple times the same script with different arguments and using time.sleep wait until it gets executed but I am trying to find a cleaner way...
You can try to use sched module.
Here is an example that I made for myself in the past to deal with the discord:
import sched
import time
import sh_executor
import sh_logger as log
import sv_discord
s = None
def init():
log.info("Initializing Scheduler...")
global s
s = sched.scheduler(time.time, time.sleep)
sh_executor.submit_task(_start)
def stop():
s.cancel(_restart_discord)
def _start():
if sv_discord.DiscordSettings.IS_BOT_ENABLED:
s.enter(30, 1, _restart_discord)
s.run()
def _restart_discord():
log.info("Scheduler: restarting Discord...")
sv_discord.reconnect()
s.enter(3600, 1, _restart_discord)
Also: Take a look here (python cron-like jobs)
Related
I am trying to run something at the exact time to the second everyday.
I have tried Schedule and used a sleep time of 1 second but it runs twice sometimes so I want to switch to APScheduler. But I have never used anything Cron like before and their webpage's "User Guide" thing doesn't resemble a detailed documentation at all so I don't know where to start. Googled but there's only answers for intervals. I want to specify a time not interval.
For example run a function at 03:00:05 everyday. How would you do that?
Thank you.
I believe what you want is the BackgroundScheduler from APScheduler using a CronTrigger.
A minimal example of the program would be the following:
from time import sleep
from apscheduler.schedulers.background import BackgroundScheduler
from apscheduler.triggers.cron import CronTrigger
def foo(bar):
print(bar)
def main():
scheduler = BackgroundScheduler()
scheduler.start()
trigger = CronTrigger(
year="*", month="*", day="*", hour="3", minute="0", second="5"
)
scheduler.add_job(
foo,
trigger=trigger,
args=["hello world"],
name="daily foo",
)
while True:
sleep(5)
if __name__ == "__main__":
main()
This would run the function foo with argument bar equal to the string "hello world" every day at 03:00:05.
The background scheduler is not blocking, therefore the while loop is needed to keep the program running forever.
To change the function to call you would just need to change the first argument of scheduler.add_job to the function you want to call and change the args keyword argument to a list containing the arguments you want your function to be called with.
The arguments to CronTrigger form a cron string, this is a well-known format used by the crontab utility in Linux (i find the following site quite useful for testing those strings.)
I would write my own function.
from time import time, ctime
def your_function():
pass
if __name__ == "__main__":
while True:
now = ctime(time())[11:19]
if now == "12:34:56":
try:
your_function()
sleep(1.25)
except:
pass # or raise
else:
sleep(0.75)
This should do the job.
apscheduler.schedulers.background import BackgroundScheduler
from apscheduler.triggers.cron import CronTrigger
I want to execute a file at a specific time as I won't be around at that time to execute it.
I was thinking if there is a way in which I can write some code in another file, that will start that code, and leave it running so that it can start that code at that specific time - Maybe using os and time, or command line.
How about something like:
import time, subprocess, sys
time.sleep(3600) # wait 1 hour
subprocess.call([sys.executable, 'another-script.py'])
You can use Python's builtin sched module:
import sched, time
def action():
print("hello world")
# Set up scheduler
s = sched.scheduler(time.localtime, time.sleep)
# Schedule when you want the action to occur
s.enterabs(time.strptime('Tue March 15 15:55:17 2020'), 0, action)
# Block until the action has been run
s.run()
I have a multi processed web server with processes that never end, I would like to check my code coverage on the whole project in a live environment (not only from tests).
The problem is, that since the processes never end, I don't have a good place to set the cov.start() cov.stop() cov.save() hooks.
Therefore, I thought about spawning a thread that in an infinite loop will save and combine the coverage data and then sleep some time, however this approach doesn't work, the coverage report seems to be empty, except from the sleep line.
I would be happy to receive any ideas about how to get the coverage of my code,
or any advice about why my idea doesn't work. Here is a snippet of my code:
import coverage
cov = coverage.Coverage()
import time
import threading
import os
class CoverageThread(threading.Thread):
_kill_now = False
_sleep_time = 2
#classmethod
def exit_gracefully(cls):
cls._kill_now = True
def sleep_some_time(self):
time.sleep(CoverageThread._sleep_time)
def run(self):
while True:
cov.start()
self.sleep_some_time()
cov.stop()
if os.path.exists('.coverage'):
cov.combine()
cov.save()
if self._kill_now:
break
cov.stop()
if os.path.exists('.coverage'):
cov.combine()
cov.save()
cov.html_report(directory="coverage_report_data.html")
print "End of the program. I was killed gracefully :)"
Apparently, it is not possible to control coverage very well with multiple Threads.
Once different thread are started, stopping the Coverage object will stop all coverage and start will only restart it in the "starting" Thread.
So your code basically stops the coverage after 2 seconds for all Thread other than the CoverageThread.
I played a bit with the API and it is possible to access the measurments without stopping the Coverage object.
So you could launch a thread that save the coverage data periodically, using the API.
A first implementation would be something like in this
import threading
from time import sleep
from coverage import Coverage
from coverage.data import CoverageData, CoverageDataFiles
from coverage.files import abs_file
cov = Coverage(config_file=True)
cov.start()
def get_data_dict(d):
"""Return a dict like d, but with keys modified by `abs_file` and
remove the copied elements from d.
"""
res = {}
keys = list(d.keys())
for k in keys:
a = {}
lines = list(d[k].keys())
for l in lines:
v = d[k].pop(l)
a[l] = v
res[abs_file(k)] = a
return res
class CoverageLoggerThread(threading.Thread):
_kill_now = False
_delay = 2
def __init__(self, main=True):
self.main = main
self._data = CoverageData()
self._fname = cov.config.data_file
self._suffix = None
self._data_files = CoverageDataFiles(basename=self._fname,
warn=cov._warn)
self._pid = os.getpid()
super(CoverageLoggerThread, self).__init__()
def shutdown(self):
self._kill_now = True
def combine(self):
aliases = None
if cov.config.paths:
from coverage.aliases import PathAliases
aliases = PathAliases()
for paths in self.config.paths.values():
result = paths[0]
for pattern in paths[1:]:
aliases.add(pattern, result)
self._data_files.combine_parallel_data(self._data, aliases=aliases)
def export(self, new=True):
cov_report = cov
if new:
cov_report = Coverage(config_file=True)
cov_report.load()
self.combine()
self._data_files.write(self._data)
cov_report.data.update(self._data)
cov_report.html_report(directory="coverage_report_data.html")
cov_report.report(show_missing=True)
def _collect_and_export(self):
new_data = get_data_dict(cov.collector.data)
if cov.collector.branch:
self._data.add_arcs(new_data)
else:
self._data.add_lines(new_data)
self._data.add_file_tracers(get_data_dict(cov.collector.file_tracers))
self._data_files.write(self._data, self._suffix)
if self.main:
self.export()
def run(self):
while True:
sleep(CoverageLoggerThread._delay)
if self._kill_now:
break
self._collect_and_export()
cov.stop()
if not self.main:
self._collect_and_export()
return
self.export(new=False)
print("End of the program. I was killed gracefully :)")
A more stable version can be found in this GIST.
This code basically grab the info collected by the collector without stopping it.
The get_data_dict function take the dictionary in the Coverage.collector and pop the available data. This should be safe enough so you don't lose any measurement.
The report files get updated every _delay seconds.
But if you have multiple process running, you need to add extra efforts to make sure all the process run the CoverageLoggerThread. This is the patch_multiprocessing function, monkey patched from the coverage monkey patch...
The code is in the GIST. It basically replaces the original Process with a custom process, which start the CoverageLoggerThread just before running the run method and join the thread at the end of the process.
The script main.py permits to launch different tests with threads and processes.
There is 2/3 drawbacks to this code that you need to be carefull of:
It is a bad idea to use the combine function concurrently as it performs comcurrent read/write/delete access to the .coverage.* files. This means that the function export is not super safe. It should be alright as the data is replicated multiple time but I would do some testing before using it in production.
Once the data have been exported, it stays in memory. So if the code base is huge, it could eat some ressources. It is possible to dump all the data and reload it but I assumed that if you want to log every 2 seconds, you do not want to reload all the data every time. If you go with a delay in minutes, I would create a new _data every time, using CoverageData.read_file to reload previous state of the coverage for this process.
The custom process will wait for _delay before finishing as we join the CoverageThreadLogger at the end of the process so if you have a lot of quick processes, you want to increase the granularity of the sleep to be able to detect the end of the Process more quickly. It just need a custom sleep loop that break on _kill_now.
Let me know if this help you in some way or if it is possible to improve this gist.
EDIT:
It seems you do not need to monkey patch the multiprocessing module to start automatically a logger. Using the .pth in your python install you can use a environment variable to start automatically your logger on new processes:
# Content of coverage.pth in your site-package folder
import os
if "COVERAGE_LOGGER_START" in os.environ:
import atexit
from coverage_logger import CoverageLoggerThread
thread_cov = CoverageLoggerThread(main=False)
thread_cov.start()
def close_cov()
thread_cov.shutdown()
thread_cov.join()
atexit.register(close_cov)
You can then start your coverage logger with COVERAGE_LOGGER_START=1 python main.y
Since you are willing to run your code differently for the test, why not add a way to end the process for the test? That seems like it will be simpler than trying to hack coverage.
You can use pyrasite directly, with the following two programs.
# start.py
import sys
import coverage
sys.cov = cov = coverage.coverage()
cov.start()
And this one
# stop.py
import sys
sys.cov.stop()
sys.cov.save()
sys.cov.html_report()
Another way to go would be to trace the program using lptrace even if it only prints calls it can be useful.
Hello I am very new to programming and i'm trying to make an auto-posting bot for my subreddit.I'm using praw and I need to run this script at certain times and have it input and work
import praw
r = praw.Reddit(user_agent="UA")
r.login("username", "password")
sub = r.get_subreddit("Sub")
sub.submit("Title", text="Post text")
I'm running windows and someone said to use the task scheduler but I haven't been able to figure it out. Any help would be great. Thank You.
I would suggest to look into sched, a general purpose event scheduler. It is described, with appropriate examples to start you, in the Python's documentation.
Sample:
import time
import sched
scheduler = sched.scheduler(time.time, time.sleep)
def reddit():
<your code>
def scheduler_reddit():
scheduler.enter(0, 1, reddit, ())
scheduler.run()
time.sleep(3600)
for i in range(100):
scheduler_reddit()
Change 3600 with the desired time in seconds.
I'm working on a python script that needs to run between two given times. I'm required to use the build in sched module as this script needs to be able to run directly on any machine that has python 2.7 as to reduce configuration time. (SO CRON IS NOT AN OPTION)
A few variables define the settings for the time to run, here set_timer_start=0600 and set_timer_end=0900 are written in HHMM. I'm able to stop the script at the right time.
I don't know exactly how sched works (the python doc page doesn't make to much sense to me), but as far as I understand It runs at a date/time (epoch) while I only want it to run at a given time (HHMM).
Can anyone give me an example (or link) on how to use the scheduler and perhaps calculate the next run date/time?
If I got your requirements right, what you need is probably a loop, that will re-enter a task in the queue every time it will be executed. Something along the lines of:
# This code assumes you have created a function called "func"
# that returns the time at which the next execution should happen.
s = sched.scheduler(time.time, time.sleep)
while True:
if not s.queue(): # Return True if there are no events scheduled
time_next_run = func()
s.enterabs(time_next_run, 1, <task_to_schedule_here>, <args_for_the_task>)
else:
time.sleep(1800) # Minimum interval between task executions
However, using the scheduler is - IMO - overkilling. Using datetime objects could suffice, for example a basic implementation would look like:
from datetime import datetime as dt
while True:
if dt.now().hour in range(start, stop): #start, stop are integers (eg: 6, 9)
# call to your scheduled task goes here
time.sleep(60) # Minimum interval between task executions
else:
time.sleep(10) # The else clause is not necessary but would prevent the program to keep the CPU busy.
HTH!