I have a python code, using Dash, which reads a CSV file and builds up a dashboard. Now I want the code to get stopped daily at 9:15 AM and restart at 9:30 AM (so that in these 15 minutes, the CSV file gets updated).
How to achieve the same ?
The code runs on one base system, the other users then access the dashboard using link. The code continues to run on the same base system. And SAS is ued to store the file in that base system. So that is why I want that the code stops at a particular time. The file gets refreshed in that system and then the code restarts.
Depending on the system it's running on, there's different solutions to your problem.
If you've system access and are on a Linux-based OS, the most straightforward way would be to use cron, https://en.wikipedia.org/wiki/Cron. If you Google you'll find tons of documentation and "how to" guides.
Related
I was able to write a script to scrape using selenium, right now I'm trying to automate it so it can work periodically on a server so I don't bother myself by running it from my local, I did a lot of googling but I got no clue of how I can do that, can anyone simplify things for me ..
In order to run a python script on a linux server periodically you can make a cronjob, since you already have a python script which most probably fetches or scrapes data and saves it in a file. You can make a cronjob and set the exact time it has to run, say for instance after every 2 hours you can do it using something like this,
crontab -e
this will open a editor in your terminal, at the bottom of the text just write timing and the command to be executed.
* * * * */path/to/your/code.py
from this link you can find out how to fill out the stars https://crontab.guru/#*_*_*_*_1
if you need anymore help with using cronjobs take a look at this https://www.geeksforgeeks.org/scheduling-python-scripts-on-linux/
You can simply redo your script on pythonanywhere and schedule it as a task and choose the frequency that you want the script to be executed. The current frequency options available include; script runs always, hourly or daily.
i dont know if it'll really work but
while True:
whole
code
here
time.sleep(period required)
I have python code that depends on specific libraries like selenium and interaction with google chrome to extract data from the web.
my code works fine but i need a lot of records to do analysis, so i can't leave my computer on, to run the script for a month.
That's why I thought of running the script in a cloud service like aws but I don't have a clear idea of how to do it, because I need the script to not stop
and I would rather not have to pay for it (or at least not that much money)
That said, my code opens a website, looks for a specific text data and saves it in a csv document.
I thank you in advance for the help
You will have to check the terms of each cloud service as many do have downtime/restarts on their free tiers.
The kind of task you're describing shouldn't be very resource hungry, so you may be better off setting up your own server using a Raspberry Pi or similar.
I am new to python.
I created a simple Selenium bot that basically runs on a while loop.
However, it runs on an external machine that I do not have access at all times.
It runs into unexpected crashes sometimes, and it takes hours and sometimes days until I check it again only to find a crashed bot. With that said, I'd like to know if a monitoring module or program I can implement into my bot, so I can easily check if the program is running or not.
Some have suggested a WebSocket heartbeat, however, I do not know, after extensive research, how I would go about implementing such an intricate system. Thank you for taking the time to read my question.
Any answers appreciated!
You can just put a few lines of code at the appropriate point in your script (after the Selenium bot completes its work successfully) to write the current date and time into a file.
A second script can be scheduled just to read that file and ring an alarm bell (e.g. send you an email) if it is X hours since the timestamp was last updated.
Alternatively you could write a timestamp to a database table or use whatever other shared systems you have.
I have created a program using python which moves goes through each file in my downloads directory, and moves that file to another directory based on the suffix at the end of the file (.mp3, .mp4, .txt, .jpg, ...). How would I go about automating this program so that it runs in the background of my computer every couple of hours?
What you are referring to is often called a "cronjob". Python has a module called python-crontab that can do this type of thing. Here is a tutorial to help you get started.
I do not know much about python, but in nodejs, there is a tool called cronjob. When you set routine, according to the time that you set, it calls scripts. Maybe there is an equivalent version in python.
I have a Python bot running PRAW for Reddit. It is open source and thus users could schedule this bot to run at any frequency (e.g. using cron). It could run every 10 minutes, or every 6 hours.
I have a specific function (let's call it check_logs) in this bot that should not run every execution of this bot, but rather only once a day. The bot does not have a database.
Is there a way to accomplish this in Python without external databases/files?
Generally speaking, it's better (and easier) to use the external database or file. But, if you absolutely need it you could also:
Modify the script itself, e.g. store the date of the last run in commented out last line of the script.
Store the date of the last update on the web, for example, in your case it could be a Reddit post or google doc or draft email or a site like Pastebin, etc.
Change the "modified date" of the script itself and use it as a reference.
If you're using cron you can run it with command line arguments.
And define in cron eg. python3 main.py daily for the daily run that you need and python3 main.py frequent for the other version.
I'm doing it that way and it worked optimally by now.