I am new to programming. I think I'm close to solving my issue, but it finally broke my brain and I need help.
I am running a pygame script as a scheduled task in crontabs. I have one code that executes successfully and does what it should. I have another code that executes, but when it does the screen goes blank, usually displaying some lines that I usually see when the Linux boots up, and it just stays stuck there.
I have gone through both codes and absolutely everything is similar and correct. I have #!/usr/bin/env python at the start of each script.
(Elsewhere it was recommended that I give the exact version, because I use dictionaries in my script and apparently crontab could get confused with the dictionary stuff that pygame uses. I don't fully understand, I tried it but it didn't work on my raspberry pi so I don't think that's the solution.)
Each script runs fine in the terminal.
I have set the PATH variable to the one that the python script uses and also set the SHELL to /bin/bash. In the task I also have "export DISPLAY=:0" (e.g. 12 21 * * * export DISPLAY=:0 && ...). I have found this was the magic trick that got the other code to work. Which made me wonder if there is another environment variable that I need to set in the task?
The difference between the first code and the second code: The second code uses pygame.mixer and plays sound files. In the script, the sound files are in dictionaries (e.g. sound = {"word" : "/absolute/path/to/file.wav", "word2" : ...etc} As I said this code runs fine in terminal.
So why does the one script work and not the other. Both use pygame. The other just uses sound, dictionaries instead of strings, and pygame.mixer as well. My reasoning is that there is an issue with crontab getting stuck on one of these things.
I fixed it. I was running it out of root. When I ran the crontab in the user it worked. It must be because the root couldn't get at the files.
Important tips is to add 'import os' and at some point in the script add 'print(os.environ)' to check all the environment variables when it runs in terminal.
Then copy the $PATH variables to the top of the crontab jobs.
Same with $SHELL.
The $DISPLAY variables need to be put in the line, so for example '* * * * * export DISPLAY=:0 && /usr/bin/python /home/user/file.py'
See how I put absolute paths for both the command and the path to file. I also set absolute paths to files in the script, even if the files are in the same folder as the script. I'm not sure if this is necessary I will check that next. I've seen a lot of people struggle with this issue and the advice usually comes down to these things.
Related
I am new to coding and Python and have read a lot of similar questions about absolute/relative imports of modules but I cannot understand why my cronjob isn't working.
When I run from the terminal: python3 example.py the program runs without issue.
However when I schedule a cronjob to run "example.py", the program fails to run due to an ImportError. The first time it failed was due to a ModuleNotFound error, so I reinstalled the module in question (pyautogui) which seemed to solve that error.
I have tried using from . import pyautogui, os, time, smtplib, traceback instead of just import xyz but that brings up "ImportError: attempted relative import with no known parent package".
Can anyone help explain in simple terms what's going on here? Or point me in the direction of reading I can do to understand how terminal/cronjob is attempting to execute my program?
Thanks
Can anyone help explain in simple terms what's going on here? Or point me in the direction of reading I can do to understand how terminal/cronjob is attempting to execute my program?
Okay. Crontab programs are run under a different environment than the one in the terminal.
The surest way to check this is to run this bash script from crontab:
#!/bin/bash
set > /tmp/set.txt
It will dump the whole environment (probably you only need $PATH though) to "/tmp/set.txt".
Run the same script from terminal, saving to a different file. If you compare the two, you'll notice differences, for example again in the PATH.
And, you've guessed it, python uses PATH to find its various bits and pieces (and for that matter, which python is run if several are present also depends on PATH).
In the crontab file, you should notice, near the top, an assignment to both the SHELL variable and the PATH variable. Those are the values used by all following commands.
You should probably be able to set the PATH value to the same value you get from terminal, but keep in mind that you might disrupt the operation of any other script in the same crontab (that is why you should rather make use of /etc/cron.d files).
I am aware of the difference between import x and from x import yyy, in the latter case one can address x-located functions by bare name, instead of full specification, and this exactly I want to do, I do not want to write full name every time.
Also I know how to redefine default Python dir on Windows so that it starts knowing my developed modules, and this is fine. However how to combine those two things altogether?
I want IDLE to get started knowing all functions from all my modules so I do not need to import them manually.
from * import *
i.e. for all modules in IDLE home directory?
P.S. I saw this question, however it puts solution only for one import and do not scan the whole dir, also this solution with shortcut parameters does not seem beautiful to me.
Are there any more neat ways?
If you start idle with -r file it will first run the file, or with -s, it will first run the file listed in the IDLESTARTUP or PYTHONSTARTUP environmental variables. Then test by starting IDLE in command prompt with, for instance, py -m idlelib -r file. Once this works, you could create on IDLE shortcut on your desktop (drag and drop from Start menu), open Properties, and add either of the above to the end of the 'Target'.
I am trying to get this python file with an infinite loop on my raspberry pi to start on boot and stay running while the pi has power. I've tried editing my "etc/rc.local" file and adding the different command variations below. I am certainly referencing the right file and path. I also added a 10 second delay at the beginning of the scrip to make sure everything is in order before it runs. If anyone has any input that'd be great, this is seemingly a common task and I dont know why I am having difficulty.
Additionally I dont have a shebang line at the top of my python script. I dont know if that has anything to do with my problem even though i am specifying the file is a python file.
/usr/bin/python3 /home/pi/Desktop/webplants/mqtt.py &
sudo /usr/bin/python3 /home/pi/Desktop/webplants/mqtt.py &
sudo python3 /home/pi/Desktop/webplants/mqtt.py &
python3 /home/pi/Desktop/webplants/mqtt.py &
You should use systemd. This utility will easily allow you to run a python script on startup. This tutorial should cover it: https://www.dexterindustries.com/howto/run-a-program-on-your-raspberry-pi-at-startup/#systemd
I have several scripts perfectly working by themselves and i would like to create a big pyqt-script which launch those scripts when they are selected.
My only one problem is, every import done on my scripts are not working anymore because of the path. (As the new reference is the pyqt-script and no longer themselves).
I tried to make a movable path but it means changing all my scripts and I would like to be able to "copy/past" my scripts in a file without having to modify them, only the pyqt-script.
I also tried to make a function which open a new terminal with gnome-terminal and launch my script but it is not successful as it keep taking my pyqt-script for reference)
os.system("gnome-terminal --tab -e 'tcsh -c \"python /path/script.py; sleep 10)
Do you have any idea? even if they work in the same terminal.
Thank you for your help.
I'm rather green in python development but learning nevertheless.
I've written some python codes but they are mostly one-off use and to be run at command line. I still have no idea how to automate and schedule my codes. Lets say I have:
written a python script that pulls some CSV data from API and saves it in /tmp.
written another script to ingest the csv data and transform it into XML per line.
Each time I want to do this, I find myself doing:
$ python getdata.py
$ python converttoxml.py
In shell, I think one can write a wrapper script and cron it. Right? If so, how is this done in the world of python? Bear in mind we have to include all python libraries/ modules used, too.
P.S. developing in Linux environment using PyCharm.
Login to cron using crontab -e
Scroll to bottom at add a line following the following format:
m h dom mon dow command which is minutes, hours, day of month, day of week, command
So if you want to run your command on the hour every hour you would have
* /1 * * * * python path/to/file/getdata.py
whereas if you want it to run at 12 only then you would have
* 12 * * * * python path/to/file/getdata.py
Python files could be treated as executables. You just need to give them executing permissions with chmod +x my_file.py and tell the bash which interpreter should be used to parse the file by adding #!/usr/bin/python as the 1st line of the code file.
Once you've done the two things, you could just run your file using ./my_file.py and the python script should be executed.
From this point on, it's just like any ordinary program and could be used in cron/systemd/whatever other use you need. Once you have a working Python script, it should be treated no differently than any other executable on your system - in this regard, there's no "Python World".
As far as extra modules are concerned - this shouldn't be an issue as Python will still use the same way to resolve library location. It might be necessary to update $PYTHON_PATH to add a library path - but if the library is installed correctly it shouldn't be an issue while for libraries contained within your local directory (only in the case Python fails to find them) you could add:
sys.path.apped(os.path.dirname(__file__)) to the beginning of your script.