Develop Reference

Issue with Python 3 Imports when using a Shell script

I have the following scenario:
A Python3 Package (meross_iot) installed through pip and located in
~/.local/lib/python3.7/site-packages
A Python script (meross_electricity.py) that imports from this package: from meross_iot.controller.mixins.electricity import ElectricityMixin
A shell script (launcher.sh) that is meant to be a wrapper so that the .py script is run at startup:
#!/bin/sh
# launcher.sh
# navigate to home directory, then to this directory, then execute python script, then back home
cd /
cd home/pi/Documents
sudo python meross_electricity.py
cd /
If I simply execute the .py file everything works as expected, imports done, etc. If I try to run the .sh script I get the following error:
pi#home:~/Documents $ ./launcher.sh
Traceback (most recent call last):
File "meross_electricity.py", line 4, in <module>
from meross_iot.controller.mixins.electricity import ElectricityMixin
ModuleNotFoundError: No module named 'meross_iot'
Can someone please help me solve this issue?
Thanks!

Installed package with sudo worked .
Answer based on comment by #KlausD

How to run a python script taking arguments with blender?

I have a script I want to run within blender to generate AO maps (script was given to me and the source guarantees it works).
I try to run the script as follows:
blender --background --python /opt/ff/product_builder/furniture_builder/generate_ao_maps.py --input_dir /tmp/test.obj --output_dir /tmp/test.png --mode ao
Which produces:
AL lib: (EE) UpdateDeviceParams: Failed to set 44100hz, got 48000hz instead
found bundled python: /usr/share/blender/2.79/python
Traceback (most recent call last):
File "/opt/ff/product_builder/furniture_builder/generate_ao_maps.py", line 195, in <module>
main()
File "/opt/ff/product_builder/furniture_builder/generate_ao_maps.py", line 178, in main
args = parse_args()
File "/opt/ff/product_builder/furniture_builder/generate_ao_maps.py", line 21, in parse_args
return parser.parse_args(os.getenv(BLENDER_ENV).split(' '))
AttributeError: 'NoneType' object has no attribute 'split'
Error: File format is not supported in file '/tmp/test.obj'
Blender quit
If I run this same script without blender (but with the argument) it tells me:
Traceback (most recent call last):
File "/opt/ff/product_builder/furniture_builder/generate_ao_maps.py", line 5, in <module>
import bpy
ImportError: No module named bpy
What do I need to do to pass the parameters to the script and get it working?

You are seeing that error because your script is looking for the environment variable BLENDER_ENV, which is not on your system. I don't recognize BLENDER_ENV as a standard Blender related environment variable so it's probable that your friend added BLENDER_ENV to his or her environment.

Firstly, blender processes its cli args in the order they are given, so your example will start in the background, run a script, then set the input_dir... This will most likely not have the result you are after.
The problem with your script failing is that the arg passed to os.getenv() needs to be a string that is the name of a shell environment variable, if you are using bash you need to export the variable to put it into the environment before you start blender.
export BLENDER_ARGS="arg1 arg2"
blender -b myfile.blend --python myscript.py
If you are using a csh then use setenv BLENDER_ARGS "arg1 arg2"
Then in your py script, you use os.getenv('BLENDER_ARGS').split(' ')
Note that each shell instance is a separate environment, you need to set the variables in the same instance that starts blender.
You may also be interested in passing cli arguments to the script as explained in response to this question.

Calling python script from a Bash script

I'm trying to call a python script from a bash script. I get import errors only if I try to run the .py from the bash script. If I run with python myscript.py everything is fine. This is my bash script:
while true; do
python script.py
echo "Restarting...";
sleep 3;
done
The error I get:
Traceback (most recent call last):
File "script.py", line 39, in <module>
from pokemongo_bot import logger
File "/Users/Paolo/Downloads/folder/t/__init__.py", line 4, in <module>
import googlemaps
ImportError: No module named googlemaps

There is more to this story that isn't in your question.
Your PYTHONPATH variable is getting confused somewhere along the way.
Insert a couple quick test lines:
in bash:
echo $PYTHONPATH
in your python:
import os
print os.environ["PYTHONPATH"]
At some point, the path to googlemaps got lost.

Your problem is in the script itself, your bash code is OK!. If you don't have problem running python scrip.py from bash directly, you should test if you use the same interpreter for both calls. You can check the shebang line in the python script, it is the first line in the file for example #!/usr/bin/env python or #!/usr/bin/python and compare it to the output of which python command if the output is different try to change or add the shebang line in to the file. If you call directly file in bash ./some_script.py bash reads the first line and if it is shebang line he wil execute the specific command for the file. My point is that if you use two diferent interpreters for calling file directly with python script.py and indirectly ./script.py one of them may not have the proper python modules.
Howto code:
$ which python
/usr/local/bin/python
So the second line is the path for your interpreter to build a shebang from it write in the first line of your script file this.
#!/usr/local/bin/python

Cannot run a python script using crontab

I am struggling to run the a python script as a cron job.
I am logged in as root
the permission for the python script is
-rwxr-xr-x 1 root root 2374 Mar 1 22:49 k_collab_spark.2.py
I am starting the script with
#!/usr/bin/env python
I tested the pythong script
if i do "./k_collab_spark.2.py` this work fine.
on the crontab i have set the job as
15 12 * * * /opt/lampp/htdocs/testme/SPARK/k_collab_spark.2.py >> /var/log/kspark.log
I do not see any message on the log file
Once i adde 2>&1 it gives an error Traceback (most recent call last):
File "/opt/lampp/htdocs/kabeer/SPARK/k_collab_spark.2.py", line 2, in
import requests
ImportError: No module named requests but if i execute the service manually it is successful . WHen i run it manually it works fine
Tried defining the path but still the same issue
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
import requests
ImportError: No module named requests
Any idea what i am missing.. Appreciate any help around this.

Try to run script with another first line:
#!/usr/bin/python
If it's executes successfully the problem in python interpreter, because when you have several versions of Python installed, /usr/bin/env will ensure the interpreter used - is the first one on your environment's $PATH, which i guess has no requests lib.

Can you add python explicitly before the script name?
At the end of the crontab line, add 2>&1, which redirects error messages to the log file as well. See this link for a detailed description In the shell, what does " 2>&1 " mean?
There is also a possibility that your current user and root runs different versions of python.

I used a shell script to call the python script. THe anaconda on the box was causing the trouble
export PATH=/opt/anaconda3/bin:$PATH
/opt/anaconda3/bin/python /opt/lampp/htdocs/scriptme.py >/opt/lampp/htdocs/scriptme.log 2>&1

Add the following lines of code to your script and edit the crontab :
from distutils.sysconfig import get_python_lib
print(get_python_lib())
Now check the log in crontab, you will get some path
e.g. "/usr/lib/python2.7/dist-packages"
cd(change directory) to the above path and ls(list directory) to check if package exists ; if not :
sudo pip3 install requests -t . # dot indicates current directory
or else if you have a requirements.txt file then you could try:
sudo pip3 install -r requirements.txt -t "/usr/lib/python2.7/dist-packages"
#try this from the directory where "requirements.txt" file exists
Now run your scripts.

pyinstaller and python's sh module

I have a python project which i built into a single executable on Linux using pyinstaller. The python also uses python's sh module.
While running the single executable, it seems the following commands do not work:
getattr(sh, 'mount')
getattr(sh, 'ls')
Although when i check the os.environ['PATH'] that is set correctly and the following work:
sh.which('mount') -> /bin/mount
I understand that sh module behaves differently than other python packages as it dynamically import commands. etc.
Is that causing a problem with pyinstaller?

To reproduce your issue more exactly:
cat > test.py << EOF
import sh
ls = getattr(sh, 'ls')
print(ls('-d', '/tmp'))
EOF
Normally, this works fine:
$ python test.py
/tmp
But with PyInstaller, AttributeError is raised:
$ ./pyinstaller test.py
$ ./dist/test/test
Traceback (most recent call last):
File "<string>", line 3, in <module>
AttributeError: 'module' object has no attribute 'ls'
You are correct, PyInstaller has a custom module loader, which does not interplay well with the sh module. Briefly.
This can be resolved by reproducing the same "Magic" that sh.Environment.__getitem__ method performs.
We may modify test.py as follows:
import sh
ls = sh.Command._create('ls')
print(ls('-d', '/tmp'))
and this works fine.
$ ./pyinstaller test.py
$ ./dist/test/test
/tmp
sh.py overloads sys.modules[__name__] with an instance of sh.SelfWrapper. This is where it conflicts with PyInstaller, whose custom Module importer does not evaluate sys.modules as standard python.