python scripts issue (no module named ...) when starting in rc.local - python

I'm facing of a strange issue, and after a couple of hour of research I'm looking for help / explanation about the issue.
It's quite simple, I wrote a cgi server in python and I'm working with some libs including pynetlinux for instance.
When I'm starting the script from terminal with any user, it works fine, no bug, no dependency issue. But when I'm trying to start it using a script in rc.local, the following code produce an error.
import sys, cgi, pynetlinux, logging
it produce the following error :
Traceback (most recent call last):
File "/var/simkiosk/cgi-bin/load_config.py", line 3, in
import cgi, sys, json, pynetlinux, loggin
ImportError: No module named pynetlinux
Other dependencies produce similar issue.I suspect some few things like user who executing the script in rc.local (root normaly) and trying some stuff found on the web without success.
Somebody can help me ?
Thanx in advance.
Regards.
Ollie314

First of all, you need to make sure if the module you want to import is installed properly. You can check if the name of the module exists in pip list
Then, in a python shell, check what the paths are where Python is looking for modules:
import sys
sys.path
In my case, the output is:
['', '/usr/lib/python3.4', '/usr/lib/python3.4/plat-x86_64-linux-gnu', '/usr/lib/python3.4/lib-dynload', '/usr/local/lib/python3.4/dist-packages', '/usr/lib/python3/dist-packages']
Finally, append those paths to $PATH variable in /etc/rc.local. Here is an example of my rc.local:
#!/bin/sh -e
#
# rc.local
#
# This script is executed at the end of each multiuser runlevel.
# Make sure that the script will "exit 0" on success or any other
# value on error.
#
# In order to enable or disable this script just change the execution
# bits.
#
# By default this script does nothing
export PATH="$PATH:/usr/lib/python3.4:/usr/lib/python3.4/plat-x86_64-linux-gnu:/usr/lib/python3.4/lib-dynload:/usr/local/lib/python3.4/dist-packages:/usr/lib/python3/dist-packages"
# Do stuff
exit 0

The path where your modules are install is probably normally sourced by .bashrc or something similar. .bashrc doesn't get sourced when it's not an interactive shell. /etc/profile is one place that you can put system wide path changes. Depending on what Linux version/distro it may use /etc/profile.d/ in which case /etc/profile runs all the scripts in /etc/profile.d, add a new shell script there with execute permissions and a .sh extention.

Related

Cannot set pythonpath environment variable from python

I'm trying to debug a project that has a lot of additional libraries added to PYTHONPATH at runtime before launching the python file.
I was not able to add those commands with tasks.json file prior to debugging python file in Visual Studio code (see post Visual Studio Code unable to set env variable paths prior to debugging python file), so I'm just adding them via an os.system("..") command
I'm only showing 1 of the libraries added below:
# Standard library imports
import os
import sys
os.system("SET PYTHONPATH=D:\\project\\calibration\\pylibrary\\camera")
# Pylibrary imports
from camera import capture
When I debug, it fails on line from camera import capture with:
Exception has occurred: ModuleNotFoundError
No module named 'camera'
File "D:\project\main.py", line 12, in <module>
from camera.capture import capture
I also tried
os.environ['PYTHONPATH']="D:\\project\\pylibrary\\camera" and I still get the same error
Why is it not remembering the pythonpath while running the script?
How else can I define the pythonpath while running Visual Studio Code and debugging the project file?
I know I can add the pythonpath to env variables in windows, but it loads too many libraries and I want it to only remember the path while the python script is executed.
Thanks
Using os.system() won't work because it starts a new cmd.exe shell and sets the env var in that shell. That won't affect the env vars of the python process. Assigning to os.environ['PYTHONPATH'] won't work because at that point your python process has already cached the value, if any, of that env var in the sys.path variable. The solution is to
import sys
sys.path.append(r"D:\project\calibration\pylibrary\camera")

python gives ImportError: No module named "" when triggered in a perl program

In my perl program which runs the python script
I have provided the PYTHONPATH env param with the path for the lib and i have run the python script. I am getting
ImportError: No module named "....."
my $script = "/path/pythonscript.py";
$ENV{'PYTHONPATH'} = "/path/lib";
system("python $script");
Whereas when i run the same python script on command line in the same directory where the script executes in my perl program, it is working.
Can anyone give me some pointers on why this is happening.
Try printing the contents of sys.path and compare the difference e.g. change your python script to
import sys
print(sys.path)
Most likely there is a difference here and this is causing the module to not be found.
I once had a similar problem. I solved it by creating an executable script (chmod) and making that script run instead of the python script. The script simply contained a cd to the directory and a python3 program. py

Run .py script via sh import module error

This is a very basic question on how to code in python and run your script from a very beginner.
I'm writing a script using Xcode9.4.1 which is supposed to be for python3.6. I then have an sh script run.sh, in the same folder of the script (say "my_folder") which simply looks like
python my_script.py
The python script looks like
from tick.base import TimeFunction
import numpy as np
import matplotlib.pyplot as plt
v = np.arange(0., 10., 1.)
f_v = v + 1
u = TimeFunction((v, f_v))
plt.plot(v, u.value(v))
print('donne!\n')
But as I try to run my_script.sh from the terminal I get a "ImportError: No module named tick.base" error.
But the tick folder is actually present in "my_computer/anaconda3/lib/python3.6/site-packages" and up to last week I was using Spyder from anaconda navigator and everything was correctly working, so no "import error" occurred.
The question is quite trivial, in some sense it simply is "what's the typical procedure to code and run python script and how modules are supposed to be imported-downloaded when running on a given machine?"
I need it since my script is to be run on another machine through ssh and using my laptop to make some attempts. Up to last year I used to work in C and only need to move some folders with code and .h files.
Thank for help!
EDIT 1:
From the Spyder 3.2.7 setting, where the script was giving non problem, I printed the
import sys
print(sys.path)
The -manually- copied the content to the sys.path variable in my_script.py and rerun 'run.sh' and now getting a new (strange) error:
Traceback (most recent call last):
[...]
File "/Users/my_computer/anaconda3/lib/python3.6/site-packages/tick/array/build/array.py", line 106
def tick_double_array_to_file(_file: 'std::string', array: 'ArrayDouble const &') -> "void":
^
SyntaxError: invalid syntax
First, check the python which you are calling the script with is pointing to the anaconda python and it is of the same version you are expecting it to be. You can do "which python" command in Linux and Mac to which the path which points to python. It if is pointing to some different version or build of python than the one which you are expecting then add the needed path to the system environment PATH variable. In Linux and Mac this can be done by adding the following line in the .bashrc file at the /home/ folder:
export PATH=/your/python/path:$PATH
And then source the .bashrc file.
source .bashrc
If you are on a operating system like cent os ,breaking the default python path can break your yum so be careful before changing it.
I am running a script in PyCharm and under the Project Interpretor I have the path
C:\envs\conda\keras2\python.exe
When I try to run the script via ssh on the server I get a 'no module named' error. I get
/usr/bin/python as the ans to 'which python' on the server itself. Could you tell me which path I must add for the script to run properly?

Import statement works on PyCharm but not from terminal

PyCharm 2016.2.3, Mac OS X 10.11.1, Python 3.5 (Homebrew);
I have this folder structure
project
/somepackage
/subpackage
__init__.py
bar.py
__init__.py
foo.py
foo.py:
import somepackage.subpackage.bar
print("foo")
bar.py:
print("bar")
So my expected output is
bar
foo
This works fine when run from PyCharm. However, when I run it from my terminal I get an ImportError:
$ pwd
$ /home/project (not the actual path; just omitting some personal stuff)
$ python3.5 somepackage/foo.py
File "foo.py", line 1, in <module>
import somepackage.subpackage.bar
ImportError: No module named 'somepackage'
I have found this question, which is about the same problem. However, none of the suggested solutions work for me, as I am indeed using the same Python interpreter as PyCharm does and I am currently in the folder that contains the /somepackage folder.
Does anyone have any other suggestions about how to solve this issue?
You are running foo.py like a script, but you are really using it like a module. So the proper solution is to run it as a module:
python3 -m somepackage.foo
For the record, another alternative is to edit your path like:
export PYTHONPATH=.
(Or you could put the absolute directory in there, and of course you should append any other directories that are already in your PYTHONPATH.) This is closer to what PyCharm does, but is less philosophically correct.
Setting PYTHONPATH is what makes it work, as noted above. I use the following VSCODE .env content so that it works for any project:
PYTHONPATH=${PROJ_DIR}:${PYTHONPATH}
This is essentially what PyCharm does when you check "Add Content Roots to PYTHONPATH" in your run/debug configuration. It's a helpful setting, but it spoils you because your code fails outside PyCharm.
Or, if you run in terminal, first export:
export PYTHONPATH=...
Took me days to work all this out.
i solved my problem by two steps on Linux:
first step
go to the root directory of your project and set:
export PYTHONPATH=$PATHONPATH:`pwd`
second step
run python3 -m somepackage.foo
remember Without '.py' suffix
I just had the same problem using scapy.layers.http Module,
this problem occurred on my Kali (linux-Debian) but run fine on Win-10 (after few modifications.)
packet was installed (scapy-http) correctly and the program was running in PyCharm but not as script (from terminal)
I tried solving it with reinstalling in main root, and messing with the sys.path but None have worked.
Troubleshoot & Solution
I found that it looks for the http module in:
/usr/local/lib/python3.7/dist-packages/scapy/layers/init.py
and got the ImportError:
from scapy.layers import http --->
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: cannot import name 'http' from 'scapy.layers' (/usr/local/lib/python3.7/dist-packages/scapy/layers/__init__.py)
So then I checked where scapy-http is really installed, which is module called http.py, so I just copied it to:
/usr/local/lib/python3.7/dist-packages/scapy/layers/
found the http.py file in :/usr/local/lib/python3.7/dist-packages/scapy_http/
And that did it :)!
I know its a bit glitchie but that worked!

Setting up Django on an internal server (os.environ() not working as expected?)

I'm trying to setup Django on an internal company server. (No external connection to the Internet.)
Looking over the server setup documentation it appears that the "Running Django on a shared-hosting provider with Apache" method seems to be the most-likely to work in this situation.
Here's the server information:
Can't install mod_python
no root access
Server is SunOs 5.6
Python 2.5
Apache/2.0.46
I've installed Django (and flup) using the --prefix option (reading again I probably should've used --home, but at the moment it doesn't seem to matter)
I've added the .htaccess file and mysite.fcgi file to my root web directory as mentioned here.
When I run the mysite.fcgi script from the server I get my expected output (the correct site HTML output). But, it won't when trying to access it from a browser.
It seems that it may be a problem with the PYTHONPATH setting since I'm using the prefix option.
I've noticed that if I run mysite.fcgi from the command-line without setting the PYTHONPATH enviornment variable it throws the following error:
prompt$ python2.5 mysite.fcgi
ERROR:
No module named flup Unable to load
the flup package. In order to run
django as a FastCGI application, you
will need to get flup from
http://www.saddi.com/software/flup/
If you've already installed flup,
then make sure you have it in your
PYTHONPATH.
I've added sys.path.append(prefixpath) and os.environ['PYTHONPATH'] = prefixpath to mysite.fcgi, but if I set the enviornment variable to be empty on the command-line then run mysite.fcgi, I still get the above error.
Here are some command-line results:
>>> os.environ['PYTHONPATH'] = 'Null'
>>>
>>> os.system('echo $PYTHONPATH')
Null
>>> os.environ['PYTHONPATH'] = '/prefix/path'
>>>
>>> os.system('echo $PYTHONPATH')
/prefix/path
>>> exit()
prompt$ echo $PYTHONPATH
Null
It looks like Python is setting the variable OK, but the variable is only applicable inside of the script. Flup appears to be distributed as an .egg file, and my guess is that the egg implementation doesn't take into account variables added by os.environ['key'] = value (?) at least when installing via the --prefix option.
I'm not that familiar with .pth files, but it seems that the easy-install.pth file is the one that points to flup:
import sys; sys.__plen = len(sys.path)
./setuptools-0.6c6-py2.5.egg
./flup-1.0.1-py2.5.egg
import sys; new=sys.path[sys.__plen:]; del sys.path[sys.__plen:]; p=getattr(sys,'__egginsert',0); sy
s.path[p:p]=new; sys.__egginsert = p+len(new)
It looks like it's doing something funky, anyway to edit this or add something to my code so it will find flup?
In your settings you have to point go actual egg file, not directory where egg file is located. It should look something like:
sys.path.append('/path/to/flup/egg/flup-1.0.1-py2.5.egg')
Try using a utility called virtualenv. According to the official package page, "virtualenv is a tool to create isolated Python environments."
It'll take care of the PYTHONPATH stuff for you and make it easy to correctly install Django and flup.
Use site.addsitedir() not os.environ['PYTHONPATH'] or sys.path.append().
site.addsitedir interprets the .pth files. Modifying os.environ or sys.path does not. Not in a FastCGI environment anyway.
#!/user/bin/python2.6
import site
# adds a directory to sys.path and processes its .pth files
site.addsitedir('/path/to/local/prefix/site-packages/')
# avoids permissions error writing to system egg-cache
os.environ['PYTHON_EGG_CACHE'] = '/path/to/local/prefix/egg-cache'
To modify the PYTHONPATH from a python script you should use:
sys.path.append("prefixpath")
Try this instead of modifying with os.environ().
And I would recommend to run Django with mod_python instead of using FastCGI...

Categories

Resources