When running a Python CGI script on Apache2, the server is unable to import certain python modules. For example, Pandas is installed locally:
And this is the script I want to run from the /var/www/html directory:
But the python script keeps giving an import error for pandas when running from the web. I have changed the shebang line a few times thinking that it could be an issue. I've tried /usr/bin/python3, /usr/bin/env python3, /usr/bin/python3.5 and /usr/bin/env python3.5 but it doesn't seem to make a difference.
I'm wondering if it's to do with permissions for the site-packages directory seeing as it's outside the web directory. Or if a completely different python environment is used when the CGI script is executed from the web. This is the site configuration:
Are there any additional authorizations that would allow for local modules to be called or is there another solution for this?
Related
I have an issue running python script when import any packages that didn't come with python 3.9 by default. Details below:
IIS 10 running on Windows server 2019
Python 3.9.7 installed under C:\Program Files\Python397
Packages are under C:\Program Files\Python397\lib\site-packages
I went thru all steps to configure IIS to run python with Handler Mappings, etc.
Security set for IUSR full control for python install folder and the IIS folder (/pyapp)
When tested with simple forms (like https://www.tutorialspoint.com/python/python_cgi_programming.htm) and it works good on browser.
Here's the problem - I have a new .py script with the content below. When run on browser it gives me the infamous error "502 - Web server received an invalid response while acting as a gateway or proxy server." because of the "import requests" line. If I remove that line (import requests) then it works fine. All packages, like this "requests" has been installed properly and works fine in Jupyter Notebook. I heard that IIS runs on a diff acct (IUSR?) and cannot access this site-package folder so I added here but not working. I also add these paths in the system path, no help either. Thanks for your help!
import sys
import os
import site
site.addsitedir(r'C:\Program Files\Python397\lib\site-packages')
sys.path.append(r'C:\Program Files\Python397\lib\site-packages')
import requests
print('Content-type: text/html\r\n\r\n')
print('html tags here')
Thank you #furas for your help. The root cause was that, for some reason when I pip installed those packages, they were installed under diff folder (user roaming folder) instead of under the folder where python is installed. The exception errors can be seen when we put the import statement in the try/except block in the html output section. I pip uninstalled them and reinstall, and now they are installed under the python folder. My python cgi scripts runs good on the web browser now. Thanks again.
I have an executable python script which archives data from mysql server using the pymysql library. The script works well from the command line.
I call this script from a php script using escapeshellcmd function and I've gotten it to work.
I also have created a bash script that I intend to use from crontab to archive the information as well. I can make this script work as well, by making changes outlined below.
Somehow I have gotten into python versions and path problems.
if I include
#!/home/tim/anaconda3/bin/python
as the first line of the python script it works when called by the php script (using www-data as the user, I believe). It doesn't work from the bash script or the command line, giving the following error:
File "./signal_archive.py", line 22, in <module>
import pymysql
ModuleNotFoundError: No module named 'pymysql'
However, if the first line of the python script is as follows:
#!/usr/bin/python3
the script works from the bash script and the command line but not from the php script. It gives the following error:
File "/home/tim/python/commodities_related/signal_archive.py", line 23, in <module>
import pandas as pd
ModuleNotFoundError: No module named 'pandas'
Both packages are installed on my system. Thinking pointing the script to the path would help, I added the following to the python script but no luck so far.
sys.path.append('/usr/lib/python3/dist-packages:')
sys.path.append('/usr/local/lib/python3.5/dist-packages:')
There is obviously something I'm missing; I think it is that php script is called by www-user and I don't know the default path. The bash file is called by my user with the path specified in the .bashrc file. However, I may need to point the apache or php (www-user) to use a specific installation of python.
EDIT-
To be more clear, a php script (phpfile1.php) calls the python script. When I call phpfile1.php from another php script (phpfile2.php) running on apache2 I everything works using the
#!/home/tim/anaconda3/bin/python
When I call the same file (phpfile1.php) from a different php script (phpfile3.php) from a bash script it fails.
Additionally, if I run the file in place using the following
./signal_archive.py
I get the error but if I run it using the following command it works:
python signal_archive.py
Any ideas if this is right or how to do it? Thanks.
I fixed this in 2 steps:
It turns out that I needed to add the path to anaconda3 to my .bash_profile file.
export PATH="/home/tim/anaconda3/bin:$PATH"
When anaconda3 is installed it modifies the .bashrc file with the previous code snippet. However .bash_profile made the difference in this case.
I also modified the top of the python file to use the anaconda path for execution, as well as add the path for the specific python packages.
#!/home/tim/anaconda3/bin/python
import sys
sys.path.insert(1, '/usr/local/lib/python3.5/dist-packages')
I have a problem with my apache server (Shared Hosting, no root).
I've installed a python package via SSH-Terminal (with pip install) under ./local/lib/python2.7/site-packages/.
There is a python file which imports this package. When I run the python file via the SSH-Terminal everything works fine.
But when I run the python file within a php exec command
exec("/usr/bin/python myscript.py 2>&1", $out, $result);
an error occurs:
"ImportError: No module named XXX".
Do you know what went wrong here or what I can do to make the script work in the browser as well?
Thanks in advance
This question already has answers here:
Getting Flask to use Python3 (Apache/mod_wsgi)
(4 answers)
Closed 6 years ago.
I have my flask app working just great with the dev server when I run it like this:
python manage.py runserver
But when I try to run it with wsgi I get all kinds of "module not installed" errors, and even syntax errors. Strangely, it shows a different error each time I hit an app route.
on some of the errors, from the output in the apache error log, I can see that it's trying to run 2.7 versions of packages, when I am using 3.4.
File "/usr/lib/python2.7/dist-packages/requests/__init__.py", line 55, in <module>
I don't know why this is, python 3 is the default on the server and
$ python --version
Python 3.4.2
is the output for every user on the system. I am using virtualenv, but I did follow these instructions for setting up mod_wsgi with virtualenv: http://flask.pocoo.org/docs/0.11/deploying/mod_wsgi/
my wsgi file looks like this:
import sys
activate_this = '/home/flask-dev/es_app/venv/bin/activate_this.py'
with open(activate_this) as file_:
exec(file_.read(), dict(__file__=activate_this))
print(sys.path)
from searchapp import app as application
and that print(sys.path) outputs this when I run it like 'python run.wsgi':
['/home/flask-dev/es_app/venv/lib/python3.4/site-packages', '/home/flask-dev/es_app', '/usr/lib/python3.4', '/usr/lib/python3.4/plat-x86_64-linux-gnu', '/usr/lib/python3.4/lib-dynload', '/usr/lib/python3/dist-packages']
i.e. a path to the 2.7 packages is nowhere in sight. Also, all the required modules are installed in my virtualenv, which is why it works fine when I run the dev server.
Totally stumped on this. Any help appreciated.
You need to tell Apache which Python executable it should use to launch python processes. Without explicit configuration, Apache probably takes the first 'python' present in the system path. (In fact no, see comments below. The distribution default python is used.)
Unfortunately, it looks like you need to re-compile mod_wsgi, as the python exe can only be changed by the configure script.
http://modwsgi.readthedocs.io/en/develop/user-guides/installation-issues.html#multiple-python-versions
In your case: ./configure --with-python=/home/flask-dev/es_app/venv/bin/python
That's a lot of work for something that should be simple. Usually i just use Apache in reverse-proxy mode (thanks to mod_proxy), and point it to an external WSGI server like Gunicorn (http://gunicorn.org/) or Waitress (http://docs.pylonsproject.org/projects/waitress/en/latest/).
I'm want to use peewee library installed in /home/user/.local/lib/python2.7/site-packages directory in Python CGI script.
When I'm running the script from console everything works good, but when I'm running the script form CGI Python can't find the peewee. It happens because CGI scripts runs as a different user and has different environment: sys.path doesn't contain /home/user/.local/lib/python2.7/site-packages in this case.
I've tried to add /home/user/.local/lib/python2.7/site-packages manually to the sys.path before import peewee but it doesn't work. What should I do to make Python find this package?