Django fails to find R when loading RPy2 under virtualenv - python

This is kind of confusing, apologies in advance. I am configuring a new (temporary) internal production web server with the following specs:
OSX 10.8.4 (Mountain Lion), Apache 2.2.25, PostgreSQL 9.2, Python 2.7.5, Django 1.5.2, R 3.0.1, RPy2 2.3.7, virtualenv 1.10.4, virtualenvwrapper 4.1.1, mod_wsgi 3.4
This was set up on a new machine using Homebrew and pip. I removed original, Apple-installed versions of Apache and Postgres before doing the installs. Both Apache and mod_wsgi work. mod_wsgi is installed to run as daemon process.
Virtual hosts is configured in Apache to access django via mod_wsgi and the wsgi.py file contains the following lines:
import os,sys,site
site.addsitedir('/Users/kaiju/.virtualenvs/nauru_dev/lib/python2.7/site-packages')
# Activate the virtual environment
# Taken from http://thecodeship.com/deployment/deploy-django-apache-virtualenv-and-mod_wsgi/
activate_env = os.path.expanduser('/Users/kaiju/.virtualenvs/nauru_dev/bin/activate_this.py')
execfile(activate_env,dict(__file__=activate_env))
All python packages (including django and RPy2) have been installed using the virtual environment's python site-packages.
R was installed using Homebrew and works fine from the command line or from python interpreter when invoked within the virtual env (most of the time, more later).
homebrew.mxcl.httpd.plist has been added to /Library/LaunchDaemons instead of ~/Library/LaunchAgents so that it would automatically start when the machine is rebooted. homebrew.mxcl.postgres.plist has been added to ~/Library/LaunchAgents (probably should also move it to /Library/LaunchDaemons).
So here is the problem. When everything was installed, it worked together fine. To verify everything was cool, I restarted the machine and tested the django application. Apache is running, as is django, but now I get the following error:
RuntimeError at /admin/
R_HOME not defined, and no R command in the PATH.
Request Method: GET
Request URL: http://nauru.xoma.com/admin/
Django Version: 1.5.2
Exception Type: RuntimeError
Exception Value: R_HOME not defined, and no R command in the PATH.
Exception Location: /Users/kaiju/.virtualenvs/nauru_dev/lib/python2.7/site-packages/rpy2/rinterface/__init__.py in <module>, line 48
Python Executable: /usr/local/opt/python/bin/python2.7
Python Version: 2.7.5
Python Path: ['usr/local/var/django/code/ri',
'/usr/local/var/django/code',
'/usr/local/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/setuptools-0.9.8-py2.7.egg',
'/usr/local/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pip-1.4-py2.7.egg',
'/usr/local/Frameworks/Python.framework/Versions/2.7/lib/python27.zip',
'/usr/local/Frameworks/Python.framework/Versions/2.7/lib/python2.7',
'/usr/local/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-darwin',
'/usr/local/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac',
'/usr/local/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac/lib-scriptpackages',
'/usr/local/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk',
'/usr/local/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-old',
'/usr/local/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload',
'/usr/local/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages',
'/Library/Python/2.7/site-packages',
'/Users/kaiju/.virtualenvs/nauru_dev/lib/python2.7/site-packages',
'/Users/kaiju/.virtualenvs/nauru_dev/lib/python2.7/site-packages/PIL']
But if I open a terminal window and type
sudo apachectl stop
sudo apachectl start
When the server comes back, everything works. This does not happen if I use sudo apachectl restart, in that case the error remains. What is going on? Too many moving parts for me.

The best way to resolve this issue is to add
os.environ['R_HOME']='/usr/local/Cellar/r/3.0.1/R.framework/Resources'
to the beginning of the wsgi.py file that is being used for mod_wsgi / django. It should be added before activating the virtual environment. Any other environ variables should also be added here. So now the wsgi.py file looks like this:
import os,sys,site
os.environ["R_HOME"]='/usr/local/Cellar/r/3.0.1/R.framework/Resources'
site.addsitedir('/Users/kaiju/.virtualenvs/nauru_dev/lib/python2.7/site-packages')
# Activate the virtual environment
# Taken from http://thecodeship.com/deployment/deploy-django-apache-virtualenv-and-mod_wsgi/
activate_env = os.path.expanduser('/Users/kaiju/.virtualenvs/nauru_dev/bin/activate_this.py')
execfile(activate_env,dict(__file__=activate_env))
sys.path.insert(0,'/usr/local/var/django/code')
sys.path.insert(0,'/usr/local/var/django/code/ri')
# We defer to a DJANGO_SETTINGS_MODULE already in the environment. This breaks
# if running multiple sites in the same mod_wsgi process. To fix this, use
# mod_wsgi daemon mode with each site in its own daemon process, or use
# os.environ["DJANGO_SETTINGS_MODULE"] = "ri.settings"
os.environ.setdefault("DJANGO_SETTINGS_MODULE","ri.settings")
# This application object is used by any WSGI server configured to use this
# file. This includes Django's development server, if the WSGI_APPLICATION
# setting points here.
from django.core.wsgi import get_wsgi_application
application = get_wsgi_application()
Many thanks to those who contributed to this discussion!

I believe that sys.path has little to do with the environment variable PATH, which the one that should be set (along with LD_LIBRARY_PATH if needed).
I am guessing that Apache is running as a dedicated user (not your own UID for obvious security reasons), and that user does not have R in its path (other security step - by default Apache should not be able to run any executable on your system).
You'll have to either allow Apache to run the system's R, or have an R install only for your server (the later being the better if considering anything beyond a prototype to play with)

Related

How to solve Local path <path> is not registered within uploads in the request in PyCharm 2022.2.1 (Professional Edition)?

I want to set up a Django project with docker-compose and PyCharm on my PC with Ubuntu 22.04 OS. Using PyCharm 2022.2.1 (Professional) I get the following error
How to solve Local path is not registered within uploads in the request
I added a Python interpreter from Settings > project > Python interpreter and then add interpreter > on SSH after that entered ssh credentials and on the system interpreter finally I created the Python interpreter.
I have docker-compose run on another terminal.
After I run the runserver command it shows this error:
this is the runserver command configuration:
I have recreated the interpreter, and explored the same problems on JetBrains website but couldn't solve the issue.
This bug was reported as PY-55396 on the JetBrains bug tracker.
The bug was solved in PyCharm 2022.2.2, the solution is to upgrade to that version or downgrade to PyCharm 2021.3.

OSError: Cannot load native module 'Crypto.Cipher._raw_ecb' on Apache mod_wsgi CentOS 8

I'm trying to run a django project on an apache server. The django server runs fine on its own but fails when running through mod_wsgi. It returns the error as follow :
OSError: Cannot load native module 'Crypto.Cipher._raw_ecb': Trying '_raw_ecb.cpython-39-x86_64-linux-gnu.so': /home/user/django/centos_env/lib/python3.9/site-packages/Cryptodome/Util/../Cipher/_raw_ecb.cpython-39-x86_64-linux-gnu.so: failed to map segment from shared object,
Trying '_raw_ecb.abi3.so': /home/user/django/centos_env/lib/python3.9/site-packages/Cryptodome/Util/../Cipher/_raw_ecb.abi3.so: cannot open shared object file: No such file or directory,
Trying '_raw_ecb.so': /home/user/django/centos_env/lib/python3.9/site-packages/Cryptodome/Util/../Cipher/_raw_ecb.so: cannot open shared object file: No such file or directory
I checked that the file were there. I checked Python home variable and tried to import Crypto.Cipher from the python interpreter(which worked). Everything seems fine.
I tried to compile pycryptodome from source but it didn't help either.
I had the same problem (I'm on Gentoo-Linux) with flask.
Reason were different versions of Python in the virtual environment and mod_wsgi compiled against.
You can see the version of mod_wsgi when you restart apache and look into the error_log, something like
AH00163: Apache/2.4.46 (Unix) OpenSSL/1.1.1i PHP/7.2.34 mod_wsgi/4.7.1 Python/3.7 configured -- resuming normal operations
shows the version Python mod_wsgi was compiled against.
My virtual environment used Python 3.7 while mod_wsgi was on 3.8 .

Develop and Debug Python with Remote Server

There is a Ubuntu server with Python 3.6 and many other required Python modules installed.
When using Eclipse for Python development, can I connect Eclipse to the full Python environment installed on the Ubuntu server?
I don't want to install the whole Python environment in the local PC client.
Thanks.
Latest Update:
I have installed the latest Eclipse (2018-12) and PyDev (7.0.3) plugin and Python (3.6.7) on Windows 10. And I have also created a test file called Test.py with the following content:
import my_server_module as my;
print(my.version);
It doesn't work at all because the "my_server_module" is not installed on Windows 10, it just installed on the Ubuntun server.
How can I debug the Test.py with connecting to the python environment on Ubuntu server?
Always use the virtualenv for python app and load into the eclipse - https://www.michaelpollmeier.com/eclipse-pydev-and-virtualenv
It seems it is impossible for remote debugging.
I have installed all the modules in local PC for python development. This is the only way.

Google App Engine local development server SSL warning (Python 2.7)

Each time I upload my app to Google App Engine, the logs always show this warning:
WARNING appengine_rpc.py:435 ssl module not found. Without the ssl
module, the identity of the remote host cannot be verified, and
connections may NOT be secure. To fix this, please install the ssl
module from http://pypi.python.org/pypi/ssl .
I'm running a virtualenv with Python 2.7. When I'm in it, I try to run
$ pip install ssl
but this produces an error:
ValueError: This extension should not be used with Python 2.6 or later
(already built in), and has not been tested with Python 2.3.4 or earlier.
If ssl is built in to Python 2.7, how do I tell the local development server to use the built in ssl module?
It sounds like appcfg.py is not using your virtualenv correctly. You can try editing the appcfg.py script so that it prints sys.version and sys.path, to confirm that it's using your virtualenv correctly. (The shebang line is "#!/usr/bin/env python", so that should use your active environment, but it's worth checking.)
If you're using the Launcher, you need to tell it explicitly where to look for your virtualenv's Python. Go to Preferences, and set the Python Path to your virtualenv's Python 2.7. The Launcher prints which Python path it is using in the Logs (at least when starting the dev server).

WSGI: _mysql.so Image not found

Environment: MacBook Pro, OSX 10.6, Python 2.6.1 64 bit, MySQL 5.X 64 bit
I am having problems installing MySQLdb for use with Django. After installing everything I am able to open the python interpreter and execute:
import MySQLdb
This works fine. I am also able to create and run django apps and host them using the development server:
./manage.py runserver
But I am now trying to host this django app in my local apache. So I installed and configured mod_wsgi and added a virtual host to httpd.conf setting the location of my *.wsgi script. From that script I am able to define the application function en return a simple Hello World string.
But when I try to run:
import MySQLdb
inside a try block in my *.wsgi script, the caught exception says the following:
dlopen(/Library/Python/2.6/site-packages/_mysql.so, 2): Library not loaded:
libmysqlclient.18.dylib Referenced from: /Library/Python/2.6/site-packages/_mysql.so
Reason: image not found
So, simply put, the problem is that my MySQLdb adapter works when using it in the python interpreter and django development server, but not through Apache and mod_wsgi.
Any suggestions?

Categories

Resources