Develop Reference

Cannot install external package insde python when uwsgi run

I run a Python application within uwsgi, and then my application is trying to install an external package with this code:
subprocess.call([sys.executable, "-m", "pip", "install", module])
But there is an error:
unable to load configuration from pip
This is my uwsgi configuration:
[uwsgi]
module = bin.run:app
manage-script-name = true
virtualenv = venv
master = true
processes = 5
http-socket = 0.0.0.0:9002
vacuum = true
die-on-term = true
enable-threads=true
lazy=true
How can I fix this? Ive tried with flask run and it worked normally.

This is because sys.executable isn't python like it normally would be. It points to the uwsgi executable instead. So your code is actually trying to start up another uwsgi process.
The first argument of uwsgi is a configuration file to load from (https://uwsgi-docs.readthedocs.io/en/latest/Configuration.html#ini-files).
For convenience, uWSGI recognizes bare .ini arguments specially, so the invocation uwsgi myconf.ini is equal to uwsgi --ini myconf.ini
Since the first argument that you gave uwsgi is pip, it tries to load a file called pip, can't, and then dies.
For me, hard coding "python" (or "python3" if that's where it is on your system) as the executable to run fixed it.

Django, uwsgi, nginx, virtualenv, ImportError: No module named site

I have a Django app and a virtual env and I try to run it under nginx+uwsgi. I've configured the whole system as is described here. I have the CentOS 7 as well.
However, I get the famous "ImportError: No module named site". No other topic here on stack overflow helped me to solve this.
In the log of uwsgi I have these two lines:
Set PythonHome to /hedgehog/.virtualenv/hedgehog
ImportError: No module named site
/etc/uwsgi/sites/hedgehog.ini:
[uwsgi]
project = hedgehog
username = hedgehog
base = /var/www/%(username)/code
chdir = /var/www/hedgehog/code
home = /%(username)/.virtualenv/%(username)
module = %(username).wsgi:application
master = true
processes = 5
uid = %(username)
socket = /run/uwsgi/%(project).sock
chown-socket = %(username):nginx
chmod-socket = 660
vacuum = true
logto = /var/www/%(username)/log/uwsgi.log
/etc/systemd/system/uwsgi.service:
[Unit]
Description=uWSGI Emperor service
[Service]
ExecStartPre=/usr/bin/bash -c 'mkdir -p /run/uwsgi; chown hedgehog:nginx/run/uwsgi'
ExecStart=/usr/bin/uwsgi --emperor /etc/uwsgi/sites
Restart=always
KillSignal=SIGQUIT
Type=notify
NotifyAccess=all
[Install]
WantedBy=multi-user.target
The app is in /var/www/hedgehog/code.
Seems that uwsgi somehow doesn't use the virtualenv. I've tried adding this to the ini file:
plugins = python
virtualenv = %(home)
It didn't help.
However, if I run "import site" in python interpreter in this virtualenv it gives no error:
[rreimche#rreimche-web sites]$ sudo -u hedgehog -H bash -l
[sudo] password for rreimche:
[hedgehog#rreimche-web sites]$ python
Python 2.7.5 (default, Jun 24 2015, 00:41:19)
[GCC 4.8.3 20140911 (Red Hat 4.8.3-9)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import site
>>> exit()

I had the same error and a mistake:
I used a Python3 virtualenv by mistake. Once I used a Python2.7 virtualenv it worked.
Cheers

You don't really need the virtualenv entry in your config since it's the same option as home entry. /%(username)/.virtualenv/%(username) should resolve as /hedgehog/.virtualenv/hedgehog. What does ls -l /hedgehog/.virtualenv/hedgehog say?
You may also want to check that you are loading the python plugin for the very same version of python you are using creating your virtualenv.
Please paste more log, there may be more hints on what is wrong.

This also happened to me when I switch my virtualenv to use Python 3, it had been working well under Python 2.7. Here's how I solve it:
replace
plugins = python
with
plugins = python3

Flask+Nginx+uWSGI : ImportError: No module named site

I installed as the http://www.reinbach.com/uwsgi-nginx-flask-virtualenv-mac-os-x.html link's tutorial and when executing the command uwsgi --ini deploy/deploy.ini, the terminal says there was an import error:
Set PYTHONHOME to /virtualenv/sample/
ImportError: No module named site
I have set my PYTHONHOME and PYTHONPATH as
export PYTHONPATH=$PYTHONPATH:/Library/Python/2.7/site-packages
export PYTHONHOME=$PYTHONHOME:/Library/Python/2.7
I cannot figure out what wrong with it.
Could someone help me with the problem?
The whole info in the terminal is shown as below if it is helpful:
(env)ios-devmatoMacBook-Pro:hello ios_dev$ uwsgi --ini deploy/deploy.ini
[uWSGI] getting INI configuration from deploy/deploy.ini
*** Starting uWSGI 1.9.10 (64bit) on [Fri May 17 16:42:22 2013] ***
compiled with version: 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2336.11.00) on 17 May 2013 12:41:07
os: Darwin-11.4.2 Darwin Kernel Version 11.4.2: Thu Aug 23 16:25:48 PDT 2012; root:xnu-
1699.32.7~1/RELEASE_X86_64
nodename: ios-devmatoMacBook-Pro.local
machine: x86_64
clock source: unix
detected number of CPU cores: 4
current working directory: /Users/ios_dev/Desktop/sample/hello
detected binary path: /Users/ios_dev/Documents/little/little-web/little_web_dev/env/bin/uwsgi
your processes number limit is 709
your memory page size is 4096 bytes
detected max file descriptor number: 256
lock engine: OSX spinlocks
uwsgi socket 0 bound to TCP address 127.0.0.1:3031 fd 3
Python version: 2.7.1 (r271:86832, Jun 16 2011, 16:59:05) [GCC 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2335.15.00)]
Set PYTHONHOME to /virtualenv/sample/
ImportError: No module named site

Here's how I resolved the same error message (ImportError: No module named site) that I got while trying the Django and NGINX tutorial — uWSGI 2.0 documentation.
Deactivate the virtualenv:
deactivate
Install uWSGI system-wide (if not already installed system-wide)
sudo pip install uwsgi
Edit the uwsgi.ini file. I commented out the line with the:
home = /path/to/virtualenv
Run uWSGI --ini mysite_uwsgi.ini.

I read a lot of document about the question, but get no answer.
By coincidentally, I fix this problem by edit uid and gid as root.
It seem like a permissions problem. I don't know why, but it just work. Remember, it is very unsafe to run a product environment as root.

1、active the virtual you used
2、 pip install uwsgi
this the key action，then the
command -v wsgi
show this
/virtual-path/bin/uwsgi
3、use current user to run uwsgi, because other user is not active the virtualenv

Im My case I wasn't using virtualEnv. Just using django + ngnix. My solution was removing HOME variable into the *.ini configuration file:
sudo nano /etc/uwsgi/sites/c_app.ini
[uwsgi]
project = c_app
uid = ubuntu
base = /home/%(uid)
chdir = %(base)/%(project)
**home = %(base)/%(project)** (REMOVED IT)
module = %(project).wsgi:application
master = true
processes = 5
socket = /run/uwsgi/%(project).sock
chown-socket = %(uid):www-data
chmod-socket = 660
vacuum = true
then it works.

Calling Python from Maven in a Cygwin shell

When building with Maven from a Cygwin shell (bash on Windows via Cygwin), the path to /usr/bin is not resolving properly.
Let me explain. Cygwin comes with Python, which is accessible as a symlink from /usr/bin. This symlink should be accessible to Maven, because its location is in the PATH environment variable. Cygwin adds /usr/bin to the PATH environment variable. However, Maven is unable to find Python. E.g.
[ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.2.1:exec (default) on project cloud-devcloud: Command execution failed. Cannot run program "python" (in directory "C:\cygwin\home\myuser\incubator-cloudstack\tools\devcloud"): CreateProcess error=2, The system cannot find the file specified -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException
But Python is there:
$ which python
/usr/bin/python
Albeit as a symlink.
$ ls -al /usr/bin/python
lrwxrwxrwx 1 donall Domain Users 13 Sep 19 11:38 /usr/bin/python -> python2.6.exe
Is the issue that Python is a symlink? Or that Maven cannot access files in /usr/bin?

It seems like maven is running as a regular Windows-native application, not a "cygwin-aware application" (if such thing exists :-) )
This is indicated by the path used internally, which is printed in the log: C:\cygwin\home\myuser\incubator-cloudstack\tools\devcloud
For this reason the call from Maven will not recognize the cygwin path when locating the python executable. One possible solution is to add the directory containing python.exe to the system PATH (which means, the Windows PATH variable)

uWSGI + Django + Virtualenv unable to pick up _functools (import error)

OK, So I have tried this with & without a virtualenv:
uwsgi --home /home/auston/new_proj/ --socket /tmp/uwsgi2.sock --chmod-socket --module app_wsgi --pp /home/auston/new_proj/nikeshere --logto /tmp/uwsgi.log --master --processes 4 -P
Pretty much no matter what, I get this:
*** Starting uWSGI 0.9.6.5 (32bit) on [Thu Oct 21 08:05:44 2010] ***
compiled with version: 4.4.3
Python version: 2.6.6 (r266:84292, Oct 21 2010, 04:07:38)
[GCC 4.4.3]
your memory page size is 4096 bytes
allocated 412 bytes (0 KB) for 1 request's buffer.
Setting PythonHome to /home/auston/new_proj/...
binding on UNIX socket: /tmp/uwsgi2.sock
chmod() socket to 666 for lazy and brave users
your server socket listen backlog is limited to 64 connections
added /home/auston/new_proj/nikeshere to pythonpath.
initializing hooks...done.
['/home/auston/new_proj/nikeshere', '.', '', '/home/auston/new_proj/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg', '/home/auston/new_proj/lib/python2.6/site-packages/pip-0.8.1-py2.6.egg', '/home/auston/new_proj/lib/python26.zip', '/home/auston/new_proj/lib/python2.6', '/home/auston/new_proj/lib/python2.6/plat-linux2', '/home/auston/new_proj/lib/python2.6/lib-tk', '/home/auston/new_proj/lib/python2.6/lib-old', '/home/auston/new_proj/lib/python2.6/lib-dynload', '/usr/lib/python2.6', '/usr/lib/python2.6/plat-linux2', '/usr/lib/python2.6/lib-tk', '/home/auston/new_proj/lib/python2.6/site-packages', '/usr/local/lib/python2.6/dist-packages/pip-0.8.1-py2.6.egg', '/usr/local/lib/python2.6/site-packages', '/usr/local/lib/python2.6/dist-packages', '/usr/lib/python2.6/dist-packages', '/home/auston/new_proj/nikeshere', '/usr/local/lib/python2.6']
Traceback (most recent call last):
File "/home/auston/new_proj/nikeshere/app_wsgi.py", line 11, in <module>
import django.core.handlers.wsgi
File "/usr/local/lib/python2.6/site-packages/django/core/handlers/wsgi.py", line 1, in <module>
from threading import Lock
File "/usr/lib/python2.6/threading.py", line 13, in <module>
from functools import wraps
File "/usr/lib/python2.6/functools.py", line 10, in <module>
from _functools import partial, reduce
ImportError: No module named _functools
If I change --home to /usr/local/lib/python/2.6 I get fail on my app_wsgi.py import of os. Here it is, below, just in case:
import sys
import os
sys.path.append(os.path.abspath(os.path.dirname(__file__)))
import django.core.handlers.wsgi
application = django.core.handlers.wsgi.WSGIHandler()
Essentially I am asking, how can I get uWSGI to recognize functools OR get on the right path (path is in output above). I would appreciate any help you guys can give!!
P.S. Ubuntu 10.04 - uWSGI 0.9.6.5 - NGINX 0.8.53 - virtual env Python 2.6.5 - "regular (or system)" Python 2.6.6 - Django 1.2.3
UPDATE:
I was able to get uwsgi to start accepting requests if I omit the "--module" like so:
uwsgi --home /home/auston/new_proj --socket /tmp/uwsgi2.sock --chmod-socket --pp /home/auston/new_proj/nikeshere --logto /tmp/uwsgi.log --master --processes 4 -P
but now I get a app not found error:
"uWSGI Error
wsgi application not found"
I'm closer but I would still appreciate suggestions as the app is not found because i cannot include the module needed to load it!

So as noted above, the problem has been with the pythonpath & it's inability to find a module named _functools.
Apparently, _functools is a c module & I needed to append the it's path to the pythonpath in order for it to be found, so the difference from the original wsgi.py, is now:
import sys
sys.path.append('/usr/local/lib/python2.6/lib-dynload') # to load _functools
sys.path.append('/usr/local/lib/python2.6/site-packages') # to load django
sys.path.append('/usr/local/lib/python2.6/dist-packages') # cautionary to load django
sys.path.append('/usr/lib/python2.6') # to load os
import os
os.environ['DJANGO_SETTINGS_MODULE'] = 'iwin.settings'
import django.core.handlers.wsgi
application = django.core.handlers.wsgi.WSGIHandler()
Very hacky, but it works for now...

I know it's old topic and versions of stack building blocks changed, but I had the same problem with not recognizing under uWSGi installed libs in virtualenv. The solution is to point home parameter to virtualenv, as shown below (taken from https://uwsgi.readthedocs.org/en/latest/tutorials/Django_and_nginx.html).
So for me command:
uwsgi --http :8000 --module ii.wsgi --home /home/dev/.virtualenvs/ii_env/
worked, while being in the django application (ii) directory.
# mysite_uwsgi.ini file
[uwsgi]
# Django-related settings
# the base directory (full path)
chdir = /path/to/your/project
# Django's wsgi file
module = project.wsgi
# the virtualenv (full path)
home = /path/to/virtualenv
# process-related settings
# master
master = true
# maximum number of worker processes
processes = 10
# the socket (use the full path to be safe
socket = /path/to/your/project/mysite.sock
# ... with appropriate permissions - may be needed
# chmod-socket = 664
# clear environment on exit
vacuum = true

Check out http://blog.zacharyvoase.com/2010/03/05/django-uwsgi-nginx/. He's using very similar set up.

I've been having very similar problem and I found this:
When you install virtuelenv, it 'installs' the Python standard library by creating symlinks to the original one (in like /usr/lib/python2.7). But when you check your virtualenv Python lib directory, there are symlinks created for only a few basic libraries. Your functools is probably not among them.
So the solution is to create the symlink manually. It is a PITA, because you may have to create a lot of symlinks, but it seems like a cleaner solution to me. You don't have to hack any source files and it's transparent.
The symlink should be created not in the root of the venv_directory, but in e.g.
venv_directory/lib/python2.7/site-packages/
Hope it works for you!