Develop Reference

Error importing psutil in PyCharm Google App Engine Project

I have recently run into an import error with psutils in the types.py called from google.cloud.pubsub_v1 on a GAE standard project inside Pycharm. I messed around for a day trying to find away around it, but no go. psutil imports in python run from the terminal and the console and from a vanilla python project but not from a GAE project. To double up on this I made a basic test of the two as seen below: Both using the same virtualenv...
Standard Vanilla Python Project:
main.py contains
import psutil
def x():
print "hello world"
if __name__ == "__main__":
x()
Google App Engine Project -
main.py contains
import webapp2
import psutil
class MainHandler(webapp2.RequestHandler):
def get(self):
self.response.write('Hello world!')
app = webapp2.WSGIApplication([
('/', MainHandler)
], debug=True)
The import in the vanilla python project works fine, but the import in the GAE app returns the traceback:
Traceback (most recent call last):
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/runtime/wsgi.py", line 240, in Handle
handler = _config_handle.add_wsgi_middleware(self._LoadHandler())
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/runtime/wsgi.py", line 299, in _LoadHandler
handler, path, err = LoadObject(self._handler)
File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/runtime/wsgi.py", line 85, in LoadObject
obj = __import__(path[0])
File "/Users/chez 1/google-app-engine-projects/untitled/main.py", line 18, in <module>
import psutil
ImportError: No module named psutil
Has anyone had similar problems and found a fix?

You're likely having the psutil package installed on your local system (thus working in terminal/console), but not vendored into your app (i.e installed in your app's lib dir) as required for standard env apps (the SDK emulates the real GAE, for which whatever is installed into your local python installation means nothing). From Installing a third-party library:
Create a directory to store your third-party libraries, such as lib/.
mkdir lib
Use pip (version 6 or later) with the -t <directory> flag to copy the libraries into the folder you created in the previous
step. For example:
pip install -t lib/ <library_name>
So just vendor into your app the psutil package.

In my experience this and similar 'No module named ...' errors have been caused by an incorrectly defined python interpreter. See the answers at: ImportError: No module named 'bottle' - PyCharm

trouble installing pip and or queue for python

I have apache ampps which comes with version 3.6.1 of python.
I was given various directions for installing pip.
None seemed to work.
For example,
link https://packaging.python.org/tutorials/installing-packages/
says that I can run:
python -m pip install -U pip setuptools
Get a whole bunch of errors. It might amount to: no module named queue.
Similar errors happen when I download the file they mentioned (get-pip.py) and run it from python.
Now, when I look at directions for installing queue, some point me to use pip. But when I try to install pip, it is complaining that queue is not there....
Hmmm...?
Now what?
ERROR:
File "C:\Users\Nima\AppData\Local\Temp\tmp1v2hpnae\pip.zip\pip\compat\__init__.py", line 11, in <module>
File "C:\Program Files (x86)\Ampps\python\lib\logging\config.py", line 30, in <module>
import logging.handlers
File "C:\Program Files (x86)\Ampps\python\lib\logging\handlers.py", line 28, in <module>
import queue
ModuleNotFoundError: No module named 'queue'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "getpip.py", line 20061, in <module>
main()
File "getpip.py", line 194, in main
bootstrap(tmpdir=tmpdir)
File "getpip.py", line 82, in bootstrap
import pip
File "C:\Users\Nima\AppData\Local\Temp\tmp1v2hpnae\pip.zip\pip\__init__.py", line 26, in <module>
File "C:\Users\Nima\AppData\Local\Temp\tmp1v2hpnae\pip.zip\pip\utils\__init__.py", line 22, in <module>
File "C:\Users\Nima\AppData\Local\Temp\tmp1v2hpnae\pip.zip\pip\compat\__init__.py", line 13, in <module>
File "C:\Users\Nima\AppData\Local\Temp\tmp1v2hpnae\pip.zip\pip\compat\dictconfig.py", line 22, in <module>
File "C:\Program Files (x86)\Ampps\python\lib\logging\handlers.py", line 28, in <module>
import queue
ModuleNotFoundError: No module named 'queue'

From the format i see here.
Firstly, pip is an installer basically what you've confused yourself with is that pip = queue which is not the case. Pip is just a packager that helps you install packages. Queue is a different module
For your case here Queue is a part of multiprocessing module so you just put this at the top of your code:
from multiprocessing import Queue
and you do not need to add import pip into your code
hope that explained things better for you :)

This line:
python -m pip install -U pip setuptools
means use pip to upgrade the installations of pip and setuptools. Only works if you already have pip - which you probably do. It is a useful step to make sure your install environment is up to date, though.
pip does in places use queue. Note it's been renamed between python2 and python3 - if you have py3, which you claim, you have queue (it was Queue in py2). So I wonder if there's a version mismatch in something.
Windows installs always create problems. You might be safer installing and experimenting with a virtualenv so your experiments don't mess up the python install from the package you mention - ampps. There are plenty of notes on that elsewhere on stackoverflow... e.g. Python and Virtualenv on Windows

pip is already included in 3.6.1, but it is in subfolder Scripts.
it is not automatically a part of the path variable.
you have to change directory and run pip or you can change environment variable so that the location of pip becomes part of the path search.

Install custom sklearn

I forked the scikit-learn repository, added a file that I need and downloaded the repository. I am not sure how to use this custom library. I cd'ed to the scikit-learn-master folder and tried to use it but it throws errors. So after reading the errors I installed it using python3 setup.py install. There were two setup.py files. One in the scikit-learn-master folder and sklearn folder so I ran python3 setup.py install at both these locations. They threw some warnings but no errors. I opened the python terminal in the scikit-learn-master folder and used import sklearn which returns the following -
Traceback (most recent call last):
File "/Users/shubhamgandhi/Desktop/scikit-learn-master/sklearn/__check_build/__init__.py", line 44, in <module>
from ._check_build import check_build # noqa
ModuleNotFoundError: No module named 'sklearn.__check_build._check_build'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/shubhamgandhi/Desktop/scikit-learn-master/sklearn/__init__.py", line 128, in <module>
from . import __check_build
File "/Users/shubhamgandhi/Desktop/scikit-learn-master/sklearn/__check_build/__init__.py", line 46, in <module>
raise_build_error(e)
File "/Users/shubhamgandhi/Desktop/scikit-learn-master/sklearn/__check_build/__init__.py", line 41, in raise_build_error
%s""" % (e, local_dir, ''.join(dir_content).strip(), msg))
ImportError: No module named 'sklearn.__check_build._check_build'
___________________________________________________________________________
Contents of /Users/shubhamgandhi/Desktop/scikit-learn-master/sklearn/__check_build:
__init__.py __pycache__ _check_build.c
_check_build.pyx setup.py setup.pyc
___________________________________________________________________________
It seems that scikit-learn has not been built correctly.
If you have installed scikit-learn from source, please do not forget
to build the package before using it: run `python setup.py install` or
`make` in the source directory.
If you have used an installer, please check that it is suited for your
Python version, your operating system and your platform.
I am not sure how to proceed. Is there anything I am missing?

It appears the module has not been built correctly. And so it shows importError for check_build.
Before installing the module using pip, make sure you have installed all of the dependencies. On the README file, the mentioned packages are:
Python (>= 2.7 or >= 3.3)
NumPy (>= 1.8.2)
SciPy (>= 0.13.3)
For running the examples Matplotlib >= 1.1.1 is required.
If you are not planning on contributing to the project, but only using it, it is recommended that you download from https://pypi.python.org/pypi/scikit-learn instead of forking it.
View detailed instructions on how to install here.
After installing and building this way, if you still have issues, you can refer http://scikit-learn.org/stable/faq.html for FAQs.

Cannot import flask from project directory but works everywhere else

So I have run into a funny problem when trying to use Flask, I can only run it from ~/ (home) and not from ~/Projects/projectfolder. I'm using Python 2.7.4 installed via their homepage, virtualenv and virtualenvwrapper. Every time it's the same:
$ mkvirtualenv project
New python executable in project/bin/python
Installing setuptools............done.
Installing pip...............done.
Then I install Flask:
$ pip install flask
[...]
Successfully installed flask Werkzeug Jinja2
Cleaning up...
Then I open Python from my home directory:
(project) $ python
>>> from flask import Flask
>>>
Then I quit and go to my project folder:
(project) $ cd ~/Projects/example
(project) $ python
>>> from flask import Flask
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "flask.py", line 1, in <module>
from flask import Flask
ImportError: cannot import name Flask
And I'm a bit lost as to why this is happening, anybody have any ideas?

According to you traceback, you have a module of your own called flask.py in ~/Projects/example.
The current directory is searched before the actual package installation path, so it shadows the "real" Flask.

AttributeError: 'module' object has no attribute 'wraps'

I am trying to get Django running on a shared hosting account for which I have ssh access, but I have been having issues getting it working. When I first tried running django-admin.py, I received an error about the module 'functools' being missing. I had the hosting company install the functools package, but now I get the following traceback when I try to run django-admin.py.
Traceback (most recent call last):
File "manage.py", line 2, in <module>
from django.core.management import execute_manager
File "/usr/local/lib/python2.6/site-packages/Django-1.3-py2.6.egg/django/core/management/__init__.py", line 3, in <module>
from optparse import OptionParser, NO_DEFAULT
File "/usr/local/lib/python2.6/optparse.py", line 90, in <module>
from gettext import gettext
File "/usr/local/lib/python2.6/gettext.py", line 49, in <module>
import locale, copy, os, re, struct, sys
File "/usr/local/lib/python2.6/locale.py", line 98, in <module>
#functools.wraps(_localeconv)
AttributeError: 'module' object has no attribute 'wraps'
I opened a support ticket and was told by the company that they reinstalled Python 2.6, but that didn't change anything. I also tried with 2.4, which is also installed on the machine, but that resulted in the earlier ImportError: No module named functools error.
Does anybody have any ideas?

A likely cause is something shadowing the stdlib functools module. Do you have a functools.py alongside the main script (manage.py)? See what import functools; print functools.__file__ prints, inside manage.py (before any of the other imports.) If it doesn't print /usr/local/lib/python2.6/functools.py, there's probably a file shadowing the standard library module. If it does print that, something is wrong with that file, reinstalls notwithstanding.

this solution worked for me .
mv /usr/lib/python2.7/site-packages/pip* ./
yum reinstall python-pip
sudo pip install django
or try an another solution .
its too risky and this might not be a safest solution.but it worked fine for me .
uninstall pip.
pip uninstall pip
if you installed by get-pip.py .
python -m pip uninstall pip setuptools
navigate to the path.
/usr/lib/python2.7/site-packages/
and find out two pip directories pip and pip-6.0.8.dist-info.
remove both pip and pip-6.0.8.dist-info.
then try again.
yum install python-pip
or
python get-pip.py