Develop Reference

Add a Python module to a Google App Engine project

I'm trying to use the feed-parser module for this project im working on. When I upload the files to App Engine and I run the script it comes back with the error that the there is no module named feed-parser.
So I'm wondering if and how can I install this module on App Engine, so that I can fix this error and use RSS.
Error:
Traceback (most recent call last):
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 240, in Handle
handler = _config_handle.add_wsgi_middleware(self._LoadHandler())
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 299, in _LoadHandler
handler, path, err = LoadObject(self._handler)
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 85, in LoadObject
obj = __import__(path[0])
File "/base/data/home/apps/s~vlis-mannipulus-bot/1.391465315184045822/main.py", line 7, in <module>
import feedparser
ImportError: No module named feed parser
Development 1:
So I've tried installing the module in the lib directory i created(in this fail example i forgot the /lib at the --prefix=..). And i get PYTHONERROR as is shown in the shell. Ive done some research on python paths and the solutions i tried didn't work for me.
kevins-MacBook-Pro-2:~ KevinH$ cd /Users/KevinH/Downloads/feedparser -5.2.1
kevins-MacBook-Pro-2:feedparser-5.2.1 KevinH$ sudo python setup.py install --prefix=/Users/KevinH/Documents/Thalia\ VMbot/Thalia-VMbot/
Password:
running install
Checking .pth file support in /Users/KevinH/Documents/Thalia VMbot/Thalia-VMbot//lib/python2.7/site-packages/
/usr/bin/python -E -c pass
TEST FAILED: /Users/KevinH/Documents/Thalia VMbot/Thalia- VMbot//lib/python2.7/site-packages/ does NOT support .pth files
error: bad install directory or PYTHONPATH
You are attempting to install a package to a directory that is not
on PYTHONPATH and which Python does not read ".pth" files from. The
installation directory you specified (via --install-dir, --prefix, or
the distutils default setting) was:
/Users/KevinH/Documents/Thalia VMbot/Thalia-VMbot//lib/python2.7/site- packages/
and your PYTHONPATH environment variable currently contains:
''
Here are some of your options for correcting the problem:
* You can choose a different installation directory, i.e., one that is
on PYTHONPATH or supports .pth files
* You can add the installation directory to the PYTHONPATH environment
variable. (It must then also be on PYTHONPATH whenever you run
Python and want to use the package(s) you are installing.)
* You can set up the installation directory to support ".pth" files by
using one of the approaches described here:
https://pythonhosted.org/setuptools/easy_install.html#custom- installation-locations
Please make the appropriate changes for your system and try again.
Then i tried with the "pip" command but then i get this:
can't open file 'pip': [Errno 2] No such file or directory
According to what I have read "pip" should be a default program installed with python 2.7 and up. So to be sure i did install python3.5 and ran it with that and still get the same error. I typed this with both pythons:
kevins-MacBook-Pro-2:feedparser KevinH$ python3 pip -m install feedparse
--
Not sure if this would work, but via terminal i went to the default directory where feed parser has been installed on my system and copied it to the lib directory i made. Then I've created the config file with the following:
from google.appengine.ext import vendor
# Add any libraries installed in the "lib" folder.
vendor.add('lib')
Deployed it and im still getting the same error as above no module named feeedparser.
Apologies if im doing something stupidly wrong, im still in the learning process.

The App Engine documentation that explains how to add third party modules is here
In summary, you will need to add a folder, usually named 'lib', to the top level off your app, and then install feedparser into that folder using the commands described in the documentation. You will also need to create an appengine_config.py file as descibed in the documentation.
Note that not all third party packages can be uploaded to App Engine - those with C extensions are forbidden. Feedparser looks OK to me though.
EDIT: further comments based on edit "development1" to the question.
Your appengine_config.py looks good.
You "lib" folder should be your application folder, that is the same folder as your app.yaml and appengine_config.py files.
You need to install the feedparser package into the lib folder. The Google docs recommend that you do this by running the command
pip install -t lib feedparser
This command will install the feedparser package into your lib folder.
You need to install and run a version of pip that works with Python2.x - the Python3 version will create a version of feedparser that only runs under Python3.
If you can't install a pip for Python2 this question might provide the right solution, otherwise I'd suggest you ask a separate question about how to install feedparser into a custom install directory on a Mac.I don't have a Mac so I can't help with this.
Once you have feedparser installed in your lib folder, verify that your app works locally; in particular verify that it's using your lib/feedparser installation: try logging feedparser.__file__ after importing feedparser to check the location of the file being imported.
Once everything is working locally you should be able to upload to GAE.
So in summary, your appengine_config.py looks good, but you need to make sure that the right version of feedparser is installed into the lib folder in your app folder before uploading to App Engine.

Python GUI2Exe Application Standalone Build (Using Py2Exe)

I am trying to build a Python Script into a stand alone application. I am using GUI2Exe. My script uses selenium package. I have it installed.
Project compiles fine and runs on python command line directly but fails to build a stand alone because it is referring to folder:
ERROR: test_file_data_extract (__main__.FileDataExtract)
----------------------------------------------------------------------
Traceback (most recent call last):
File "File_data_extract.py", line 18, in setUp
File "selenium\webdriver\firefox\firefox_profile.pyc", line 63, in __init__
IOError: [Errno 2] No such file or directory: 'C:\\users\\username\\PycharmProjects\\Python_27_32bit\\file_data_extract\\dist\\File_data_extract.exe\\selenium\\webdriver\\firefox\\webdriver_prefs.json'
It is looking for selenium package is located at :
C:\Users\username\Anaconda2_Py27_32bit\Lib\site-packages\selenium-2.48.0-py2.7.egg\selenium\webdriver\firefox
where C:\Users\username\Anaconda2_Py27_32bit is where I installed Anaconda Python 2.7, 32 bit version. By default it is looking for in \dist\filename.exe folder.

I was able to build it using bbfreeze. It works great.
First I had to install bbfreezee via pip (one time only):
pip install bbfreeze
Create a build_package.py file as:
from bbfreeze import Freezer
f = Freezer("project_name", includes=("selenium","SendKeys",)) #list problem packages here to manually include
f.addScript("project_name_script.py")
f() # starts the freezing process
Build project:
python build_package.py bdist_bbfreezee
in folder project_name where project_name_script.py sits you find project_name_script.exe with all the include packages including selenium and sendkeys. When you distribute the package you need to distribute entire project_name because it contains all dependent library dlls (python .pyd).
More details refer official bbfreezee here:
https://pypi.python.org/pypi/bbfreeze/#downloads

not able to install netCDF4 in python without HDF5

i have been trying to install the python package for netcdf4 using the package provided in
ftp://ftp.unidata.ucar.edu/pub/netcdf/
i have installed the netcdf-4.3.3.1.zip
extracted it and followed the instructions given in
https://www.unidata.ucar.edu/software/netcdf/docs/netcdf-install/Quick-Instructions.html
2.1 Building NetCDF Without HDF5
If you don't want netCDF-4/HDF5, then build like this:
./configure --prefix=/home/ed/local --disable-netcdf-4
make check install
(Replace “/home/ed/local” with the name of the directory where netCDF is to be installed.)
i replaced '/home/ed/local' with '/usr/lib/python2.7/dist-packages'
after this i ran the command ' nc-config --version'
netCDF 4.3.3.1
but i am not able to import the module in python i am geting that
ImportError: No module named netCDF4
nor i am able to find any folder related to netcdf4 in /usr/lib/python2.7/dist-packages
i just need to read the .nc file data if their is any other way help me
thank you
EDIT -------
i realised that you require hdf5 so i got the package
then when i follow the instructions given in the link
when i run the instructions for hdf5 folder i got some errors related to conversion of dos files to unix i fixed them by saving them with ending in unix/linux . but i did not do all the files as after the first two files there where no errors
then i use the commands
make
sudo make install
i get multiple output as
make[1]: Entering directory `/home/akhil/hdf5-1.8.16/src'
Makefile:972: .deps/H5.Plo: No such file or directory
Makefile:973: .deps/H5A.Plo: No such file or directory
Makefile:974: .deps/H5AC.Plo: No such file or directory
Makefile:975: .deps/H5Abtree2.Plo: No such file or directory
Makefile:976: .deps/H5Adense.Plo: No such file or directory
and many errors of the same format
so i ignore them since there was no specific error
igo do the commands for netdf4 file
once i do the first command
LDFLAGS=-L/usr/local/lib CPPFLAGS=-I/usr/local/include ./configure -- enable-netcdf-4 --enable-dap --enable-shared --prefix=/usr/local
i get an error
configure: error: Can't find or link to the hdf5 library. Use --disable-netcdf-4, or see config.log for errors.
then i go into the netcdf4 python
i run sudo python setup.py install
error
netCDF4/_netCDF4.c:237:22: fatal error: H5public.h: No such file or directory
#include "H5public.h"
^
compilation terminated.
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
help

Workaround read-only site-packages for cx_freeze

I'm currently trying to make cx_freeze to work on a Solaris workstation I have to work with, in order to make an executable from a Python script I have. Problem is, I'm not administrator of this machine, and installation of cx_freeze requests write to site-packages, which is read-only for me.
So, obviously, I get this error:
creating /usr/local/lib/python2.6/site-packages/cx_Freeze
error: could not create '/usr/local/lib/python2.6/site-packages/cx_Freeze': Read-only file system
And if I try to run it anyway, it fails:
bash-3.00$ python /home/xxxx/cx_freeze-4.2.3/cxfreeze --target-dir cx_dist src/p_tool.py
Traceback (most recent call last):
File "/home/xxxx/cx_freeze-4.2.3/cxfreeze", line 5, in <module>
main()
File "/home/xxxx/cx_freeze-4.2.3/cx_Freeze/main.py", line 187, in main
silent = options.silent)
File "/home/xxxx/cx_freeze-4.2.3/cx_Freeze/freezer.py", line 91, in __init__
self._VerifyConfiguration()
File "/home/xxxx/cx_freeze-4.2.3/cx_Freeze/freezer.py", line 371, in _VerifyConfiguration
self._GetInitScriptFileName()
File "/home/xxxx/cx_freeze-4.2.3/cx_Freeze/freezer.py", line 283, in _GetInitScriptFileName
raise ConfigError("no initscript named %s", name)
cx_Freeze.freezer.ConfigError: no initscript named Console
Obviously, this is linked to the failed installation. So, here's my question:
Without installation of virtualenv, could I avoid the writing to site-packages, and make cx_freeze to execute from my home folder?
EDIT I had a look at site.py documentation, and PYTHONPATH filling should be equivalent to use of site-packages. So my question is now more something like: what is the path to be added to PYTHONPATH, so that cx_freeze could be executed from any location?
Notes:
I would like to avoid to deal with virtualenv, as I'm already struggling to understand the executable tools...
I saw this question, but this still requires access to site-packages folder, plus it's not user-specific;
I tried adding the following path to PYTHONPATH, but this does not work: /home/xxxx/cx_freeze-4.2.3/build/lib.solaris-2.10-sun4v-2.6;
I'm also trying to use PyInstaller but have dependency problems (and the administrator is not really helping me).

This works like a charm for me :
$ python setup.py install --home=$HOME
Run in the source directory of cx_freeze found on the Sourceforge download page.

Django, PIP, and Virtualenv

Got this django project that I assume would run on virtualenv. I installed virtualenv through pip install and created the env but when I try to feed the pip requirements file, I got this:
Directory 'tagging' is not installable. File 'setup.py' not found.
Storing complete log in /Users/XXXX/.pip/pip.log
Here's the entry on the log file:
------------------------------------------------------------
/Users/XXXX/Sites/SampleProject/bin/pip run on Wed Jul 21 06:35:02 2010
Directory 'tagging' is not installable. File 'setup.py' not found.
Exception information:
Traceback (most recent call last):
File "/Users/XXXX/Sites/SampleProject/lib/python2.6/site-packages/pip-0.7.2-py2.6.egg/pip/basecommand.py", line 120, in main
self.run(options, args)
File "/Users/XXXX/Sites/SampleProject/lib/python2.6/site-packages/pip-0.7.2-py2.6.egg/pip/commands/install.py", line 158, in run
for req in parse_requirements(filename, finder=finder, options=options):
File "/Users/XXXX/Sites/SampleProject/lib/python2.6/site-packages/pip-0.7.2-py2.6.egg/pip/req.py", line 1395, in parse_requirements
req = InstallRequirement.from_line(line, comes_from)
File "/Users/XXXX/Sites/SampleProject/lib/python2.6/site-packages/pip-0.7.2-py2.6.egg/pip/req.py", line 87, in from_line
% name)
InstallationError: Directory 'tagging' is not installable. File 'setup.py' not found.
Also, here's the requirements file I'm trying to feed:
# to use:
# mkvirtualenv %PROJECT% (or workon %PROJECT%)
# export PIP_RESPECT_VIRTUALENV=true
# pip install -r requirements.txt
# you'll also need:
# mongodb1.1.4
# imagemagick > 6.3.8
# -e svn+http://code.djangoproject.com/svn/django/trunk#egg=djangoipython
ipdb
PIL
django-extensions
django-debug-toolbar
pytz
tagging
Could it be a problem with PIP? I have installed it through easy_install and used it already to install some modules such as fabric and etc. with no problems.
Hope someone could lend a hand :) BTW, here's my local setup: OSX 10.6.4, Python 2.6.1, Django 1.3 alpha. Thanks!

Sounds like you have a tagging/ directory in the directory from which you are running pip, and pip thinks this directory (rather than the django-tagging project on PyPI) is what you want it to install. But there's no setup.py in that directory, so pip doesn't know how to install it.
If the name of the project you wanted to install from PyPI were actually "tagging", you'd need to move or rename the tagging/ directory, or else run pip from a different directory. But it's not; it's actually django-tagging: http://pypi.python.org/pypi/django-tagging So if you just change the entry in your requirements file from "tagging" to "django-tagging," it should work.
All of this is a bug in pip, really: it should assume something is a PyPI project name rather than a local directory, unless the name you give has an actual slash in it or appended to it.

Is it possible you've copied the "tagging" directory from this location in the django-tagging source? In that case, you actually need the root from this location which has "tagging" as a sub-directory and a setup.py file. Just checkout from trunk or unzip to a "django-tagging" directory and make sure that your requirements file points to the "django-tagging" directory.