I was seeing this in my Google App Engine Log:
...
File "/base/data/home/apps/s~just-terminus-94303/1.384249106864280829/appengine_config.py", line 5, in <module>
vendor.add('google-api-python-client')
...
ValueError: virtualenv: cannot access google-api-python-client: No such virtualenv or site directory
I edited the appengine_config.py file from
vendor.add('google-api-python-client')
to
vendor.add('googleapiclient')
and I commit the change but I am still getting the same exact error as above. How do I get my change to update?
The vendor.add() method is used to add a folder to your application's python path. It should be a root directory that contains python packages that you may have installed using pip's target option. It should not point directly to a python module.
vendor.add('lib') # lib contains googleapiclient/
Related
I'm trying to integrate python3-functionfs module into buildroot.
I'm able to select it with make menuconfig but when I'm running make the package isn't even downloaded.
The package is available here : functionfs-0.3 pypi page
And the download url here : functionfs-0.3 download link
There is also the github repo here : functionfs git repository
I'm using Buildroot 2017.02 version.
Here is my Config.in file :
config BR2_PACKAGE_PYTHON3_FUNCTIONFS
bool "python3-functionfs"
depends on BR2_PACKAGE_PYTHON3
help
Pythonic API for linux’s functionfs.
functionfs is part of the usb gadget subsystem. Together with usb_gadget’s configfs integration, allows userland to declare and implement an USB device.
https://pypi.python.org/pypi/functionfs
And here is my .mk file :
################################################################################
#
# python3-functionfs
#
################################################################################
PYTHON_FUNCTIONFS_VERSION = 0.3
PYTHON_FUNCTIONFS_SOURCE = functionfs-$(PYTHON_FUNCTIONFS_VERSION).tar.gz
PYTHON_FUNCTIONFS_SITE = https://pypi.python.org/packages/e3/2d/56e0d9ffe0da7c116a6724ee538375689dd59e34dbe1676edf6b66b52be4
PYTHON_FUNCTIONFS_LICENSE = GPLv3+
PYTHON_FUNCTIONFS_LICENSE_FILE = COPYING
PYTHON_FUNCTIONFS_SETUP_TYPE = setuptools
$(eval $(python-package))
The documentation also mentions at 17.8.3. Generating a python-package from a PyPI repository
If the Python package for which you would like to create a Buildroot
package is available on PyPI, you may want to use the scanpypi tool
located in utils/ to automate the process.
You can find the list of existing PyPI packages here.
scanpypi requires Python’s setuptools package to be installed on your
host.
When at the root of your buildroot directory just do :
utils/scanpypi foo bar -o package
This will generate packages python-foo and python-bar in the package
folder if they exist on https://pypi.python.org.
Find the external python modules menu and insert your package inside.
Keep in mind that the items inside a menu should be in alphabetical
order.
Please keep in mind that you’ll most likely have to manually check the
package for any mistakes as there are things that cannot be guessed by
the generator (e.g. dependencies on any of the python core modules
such as BR2_PACKAGE_PYTHON_ZLIB). Also, please take note that the
license and license files are guessed and must be checked. You also
need to manually add the package to the package/Config.in file.
If your Buildroot package is not in the official Buildroot tree but in
a br2-external tree, use the -o flag as follows:
utils/scanpypi foo bar -o other_package_dir
This will generate packages python-foo and python-bar in the
other_package_directory instead of package.
Option -h will list the available options:
utils/scanpypi -h
However I don't have an util/ folder in buildroot main directory.
The script is located at support/scripts/scanpypi but when I'm running it I have the following error :
$ support/scripts/scanpypi functionfs -o package
Traceback (most recent call last):
File "support/scripts/scanpypi", line 47, in <module>
import setuptools
File "/usr/local/lib/python2.7/dist-packages/setuptools/__init__.py", line 11, in <module>
from setuptools.extern.six.moves import filterfalse, map
File "/usr/local/lib/python2.7/dist-packages/setuptools/extern/__init__.py", line 1, in <module>
from pkg_resources.extern import VendorImporter
File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 40, in <module>
from pkgutil import get_importer
ImportError: cannot import name get_importer
This can be solved by renaming both support/scripts/pkgutil.py and support/scripts/pkgutil.pyc.
However I would like to understand what's going on when I'm trying to create the package by myself and it doesn't download.
Does someone know why functionfs-0.3.tar.gz is not downloaded when running make ?
Your package is not downloaded because there is a mismatch between your package name and the name of the variables in the .mk files. Basically, you have three things that must match:
The BR2_PACKAGE_<FOO> option in Config.in
The name of the file and directory must be package/<foo>/<foo>.mk
The variables in the .mk files must be named <FOO>_SOMETHING
You didn't say what the name of the .mk file was, but at the very least your option is named BR2_PACKAGE_PYTHON3_FUNCTIONFS while the make variables are named PYTHON_FUNCTIONFS_SOMETHING.
That explains why it isn't downloaded.
Then, regarding the scanpypi script, it is definitely in the utils/ directory in recent versions of Buildroot. It used to be in support/scripts a few releases ago. So basically you're looking at the Buildroot documentation that is online (and matches the latest release), but you're using an older Buildroot version. You can build the Buildroot documentation matching your Buildroot release by running make manual-html.
I am trying to run my GAE app locally inside a virtual environment.
I've followed these two articles [1], [2] as reference to setup, but when I source evn/bin/activate and then dev_appserver.py ., it keeps raising the error of ImportError: No module named warnings (more trace below)
Surprisingly, if I start it without activating virtual env by just running dev_appserver.py . inside root of project it runs without any issue.
Is there any solution or workaround for this problem?
INFO 2017-08-31 14:09:36,293 devappserver2.py:116] Skipping SDK update check.
INFO 2017-08-31 14:09:36,354 api_server.py:313] Starting API server at: http://localhost:52608
INFO 2017-08-31 14:09:36,357 dispatcher.py:226] Starting module "default" running at: http://localhost:8080
INFO 2017-08-31 14:09:36,359 admin_server.py:116] Starting admin server at: http://localhost:8000
Traceback (most recent call last):
File "/usr/local/share/google/google-cloud-sdk/platform/google_appengine/_python_runtime.py", line 103, in <module>
_run_file(__file__, globals())
File "/usr/local/share/google/google-cloud-sdk/platform/google_appengine/_python_runtime.py", line 97, in _run_file
execfile(_PATHS.script_file(script_name), globals_)
File "/usr/local/share/google/google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/python/runtime.py", line 192, in <module>
main()
File "/usr/local/share/google/google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/python/runtime.py", line 172, in main
sandbox.enable_sandbox(config)
File "/usr/local/share/google/google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/python/sandbox.py", line 326, in enable_sandbox
__import__('%s.threading' % dist27.__name__)
File "/usr/local/share/google/google-cloud-sdk/platform/google_appengine/google/appengine/dist27/threading.py", line 11, in <module>
import warnings
File "/usr/local/share/google/google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/python/sandbox.py", line 1076, in load_module
raise ImportError('No module named %s' % fullname)
ImportError: No module named warnings
ERROR 2017-08-31 14:09:39,070 instance.py:280] Cannot connect to the instance on localhost:52366
I've solved it by removing from the skip_files of my .yaml a - venv line. venv is my virtualenv folder, and I still have a - ^venv$ line. In case that helps someone ;-)
All dependencies for standard environment GAE apps (which aren't provided by GAE) must be installed in the app itself, not on your local system. See Using third-party libraries.
Since GAE doesn't care about your local system libraries (besides the basic python 2.7 installation needed to run the development server), using a virtualenv for developing standard env GAE apps doesn't make a lot of sense.
The use of the virtualenv as suggested by the articles you mentioned can actually cause trouble:
local python libraries can interfere with the GAE runtime-provided equivalent libraries (from the SDK) when running locally (I suspect this is somehow what you're experiencing)
at deployment time the content of the lib directory (which includes all the site packages for your python installation) will be uploaded to GAE, potentially causing clashes with the GAE runtime-provided libraries or exceeding the app file quota.
So my suggestion is to drop virtualenv (which, in a way, you did when you skipped the virtualenv activation) and follow the official documentation instead.
I had this problem when running two AppEngine projects in separate directories at once using dev_appserver.py in a virtualenv.
I found it works correctly if I use absolute paths instead.
I'm getting ImportError: No module named when I try to deploy my app to heroku but the app builds fine locally. Here are the logs from Heroku
Traceback (most recent call last):
File "mr_meeseeks.py", line 4, in <module>
import Helpers.Plugin_Handler as Plugin_Handler
File "/app/Helpers/Plugin_Handler.py", line 5, in <module>
from Utils.constants import Plugin_Type
ImportError: No module named 'Utils.constants'
Here is my file structure:
As far as I can tell, Utils/constants.py exists. In case it's relevant, this is a SlackBot. The rest of the code can be found here.
The Python Interpreter looks for modules under the $PYTHONPATH environment variable. It looks like you or your editor (my editor does this when I mark a directory as sources) root has added SlackBot/ to $PYTHONPATH.
I myself encountered this error when I marked a directory as sources root.
You have a few options:
Make the bot launch script append SlackBot/ to the $PYTHONPATH
Use relative imports - from ..Utils import constants
Dynamically add SlackBot/ to the sys.path variable
Also, a note on style: python classes should be CamelCase and python modules should be lowercase_with_underscores. If you have an editor like PyCharm, your editor can fix these issues automatically, and more.
PEP 8 is the official python style guide, although I recommend using a linter so these issues can be automatically detected and fixed.
In your Procfile, append Slackbot to PYTHONPATH, try adding
--pythonpath SlackBot
as a argument. This link address the exact same issue as well.
I'm trying to use the feed-parser module for this project im working on. When I upload the files to App Engine and I run the script it comes back with the error that the there is no module named feed-parser.
So I'm wondering if and how can I install this module on App Engine, so that I can fix this error and use RSS.
Error:
Traceback (most recent call last):
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 240, in Handle
handler = _config_handle.add_wsgi_middleware(self._LoadHandler())
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 299, in _LoadHandler
handler, path, err = LoadObject(self._handler)
File "/base/data/home/runtimes/python27/python27_lib/versions/1/google/appengine/runtime/wsgi.py", line 85, in LoadObject
obj = __import__(path[0])
File "/base/data/home/apps/s~vlis-mannipulus-bot/1.391465315184045822/main.py", line 7, in <module>
import feedparser
ImportError: No module named feed parser
Development 1:
So I've tried installing the module in the lib directory i created(in this fail example i forgot the /lib at the --prefix=..). And i get PYTHONERROR as is shown in the shell. Ive done some research on python paths and the solutions i tried didn't work for me.
kevins-MacBook-Pro-2:~ KevinH$ cd /Users/KevinH/Downloads/feedparser -5.2.1
kevins-MacBook-Pro-2:feedparser-5.2.1 KevinH$ sudo python setup.py install --prefix=/Users/KevinH/Documents/Thalia\ VMbot/Thalia-VMbot/
Password:
running install
Checking .pth file support in /Users/KevinH/Documents/Thalia VMbot/Thalia-VMbot//lib/python2.7/site-packages/
/usr/bin/python -E -c pass
TEST FAILED: /Users/KevinH/Documents/Thalia VMbot/Thalia- VMbot//lib/python2.7/site-packages/ does NOT support .pth files
error: bad install directory or PYTHONPATH
You are attempting to install a package to a directory that is not
on PYTHONPATH and which Python does not read ".pth" files from. The
installation directory you specified (via --install-dir, --prefix, or
the distutils default setting) was:
/Users/KevinH/Documents/Thalia VMbot/Thalia-VMbot//lib/python2.7/site- packages/
and your PYTHONPATH environment variable currently contains:
''
Here are some of your options for correcting the problem:
* You can choose a different installation directory, i.e., one that is
on PYTHONPATH or supports .pth files
* You can add the installation directory to the PYTHONPATH environment
variable. (It must then also be on PYTHONPATH whenever you run
Python and want to use the package(s) you are installing.)
* You can set up the installation directory to support ".pth" files by
using one of the approaches described here:
https://pythonhosted.org/setuptools/easy_install.html#custom- installation-locations
Please make the appropriate changes for your system and try again.
Then i tried with the "pip" command but then i get this:
can't open file 'pip': [Errno 2] No such file or directory
According to what I have read "pip" should be a default program installed with python 2.7 and up. So to be sure i did install python3.5 and ran it with that and still get the same error. I typed this with both pythons:
kevins-MacBook-Pro-2:feedparser KevinH$ python3 pip -m install feedparse
--
Not sure if this would work, but via terminal i went to the default directory where feed parser has been installed on my system and copied it to the lib directory i made. Then I've created the config file with the following:
from google.appengine.ext import vendor
# Add any libraries installed in the "lib" folder.
vendor.add('lib')
Deployed it and im still getting the same error as above no module named feeedparser.
Apologies if im doing something stupidly wrong, im still in the learning process.
The App Engine documentation that explains how to add third party modules is here
In summary, you will need to add a folder, usually named 'lib', to the top level off your app, and then install feedparser into that folder using the commands described in the documentation. You will also need to create an appengine_config.py file as descibed in the documentation.
Note that not all third party packages can be uploaded to App Engine - those with C extensions are forbidden. Feedparser looks OK to me though.
EDIT: further comments based on edit "development1" to the question.
Your appengine_config.py looks good.
You "lib" folder should be your application folder, that is the same folder as your app.yaml and appengine_config.py files.
You need to install the feedparser package into the lib folder. The Google docs recommend that you do this by running the command
pip install -t lib feedparser
This command will install the feedparser package into your lib folder.
You need to install and run a version of pip that works with Python2.x - the Python3 version will create a version of feedparser that only runs under Python3.
If you can't install a pip for Python2 this question might provide the right solution, otherwise I'd suggest you ask a separate question about how to install feedparser into a custom install directory on a Mac.I don't have a Mac so I can't help with this.
Once you have feedparser installed in your lib folder, verify that your app works locally; in particular verify that it's using your lib/feedparser installation: try logging feedparser.__file__ after importing feedparser to check the location of the file being imported.
Once everything is working locally you should be able to upload to GAE.
So in summary, your appengine_config.py looks good, but you need to make sure that the right version of feedparser is installed into the lib folder in your app folder before uploading to App Engine.
I've just cloned a Django project using Mercurial to a Windows machine that I have set up Python 2.7 on. When I try to run manage.py (with or without a command), I get the following error:
Traceback (most recent call last):
File "C:\Users\jes.000\Documents\project\manage.py", line 30, in <module>
import django.core.servers.basehttp
File "C:\Python27\lib\site-packages\django\core\servers\basehttp.py", line 26, in <module>
from django.views import static
File "C:\Python27\lib\site-packages\django\views\static.py", line 95, in <module>
template_translatable = ugettext_noop(u"Index of %(directory)s")
File "C:\Python27\lib\site-packages\django\utils\translation\__init__.py", line 75, in gettext_noop
return _trans.gettext_noop(message)
File "C:\Python27\lib\site-packages\django\utils\translation\__init__.py", line 48, in __getattr__
if settings.USE_I18N:
File "C:\Python27\lib\site-packages\django\utils\functional.py", line 184, in inner
self._setup()
File "C:\Python27\lib\site-packages\django\conf\__init__.py", line 40, in _setup
raise ImportError("Settings cannot be imported, because environment variable %s is undefined." %ENVIRONMENT_VARIABLE)
ImportError: Settings cannot be imported, because environment variable DJANGO_SETTINGS_MODULE is undefined.
I understand that manage.py is supposed to set that environment variable. I also understand that I might be getting this error because the project is not in the python path... but adding it manually seems like it shouldn't be necessary. Isn't manage.py supposed to resolve this issue as well?
Edit: I've just tested and found that it does the exact same thing under Linux.
Edit: Here's a useful discovery: I only get this error if I install Django via PIP, it works fine if I install the python-django package on Ubuntu. So there's something that the debian package is doing that PIP isn't. Maybe the debian package sets some environment variables? I need to figure that out, since I'm trying to develop on Windows.
Edit: I think I've found the problem, but not a solution. Working on an Ubuntu machine, when I install Django via PIP (pip install django), this project doesn't work with the error I gave. If I create a new project (django-admin startproject testproject) and try that it works, and it consists of manage.py in the created folder, with another folder that contains everything else. Like this:
+-testproject
+-manage.py
+-testproject
+-__init__.py
+-etc.
When I install the Debian package (apt-get install python-django) the project I'm trying to work on works, and if I create a test project (django-admin startproject testproject) it has a folder structure like this:
+-testproject
+-manage.py
+-__init__.py
+-etc.
Notice the difference: the PIP Django package seems to put the manage.py outside of the python module that is the app, while the Debian package puts manage.py inside the module. I assume that this is due to the two packages being of a different version of Django, in between which they changed the structure, because the two packages doing this differently doesn't make any sense.
So my problem is that the project I'm trying to develop was originally generated using the version that's been Debian packaged, while on my Windows machine I'm trying to use the version of Django from pip.
The trouble is that I'm not certain how to fix the problem. Naively moving the manage.py up a directory so it's above the project folder doesn't work, because this 'debian-package version' manage.py tries to just import settings, which fails if it's not in the same folder as settings.py. But somewhere else in Django, something seems to be expecting the project itself to be at //settings.py (as would be the case if the pip version of django-admin was used) rather than at /settings.py (as is the case if the debian-package version is used).
If I look it up, the version in the Ubuntu repositories is 1.3.1, while the version in PyPI is 1.4. Look at the release notes for 1.4, and hey:
An updated default project layout and manage.py that removes the
“magic” from prior versions. And for those who don’t like the new
layout, you can use custom project and app templates instead!
But... shouldn't projects generated with the 1.3 layout still work under 1.4..?
I'm not sure if this has changed in more recent version of django, but the default manage.py simply tries to import the settings.py file and start the manager. You would be expected to put the project in your PYTHONPATH, or to put the settings.py into your DJANGO_SETTINGS_MODULE variable, by default.
You can modify the manage.py to be a bit smarter with this snippet:
manage.py
#!/usr/bin/env python
from django.core.management import execute_manager
...
import settings
import os
project = os.path.basename(os.path.dirname(__file__))
os.environ['DJANGO_SETTINGS_MODULE'] = '%s.settings' % project
if __name__ == "__main__":
execute_manager(settings)
Now you can run manage.py from any location, since it will always explicitly set up the path to the settings.py file.
You may add the following code to the file manage.py ,like this:
from django.conf import settings
settings.configure()
This is indeed due to the difference in default layouts between 1.3 and 1.4. Normally Django 1.3 projects will work under 1.4, so something else is going on here that I can't quite figure out. I have established that the problem isn't platform-dependent.
I couldn't figure out the exact problem but I did figure out a fix. That was:
Create a new project under Django 1.4 and poach its manage.py file. After changing the project name, this file works as a manage.py. I named it 'manage14.py' so that the original manage.py would be left for my colleagues using Django 1.3.
Add the directory above the one that the project was in to the global PYTHONPATH. This is not an ideal solution, but it allowed an unmodified Django 1.4 manage.py to load the project without having to convert the project to Django 1.4's organization.
As soon a practical, I plan to convert the project to the Django 1.4 format since it seems to be better anyway, particularly from the perspective of avoiding this kind of problem. Unfortunately I'll have to wait until the Ubuntu repo's get 1.4 so that my colleagues will be up to the same version I am on Windows using PyPI.