The latest Python2.7 has a google directory within dist-packages, making it impossible to import the google directory which contains appengine and other packages from another location. Such directory is required to work to effect imports from GoogleAppEngine (GAE) code on the dev_server. Otherwise such imports fail. An example of such import is:
from google.appengine.api import mail
which yields
ImportError: No module named appengine.api
This issue is similar to the one in here and indeed following Alex Martelli's reply the location of my google import is
In [1]: import google
In [2]: google.__file__
Out[2]: '/usr/lib/python2.7/dist-packages/google/__init__.pyc'
rather than the one where I placed the GAE unzipped files.
Any recommended way to fix this? I already thought about dirty hacks to fix this, such as putting soft links in the dist-packages google directory, but again, that's dirty.
Packages have a special attribute, __path__, which tells the Python interpreter where to look for modules and subpackages. By modifying this, you can let Python find contents from both google directories. Using the pkgutil module, this ought to work (untested):
import pkgutil
import google
google.__path__ = pkgutil.extend_path(google.__path__, google.__name__)
Are you sure this google directory came with Python 2.7? I've seen it too, but it appeared for the first time when I installed some utility from Google (I think it was Google Storage for Developers). I also think there's a .pth file related to this.
Related
I am testing my own package, but I am struggling to import it. My package file structure is as follows:
(You can also alternatively view my Github repository here)
In PyAdventures is my init.py with the content of
name="pyadventures"
So my question is, when I import it in another python file, how come it doesn't work?
I run:
import pyadventures
But I get the following error:
No module named pyadventures
It'd be great if you could answer my question!
It's important to note that the package is in my Python package directory, not the test one I showed
New discovery! In my PyAdventures folder (the one Python actually uses) the folder only has a few files, not the ones in the screenshot above.
You can install this with pip install pyadventures
Ah, like others remark in comments and answer: The camelcase might be the problem. (Why not name it just all in lower case: pyadventures? - Else users will be as confused as its developer now :D .)
Before, I thought, it might be a problem that you want to use your module without having installed it (locally). And my following answer is for this case (using a not installed package from a local folder):
In the docs you can read:
The variable sys.path is a list of strings that determines the
interpreter’s search path for modules. It is initialized to a default
path taken from the environment variable PYTHONPATH, or from a
built-in default if PYTHONPATH is not set. You can modify it using
standard list operations:
import sys
sys.path.append('/ufs/guido/lib/python')
thus, before import do:
import sys
sys.path.append('/absolute/or/relative/path/to/your/module/pyadventures')
# Now, your module is "visible" for the module loader
import pyadventures
Alternatively, you could just place your pyadventures module folder locally in your working directory where you start python and then just do
import pyadventures
However, it is much easier to manage to keep only one version in one place and refer to this from other places/scripts. (Else you have multiple copies, and have difficulties to track changes on the multiple copies).
As far as I know, your __init__.py doesn't need to contain anything. It works if it is empty. It is there just to mark your directory as a module.
Simple. Even though the package listed on pip is namd pyadventures, in your code the directory is called PyAdventures. So that's what python knows it as. I ran import PyAdventures and it worked fine.
I am following the tutorial here:
http://www.prokopyshen.com/create-custom-zipline-data-bundle
and trying to set up a custom bundle to get price from custom, non US financial assets. I am stuck on the line that says:
Advise zipline of our bundle by registering it via .zipline/extension.py
My extension.py file is located in the .zipline/ directiory and has the following code:
from zipline.data.bundles import register
from zipline.data.bundles.viacsv import viacsv
eqSym = {
"CBA"
}
register(
'CBA.csv', # name this whatever you like
viacsv(eqSym),
)
I don't get what it means to register the bundle via .zipline/extension.py though? I thought it might mean to just run the extension.py file from my terminal via a:
python extenion.py
but that fails and says:
ImportError: No module named viacsv
How do i register this bundle?
I also followed this tutorial and I must confess this part is a little confusing.
First of all, I don't think it's necessary to run:
$ python extension.py
The error message you get probably comes from the fact that Python cannot find the viacsv.py file in sys.path (the places where it looks for modules, etc.). In the tutorial you mentioned, it's not really clear what to do with this file. As far as I am concerned, I just saved the viacsv.py file in my local site-packages directory. As I am on Linux I put it there ~/.local/lib/python2.7/site-packages but it might different for you. You can run the following python script to find out:
import sys
for dr in sys.path:
print dr
Then I just substituted from zipline.data.bundles.viacsv import viacsv with from viacsv import viacsv in extension.py.
I suspect you might be looking for the wrong place for the extension.py file.
For windows machine, the file is under "~\.zipline\extension.py". In my case, it's under "C:\Users\XXXX\.zipline\extension.py".
I had been looking at zipline folder under conda's site-packages folder, and couldn't find it. Then created an extension.py myself wondering why it's not called.
Check a related post here https://www.quantopian.com/posts/zipline-issue-while-creating-custom-bundle-to-bring-yahoo-data.
Same issue here, #Gillu13 pointed me to this solution.
I installed zipline through conda. So zipline is installed in
home/me/anaconda3/envs/krakenex/lib/python3.6/site-packages
in there you will find zipline/data/bundles and you can put viacsv.py in there...
then
from zipline.data.bundles.viacsv import viacsv
works
I'm working on an XBMC plugin which requires a few Python modules not available via a requires tag in addon.xml (they are not in the XBMC repository as far as I'm aware). The documentation for plugin development seems to indicate that you can do this by adding the module(s) to the resources/lib/ subdirectory of your plugin directory.
I've done this and when testing it in XBMC I get an import error trying to import this module because it cannot be found.
I've read the other question I found on SO regarding this topic entitled Importing a python module into XBMC for use within an addon but the proposed solution there, of adding the module directory to the path before importing, doesn't work for me either. I get the same import error.
In fact I don't think that answer is correct because os.getcwd() in XBMC does not return your plugin directory path when called from within your plugin; so concatenating the path it gives with /resources/lib as the answer suggests won't yield a valid path. I modified the example to use getAddonInfo to find the plugin path from an Addon object via the xbmcaddon module and added that to the path concatenated with /resources/lib but it still did not work.
Putting the modules into the root of the plugin directory also doesn't work. I haven't been able to find specific documentation about how to do this correctly beyond the initial tutorial saying that I should add it to the resources/lib subdirectory.
So does anyone know how to do this or have an example of this being done successfully in another XBMC plugin?
Figured out my mistake. I wasn't paying attention to the path I was adding. I was adding the addon profile directory to sys.path using getAddonInfo('profile') when I should have been using getAddonInfo('path')
For future reference, if you want to add a subdirectory of your addon to the path, this is what I did:
import xbmcaddon
import os
...
my_addon = xbmcaddon.Addon('plugin.video.my_plugin')
addon_dir = xbmc.translatePath( my_addon.getAddonInfo('path') )
sys.path.append(os.path.join( addon_dir, 'resources', 'lib' ) )
import <whatever modules you want>
I guess that's yet another lesson in paying close attention to the content of an error message.
I'm trying to deploy a Flask app on GAE using Windows. It runs fine locally but runs into problems when I try to run it on GAE.
First I get this error in flask\json.py:
from itsdangerous import json as _json
ImportError: No module named itsdangerous
Downloading and unpacking https://pypi.python.org/pypi/itsdangerous in the same directory doesn't fix the issue. If I just grab itsdangerous.py and put it in the flask directory, I get:
_slash_escape = '\\/' not in _json.dumps('/')
AttributeError: 'module' object has no attribute 'dumps'
I've read that it may be due to conflicting json.py files but I've also tried using absolute paths for the import json and it doesn't seem to make a difference.
You put the itsdangerous.py in the wrong directory. Because json.py and itsdangerous.py both exist in the /flask directory, itsdangerous.py will import /flask/json.py intead of the right one.
The GAE official doc mentioned a way to include 3rd-party libraries:
You can include other pure Python libraries with your application by
putting the code in your application directory. If you make a symbolic
link to a module's directory in your application directory, appcfg.py
will follow the link and include the module in your app.
Obviously, it's a poor solution because we don't want to mix the libraries we used with the code we write. The community have found better ways.
I suggest you to use a gae-flask project template(e.g. flask-appengine-template) or at least follow some of its project structure. You can put all these 3rd-party libraries under a directory like /lib and add '/lib' to sys.path. Actually flask-appengine-template include common flask modules like itsdangerous for you by default.
sample code:
import os
import sys
sys.path.insert(1, os.path.join(os.path.abspath('.'), 'lib'))
import application
Google now makes it super-easy:
https://console.developers.google.com/start/appengine
I have a Flask project which I've put the flask module (version 0.9) directly beside my app.py file. I've done this so that I can bundle everything into a version control repository that won't require anyone else using it to install additional Python modules.
I want to use flask-login so I've tried to manually install it by downloading the latest version and putting the flask_login.py file in my "local" flask/ext/ directory. However, while I can import flask and import flask.ext, I am unable to import flask.ext.login with Python throwing ImportError: No module named flask.ext.login. import flask.ext.flask_login throws an import error as well.
Is there something I have to do differently if Flask and it's extensions are local to the app.py?
The solution is to put the flask_login.py file in the same directory as my app.py file. No modification of the flask/ext/__init__.py file is necessary.
The flask.ext module is intended only as a an extension importer, not a repository for installed extensions. Based on the import path, I thought the flask/ext folder was where extensions were to be copied. This was incorrect. Extensions simply need to be somewhere on the python path.
Python doesn't throw the exception in your case, this Flask module is responsible for modifying the standard import hook in the ext folder:
https://github.com/mitsuhiko/flask/blob/master/flask/exthook.py
So, I don't think putting flask_login.py into flask/ext is the right way to use extensions in Flask. The documentation recommends to use pip install flask-login or python setup.py install. After that you can do:
import flask_login
If you still want to do it manually, remove the setup() call from ext/__ init__.py.
It probably has a good reason why the Flask guys did it this way though :-)
Right, but the point was to manually install everything so it could be self-contained in a version control repository that won't require additional installs. I've looked at the setup.py file but can't determine how it achieves the installation. – Soviut Oct 25 at 10:06
Why not use virtualenv to achieve this? You can push your env to source control as well, although that's not normal usage.
See: http://www.virtualenv.org/en/latest/