I have a Flask project which I've put the flask module (version 0.9) directly beside my app.py file. I've done this so that I can bundle everything into a version control repository that won't require anyone else using it to install additional Python modules.
I want to use flask-login so I've tried to manually install it by downloading the latest version and putting the flask_login.py file in my "local" flask/ext/ directory. However, while I can import flask and import flask.ext, I am unable to import flask.ext.login with Python throwing ImportError: No module named flask.ext.login. import flask.ext.flask_login throws an import error as well.
Is there something I have to do differently if Flask and it's extensions are local to the app.py?
The solution is to put the flask_login.py file in the same directory as my app.py file. No modification of the flask/ext/__init__.py file is necessary.
The flask.ext module is intended only as a an extension importer, not a repository for installed extensions. Based on the import path, I thought the flask/ext folder was where extensions were to be copied. This was incorrect. Extensions simply need to be somewhere on the python path.
Python doesn't throw the exception in your case, this Flask module is responsible for modifying the standard import hook in the ext folder:
https://github.com/mitsuhiko/flask/blob/master/flask/exthook.py
So, I don't think putting flask_login.py into flask/ext is the right way to use extensions in Flask. The documentation recommends to use pip install flask-login or python setup.py install. After that you can do:
import flask_login
If you still want to do it manually, remove the setup() call from ext/__ init__.py.
It probably has a good reason why the Flask guys did it this way though :-)
Right, but the point was to manually install everything so it could be self-contained in a version control repository that won't require additional installs. I've looked at the setup.py file but can't determine how it achieves the installation. – Soviut Oct 25 at 10:06
Why not use virtualenv to achieve this? You can push your env to source control as well, although that's not normal usage.
See: http://www.virtualenv.org/en/latest/
Related
I am following the tutorial here:
http://www.prokopyshen.com/create-custom-zipline-data-bundle
and trying to set up a custom bundle to get price from custom, non US financial assets. I am stuck on the line that says:
Advise zipline of our bundle by registering it via .zipline/extension.py
My extension.py file is located in the .zipline/ directiory and has the following code:
from zipline.data.bundles import register
from zipline.data.bundles.viacsv import viacsv
eqSym = {
"CBA"
}
register(
'CBA.csv', # name this whatever you like
viacsv(eqSym),
)
I don't get what it means to register the bundle via .zipline/extension.py though? I thought it might mean to just run the extension.py file from my terminal via a:
python extenion.py
but that fails and says:
ImportError: No module named viacsv
How do i register this bundle?
I also followed this tutorial and I must confess this part is a little confusing.
First of all, I don't think it's necessary to run:
$ python extension.py
The error message you get probably comes from the fact that Python cannot find the viacsv.py file in sys.path (the places where it looks for modules, etc.). In the tutorial you mentioned, it's not really clear what to do with this file. As far as I am concerned, I just saved the viacsv.py file in my local site-packages directory. As I am on Linux I put it there ~/.local/lib/python2.7/site-packages but it might different for you. You can run the following python script to find out:
import sys
for dr in sys.path:
print dr
Then I just substituted from zipline.data.bundles.viacsv import viacsv with from viacsv import viacsv in extension.py.
I suspect you might be looking for the wrong place for the extension.py file.
For windows machine, the file is under "~\.zipline\extension.py". In my case, it's under "C:\Users\XXXX\.zipline\extension.py".
I had been looking at zipline folder under conda's site-packages folder, and couldn't find it. Then created an extension.py myself wondering why it's not called.
Check a related post here https://www.quantopian.com/posts/zipline-issue-while-creating-custom-bundle-to-bring-yahoo-data.
Same issue here, #Gillu13 pointed me to this solution.
I installed zipline through conda. So zipline is installed in
home/me/anaconda3/envs/krakenex/lib/python3.6/site-packages
in there you will find zipline/data/bundles and you can put viacsv.py in there...
then
from zipline.data.bundles.viacsv import viacsv
works
Well I'm trying to use Datastore in a personal project using the Google App Engine. Though, I can't import the datastore module, no matter how hard I try.
I've been using the online console during the whole time (in order to avoid to have to solve problems first on my PC and then on GCloud...)
So, I'm using
from google.cloud import datastore
Unfortunately, that's not working at all. The last error I have is
ImportError: No module named google.protobuf
But before I had things like Can't import Datastore.
What I did was removing the integrality of /lib, and reinstalling every dependancy with pip. Here is my requirements.txt:
# This requirements file lists all third-party dependencies for this project.
#
# Run 'pip install -r requirements.txt -t lib/' to install these dependencies
# in `lib/` subdirectory.
#
# Note: The `lib` directory is added to `sys.path` by `appengine_config.py`.
Flask==0.10
google.cloud==0.25.0
protobuf==3.3.0
(The last line was added to try to resolve the last error I got). Before having this error, I got
Also, a little clarification question: I've seen (while looking for answers) people using gcloud and some using google.cloud. What's the difference? What should I use?
Also, pip show google.cloud shows nothing.
What am I missing?
Thank you
Well, if anyone is wondering, here's how I solved the problem. What fixed it was changing the Flask version to 0.12 (don't know why, but that's what happened).
I deleted lib to be sure I was starting from scratch. Then, I used this requirements.txt file:
Flask==0.12
google-cloud==0.25.0
click==5.1
(click is needed by Flask 0.12).
I keep getting ImportError: No module named twitter when it's in my lib module
Details:
the twitter module is located in the lib directory.
The lib directory has an empty init.py
I'm getting the error on a page just outside the directory
When I take the twitter module out of the lib directory and just say import twitter, it works
When I put the twitter module back inside the lib and say from lib import twitter, or when I say import twitter, or saying import lib.twitter it says, go to **** and like it
I tried doing this: https://cloud.google.com/appengine/docs/python/tools/libraries27?hl=en , which suggests to:
To tell your app how to find libraries in this directory, create (or modify) a file named appengine_config.py in the root of your project, then add these lines:
from google.appengine.ext import vendor
# Add any libraries installed in the "lib" folder.
vendor.add('lib')
and then tried import twitter, and tried from lib import twitter, and it says, "haha, nice try."
Now what?
tl;dr - the comments are right. 'lib' is not a module, it's a folder containing modules. It needs to be on your Python import path before your import twitter, using the vendor.add('lib'), which should set up your import path correctly when you deploy or use dev_appserver.py. If you place the vendor.add(lib) snippet in appengine_config.py file and run dev_appserver.py or deploy, and the twitter module is inside lib, you should be able to import twitter. If you're running outside of App Engine/devappserver, you'll need to make sure lib is on the PYTHONPATH.
Here is the longer explanation of what the lib folder is; what vendoring is; and how they fit into the bigger picture of managing dependencies on App Engine.
With App Engine, you can specify some dependencies provided by the sandbox in app.yaml. You can se the list of available libraries here. For any other dependencies (which can only be pure Python libraries), the libraries need to be directly included inside your project. Directly including the code for these dependencies as part of your project (rather than just listing them in requirements.txt and expecting them to get installed at deploy time) is commonly called 'vendoring'.
Now, even though we are including dependencies directly, we'd rather just use pip and a requirements.txt, for a few reasons I will mention below. But pip is usually used to install into libraries into the system libraries or a virtualenv, not into the project itself. So a feature was added to pip and a feature was added to App Engine to fix this.
The pip feature install dependencies into a folder rather than your system libraries or a virtualenv. You use the -t flag like this:
pip install -r requirements.txt -t lib
Where 'lib' is the folder to install into. This way, you still specify your dependencies in requirements.txt, and use pip to install them, they just get installed right into the directory specified. As the comments noted, the lib folder is itself not a module - it just contains them. So your lib folder should not have an __init__.py, it should just contain a folder like 'twitter' that does have an __init__.py. Since lib is not a module, from lib import twitter doesn't actually make sense. Also note that lib is an arbitrary name, but the one we usually pick by convention.
There are a few big advantages to vendoring using pip rather than just manually downloading dependencies and adding them to the project. One advantage is you don't need to check the dependencies into source control - just add the requirements.txt, and tell other users to use the pip -t command to vendor the dependencies too. Another advantage is it's just more organized to cleanly separate your code from third-party modules. The point of the vendoring features is to keep these advantages while still following App Engine requirements to include the dependencies in the directory at deploy time.
The App Engine vendor extension which you are using was made to recognize that a folder contains modules that have been 'vendored' and add it to the path. That is the vendor extension you are using in your snippet. It makes sure the lib folder is on your import path so you can import the modules you installed into it.
Once you run the vendor.add command, you should be able to import your modules in the lib folder. As the comment notes, you need to make sure it runs before import twitter. A good practice is to make sure those vendor commands run before anything else. You can accomplish this by putting that code inside a file named appengine_config.py in your directory. That's a special file that runs before anything else. Take a look at an example of doing that here.
A few last notes that might be helpful:
appengine_config.py will run in the GAE environment or when you run dev_appserver, which emulates the GAE environment. However if you're ever running outside of a GAE environment, make sure your PYTHONPATH includes the lib folder you want to import from.
Since you can vendor libraries using pip, you might ask why would you ever include a dependency using app.yaml? The answer is that since only pure Python libraries can be vendored, you still should use app.yaml dependencies for any libraries which need C libraries, such as MySQL.
If you have vendored libraries and app.yaml libaries, you don't want to vendor libraries that will be in the GAE sandbox since the versions might conflict. In that case, it's a good idea to have a separate requirements.txt for dependencies you want to vendor, and dependencies you want included only when running locally but that GAE will provide in the sandbox. Here is an example of that for MySQL.
Create a folder called "libs" (in the home directory of your webapp)
If one library, then just do: pip install -t libs/ your_library_name
if multiple libraries, do:
pip freeze > requirements.txt,
pip install -r requirements.txt -t libs
Now, create a file called appengine_config.py (in the home directory, again)
In the above file, add:
from google.appengine.ext import vendor
vendor.add('libs')
These get uploaded onto your app engine environment and you'll be able to run it!
I'm trying to deploy a Flask app on GAE using Windows. It runs fine locally but runs into problems when I try to run it on GAE.
First I get this error in flask\json.py:
from itsdangerous import json as _json
ImportError: No module named itsdangerous
Downloading and unpacking https://pypi.python.org/pypi/itsdangerous in the same directory doesn't fix the issue. If I just grab itsdangerous.py and put it in the flask directory, I get:
_slash_escape = '\\/' not in _json.dumps('/')
AttributeError: 'module' object has no attribute 'dumps'
I've read that it may be due to conflicting json.py files but I've also tried using absolute paths for the import json and it doesn't seem to make a difference.
You put the itsdangerous.py in the wrong directory. Because json.py and itsdangerous.py both exist in the /flask directory, itsdangerous.py will import /flask/json.py intead of the right one.
The GAE official doc mentioned a way to include 3rd-party libraries:
You can include other pure Python libraries with your application by
putting the code in your application directory. If you make a symbolic
link to a module's directory in your application directory, appcfg.py
will follow the link and include the module in your app.
Obviously, it's a poor solution because we don't want to mix the libraries we used with the code we write. The community have found better ways.
I suggest you to use a gae-flask project template(e.g. flask-appengine-template) or at least follow some of its project structure. You can put all these 3rd-party libraries under a directory like /lib and add '/lib' to sys.path. Actually flask-appengine-template include common flask modules like itsdangerous for you by default.
sample code:
import os
import sys
sys.path.insert(1, os.path.join(os.path.abspath('.'), 'lib'))
import application
Google now makes it super-easy:
https://console.developers.google.com/start/appengine
Im trying to import a company module into my software and I get the error:
ImportError: No module named config
from:
from pylons.config import config
So obviously, the module that im importing requires pylons.config but cant find it in my virtual environment.
If I go to the terminal and try some Python scripts I can seem to find the config file if I try:
from pylons import config
but will error if I try:
import pylons.config
Why is this?
And does anybody how or where I can get:
from pylons.config import config
to work. Bearing in mind that I cannot change the code for this module, only mine which is importing it or my own system files.
UPDATE
If anyone finding this page has a similar problem you may find that you are trying to run two modules with different versions of Pylons.
For example, you are creating a login application called myApp. You have some Python modules which help with login handling called pyLogin.
First you install pyLogin with python setup.py install. This adds the libraries to your site packages and updates any libraries it depends on, such as SqlAlchemy.
Next you install myApp in the same way which again updates libraries and dependencies.
This problem will occur if pyLogin and myApp are using different versions of Pylons. If pyLogin is using Pylons 0.9.6 and myApp is using Pylons 1.0 for example, then the pyLogin code will be called from myApp but it will be running in the wrong Pylons framework and hence will require EITHER from pylons import config or from pylons.config import config, but will only work with one. If it is using the wrong call for Pylons then you will find yourself with this error message.
So the only solution to this error is to either find earlier or later libraries which use the same Pylons version as your application or to convert your application to the same Pylons version as the libraries you are using.
There is a diffrence between two usages...
import loads a Python module into its own namespace, while from loads a Python module into the current namespace.
So, using from pylons import config imports config to to your current namespace. But trying to import a class or function using import is not possible since there is no namespace to keep them... You can only import modules, and use functions or classes via calling them with their own namespace like
import pylons
....
pylons.config #to retreive config
More about import in Python