(UPDATE: I've made a better question with a better answer here. I was going to delete this question, but some of the answers might prove useful to future searchers.)
My question is just about identical to this, but that answer is ugly (requires a dir structure including sharedtemplates/templates/templates/), incomplete as posted (user "answered" his own question), and assumes some knowledge I don't have.
I'm working on my first python-backed web application. The javascript component is well under development using a static HTML page. Now I want a server-side python component to handle AJAX calls and render an HTML template using jinja2.
I've had success with python before, creating GUI apps using tkinter/requests. Love the language, but the python environment (environments?) is confusing. I'm not working in a virtualenv.
According to jinja2 docs, HTML templates have to be in something called a package. Then you create an Environment with a PackageLoader that knows the name of the package and the template dir:
from jinja2 import Environment, PackageLoader
env = Environment(loader=PackageLoader('yourapplication', 'templates'))
So, here's my index.py (it's just a stub and doesn't even try to render anything, but you can at least tell if it crashes).
#!/usr/bin/python
from jinja2 import Environment, PackageLoader # no prob, jinja2 correctly installed using pip
env = Environment(loader=PackageLoader('mypkg', 'template')) # causes server error
# if it doesn't crash, just put up a basic html page for now
print ("Content-type: text/html\r\n\r\n")
print("<html><head><title>hello</title></head><body>hello wuld</body></html>")
Here's the directory structure:
index.py
mypkg/
mypkg/template/index.html
mypkg/__init__.py # empty
Relevant line from error log:
ImportError: No module named mypkg
Maybe I need to structure this differently, and I'm pretty sure I'll need to create and invoke a setup.py to install the module. That's part of what the other answer left out: what's in setup.py and how does it work in this case? I've looked at dozens of resources on setup.py and none of them seems to pertain to the question of installing HTML templates.
Thanks in advance for your help!
UPDATE: fragilewindows pointed to a resource that tells about "developer mode", which is probably part of the answer. The difficulty here is, I'm looking to package this template for local deployment, not for distribution. But 99% of the online documentation is about packaging projects for PyPi. I don't need to package a "project", just a dinky HTML template. Indeed, the only reason I need to package the template is because that's the default way for jinja2 to access templates (and I do want to go native in python).
I just need to convince the environment that "mypkg" is installed, and that "template" is a directory within the install. You can see that my efforts so far are naive; I expect the right answer will be correspondingly lightweight.
I've discovered a WORKAROUND, here. In this usage, the template is not part of a module or package; it is loaded directly from the file system. File system:
./index.py
./template.html
index.py:
#!/usr/bin/python
import jinja2
templateLoader = jinja2.FileSystemLoader( searchpath="." )
templateEnv = jinja2.Environment( loader=templateLoader )
TEMPLATE_FILE = "template.html"
template = templateEnv.get_template( TEMPLATE_FILE )
outputText = template.render( ) # this is where to put args to the template renderer
print ("Content-type: text/html\r\n\r\n")
print(outputText)
I don't know the process involved with packaging but I figure since Jinja2 is written in Python, the process would be the same as packaging any other application in Python.
Here are a few links that may be useful to you:
The Hitchhiker's Guide to Python (great resource): Packaging Explained
Alternatives to Packaging: freeze your application
Python Packaging User Guide (probably the most useful to you)
I hope this helps.
Maybe an issue is in PYTHONPATH . According to Python documantation https://docs.python.org/2.7/tutorial/modules.html#the-module-search-path
it searches for modules in :
the directory containing the input script (or the current directory).
PYTHONPATH (a list of directory names, with the same syntax as the shell variable PATH).
the installation-dependent default.
You can add some code to your application - printing sys.path variable to make sure your applcation's path is also there.
import sys
from jinja2 import Environment, PackageLoader # no prob, jinja2 correctly installed using pip
print sys.path # printing your sys.path
env = Environment(loader=PackageLoader('mypkg', 'template')) # causes server error
If it's not You can add by
sys.path.append(path_to_your_application)
Related
I'm creating a Gtk4 application with python bindings on Linux. I'm using flatpak as well.
So I'm creating like an extensions/plugins systems where user can define his main module to call in a specific file and later on ill load it. Every thing works but when the user uses imports for external libraries like NumPy or pandas it will start looking inside the flatpak sandbox and this is normal. I'm wondering how can I tell the python interpreter to use the modules of the system for the imported plugins instead of looking in the modules of the app.
User's code and requirements should be independent of the app's requirements.
This is how I'm loading the modules:
extension_module = SourceFileLoader(file_name, module_path).load_module()
extension_class = getattr(dataset_module, class_name)
obj = extension_class()
This is an example of the loaded class
the absolute path of this module is /home/user/.extensions/ext1/module.py
import numpy as np
class Module1:
def __init__(self):
self.data = np.array([1, 2, 3, 4, 5, 6])
def get_data(self):
return self.data
I tried using
os.path.append('/usr/lib64/python-10.8/site-packages')
It's added, but in the sandbox environment.
I thought about looking for user imports manually like when a user import pandas ill try to look for installed python packages in the system and use the importlib or the SourceFileLoader to load it, but I don't think it's a good way to do it.
So, finally, after a day of reading the Flatpak docs, I found a way to do it.
I had to add the argument --filesystem=home. This argument will give you access to the user's directory. When you use pip to install packages, the packages will be installed in the following directory ~/.local/lib/python3.10/site-packages/. To let python interpreter search for packages in that folder, you can add it to the path like this.
import sys
sys.path.append('~/.local/lib/python3.10/site-packages/')
Note1: In my case, this is enough because the app is for learning and not serious, so it's 0 security concerns.
I am using openSUSE Tumbleweed, so I have python 3.10.11 and on the Flatpak runtime there is python 3.10.6. So some users who have an old distro or a distro like Ubuntu or Debian maybe you don't have the latest python version, and you may have compatibility issues.
A better solution is to create a simple folder in the user local directory f.g ~/.cache/myapp/packages and then you can add it to the flatpak manifest file --filesystem=~/.cache/myapp:create this way you're mapped your folder to be accessed from the sandbox and the option :create will create the folder if it doesn't exist. Then i your python script to install required packages based on the imports used in external scripts. In your python, add the folder path to the path sys.path.append('~/.cache/app/packages/').
Note2: It's not safe to import directly scripts to your code when you want to create a plugin system it's better to create a sub process that executes these scripts, this way you isolate your code from external code. You can use IPC protocol or other techniques to change data between the main process and the subprocess.
I am following the tutorial here:
http://www.prokopyshen.com/create-custom-zipline-data-bundle
and trying to set up a custom bundle to get price from custom, non US financial assets. I am stuck on the line that says:
Advise zipline of our bundle by registering it via .zipline/extension.py
My extension.py file is located in the .zipline/ directiory and has the following code:
from zipline.data.bundles import register
from zipline.data.bundles.viacsv import viacsv
eqSym = {
"CBA"
}
register(
'CBA.csv', # name this whatever you like
viacsv(eqSym),
)
I don't get what it means to register the bundle via .zipline/extension.py though? I thought it might mean to just run the extension.py file from my terminal via a:
python extenion.py
but that fails and says:
ImportError: No module named viacsv
How do i register this bundle?
I also followed this tutorial and I must confess this part is a little confusing.
First of all, I don't think it's necessary to run:
$ python extension.py
The error message you get probably comes from the fact that Python cannot find the viacsv.py file in sys.path (the places where it looks for modules, etc.). In the tutorial you mentioned, it's not really clear what to do with this file. As far as I am concerned, I just saved the viacsv.py file in my local site-packages directory. As I am on Linux I put it there ~/.local/lib/python2.7/site-packages but it might different for you. You can run the following python script to find out:
import sys
for dr in sys.path:
print dr
Then I just substituted from zipline.data.bundles.viacsv import viacsv with from viacsv import viacsv in extension.py.
I suspect you might be looking for the wrong place for the extension.py file.
For windows machine, the file is under "~\.zipline\extension.py". In my case, it's under "C:\Users\XXXX\.zipline\extension.py".
I had been looking at zipline folder under conda's site-packages folder, and couldn't find it. Then created an extension.py myself wondering why it's not called.
Check a related post here https://www.quantopian.com/posts/zipline-issue-while-creating-custom-bundle-to-bring-yahoo-data.
Same issue here, #Gillu13 pointed me to this solution.
I installed zipline through conda. So zipline is installed in
home/me/anaconda3/envs/krakenex/lib/python3.6/site-packages
in there you will find zipline/data/bundles and you can put viacsv.py in there...
then
from zipline.data.bundles.viacsv import viacsv
works
I'm working on an XBMC plugin which requires a few Python modules not available via a requires tag in addon.xml (they are not in the XBMC repository as far as I'm aware). The documentation for plugin development seems to indicate that you can do this by adding the module(s) to the resources/lib/ subdirectory of your plugin directory.
I've done this and when testing it in XBMC I get an import error trying to import this module because it cannot be found.
I've read the other question I found on SO regarding this topic entitled Importing a python module into XBMC for use within an addon but the proposed solution there, of adding the module directory to the path before importing, doesn't work for me either. I get the same import error.
In fact I don't think that answer is correct because os.getcwd() in XBMC does not return your plugin directory path when called from within your plugin; so concatenating the path it gives with /resources/lib as the answer suggests won't yield a valid path. I modified the example to use getAddonInfo to find the plugin path from an Addon object via the xbmcaddon module and added that to the path concatenated with /resources/lib but it still did not work.
Putting the modules into the root of the plugin directory also doesn't work. I haven't been able to find specific documentation about how to do this correctly beyond the initial tutorial saying that I should add it to the resources/lib subdirectory.
So does anyone know how to do this or have an example of this being done successfully in another XBMC plugin?
Figured out my mistake. I wasn't paying attention to the path I was adding. I was adding the addon profile directory to sys.path using getAddonInfo('profile') when I should have been using getAddonInfo('path')
For future reference, if you want to add a subdirectory of your addon to the path, this is what I did:
import xbmcaddon
import os
...
my_addon = xbmcaddon.Addon('plugin.video.my_plugin')
addon_dir = xbmc.translatePath( my_addon.getAddonInfo('path') )
sys.path.append(os.path.join( addon_dir, 'resources', 'lib' ) )
import <whatever modules you want>
I guess that's yet another lesson in paying close attention to the content of an error message.
I have a Flask project which I've put the flask module (version 0.9) directly beside my app.py file. I've done this so that I can bundle everything into a version control repository that won't require anyone else using it to install additional Python modules.
I want to use flask-login so I've tried to manually install it by downloading the latest version and putting the flask_login.py file in my "local" flask/ext/ directory. However, while I can import flask and import flask.ext, I am unable to import flask.ext.login with Python throwing ImportError: No module named flask.ext.login. import flask.ext.flask_login throws an import error as well.
Is there something I have to do differently if Flask and it's extensions are local to the app.py?
The solution is to put the flask_login.py file in the same directory as my app.py file. No modification of the flask/ext/__init__.py file is necessary.
The flask.ext module is intended only as a an extension importer, not a repository for installed extensions. Based on the import path, I thought the flask/ext folder was where extensions were to be copied. This was incorrect. Extensions simply need to be somewhere on the python path.
Python doesn't throw the exception in your case, this Flask module is responsible for modifying the standard import hook in the ext folder:
https://github.com/mitsuhiko/flask/blob/master/flask/exthook.py
So, I don't think putting flask_login.py into flask/ext is the right way to use extensions in Flask. The documentation recommends to use pip install flask-login or python setup.py install. After that you can do:
import flask_login
If you still want to do it manually, remove the setup() call from ext/__ init__.py.
It probably has a good reason why the Flask guys did it this way though :-)
Right, but the point was to manually install everything so it could be self-contained in a version control repository that won't require additional installs. I've looked at the setup.py file but can't determine how it achieves the installation. – Soviut Oct 25 at 10:06
Why not use virtualenv to achieve this? You can push your env to source control as well, although that's not normal usage.
See: http://www.virtualenv.org/en/latest/
I am trying to use the oauth2app library inside my Django app. I've tried installing the library several ways (easy_install, pip, pip via a requirements file, etc.), and every time it installs just fine. Also every time I can import the library from the Django shell (manage.py shell).
However, when I try to use a view from the library:
(r'^oauth2/token/?$', 'oauth2app.token.handler'),
I get a "No module named oauth2app" import error. I've tried comparing the Python path from the Django debug page to the one from "print sys.path" inside the shell, and the seem to be the same, so I can't for the life of me figure out why one works and the other doesn't.
Can anyone help explain what's going on? I thought the Django shell was an equivalent environment to the Django instance ...
Nevermind, it looks like this was just a case of bad documentation on the library's site; when I changed the urls.py line to:
from oauth2app import token
...
(r'^oauth2/token/?$', token.handler),
it worked, sigh.