Develop Reference

Is is possible to run a pyramid application without installing it as an egg?

I am building an application using the Pyramid web framework and in all of the documentation it assumes that you will be installing your application as an egg using setup.py. While this makes sense for a distributable package, it adds a lot of overhead and unnecessary packaging code for an application that isn't meant to be shared as a library or extension.
Pyramid uses PasteDeploy to read application configuration files (.ini) and requires a section to define which application is to be run:
[app:blog]
use = egg:MyBlog#main
database = mysql://localhost/blogdb
blogname = This Is My Blog!
This tells the app loader to import a library called MyBlog found on the PYTHONPATH and use an entry point named main that has been defined in a setup.py configuration.
Is there any way to directly refer to the application by path and specify the WSGI entry point (similar to how you would run an application directly via:
cd /path/to/MyBlog
gunicorn --bind etc... app:main
There is an additional syntax starting with call: that allows you specify an entry point that hasn't been registered with setuptools:
[app:mythirdapp]
use = call:my.project:myapplication
However, it appears that the loader still expects a package installed on the python path rather than a bare python package/executable.

Is it possible to run a pyramid application without installing it as an egg?
Yes, it is, you can find a lot of examples on https://trypyramid.com/ site. You are not obliged to use ini-style declaration for runnig your WSGI application. For example you can create myapp_wsgi.py file:
from pyramid.paster import get_app, setup_logging
ini_path = 'production.ini'
setup_logging(ini_path)
application = get_app(ini_path, 'main')
And pass it to gunicorn:
gunicorn myapp_wsgi:application
You can find more information on modwsgi and gunicorn. pages.

I actually think the call: method is able to find a package that hasn't been installed using setuptools. This is due to the fact that the python path starts in the current directory, so if you launch your app from the directory containing it, then everything should work as expected.
Problems arise when you want to compose apps using the [composite:] directive. It would be nice if [app:] sections could include a directory = ... parameter that would add that directory to the python path, but I guess this is more of a feature request to raise with the PasteDeploy developers...

Serving static files with App Engine Python 37 without any Python script

With App Engine Python27(standard env), it was possible to create a webapp that only serves static resources from the app.yaml. Without writing any Python script.
With Python37 (standard env), I see that it's still possible to have the static_dir in the handlers section of app.yaml but I'm wondering if this will work the same way. Cause I don't define any entrypoint, I don't set a webserver. I have only the app.yaml at the moment.
And the first that blocks me at the moment: if doing so, how do I locally run this app ? (no Python script so no python main.py and dev_appserver.py does not work for Python37).
Thanks for your answers

Yes, the static_dir and static_file configurations work the same way, their sections in the app.yaml reference docs for 2.7 and 3.7 are identical except for the references to the 2.7-specific application_readable option.
Since serving the static content on GAE is identical (the static content is uploaded and served separately from the application code - equivalent, if you want, to serving from a CDN) it doesn't really matter if you're using the 2.7 runtime or the 3.7 one - you're not actually executing any python code, right?
So one option would be to just use 2.7 instead (adding a minimal app skeleton to keep the runtime happy, say by just returning 404 or redirecting to one of the static pages - you can get it from the Quickstart). You can then continue to use the 2.7 development server for local execution.
Another option would be to use 2.7 (as in option 1) just for local development, but switch back to 3.7 for deployment (i.e. update the app.yaml file, drop the skeleton app code or update it for 3.7). A bit more tedious and brittle, but it can be done.
A 3rd option would be to try the updated development server which has some limited support for 3.7, see Python 3.7 Local Development Server Options for new app engine apps. Serving static content might be included in that limited support. You'd need to meet its specific requirements. You'd also need a 3.7-compatible skeleton app, you can get one from the 3.7 Quickstart.
Finally, you could also use some other tool locally during development, if you have one (same advice as for running an actual app locally). It could simply be your browser for static-only content :) Again, the goal is to just develop your static content, GAE will serve it the same way. You won't need any skeleton app in this case.
Side note: I saw this in the Node.js standard env app.yaml reference, not sure if it's applicable to python 3.7 as well, just be aware of it:
In order to use static handlers, at least one of your handlers must
contain the line script: auto to deploy successfully.

Compiling Coffeescript with Flask in Production

I have a Flask app that I'm running in production. Right now it has a big ugly js file that I'd like to break out and rewrite in something like Coffeescript. I was considering something like Flask-Cake to simplify the CoffeeScript compilation. However, I don't know how something like that would work for production. I should probably have a script that compiles the coffeescript files before deploying, right? I've never worked on a system with this particular layout -- uncompiled server-side but compiled client-side. What's the standard procedure here?

You are probably looking for Flask-Assets.
Example from the website:
from flask import Flask
from flask.ext.assets import Environment, Bundle
app = Flask(__name__)
assets = Environment(app)
js = Bundle('jquery.js', 'base.js', 'widgets.js',
filters='jsmin', output='gen/packed.js')
assets.register('js_all', js)
This would automatically concatenate jquery.js, base.js and widgets.js in your static folder, pipe them through jsmin and save the result in static/gen/packed.js.
This compilation is by default always happening when one of the source files changes. Watching the files in production is kinda expensive in production (and would require a coffeescript compiler to be installed on the server!), so there is a configuration value to disable the monitoring.
Another plugin that is more lightweight, but in my experience also less powerful is Flask-Makestatic.

Take a look at DukPy! It's a simple javascript interpreter for Python and can compile CoffeeScript, TypeScript, BabelJS and JSX. Usage is very simple:
import dukpy
dukpy.coffee_compile("CoffeeScript goes here!")
DukPy is the successor to the Python-CoffeeScript package witch is no longer maintained.

Web2Py Working Directory

Well I want to use WEb2Py because it's pretty nice..
I just need to change the working directory to the directory where all my modules/libraries/apps are so i can use them. I want to be able to import my real program when I use the web2py interface/applications. I need to do this instead of putting all my apps and stuff inside the Web2Py folder... I'm trying to give my program a web frontend without putting the program in the Web2Py folder.. Sorry if this is hard to understand .

In any multi-threaded Python program (and not only Python) you should not use os.chdir and you should not change sys.path when you have more than one thread running. It is not safe because it affects other threads. Moreover you should not sys.path.append() in a loop because it may explode.
All web frameworks are multi-threaded and requests are executed in a loop. Some web frameworks do not allow you to install/un-install applications without restarting the web server and therefore IF os.chdir/sys.path.append are only executed at startup then there is no problem.
In web2py we want to be able to install/uninstall applications without restarting the web server. We want apps to be very dynamical (for example define models based on information provided with the http request). We want each app to have its own models folder and we want complete separation between apps so that if two apps need to different versions of the same module, they do not conflict with each other, so we provide APIs to do so (request.folder, local_import).
You can still use the normal os.chdir and sys.path.append but you should do it outside threads (and this is not a web2py specific issue). You can use import anywhere you like as you would in any other Python program.
I strongly suggest moving this discussion to the web2py mailing list.

os.chdir lets you change the OS's working directory, but for your purposes (enabling imports of a bunch of modules &c which are constrained to live in some strange place) it seems better to add the needed directories to sys.path instead.

I had to do this very thing. I have few modules that I wanted to use from my controllers. If you want to be able to use the code that resides in the modules directory in the controller, you can use:
# use this in your controller code
impname = local_import('module_in_modules', reload=True)
# reload true will ensure that it will re load whenever
# there are changes to the module
Jay

How to re-use a reusable app in Django

I am trying to create my first site in Django and as I'm looking for example apps out there to draw inspiration from, I constantly stumble upon a term called "reusable apps".
I understand the concept of an app that is reusable easy enough, but the means of reusing an app in Django are quite lost for me. Few questions that are bugging me in the whole business are:
What is the preferred way to re-use an existing Django app? Where do I put it and how do I reference it?
From what I understand, the recommendation is to put it on your "PYTHONPATH", but that breaks as soon as I need to deploy my app to a remote location that I have limited access to (e.g. on a hosting service).
So, if I develop my site on my local computer and intend to deploy it on an ISP where I only have ftp access, how do I re-use 3rd party Django apps so that if I deploy my site, the site keeps working (e.g. the only thing I can count on is that the service provider has Python 2.5 and Django 1.x installed)?
How do I organize my Django project so that I could easily deploy it along with all of the reusable apps I want to use?

In general, the only thing required to use a reusable app is to make sure it's on sys.path, so that you can import it from Python code. In most cases (if the author follows best practice), the reusable app tarball or bundle will contain a top-level directory with docs, a README, a setup.py, and then a subdirectory containing the actual app (see django-voting for an example; the app itself is in the "voting" subdirectory). This subdirectory is what needs to be placed in your Python path. Possible methods for doing that include:
running pip install appname, if the app has been uploaded to PyPI (these days most are)
installing the app with setup.py install (this has the same result as pip install appname, but requires that you first download and unpack the code yourself; pip will do that for you)
manually symlinking the code directory to your Python site-packages directory
using software like virtualenv to create a "virtual Python environment" that has its own site-packages directory, and then running setup.py install or pip install appname with that virtualenv active, or placing or symlinking the app in the virtualenv's site-packages (highly recommended over all the "global installation" options, if you value your future sanity)
placing the application in some directory where you intend to place various apps, and then adding that directory to the PYTHONPATH environment variable
You'll know you've got it in the right place if you can fire up a Python interpreter and "import voting" (for example) without getting an ImportError.
On a server where you have FTP access only, your only option is really the last one, and they have to set it up for you. If they claim to support Django they must provide some place where you can upload packages and they will be available for importing in Python. Without knowing details of your webhost, it's impossible to say how they structure that for you.

An old question, but here's what I do:
If you're using a version control system (VCS), I suggest putting all of the reusable apps and libraries (including django) that your software needs in the VCS. If you don't want to put them directly under your project root, you can modify settings.py to add their location to sys.path.
After that deployment is as simple as cloning or checking out the VCS repository to wherever you want to use it.
This has two added benefits:
Version mismatches; your software always uses the version that you tested it with, and not the version that was available at the time of deployment.
If multiple people work on the project, nobody else has to deal with installing the dependencies.
When it's time to update a component's version, update it in your VCS and then propagate the update to your deployments via it.