Develop Reference

Why is django running from virtualenv unable to write to that path?

Our client has a web application running a Django instance from virtualenv on a Ubuntu server. We did a security audit for that service and found a path traversal vulnerability in a file upload form that could allow the attacker to write arbitrary files in the django user owned paths. Example:
A parameter "Import Name" is supplied with value
../some/path/to/create
Then the form file field is supplied with arbitary filename and the correct file contents
The application then does
try:
path = os.path.join(DEFAULT_UPLOAD_DIR, <Import Name>)
os.mkdir(path)
...
with open(os.path.join(path, <Filename From Form>)) as upload_file:
upload_file.write(<File Contents>)
...
The unsafe os.path.join allows the attacker to walk up in the directory tree and upload to other directories than the DEFAULT_UPLOAD_DIR. So basically if the attacker is able to find a path that doesn't yet exist on the server he's able to create that folder avoiding the failure of os.mkdir() in the try...except and the file is uploaded there.
Now this translates to a real exploit if the attacker is able to write to
../virtualenvs/<env name>/lib/python2.7/
Since e.g Django modules are loaded from the subdirectory site-packages within the virtualenv python directory and pythonpath tells us whatever is directly under lib/python2.7 gets loaded first, essentially the module loading order allows the attacker to 'overwrite' a module and ensure their code is run on import.
We did a proof-of-concept penetration test and wrote to
../virtualenvs/somepath/__init__.py
Which succeeded but for some reason we are unable to write to
../virtualenvs/<actual env name>/
Which is strange cause the permissions are exactly the same as with somepath and owner / group is in both cases the Django user. Enabling the virtualenv for the Django user and going to the python shell it allows me to do the write so it seems weird that it can't when called from the vulnerable form view.
The question is: Is there something special about the virtualenv path from which the Django instance is running that makes it unable to write to that path? Or am I missing something?

Django deployed app/project can't write to a file

I am working on a Django based application whose location on my disk is home/user/Documents/project/application. Now this application takes in some values from the user and writes them into a file located in a folder which is under the project directory i.e home/user/Documents/project/folder/file. While running the development server using the command python manage.py runserver everything worked fine, however after deployment the application/views.py which accesses the file via open('folder/path','w') is not able to access it anymore, because by default it looks in var/www folder when deployed via apache2 server using mod_wsgi.
Now, I am not putting the folder into /var/www because it is not a good practise to put any python code there as it might become readable clients which is a major security threat. Please let me know, how can I point the deployed application to read and write to correct file.

The real solution is to install your data files in /srv/data/myapp or some such so that you can give the webserver user correct permissions to only those directories. Whether you choose to put your code in /var/www or not, is a separate question, but I would suggest putting at least your wsgi file there (and, of course, specifying your <DocumentRoot..> correctly.

Dynamically adding new files to existing python process

We use Django for a project. Prior to 1.7+, we were able to dynamically pull in files to our python environment when setup.py was being executed. For example, we have this directory structure:
/foo/bar
/foo/bar/__init__.py
/foo/bar/my_functions.py
Our Django project doesn't know anything about those files to start off. When the web server starts and setup.py is read, we look at a configuration which tells us where to find files to add to our environment. Again, let's assume /foo/bar is in our configuration to be dynamically loaded. We would then use import_module() to import it so that anything under /foo/bar essentially becomes part of the project and can be used. It's basically a "plugins" feature.
After Django >=1.7, this causes huge problems, mainly:
django.core.exceptions.AppRegistryNotReady: The translation infrastructure
cannot be initialized before the apps registry is ready. Check that you
don't make non-lazy gettext calls at import time.
It also limits our ability to add new files dynamically as you always have to put them in place and restart your web server. You couldn't have an "Upload Plugin" page to add new files to the server, for example.
Is there a way to add files like this both during the web server startup as well as after the startup without restarting it?

Deploying Flask on Windows in production

I have found quite a few guides for running Flask on Linux/Unix with various technologies (nginx/apache/uWSGI/gunicorn/etc.) but all of them appear to work best on Linux, and only incidentally work on Windows, or not work at all on Windows. Are there any recommended ways to serve Flask apps in production in a Windows environment?

I have done this a few times. It can be done with only moderate hits to performance. You will want to leverage IIS and FastCGI.
Here is a link to a blog post detailing a method: https://medium.com/#bilalbayasut/deploying-python-web-app-flask-in-windows-server-iis-using-fastcgi-6c1873ae0ad8
Here is a link to a SO post detailing the same method: https://stackoverflow.com/a/22107980/8508792
and the contents, just in case the post gets taken down...
High Level Overview
HTTP -> IIS -> ISAPI -> FastCGI -> WSGI (Flask application)
Setup Steps
Step 1: Install Required Binaries
Install Python (2.7 or 3.x -- I used 3.3)
Install pip-Win (I used version 1.6)
Install pywin32 (I used version 218)
Install the IIS FastCGI extension with fcgisetup 1.5
Step 2: Install Optional Binary Packages
I installed pyodbc using the installer .exe from this site. Installing from source (e.g. pip, for installing into a virtual environment) requires a C/C++ compiler.
Step 3: Get a Copy of wfastcgi.py
Choose a version that will work for you, preferably one that supports Python 3.3 (I used David Ebbo's). You may want the "official" version from here.
Install the wfastcgi.py script into C:\Inetpub\wwwroot and make sure the account that will serve your application ("Network Service" by default) has read access to it.
Step 4: Install virtualenv Into the System site-packages
C:\Python33\Scripts\pip.exe install virtualenv
(if you are using Python 3.3 and installed everything in the default location)
Step 5: Install Your Flask Application
You may install the application just about anywhere on the system. You may want to install it under C:\Inetpub. For this tutorial, we'll call the root folder of your application install %APPROOT%. (Don't put quotation marks in the environment variable.)
Make sure that the account that will serve your application ("Network Service" by default) has read access to all of the script files. This command:
cacls "%APPROOT%" /S:"D:PAI(A;OICI;FA;;;BA)(A;OICIIO;FA;;;CO)(A;OICI;0x1200a9;;;NS)(A;OICI;FA;;;SY)"
will give your application directory the following permissions:
BUILTIN\Administrators: Full control of this folder, subfolders, and files
CREATOR OWNER: Full control for subfolders and files only
NT AUTHORITY\NETWORK SERVICE: Read permissions for this folder, subfolders, and files
NT AUTHORITY\SYSTEM: Full control of this folder, subfolders, and files
Add any local configuration necessary (my application uses a local.cnf file that is ignored by the version control system) -- e.g. database URLs.
Make sure your application contains a Web.config file in %APPROOT% -- see the section below for information on the file format.
Step 6: Create a virtualenv For Your Application
C:\Python33\Scripts\virtualenv.exe --system-site-packages "%APPROOT%\env"
(Pick a name other than env if your application already uses that directory.)
Step 7: Install The Packages Required By Your Application to the virtualenv
cd "%APPROOT%"
env\Scripts\activate
pip install -r Packages
(My project keeps the requirements spec in a file named Packages.)
Step 8: Create a Web Site Or Virtual Directory For Your Application
Use inetmgr.msc (Start -> Run…, then enter inetmgr in the edit box and press ENTER) to launch Internet Information Services (IIS) Manager. Make sure to set the local path for the node (Web Site or Virtual Directory) you create to the root folder of your Flask application. wfastcgi.py uses the local path to identify the Flask application to handle the requests.
Give both Read and Script (Run Scripts) permissions for the node.
Step 9: Configure fcgiext.ini
This file is located in the same directory as the fcgiext.dll installed in Step 1 (by default, %SYSTEMROOT%\system32\inetsrv).
In configuring this file, you need several parameters:
{site id}: the numeric Site ID you can find in the detail (right-hand) pane of Internet Information Services (IIS) Manager when “Web Sites” is selected from the tree on the left side of the window.
{application name}: the name of the section within fcgiext.ini that provides the parameters for the FastCGI (ISAPI) handler. You choose this value -- select something that represents your application.
{path to app}: for a Virtual Directory, the URL path within the Web Site to the Virtual Directory to be handled.
{approot}: the path to the root directory of your application.
Use those parameters to:
Map the FastCGI requests to a handling section:
For a whole Web Site, add *:{site id}={application name} to the [Types] section.
For a Virtual Directory, add *:/lm/w3svc/{site id}/root/{path to app}={application name} to the [Types] section.
Add a handling section ([{application name}]) with parameters for this application (full reference):
ExePath={approot}\env\python.exe
Arguments=C:\Inetpub\wwwroot\wfastcgi.py (or wherever the wfastcgi.py adapter script is installed)
EnvironmentVars=ENV_VAR1:value,ENV_VAR2:value,etc. (see the full reference for quoting rules). This is a good place to set your WSGI_LOG environment variable -- make sure the account serving the site (“Network Service” by default) has write permissions for the file and (if the file doesn’t exist) permission to add a file to the containing directory.
Step 10: Configure FastCGI Handling for the Target URLs
Using Internet Information Services (IIS) Manager, select “Properties...” from the context (right-click) menu of the node (Web Site or Virtual Directory) to be served by your Flask application and:
In the “Home Directory” tab (Web Site) or “Virtual Directory” tab (Virtual Directory), click the “Configuration..." button.
In the “Wildcard application maps” section, use the “Insert..." button to add a wildcard mapping:
The executable is the FastCGI extension DLL installed in Step 1. Its default location is %SYSTEMROOT%\system32\inetsrv\fcgiext.dll.
Make sure “Verify that file exists” is unchecked. Flask applications do their own routing that doesn’t necessarily have anything to do with the files on the disk.
Web.config
This file is (in this setup) read by wfastcgi.py, not by IIS.
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<applicationSettings>
<add key=“PYTHONPATH” value=“”/>
<add key=“WSGI_HANDLER” value=“module.application”/>
</applicationSettings>
</configuration>
<add> elements add environment variables (os.environ in Python).
WSGI_HANDLER must be specified -- it tells wfastcgi.py how to locate the WSGI application object. If the value ends in "()", wfastcgi.py will call the named object, expecting it to return a WSGI application object.
PYTHONPATH is handled specially -- wfastcgi.py performs (environment) variable expansion (using the Windows standard %VAR% notation) on the value of PYTHONPATH, then splits the result at semicolons and appends the entries to sys.path before invoking the WSGI application. Because wfastcgi.py changes the current directory to the path specified as the local path of the Web Site or Virtual Directory before importing the module containing the WSGI application object, including an empty string in the PYTHONPATH will cause the search to include your Flask application directory as a starting point. You can also set PYTHONPATH in fcgiext.ini (in which case it is included in sys.path by the interpreter and then again by wfastcgi.py).
WSGI_RESTART_FILE_REGEX gives a Python regular expression used to filter file-change notifications for paths that should trigger FastCGI handler process restarts. Set this to trigger when source files or configuration files change. I use (?i).*\.(py|cnf|config)$.
WSGI_LOG may be set here, but I think it is better set in fcgiext.ini.
For IIS 7
Some things with FastCGI changed dramatically with IIS 7. Beginning with this version, FastCGI has support directly through IIS and is not configured through an extension (i.e. Step 1.4 is not necessary and fcgiext.ini does not control FastCGI behavior for IIS 7+ and there is no need to create/edit it). Instead, make sure that CGI is enable under Internet Information Services in Control Panel > Programs and Features > Turn Windows Features on or off.
Web.config
IIS 7 is the first version of IIS to read configuration settings related to FastCGI from the Web.config file. Your Web.config file will need to contain, within the <configuration> element, a <system.webServer> element containing a <handlers> element containing an <add> element with the attributes:
path: *
verb: *
modules: FastCgiModule
resourceType: Unspecified
requireAccess: Script
scriptProcessor: the tricky one
The scriptProcessor Attribute
This attribute of the <add> element must contain the full path to the Python interpreter .exe file you want to use (the one in the Scripts subfolder of your Python virtualenv) followed by a | and then the full path to the wfastcgi.py file you are using. As these paths are dependent on the setup of the machine on which your app is running, you may want to have this attribute set as part of your deployment process.
IIS Server-wide Set Up
In inetmgr, click on the server node in the tree and then choose FastCGI Settings from the center pane. A list of executable/argument pairs will come up.
Add an entry for the full paths to your python.exe and the wfastcgi.py you are using. Both should be given the same way they show up in the <handlers>/<add> element in your Web.config.
Make sure to set up the PYTHONPATH environment variable in the new FastCGI application entry to include the root of your application codebase. The advice about adding an empty PYTHONPATH entry in the <applicationSettings> of your Web.config may not apply to this version of IIS.

You have hit the nail on the head there. Installing on Windows is kind of like trying to fit square pegs in round holes. Apache and mod_wsgi would probably be the best fit, but the whole experience is much smoother and straightforward (with pip, apt-get etc.) on a Linux box. Would a Linux VM running on the Windows server be a suitable compromise?

One potential pathway, though I foresee it to be very complicated, is having your flask app running in windows subsystem of linux.
There's existing tutorials on how to make calls to powershell scripts from the subsystem, e.g.,: https://www.raymondcamden.com/2017/09/25/calling-a-powershell-script-from-wsl

I am successfully using the simple Twisted Web server on Windows for Flask web sites.
Are others also successfully using Twisted on Windows, to validate that configuration?
new_app.py
if name == "main":
reactor_args = {}
def run_twisted_wsgi():
from twisted.internet import reactor
from twisted.web.server import Site
from twisted.web.wsgi import WSGIResource
resource = WSGIResource(reactor, reactor.getThreadPool(), app)
site = Site(resource)
reactor.listenTCP(5000, site)
reactor.run(**reactor_args)
if app.debug:
# Disable twisted signal handlers in development only.
reactor_args['installSignalHandlers'] = 0
# Turn on auto reload.
import werkzeug.serving
run_twisted_wsgi = werkzeug.serving.run_with_reloader(run_twisted_wsgi)
run_twisted_wsgi()
old_app.py
if name == "main":
app.run()

I recommend the following approach:
IIS
... with the IIS CGI/FastCGI module enabled
... using the wfastcgi python module to host a Python WSGI web application under IIS FastCGI
When correctly set up, deploying a Python web app on IIS is very easy, and can be done without clicking around in IIS admin, simply be editing the site/apps web.config file.
Documentation: https://pypi.org/project/wfastcgi/
Source code: https://github.com/microsoft/PTVS/tree/master/Python/Product/WFastCgi

How do I deploy a Flask application in IIS?

Can anyone help me get a Flask application running on IIS 6? I have tried to use isapi-wsgi, but when I visit the Virtual Directory address I get a page that says "The specified module could not be found." Are there other options for this?
Below is the Python script I wrote for isapi-wsgi. The Virtual Directory was made and everything looked ok in IIS Manager, but the site did not work.
from wof import app
import os
app.secret_key=os.urandom(24)
import isapi_wsgi
def __ExtensionFactory__():
return isapi_wsgi.ISAPISimpleHandler(app)
if __name__ == '__main__':
from isapi.install import *
params = ISAPIParameters()
sm = [ScriptMapParams(Extension="*", Flags=0)]
vd = VirtualDirParameters(Name="WOFPy_Sondes", Description="ISAPI-WSGI for WOFPY Sondes test", ScriptMaps=sm, ScriptMapUpdate="replace")
params.VirtualDirs = [vd]
HandleCommandLine(params)

High Level Overview
HTTP -> IIS -> ISAPI -> FastCGI -> WSGI (Flask application)
Setup Steps
Step 1: Install Required Binaries
Install Python (2.7 or 3.x -- I used 3.3)
Install pip-Win (I used version 1.6)
Install pywin32 (I used version 218)
Install the IIS FastCGI extension with fcgisetup 1.5
Step 2: Install Optional Binary Packages
I installed pyodbc using the installer .exe from this site. Installing from source (e.g. pip, for installing into a virtual environment) requires a C/C++ compiler.
Step 3: Get a Copy of wfastcgi.py
Choose a version that will work for you, preferably one that supports Python 3.3 (I used David Ebbo's). You may want the "official" version from here.
Install the wfastcgi.py script into C:\Inetpub\wwwroot and make sure the account that will serve your application ("Network Service" by default) has read access to it.
Step 4: Install virtualenv Into the System site-packages
C:\Python33\Scripts\pip.exe install virtualenv
(if you are using Python 3.3 and installed everything in the default location)
Step 5: Install Your Flask Application
You may install the application just about anywhere on the system. You may want to install it under C:\Inetpub. For this tutorial, we'll call the root folder of your application install %APPROOT%. (Don't put quotation marks in the environment variable.)
Make sure that the account that will serve your application ("Network Service" by default) has read access to all of the script files. This command:
cacls "%APPROOT%" /S:"D:PAI(A;OICI;FA;;;BA)(A;OICIIO;FA;;;CO)(A;OICI;0x1200a9;;;NS)(A;OICI;FA;;;SY)"
will give your application directory the following permissions:
BUILTIN\Administrators: Full control of this folder, subfolders, and files
CREATOR OWNER: Full control for subfolders and files only
NT AUTHORITY\NETWORK SERVICE: Read permissions for this folder, subfolders, and files
NT AUTHORITY\SYSTEM: Full control of this folder, subfolders, and files
Add any local configuration necessary (my application uses a local.cnf file that is ignored by the version control system) -- e.g. database URLs.
Make sure your application contains a Web.config file in %APPROOT% -- see the section below for information on the file format.
Step 6: Create a virtualenv For Your Application
C:\Python33\Scripts\virtualenv.exe --system-site-packages "%APPROOT%\env"
(Pick a name other than env if your application already uses that directory.)
Step 7: Install The Packages Required By Your Application to the virtualenv
cd "%APPROOT%"
env\Scripts\activate
pip install -r Packages
(My project keeps the requirements spec in a file named Packages.)
Step 8: Create a Web Site Or Virtual Directory For Your Application
Use inetmgr.msc (Start -> Run…, then enter inetmgr in the edit box and press ENTER) to launch Internet Information Services (IIS) Manager. Make sure to set the local path for the node (Web Site or Virtual Directory) you create to the root folder of your Flask application. wfastcgi.py uses the local path to identify the Flask application to handle the requests.
Give both Read and Script (Run Scripts) permissions for the node.
Step 9: Configure fcgiext.ini
This file is located in the same directory as the fcgiext.dll installed in Step 1 (by default, %SYSTEMROOT%\system32\inetsrv).
In configuring this file, you need several parameters:
{site id}: the numeric Site ID you can find in the detail (right-hand) pane of Internet Information Services (IIS) Manager when “Web Sites” is selected from the tree on the left side of the window.
{application name}: the name of the section within fcgiext.ini that provides the parameters for the FastCGI (ISAPI) handler. You choose this value -- select something that represents your application.
{path to app}: for a Virtual Directory, the URL path within the Web Site to the Virtual Directory to be handled.
{approot}: the path to the root directory of your application.
Use those parameters to:
Map the FastCGI requests to a handling section:
For a whole Web Site, add *:{site id}={application name} to the [Types] section.
For a Virtual Directory, add *:/lm/w3svc/{site id}/root/{path to app}={application name} to the [Types] section.
Add a handling section ([{application name}]) with parameters for this application (full reference):
ExePath={approot}\env\python.exe
Arguments=C:\Inetpub\wwwroot\wfastcgi.py (or wherever the wfastcgi.py adapter script is installed)
EnvironmentVars=ENV_VAR1:value,ENV_VAR2:value,etc. (see the full reference for quoting rules). This is a good place to set your WSGI_LOG environment variable -- make sure the account serving the site (“Network Service” by default) has write permissions for the file and (if the file doesn’t exist) permission to add a file to the containing directory.
Step 10: Configure FastCGI Handling for the Target URLs
Using Internet Information Services (IIS) Manager, select “Properties...” from the context (right-click) menu of the node (Web Site or Virtual Directory) to be served by your Flask application and:
In the “Home Directory” tab (Web Site) or “Virtual Directory” tab (Virtual Directory), click the “Configuration..." button.
In the “Wildcard application maps” section, use the “Insert..." button to add a wildcard mapping:
The executable is the FastCGI extension DLL installed in Step 1. Its default location is %SYSTEMROOT%\system32\inetsrv\fcgiext.dll.
Make sure “Verify that file exists” is unchecked. Flask applications do their own routing that doesn’t necessarily have anything to do with the files on the disk.
Web.config
This file is (in this setup) read by wfastcgi.py, not by IIS.
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<applicationSettings>
<add key=“PYTHONPATH” value=“”/>
<add key=“WSGI_HANDLER” value=“module.application”/>
</applicationSettings>
</configuration>
<add> elements add environment variables (os.environ in Python).
WSGI_HANDLER must be specified -- it tells wfastcgi.py how to locate the WSGI application object. If the value ends in "()", wfastcgi.py will call the named object, expecting it to return a WSGI application object.
PYTHONPATH is handled specially -- wfastcgi.py performs (environment) variable expansion (using the Windows standard %VAR% notation) on the value of PYTHONPATH, then splits the result at semicolons and appends the entries to sys.path before invoking the WSGI application. Because wfastcgi.py changes the current directory to the path specified as the local path of the Web Site or Virtual Directory before importing the module containing the WSGI application object, including an empty string in the PYTHONPATH will cause the search to include your Flask application directory as a starting point. You can also set PYTHONPATH in fcgiext.ini (in which case it is included in sys.path by the interpreter and then again by wfastcgi.py).
WSGI_RESTART_FILE_REGEX gives a Python regular expression used to filter file-change notifications for paths that should trigger FastCGI handler process restarts. Set this to trigger when source files or configuration files change. I use (?i).*\.(py|cnf|config)$.
WSGI_LOG may be set here, but I think it is better set in fcgiext.ini.
For IIS 7
Some things with FastCGI changed dramatically with IIS 7. Beginning with this version, FastCGI has support directly through IIS and is not configured through an extension (i.e. Step 1.4 is not necessary and fcgiext.ini does not control FastCGI behavior for IIS 7+ and there is no need to create/edit it). Instead, make sure that CGI is enable under Internet Information Services in Control Panel > Programs and Features > Turn Windows Features on or off.
Web.config
IIS 7 is the first version of IIS to read configuration settings related to FastCGI from the Web.config file. Your Web.config file will need to contain, within the <configuration> element, a <system.webServer> element containing a <handlers> element containing an <add> element with the attributes:
path: *
verb: *
modules: FastCgiModule
resourceType: Unspecified
requireAccess: Script
scriptProcessor: the tricky one
The scriptProcessor Attribute
This attribute of the <add> element must contain the full path to the Python interpreter .exe file you want to use (the one in the Scripts subfolder of your Python virtualenv) followed by a | and then the full path to the wfastcgi.py file you are using. As these paths are dependent on the setup of the machine on which your app is running, you may want to have this attribute set as part of your deployment process.
IIS Server-wide Set Up
In inetmgr, click on the server node in the tree and then choose FastCGI Settings from the center pane. A list of executable/argument pairs will come up.
Add an entry for the full paths to your python.exe and the wfastcgi.py you are using. Both should be given the same way they show up in the <handlers>/<add> element in your Web.config.
Make sure to set up the PYTHONPATH environment variable in the new FastCGI application entry to include the root of your application codebase. The advice about adding an empty PYTHONPATH entry in the <applicationSettings> of your Web.config may not apply to this version of IIS.

Check out Django's page on the subject. It helped me set up a working Django project, but it shouldn't be that different for a Flask app.
http://code.djangoproject.com/wiki/DjangoOnWindowsWithIISAndSQLServer

I never use IIS, but IIS supports CGI gateway, therefore you should be able to adapt CGI with WSGI.
IIS <--> CGI <--> WSGI
To run a WSGI as a CGI script, you can use the CGIHandler in Python standard library.