Develop Reference

File is not created/opened when using with open

I am using Flask and testing some code in Python. I am trying to store in a log file a Flask request and a string every time a post is done.
This is my code:
from flask import Flask, render_template, request
from vsearch import search4letters
app = Flask(__name__)
def log_request(request, results: str) -> None:
print(request)
with open('vsearch.log', 'a') as log:
print(request, results, file=log)
#app.route('/search4', methods=['POST'])
def do_search() -> 'html':
phrase = request.form['phrase']
letters = request.form['letters']
title = 'Here are your results:'
results = str(search4letters(phrase, letters))
log_request(request, results)
return render_template('results.html',
the_phrase=phrase,
the_letters=letters,
the_title=title,
the_results=results,
)
#app.route('/')
#app.route('/entry')
def entry_page() -> 'html':
return render_template('entry.html',
the_title='Welcome to search4letters on the web!')
if __name__ == '__main__':
app.run(debug=True)
This is my HTML view:
After pressing Do it!, 'vsearch.log' should contained what I have printed to it, but it does not. In addition, when the file does not exists, it does not get created.
I have tried changing the mode of open to 'a+', but I get the same results. I have also made a debug, and these lines are just executed with no errors raised.
Could somebody explain me what is going on, and how can I solve this problem ?

Since you're using Flask it's much better to use the built in logging functionality. See: http://flask.pocoo.org/docs/0.12/errorhandling/#logging-to-a-file
So, for example, on app startup you'd have:
import logging
file_handler = logging.FileHandler('/path/to/your/flask.log')
file_handler.setLevel(logging.WARNING)
app.logger.addHandler(file_handler)
Then wherever you want to log something in your application you'd log to warning or above, or whatever you set the file handler log level to:
#app.route('/whatever')
def whatever():
app.logger.warning('Whatever!')
return render_template('whatever.html')

Thanks to #AlexHall, I have been able to solve this problem. The solution is to specify the full absolute path to the file.
def log_request(request, results: str) -> None:
with open('/absolute/path/to/the/file/vsearch.log', 'a') as log:
print(request, results, file=log)
In addition, following #AlexHall suggestion to know the current working directory. I have seen that this is:
/Applications/PyCharm.app/Contents/bin
so when not specifying the full absolute path the file 'vsearch.log' was created here.
EDIT:
So, it seems that the problem was I was running my code from PyCharm. However, when I use the terminal and I just run:
$ python webapp.py
I do not need to specify the full absolute path.
EDIT:
I was able to solve this issue, and I probably screwed up the settings at some point, but after deleting all the run configurations in PyCharm, and running the program from webapp.py everything has been solved.
I really want to thank #AlexHall since he gave me all tips to solve this problem.

How to stop cherrypy auto-reload from changing the process name on Debian?

I am developing a project using cherrypy, on debian. At my work, the administrators want to see the name of the project instead of "python" displayed when using commands like ps -e. However, when cherrypy auto-reloads when modifying one source file, it automatically changes the process name.
For example, if I take the most basic cherrypy tutorial and save it under NameToSee.py:
#!/usr/bin/python
import cherrypy
class HelloWorld(object):
#cherrypy.expose
def index(self):
return "Hello world!"
if __name__ == '__main__':
cherrypy.quickstart(HelloWorld())
By adding the shebang at the start, when I launch it $ ./NameToSee.py &, I get a process (say 31051) whose name is "NameToSee.py":
$ head /proc/31051/status
Name: NameToSee.py
State: S (sleeping)
However, whenever I change the source code files (for example, by adding an empty line), the process name changes:
$ head /proc/31051/status
Name: python
State: S (sleeping)
So, my question is: Can I get both cherrypy auto-reload and custom process name ? If not, can I remove the cherrypy auto-reload?
I am running on debian wheezy, with python 2.7.3 and cherrypy 3.2.2

This sample covers both cases:
import cherrypy
from cherrypy.process.plugins import SimplePlugin
PROC_NAME = 'sample'
def set_proc_name(newname):
"""
Set the process name.
Source: http://stackoverflow.com/a/923034/298371
"""
from ctypes import cdll, byref, create_string_buffer
libc = cdll.LoadLibrary('libc.so.6')
buff = create_string_buffer(len(newname)+1)
buff.value = newname
libc.prctl(15, byref(buff), 0, 0, 0)
class NamedProcess(SimplePlugin):
"""
Set the name of the process everytime that the
engine starts.
"""
def start(self):
self.bus.log("Setting the name as '{}'".format(PROC_NAME))
set_proc_name(PROC_NAME)
class HelloWorld(object):
#cherrypy.expose
def index(self):
return "Hello world!"
def run_without_autoreload():
set_proc_name(PROC_NAME)
cherrypy.quickstart(HelloWorld(), config={
'global': {
'engine.autoreload.on': False
}
})
def run_with_autoreload():
# Work on any configuration but for the sake of the
# question this works with the autoreload.
NamedProcess(cherrypy.engine).subscribe()
cherrypy.quickstart(HelloWorld())
if __name__ == '__main__':
run_with_autoreload()
# run_without_autoreload()
You can test it with:
cat /proc/`pgrep sample`/status
(or maybe just use pgrep)
And as a final recommendation consider using the "production" environment (see the docs) which also included the disabling of the autoreload plugin.

python the hard way ex 50: "permission denied"

Per the book, I successfully installed lpthw.web, then created the module named app.py. First I typed it exactly, then tried cutting and pasting from the website, to be 100% sure.
When I run app.py on my OS Yosemite Mac, I get the message
Permission denied.
I think it has to do with the command import web. I tested this by commenting out all of the lines except this one and I got the same error. However, I made a simple file, put it in bin and was able to run it. Not sure what’s going on. How do I change this?
Thanks for your help.
For reference, here is the code for bin/app.py:
import web
urls = (
'/', 'index'
)
app = web.application(urls, globals())
class index:
def GET(self):
greeting = "hello world"
return greeting
if __name__ == "__main__":
app.run()

Why are environment variables empty in Flask apps?

I have a flask app (my_app) that calls a function in a different file (my_function):
my_app.py:
from my_functions import my_function
#app.route('/')
def index():
my_function()
return render_template('index.html')
my_functions.py:
def my_function():
try:
import my_lib
except:
print("my_lib not found in system!")
# do stuff...
if __name__ == "__main__":
my_function()
When I execute my_functions.py directly (i.e., python my_functions.py) "my_lib" is imported without error; however, when I execute the flask app (i.e., python my_app.py) I get an import error for "my_lib".
When I print the LD_LIBRARY_PATH variable at the beginning of each file:
print(os.environ['LD_LIBRARY_PATH'])
I get the correct value when calling my_functions.py, but get no value (empty) when calling my_app.py.Trying to set this value at the beginning of my_app.py has no effect:
os.environ['LD_LIBRARY_PATH'] = '/usr/local/lib'
Questions:
(1) Why is 'LD_LIBRARY_PATH' empty when called within the Flask app?
(2) How do I set it?
Any help appreciated.

LD_LIBRARY_PATH is cleared when executing the flask app, likely for security reasons as Mike suggested.
To get around this, I use subprocess to make a call directly to an executable:
import subprocess
call_str = "executable_name -arg1 arg1_value -arg2 arg2_value"
subprocess.call(call_str, shell=True, stderr=subprocess.STDOUT)
Ideally the program should be able to use the python bindings, but for now calling the executable works.

Reload Flask app when template file changes

By default, when running Flask application using the built-in server (Flask.run), it monitors its Python files and automatically reloads the app if its code changes:
* Detected change in '/home/xion/hello-world/app.py', reloading
* Restarting with reloader
Unfortunately, this seems to work for *.py files only, and I don't seem to find any way to extend this functionality to other files. Most notably, it would be extremely useful to have Flask restart the app when a template changes. I've lost count on how many times I was fiddling with markup in templates and getting confused by not seeing any changes, only to find out that the app was still using the old version of Jinja template.
So, is there a way to have Flask monitor files in templates directory, or does it require diving into the framework's source?
Edit: I'm using Ubuntu 10.10. Haven't tried that on any other platforms really.
After further inquiry, I have discovered that changes in templates indeed are updated in real time, without reloading the app itself. However, this seems to apply only to those templates that are passed to flask.render_template.
But it so happens that in my app, I have quite a lot of reusable, parametrized components which I use in Jinja templates. They are implemented as {% macro %}s, reside in dedicated "modules" and are {% import %}ed into actual pages. All nice and DRY... except that those imported templates are apparently never checked for modifications, as they don't pass through render_template at all.
(Curiously, this doesn't happen for templates invoked through {% extends %}. As for {% include %}, I have no idea as I don't really use them.)
So to wrap up, the roots of this phenomenon seems to lie somewhere between Jinja and Flask or Werkzeug. I guess it may warrant a trip to bug tracker for either of those projects :) Meanwhile, I've accepted the jd.'s answer because that's the solution I actually used - and it works like a charm.

you can use
TEMPLATES_AUTO_RELOAD = True
From http://flask.pocoo.org/docs/1.0/config/
Whether to check for modifications of the template source and reload it automatically. By default the value is None which means that Flask checks original file only in debug mode.

In my experience, templates don't even need the application to restart to be refreshed, as they should be loaded from disk everytime render_template() is called. Maybe your templates are used differently though.
To reload your application when the templates change (or any other file), you can pass the extra_files argument to Flask().run(), a collection of filenames to watch: any change on those files will trigger the reloader.
Example:
from os import path, walk
extra_dirs = ['directory/to/watch',]
extra_files = extra_dirs[:]
for extra_dir in extra_dirs:
for dirname, dirs, files in walk(extra_dir):
for filename in files:
filename = path.join(dirname, filename)
if path.isfile(filename):
extra_files.append(filename)
app.run(extra_files=extra_files)
See here: http://werkzeug.pocoo.org/docs/0.10/serving/?highlight=run_simple#werkzeug.serving.run_simple

When you are working with jinja templates, you need to set some parameters. In my case with python3, I solved it with the following code:
if __name__ == '__main__':
app.jinja_env.auto_reload = True
app.config['TEMPLATES_AUTO_RELOAD'] = True
app.run(debug=True, host='0.0.0.0')

You need to set a TEMPLATES_AUTO_RELOAD property as True in your app config:
from flask import Flask
app = Flask(__name__)
app.config["TEMPLATES_AUTO_RELOAD"] = True
See more on http://flask.pocoo.org/docs/1.0/config/

Actually for me TEMPLATES_AUTO_RELOAD = True does not work (0.12 version). I use jinja2 and what i have done:
Create function before_request
def before_request():
app.jinja_env.cache = {}
Register it in application
app.before_request(before_request)
That's it.

Updated as of March 2021:
The flask CLI is recommended over app.run() for running a dev server, so if we want to use the CLI then the accepted solution can't be used.
In Flask 1.1 or later, the environment variable FLASK_RUN_EXTRA_FILES or the option --extra-files effectively do the same thing as the accepted answer. See also this github issue.
Example usage:
flask run --extra-files "app/templates/index.html"
# or
export FLASK_RUN_EXTRA_FILES="app/templates/index.html"
flask run
in Linux. To specify multiple extra files, separate file paths with colons., e.g.
export FLASK_RUN_EXTRA_FILES="app/templates/index.html:app/templates/other.html"
Whole directories are also supported:
flask run --extra-files app/templates/

What worked for me is just adding this:
#app.before_request
def before_request():
# When you import jinja2 macros, they get cached which is annoying for local
# development, so wipe the cache every request.
if 'localhost' in request.host_url or '0.0.0.0' in request.host_url:
app.jinja_env.cache = {}
(taken from #dikkini's answer)

To reload the application on the server AND in the browser I used the livereload package. Installed through the CLI with
$ pip install livereload
and running the code
from flask import Flask, render_template
app = Flask(__name__)
#app.route("/")
def hello():
return render_template("index.html")
if __name__ == '__main__':
from livereload import Server
server = Server(app.wsgi_app)
server.serve(host = '0.0.0.0',port=5000)
all answers here using the extra_files argument or TEMPLATES_AUTO_RELOAD config work to reload it on the server but for a smooth development experience without damaging your keyboard's F5 key I'd go with livereload

Using the latest version of Flask on Windows, using the run command and debug set to true; Flask doesn't need to be reset for changes to templates to be brought in to effect. Try Shift+F5 (or Shift plus the reload button) to make sure nothing it being cached.

See http://flask.pocoo.org/docs/1.0/quickstart/
and use FLASK_ENV=development

I had the same trouble. The solution is really simple though. Instead of this:
if __name__ == '__main__':
app.jinja_env.auto_reload = True
app.config["TEMPLATES_AUTO_RELOAD"] = True
app.run(debug=True)
Put
app.jinja_env.auto_reload = True
app.config["TEMPLATES_AUTO_RELOAD"] = True
above the main function. So final output for example:
from flask import Flask, app,render_template
app= Flask(__name__)
app.jinja_env.auto_reload = True
app.config["TEMPLATES_AUTO_RELOAD"] = True
#app.route('/')
def index():
return render_template('index.html')
if __name__ == '__main__':
app.run(debug=True)

Templates are reloaded automatically, why not doing ctrl+f5 to refresh the webpage,
cause web-browsers usually save cache.

Adding app.config['TEMPLATES_AUTO_RELOAD'] = True after if __name__ == '__main__': doesn't work for me!
What works is adding app.config['TEMPLATES_AUTO_RELOAD'] = True after app = Flask(__name__)
Notice that I am using app.run(debug=True)