I have two scripts that are running in loop independently: a simple python script that generates data
myData=0
while True:
myData = get_data() # this data is now available for Flask App
and the flask application that displays data
from flask import Flask
app = Flask(__name__)
#app.route('/')
def hello_world(myData):
return str(myData)
app.run()
I wish to somehow connect the two scripts, so the application displays the data produced by the python script.
myData=0
app = Flask(__name__)
#app.route('/')
def hello_world(myData):
return str(myData)
app.run() # does not return until server is terminated
while True:
myData = get_data()
When I combine the scripts as shown above, I can see that the execution does not get to the while loop (past app.run() line) until I terminate the app.
I found a similar question here, but not not helpful, and another question here that is identical to what I am trying to do, but it also does not give me any clue. I can not find any info that tells how to make a flask application to communicate with a separately running script. Here's a similar question with no definite answer. Please, give me an insight how these two things should run together, or an example would be greatly appreciated.
Since your script keeps generating data indefinitely, I would suggest transforming it into a generator and iterating over it from the web request handler:
def my_counter():
i = 0
while True:
yield i # I'm using yield instead of return
i = i + 1
my_counter_it = my_counter()
#app.route('/')
def hello_world():
return str(next(my_counter_it)) # return next value from generator
You can also communicate with a long running separate process (external command):
import subprocess
def my_counter():
# run the yes command which repeatedly outputs y
# see yes(1) or http://man7.org/linux/man-pages/man1/yes.1.html
p = subprocess.Popen('yes', stdout=subprocess.PIPE)
# the following can also be done with just one line: yield from p.stdout
for line in p.stdout:
yield line
You can create a function that will procedure the data, which can then be served on the route:
def get_data():
i = 0
while i < 1000:
i += 1
return str(i)
#app.route('/')
def hello_world():
return get_data()
Related
Greetings stack overflow community, I am currently working on a flask app and I am trying to retrieve a file from a helper function with the send_file method in flask.
I have a route that goes like so:
#app.route("/process",methods=['GET','POST'])
def do_something():
process = threading.Thread(target=function_name,args=[arg1,arg2])
process.start()
return render_template("template.html")
The function_name (which is on a different file) function is suposed to return a file like so
def function_name():
filename = 'ohhey.pdf'
return send_file(filename,as_attachment=True,cache_timeout=0)
When I run my app like this I get the following error
RuntimeError: Working outside of application context.
This typically means that you attempted to use functionality that needed
to interface with the current application object in some way. To solve
this, set up an application context with app.app_context(). See the
documentation for more information.
So I try to change the function for the following:
def function_name():
filename = 'ohhey.pdf'
with app.app_context():
return send_file(filename,as_attachment=True,cache_timeout=0)
and get this new error
RuntimeError: Working outside of request context.
This typically means that you attempted to use functionality that needed
an active HTTP request. Consult the documentation on testing for
information about how to avoid this problem.
so I try the following:
def function_name():
filename = 'ohhey.pdf'
with app.test_request_context():
return send_file(filename,as_attachment=True,cache_timeout=0)
After making this final change my app doesn't return a file or an error. I appreciate your help.
I am using Flask and testing some code in Python. I am trying to store in a log file a Flask request and a string every time a post is done.
This is my code:
from flask import Flask, render_template, request
from vsearch import search4letters
app = Flask(__name__)
def log_request(request, results: str) -> None:
print(request)
with open('vsearch.log', 'a') as log:
print(request, results, file=log)
#app.route('/search4', methods=['POST'])
def do_search() -> 'html':
phrase = request.form['phrase']
letters = request.form['letters']
title = 'Here are your results:'
results = str(search4letters(phrase, letters))
log_request(request, results)
return render_template('results.html',
the_phrase=phrase,
the_letters=letters,
the_title=title,
the_results=results,
)
#app.route('/')
#app.route('/entry')
def entry_page() -> 'html':
return render_template('entry.html',
the_title='Welcome to search4letters on the web!')
if __name__ == '__main__':
app.run(debug=True)
This is my HTML view:
After pressing Do it!, 'vsearch.log' should contained what I have printed to it, but it does not. In addition, when the file does not exists, it does not get created.
I have tried changing the mode of open to 'a+', but I get the same results. I have also made a debug, and these lines are just executed with no errors raised.
Could somebody explain me what is going on, and how can I solve this problem ?
Since you're using Flask it's much better to use the built in logging functionality. See: http://flask.pocoo.org/docs/0.12/errorhandling/#logging-to-a-file
So, for example, on app startup you'd have:
import logging
file_handler = logging.FileHandler('/path/to/your/flask.log')
file_handler.setLevel(logging.WARNING)
app.logger.addHandler(file_handler)
Then wherever you want to log something in your application you'd log to warning or above, or whatever you set the file handler log level to:
#app.route('/whatever')
def whatever():
app.logger.warning('Whatever!')
return render_template('whatever.html')
Thanks to #AlexHall, I have been able to solve this problem. The solution is to specify the full absolute path to the file.
def log_request(request, results: str) -> None:
with open('/absolute/path/to/the/file/vsearch.log', 'a') as log:
print(request, results, file=log)
In addition, following #AlexHall suggestion to know the current working directory. I have seen that this is:
/Applications/PyCharm.app/Contents/bin
so when not specifying the full absolute path the file 'vsearch.log' was created here.
EDIT:
So, it seems that the problem was I was running my code from PyCharm. However, when I use the terminal and I just run:
$ python webapp.py
I do not need to specify the full absolute path.
EDIT:
I was able to solve this issue, and I probably screwed up the settings at some point, but after deleting all the run configurations in PyCharm, and running the program from webapp.py everything has been solved.
I really want to thank #AlexHall since he gave me all tips to solve this problem.
I'm trying to solve a problem in celery:
I have one task that queries an API for ids, and then starts a sub-task for each of these.
I do not know, ahead of time, what the ids are, or how many there are.
For each id, I go through a big calculation that then dumps some data into a database.
After all the sub-tasks are complete, I want to run a summary function (export DB results to an Excel format).
Ideally, I do not want to block my main worker querying the status of the sub-tasks (Celery gets angry if you try this.)
This question looks very similar (if not identical?): Celery: Callback after task hierarchy
So using the "solution" (which is a link to this discussion, I tried the following test script:
# test.py
from celery import Celery, chord
from celery.utils.log import get_task_logger
app = Celery('test', backend='redis://localhost:45000/10?new_join=1', broker='redis://localhost:45000/11')
app.conf.CELERY_ALWAYS_EAGER = False
logger = get_task_logger(__name__)
#app.task(bind=True)
def get_one(self):
print('hello world')
self.replace(get_two.s())
return 1
#app.task
def get_two():
print('Returning two')
return 2
#app.task
def sum_all(data):
print('Logging data')
logger.error(data)
return sum(data)
if __name__ == '__main__':
print('Running test')
x = chord(get_one.s() for i in range(3))
body = sum_all.s()
result = x(body)
print(result.get())
print('Finished w/ test')
It doesn't work for me. I get an error:
AttributeError: 'get_one' object has no attribute 'replace'
Note that I do have new_join=1 in my backend URL, though not the broker. If I put it there, I get an error:
TypeError: _init_params() got an unexpected keyword argument 'new_join'
What am I doing wrong? I'm using the Python 3.4.3 and the following packages:
amqp==1.4.6
anyjson==0.3.3
billiard==3.3.0.20
celery==3.1.18
kombu==3.0.26
pytz==2015.4
redis==2.10.3
The Task.replace method will be added in Celery 3.2: http://celery.readthedocs.org/en/master/whatsnew-3.2.html#task-replace (that changelog entry is misleading, because it suggests that Task.replace existed before and has been changed.)
I have a flask app (my_app) that calls a function in a different file (my_function):
my_app.py:
from my_functions import my_function
#app.route('/')
def index():
my_function()
return render_template('index.html')
my_functions.py:
def my_function():
try:
import my_lib
except:
print("my_lib not found in system!")
# do stuff...
if __name__ == "__main__":
my_function()
When I execute my_functions.py directly (i.e., python my_functions.py) "my_lib" is imported without error; however, when I execute the flask app (i.e., python my_app.py) I get an import error for "my_lib".
When I print the LD_LIBRARY_PATH variable at the beginning of each file:
print(os.environ['LD_LIBRARY_PATH'])
I get the correct value when calling my_functions.py, but get no value (empty) when calling my_app.py.Trying to set this value at the beginning of my_app.py has no effect:
os.environ['LD_LIBRARY_PATH'] = '/usr/local/lib'
Questions:
(1) Why is 'LD_LIBRARY_PATH' empty when called within the Flask app?
(2) How do I set it?
Any help appreciated.
LD_LIBRARY_PATH is cleared when executing the flask app, likely for security reasons as Mike suggested.
To get around this, I use subprocess to make a call directly to an executable:
import subprocess
call_str = "executable_name -arg1 arg1_value -arg2 arg2_value"
subprocess.call(call_str, shell=True, stderr=subprocess.STDOUT)
Ideally the program should be able to use the python bindings, but for now calling the executable works.
I need to store python code in a database and load it in some kind of bootstrap.py application for execution. I cannot use filesystem because I'm using GAE, so this is my only choice.
However I'm not a python experienced user.
I already was able to load 1 line of code and run it using eval, however a piece of code with two lines or more gave me a "invalid syntax" error.
I'm also thinking if it's possible to extend the "import" loader to implement the DB loading.
Thanks!
I was able to do what I intent after reading more about Python dynamic code loading.
Here is the sample code. I removed headers to be lighter:
Thanks anyway!
=============
class DynCode(db.Model):
name = db.StringProperty()
code = db.TextProperty(default=None)
=============
class MainHandler(webapp.RequestHandler):
def get(self):
dyn = DynCode()
dyn = "index"
dyn.code = """
from google.appengine.ext import webapp
class MainHandler(webapp.RequestHandler):
def get(self):
self.response.out.write("Hello World\\n")
self.response.out.write("Hello World 2\\n")
"""
dyn.put()
self.response.out.write("OK.")
def main():
application = webapp.WSGIApplication([('/update', MainHandler)], debug=True)
util.run_wsgi_app(application)
if __name__ == '__main__':
main()
==================================
def main():
query = DynCode.all()
dyncodes = query.fetch(1)
module = imp.new_module('mymodule')
for dyn in dyncodes:
exec dyn.code in module.__dict__
application = webapp.WSGIApplication([('/', module.MainHandler)], debug=True)
util.run_wsgi_app(application)
if __name__ == '__main__':
main()
=======================
If you want a more robust mechanism, you probably want to read PEP302, which describes input hooks. You can use these to import code rather than having to eval it.
I somewhat agree with the commentators above, it sounds kind of dangerous. However:
I experimented a little with App Engine Console ( http://con.appspot.com/console/ ), and eval() indeed tended to throw SyntaxError's.
Instead, the exec statement might be your friend ( http://docs.python.org/release/2.5.2/ref/exec.html ).
I managed to run this in App Engine Console:
>>> exec "def f(x):\n x = x + 1\n y = 10\n return x + y"
>>> f(10)
21
So try the exec statement, but remember the many, many (many!) perils of code coming directly from end-users.