So I am trying to create an API that constantly reads from a CSV and returns information about it when requested. So far, I have created a flask API that reads the CSV file once and returns correctly. However, I can't seem to make it constantly update. My working code is something like this.
app = flask.Flask(__name__)
app.config["DEBUG"] = True
dfchat = pd.read_csv(path)
escaper = None
# for now, this is just to make sure the program keeps running even if there is an error
def escape_route():
global escaper
while escaper != "Y":
escaper = str(input("Exit now? Enter \'Y\': \n")).strip()
os._exit(os.X_OK)
def sample_function(dfchat):
#app.route('/sample_text', methods=['GET'])
def sample_endpoint():
# this function filters dfchat and returns whatever
def main():
global dfchat
escape_route_thread = threading.Thread(target = escape_route)
escape_route_thread.start()
sample_function(dfchat)
app.run()
main()
I have tried creating another thread that updates the CSV file:
def retrieve_database():
global dfchat
while True:
time.sleep(0.1)
dfchat = pd.read_csv(path)
along with:
escape_route_thread = threading.Thread(target = retrieve_database)
escape_route_thread.start()
in the main function.
But that fails to update the dfchat data frame when the API launches. I have tested the thread by itself and it does update and return an updated data frame.
From what I understand so far, once an API runs, python code cannot change the API itself.
So,
Is there a way to update a running API with just python?
I'm asking for just python because I will not be able to manually enter a link like "/refresh" to do this. It has to be done by python.
Am I missing something?
Thank you very much for helping!
Edit:
I also tried to update the csv file for every API call. But that does but work either:
def sample_function():
dfchat = pd.read_csv(path)
#app.route('/sample_text', methods=['GET'])
def sample_endpoint():
# this function filters dfchat and returns whatever
Code defined at the root of the script (as was dfchat definition in your example) is executed once at the moment you start the flask server.
Code inside a Flask app route (function decorated with #app.route(...)) is executed at each API call to this route.
from flask import Flask
app = Flask(__name__)
path = "path/to/your/csv/file.csv"
#app.route('/sample_text', methods=['GET'])
def sample_endpoint():
dfchat = pd.read_csv(path)
# do what you have to do with the DF
Also note that Flask handles errors without stopping the API, and has great documentation to help you : https://flask.palletsprojects.com/en/2.0.x/quickstart/#a-minimal-application
So I realized that the solution is really simple. I'm new to CS and APIs in general so I did not realize how #app.route worked in Flask. Outside of #app.route cannot change a route but updating a variable inside a route does work. I accidentally kept updating dfchat outside of #app.route.
def sample_function():
# instead of putting dfchat here
#app.route('/sample_text', methods=['GET'])
def sample_endpoint():
dfchat = pd.read_csv(path) # it should go here.
# this function filters dfchat and returns whatever
Thank you yco for helping me realize this.
Related
Apologies in advance as this is probably the most basic question to be found here but I'm the greenest of newbies and cannot get my head around how to call a function in flask so it runs when I land on the URL.
My purpose is to try and get a python script to run when a GET request is made to the URL from WebCore (for those who don't know it's a program that allows you to code smart home functionality for SmartThings) or when I simply land at the URL. I will then tie this to a virtual switch which will start the code which controls a motor in a cat feeder so I can feed my cat remotely/by voice.
All very frivolous stuff but trying to learn some basics here, can anyone help?
As it stands I have two files, both in a root directory named 'CatFeeder'
catfeeder.py
from flask import Flask
from feed import start
app = Flask(__name__)
#app.route('/')
def feed()
return feed.start
if __name__ == '__main__':
app.run(host='0.0.0.0', port='5000', debug=True)
feed.py
import time
import RPi.GPIO as GPIO
def start():
# Next we setup the pins for use!
GPIO.setmode(GPIO.BCM)
GPIO.setwarnings(False)
Motor1Forward = 17
Motor1Backwards = 18
GPIO.setup(Motor1Forward,GPIO.OUT)
GPIO.setup(Motor1Backwards,GPIO.OUT)
print('Feeding Lola')
# Makes the motor spin forward for x seconds
# James Well Beloved - 11 seconds = 28g food (RDA Portion = 52g/4kg cat or 61g/5kg cat)
# XXXX - X seconds - Xg food
GPIO.output(Motor1Forward, True)
GPIO.output(Motor1Backwards, False)
time.sleep(11)
print('Lola Fed!')
GPIO.output(Motor1Forward, False)
GPIO.output(Motor1Backwards, False)
GPIO.cleanup()
quit()
When I set export FLASK_APP=catfeeder.py and then flask run the service runs but nothing happens when I land on the page. I assume there is something wrong in the way I am calling things.
I guess it would be easiest if I just integrated the code from feed.py into catfeeder.py but I wasn't sure what the syntax would be for this and it felt like a messy way to go about it.
Thanks in advance!
you've imported the function but didn't actually invoke it as you missed adding your brackets (), try return start()
In case you meant to return a function object and not invoke the function, you should return it by typing return start
A data file was getting corrupted when I terminated the program and realised that it was never properly closed.
It is quite critical that it does not get corrupted. So I added a statement to close the file.
Now, it seems like the file gets opened twice and then closed. That's one operation too many. There are of course many read-write operations in-between but it should only open and close the files once.
Here is what I have done to the standarize web.py template:
import web
import pandas as pd
store = pd.HDFStore('data_file.h5')
urls = (
'/', 'index'
)
class index:
def __init__(self):
self.__df = store['df']
def GET(self):
# several read-write, and modify operations on self.__df
return "Hello, world!"
if __name__ == "__main__":
try:
app = web.application(urls, globals())
app.run()
finally:
store.close()
Now, if I move the line which opens the store down inside the try statement at the bottom, it complains since it compiles the class but I can't find the variable store.
I tried initialising store with None at the top but it didn't work either. Then I tried putting that line up at the top in the function and calling it from the bottom, however, that didn't bring it into scope.
I was thinking of making it a global variable, which would probably do the trick, is that the right approach?
See web.py running twice. As mentioned there, avoid using globals as they don't do what you think they do... app.py runs twice, once on startup and a second time within web.appplication(urls, globals()). If you set autoreload=False in web.applications() call, it won't load the file twice.
Another solution is to attach your store to web.config, which is globally available.
if __name__ == "__main__":
try:
web.config.store = pd.HDFStore('data_file.h5')
app = web.application(urls, globals())
app.run()
finally:
web.config.store.close()
...and reference that global in your __init__
class index:
def __init__(self):
self.__df = web.config.store['df']
I come from Java background and most of my thinking comes from there. Recently started learning Python. I have a case where I want to just create one connection to Redis and use it everywhere in the project. Here is how my structure and code looks.
module: state.domain_objects.py
class MyRedis():
global redis_instance
def __init__(self):
redis_instance = redis.Redis(host='localhost', port=6379, db=0)
print("Redus instance created", redis_instance)
#staticmethod
def get_instance():
return redis_instance
def save_to_redis(self, key, object_to_cache):
pickleObj = pickle.dumps(object_to_cache)
redis_instance.set(key, pickleObj)
def get_from_redis(self, key):
pickled_obj = redis_instance.get(key)
return pickle.loads(pickled_obj)
class ABC():
....
Now I want to use this from other modules.
module service.some_module.py
from state.domain_objects import MyRedis
from flask import Flask, request
#app.route('/chat/v1/', methods=['GET'])
def chat_service():
userid = request.args.get('id')
message_string = request.args.get('message')
message = Message(message_string, datetime.datetime.now())
r = MyRedis.get_instance()
user = r.get(userid)
if __name__ == '__main__':
global redis_instance
MyRedis()
app.run()
When I start the server, MyRedis() __init__ method gets called and the instance gets created which I have declared as global. Still when the service tries to access it when the service is called, it says NameError: name 'redis_instance' is not defined I am sure this is because I am trying to java-fy the approach but not sure how exactly to achieve it. I read about globals and my understanding of it is, it acts like single variable to the module and thus the way I have tried doing it. Please help me clear my confusion. Thanks!
I'm dealing with global variables in Python. The code should work fine, but there is a problem. I have to use global variable for instance of class Back. When I run the application it says that back is None which should be not true because the second line in setup() function - 'back = Back.Back()'
# -*- coding: utf-8 -*-
from flask import Flask
from flask import request
from flask import render_template
import Search
import Back
app = Flask(__name__)
global back
back = None
#app.route('/')
def my_form():
return render_template('my-form.html')
def setup():
global back
back = Back.Back()
def is_ascii(s):
return all(ord(c) < 128 for c in s)
#app.route('/', methods=['POST'])
def search():
from time import time
pattern = request.form['text']
startTime = time()
pattern=pattern.lower()
arr = []
if len(pattern)<1:
arr.append('Incorrect input')
currentTime = time()-startTime
return render_template('my-form.html', arr=arr, time=currentTime)
arr = []
search = Search.Search(pattern,back)
results = search.getResults()
..................
return render_template('my-form.html', arr=arr, time=currentTime, pattern=pattern)
app.debug=True
if __name__ == '__main__':
setup()
app.run()
Why is the back variable None instead of instance of Back class? Thanks
The Flask development server runs your module twice. Once to run the server itself, and another time in a child process so it can reload your whole script each time you make a change to it. It is that second process that won't run the __main__ guarded code and the global is left as None.
You'll get the same problem if you used another WSGI server; it'd import your file as a module, not as the initial script and the __main__ guard is not executed.
Use a #app.before_first_request function instead; it is guaranteed to be executed when the very first request is handled for this process. This also keeps your global working if you moved to a proper WSGI container that used multiple child processes to scale your site:
#app.before_first_request
def setup():
global back
back = Back.Back()
the last few weaks, I am playing a little bit with the Web.py framework. As my application is now getting bigger and bigger, I want to restructure the sourcecode and put code fragments in different classes. Now, I don't really know where I should create my object instances if I need them in different web.py classes. Let us assume, my sourcecode looks like:
import web
import myclass
urls = (
'/', 'index',
'/test', 'test'
)
#should i make my instance global...
my = myclass.myClass()
class test:
def __init__(self):
#...or should i make my instance local: my = myclass.myClass()
pass
def GET(self):
item = my.getItem()
return item
def POST(self):
pass
class index:
def __init__(self):
#...or should i make my instance local: my = myclass.myClass()
pass
def GET(self):
date = my.getDate()
return date
if __name__ == "__main__":
app = web.application(urls, globals())
app.run()
Now, I want to access the methods getItem() and getDate() (which belong to the instance my), if the appropriate sites in my webbrowser are called. My question is now: Should I make the instance global or is it better, if I make it local? I really don't like global instances, but I don't see any other way as to make it global. Sure, it would be possible, to create a local instance, but then, every time the page loads, a new instance would be created, right? Normally, this wouldn't be a problem, but myclass accesses a serial port, so I need to make sure, that only one instance is created.
Am I missing something or is a global instance the only possible solution to accomplish this?
After some research, I came to the conclusion, that global instances are the way to go here. However, one must be careful with global instances if they are used together with the web.py auto reload mode. In auto reload mode, a global instance is created every time a new page loads. If you want to avoid that, you have to use something like this:
import web
import serial
urls = ("/(.*)", "test"
)
web.config.debug = True
if(web.config.debug):
#in debug mode, make sure that global serial instance is only created at start up
if not hasattr(serObj, "_email"):
serObj = serial.Serial(0, 9600, parity=serial.PARITY_NONE)
web._serObj = serObj
else:
serObj = web._serObj
class test:
def GET(self):
return "Test"
def POST(self):
pass
if __name__ == "__main__":
app = web.application(urls, globals())
app.run()