I have a function called getTableData() which runs another function get_table() and based on that get_table() output final function is called which renders a template and also routes to a different page.
So the problem is its not routing to a different url (/tabdata) from get_final() function
Flask code:
#app.route('/api/getTableData', methods=['POST'])
def getTableData():
value = request.json['value']
value=value[:8]
url="https://some.com"+value
df_time=get_table(url)
return get_final(df_time)
def get_table(url):
driver = webdriver.Chrome(options=options)
driver.get(url)
abv = pd.read_html(driver.find_element(By.ID,"frm_hist").get_attribute('outerHTML'))[0]
df_time = pd.DataFrame(abv)
return df_time
#app.route("/tabdata")
def get_final(df_time):
return render_template("new.html",df_time = df_time)
Code Explanation:
I am using the value from value variable then concat 2 strings to make the url and then passing the url to another function named get_table() which goes to that url and webscrapes the table and converts it into python dataframe.
So using the returned python dataframe get_final() is called to render the template in a html file and also route to the /tabdata url. Everything is working well except the page is not routing to that url
You have to return a redirect:
from flask import redirect
#app.route("/tabdata/<df_time>")
def get_final(df_time):
return redirect("http://www.example.com", code=200)
Use redirect and use it with url_for in case you decide to change your routes in the future. You also need to change your view function get_final
from flask import redirect, url_for
#app.route('/api/getTableData', methods=['POST'])
def getTableData():
value = request.json['value']
value = value[:8]
url = "https://some.com"+value
df_time = get_table(url)
return redirect(url_for('get_final', df_time=df_time))
def get_table(url):
driver = webdriver.Chrome(options=options)
driver.get(url)
abv = pd.read_html(driver.find_element(By.ID,"frm_hist").get_attribute('outerHTML'))[0]
df_time = pd.DataFrame(abv)
return df_time
#app.route("/tabdata/<df_time>") # notice change here!
def get_final(df_time):
return render_template("new.html", df_time=df_time)
In getTableData(), change
return get_final(df_time)
to
return redirect(url_for("get_final", df_time=df_time))
In get_final(), change
#app.route("/tabdata")
def get_final(df_time):
return render_template("new.html",df_time = df_time)
to
#app.route("/tabdata/<df_time>")
def get_final(df_time):
return render_template("new.html", df_time=df_time)
Although your redirected URL will look something like this; "http://localhost/tabdata/16606505". If this is not preferred, you can always redirect for GET request with query parameters (which will look like this; "http://localhost/tabdata&data=16606505") or redirect for POST request which will not show the df_time parameter in browser history.
Related
This question already has an answer here:
Capture arbitrary path in Flask route
(1 answer)
Closed 1 year ago.
I am trying to pass a website as a parameter. It works if the website does not have a "/" in it. For example: http://192.168.1.156:2434/www.cookinglight.com scrapes cooking light for all the images on it's page; however, if I pass in http://192.168.1.156:2434/https://www.cookinglight.com/recipes/chicken-apple-butternut-squash-soup then an I get an invalid response. Here is my current code:
import json
from flask import Flask, render_template
from imagescraper import image_scraper
app = Flask(__name__)
#app.route("/", methods = ['GET'])
def home():
return render_template('index.html')
#app.route("/<site>", methods = ['GET'])
def get_image(site):
return json.dumps(image_scraper(site))
if __name__ == '__main__':
app.run(host='0.0.0.0', port=2434, debug=True)
import requests
from bs4 import BeautifulSoup
def image_scraper(site):
"""scrapes user inputed url for all images on a website and
:param http url ex. https://www.cookinglight.com
:return dictionary key:alt text; value: source link"""
search = site.strip()
search = search.replace(' ', '+')
website = 'https://' + search
response = requests.get(website)
soup = BeautifulSoup(response.text, 'html.parser')
img_tags = soup.find_all('img')
# create dictionary to add image alt tag and source link
images = {}
for img in img_tags:
try:
name = img['alt']
link = img['src']
images[name] = link
except:
pass
return images
I tried urrllib but did not have any success. Any help would be greatly appreciated! I am a student so still learning!!
UPDATE:
I believe this is the issue as described in the stackoverflow post
Need to allow encoded slashes on Apache
Flask uses / as separate between arguments in url - so you can create route("/<arg1>/<arg2>/<arg3>") (or popular in blogs route("/<year>/<month>/<day>")) and you can get values in variables arg1, arg2, arg3 - and when you try to use your url with / then it also treat it as "/<arg1>/<arg2>/<arg3>" and it tries to find route like route("/<arg1>/<arg2>/<arg3>") and it can't find it and it gives error 404.
route("/<site>") can match only string without /. site is only variable name - it doesn't mean that it will treat it as url with /
If you want to use / as part of single argument, not as separator between arguments, then you need <path:site>.
from flask import Flask
app = Flask(__name__)
#app.route("/")
def home():
return "Hello World"
#app.route("/<path:site>")
def get_image(site):
return f"OK: {site}"
if __name__ == '__main__':
app.run(host='0.0.0.0', port=2434)#, debug=True)
See also Variable Rules
EDIT:
It has nothing to do with issue. Flask was specially created to use / as special char to separate values.
I'm writing a script to collect the emails of those users that didn't receive an email confirmation email and resend it to them. The script works obviously outside of flask app context. I would like to use url_for() but can't get it right.
def resend(self, csv_path):
self.ctx.push()
with open(csv_path) as csv_file:
csv_reader = csv.reader(csv_file)
for row in csv_reader:
email = row[0]
url_token = AccountAdmin.generate_confirmation_token(email)
confirm_url = url_for('confirm_email', token=url_token, _external=True)
...
self.ctx.pop()
The first thing I had to do was to set SERVER_NAME in config. But then I get this error message:
werkzeug.routing.BuildError: Could not build url for endpoint
'confirm_email' with values ['token']. Did you mean 'static' instead?
This is how it's defined, but I don't think it can even find this, because it's not registered when ran as script:
app.add_url_rule('/v5/confirm_email/<token>', view_func=ConfirmEmailV5.as_view('confirm_email'))
Is there a way to salvage url_for() or do I have to build my own url?
Thanks
It is much easier and proper to get the URL from the application context.
You can either import the application and manually push context with app_context
https://flask.palletsprojects.com/en/2.0.x/appcontext/#manually-push-a-context
from flask import url_for
from whereyoudefineapp import application
application.config['SERVER_NAME'] = 'example.org'
with application.app_context():
url_for('yourblueprint.yourpage')
Or you can redefine your application and register the wanted blueprint.
from flask import Flask, url_for
from whereyoudefineyourblueprint import myblueprint
application = Flask(__name__)
application.config['SERVER_NAME'] = 'example.org'
application.register_blueprint(myblueprint)
with application.app_context():
url_for('myblueprint.mypage')
We can also imagine different ways to do it without the application, but I don't see any adequate / proper solution.
Despite everything, I will still suggest this dirty solution.
Let's say you have the following blueprint with the following routes inside routes.py.
from flask import Blueprint
frontend = Blueprint('frontend', __name__)
#frontend.route('/mypage')
def mypage():
return 'Hello'
#frontend.route('/some/other/page')
def someotherpage():
return 'Hi'
#frontend.route('/wow/<a>')
def wow(a):
return f'Hi {a}'
You could use the library inspect to get the source code and then parse it in order to build the URL.
import inspect
import re
BASE_URL = "https://example.org"
class FailToGetUrlException(Exception):
pass
def get_url(function, complete_url=True):
source = inspect.getsource(function)
lines = source.split("\n")
for line in lines:
r = re.match(r'^\#[a-zA-Z]+\.route\((["\'])([^\'"]+)\1', line)
if r:
if complete_url:
return BASE_URL + r.group(2)
else:
return r.group(2)
raise FailToGetUrlException
from routes import *
print(get_url(mypage))
print(get_url(someotherpage))
print(get_url(wow).replace('<a>', '456'))
Output:
https://example.org/mypage
https://example.org/some/other/page
https://example.org/wow/456
I want to send data to html in flask framework, i have a function which receives dictionary as a parameter, then there are several functions applied on dictionary. after that final result i want to render in to html page. Its been 48 hours i am trying from different blogs but didn't get precise solution.
imports ...
from other file import other_functions
from other file import other_functions_2
from other file import other_functions_3
app = Flask(__name__, template_folder='templates/')
#app.route("/dashboard")
def calculate_full_eva_web(input:dict):
calculate_gap = other_functions(input)
calculate_matrix = other_functions_2(input)
average = other_functions_3(input)
data = dict{'calculate_gap':calculate_gap, 'calculate_matrix':calculate_matrix,'average':average}
return render_template('pages/dashboard.html', data = data)
if __name__ == "__main__":
app.run(debug=True)
TRACEBACK
Methods that Flask can route to don't take dictionaries as inputs, and the arguments they do take need to be matches by a pattern in the route. (See https://flask.palletsprojects.com/en/1.1.x/api/#url-route-registrations)
You'd get the same error if you changed
#app.route("/dashboard")
def calculate_full_eva_web(input:dict):
to
#app.route("/dashboard")
def calculate_full_eva_web(input):
Your path forward depends on how you want to pass data when you make the request. You can pass key/value pairs via URL parameters and retrieve them via the request.args object. That might be close enough to what you want. (You'll need to remove the argument declaration from calculate_full_eva_web())
Something like
from flask import request
#app.route('/dashboard')
def calculate_full_eva_web():
input = request.args
...
I need to enable some pages to write an arbitrary URL that does not depend on the structure of the site.
For example I have structure:
/
/blog
/blog/blogpost1
/blog/blogpost2
But, for example, I need change url from /blog/blbogpost2 to /some/blogpost/url1
For this, I decided to give the opportunity to handle any URL of the main page of the site.
class IndexPage(RoutablePageMixin, Page):
...
#route(r'^(?P<path>.*)/$')
def render_page_with_special_path(self, request, path, *args, **kwargs):
pages = Page.objects.not_exact_type(IndexPage).specific()
for page in pages:
if hasattr(page, 'full_path'):
if page.full_path == path:
return page.serve(request)
# some logic
But now, if this path is not found, but I need to return this request to the standard handler. How can I do it?
This isn't possible with RoutablePageMixin; Wagtail treats URL routing and page serving as two distinct steps, and once it's identified the function responsible for serving the page (which, for RoutablePageMixin, is done by checking the URL route given in #route), there's no way to go back to the URL routing step.
However, it can be done by overriding the page's route() method, which is the low-level mechanism used by RoutablePageMixin. Your version would look something like this:
from wagtail.core.url_routing import RouteResult
class IndexPage(Page):
def route(self, request, path_components):
# reconstruct the original URL path from the list of path components
path = '/'
if path_components:
path += '/'.join(path_components) + '/'
pages = Page.objects.not_exact_type(IndexPage).specific()
for page in pages:
if hasattr(page, 'full_path'):
if page.full_path == path:
return RouteResult(page)
# no match found, so revert to the default routing mechanism
return super().route(request, path_components)
I have a 3 level json file. I am fetching the values of some of the attributes from each of the 3 levels of json. At the moment, the execution time of my code is pathetic as it is taking about 2-3 minutes to get the results on my web page. I will be having a much larger json file to deal with in production.
I am new to python and flask and haven't done much of web programming. Please suggest me ways I could optimise my below code! Thanks for help, much appreciated.
import json
import urllib2
import flask
from flask import request
def Backend():
url = 'http://localhost:8080/surveillance/api/v1/cameras/'
response = urllib2.urlopen(url).read()
response = json.loads(response)
components = list(response['children'])
urlComponentChild = []
for component in components:
urlComponent = str(url + component + '/')
responseChild = urllib2.urlopen(urlComponent).read()
responseChild = json.loads(responseChild)
camID = str(responseChild['id'])
camName = str(responseChild['name'])
compChildren = responseChild['children']
compChildrenName = list(compChildren)
for compChild in compChildrenName:
href = str(compChildren[compChild]['href'])
ID = str(compChildren[compChild]['id'])
urlComponentChild.append([href,ID])
myList = []
for each in urlComponentChild:
response = urllib2.urlopen(each[0]).read()
response = json.loads(response)
url = each[0] + '/recorder'
responseRecorder = urllib2.urlopen(url).read()
responseRecorder = json.loads(responseRecorder)
username = str(response['subItems']['surveillance:config']['properties']['username'])
password = str(response['subItems']['surveillance:config']['properties']['password'])
manufacturer = str(response['properties']['Manufacturer'])
model = str(response['properties']['Model'])
status = responseRecorder['recording']
myList.append([each[1],username,password,manufacturer,model,status])
return myList
APP = flask.Flask(__name__)
#APP.route('/', methods=['GET', 'POST'])
def index():
""" Displays the index page accessible at '/'
"""
if request.method == 'GET':
return flask.render_template('index.html', response = Backend())
if __name__ == '__main__':
APP.debug=True
APP.run(port=62000)
Ok, caching. So what we're going to do is start returning values to the user instantly based on data we already have, rather than generating new data every time. This means that the user might get slightly less up to date data than is theoretically possible to get, but it means that the data they do receive they receive as quickly as is possible given the system you're using.
So we'll keep your backend function as it is. Like I said, you could certainly speed it up with multithreading (If you're still interested in that, the 10 second version is that I would use grequests to asynchronously get data from a list of urls).
But, rather than call it in response to the user every time a user requests data, we'll just call it routinely every once in a while. This is almost certainly something you'd want to do eventually anyway, because it means you don't have to generate brand new data for each user, which is extremely wasteful. We'll just keep some data on hand in a variable, update that variable as often as we can, and return whatever's in that variable every time we get a new request.
from threading import Thread
from time import sleep
data = None
def Backend():
.....
def main_loop():
while True:
sleep(LOOP_DELAY_TIME_SECONDS)
global data
data = Backend()
APP = flask.Flask(__name__)
#APP.route('/', methods=['GET', 'POST'])
def index():
""" Displays the index page accessible at '/'
"""
if request.method == 'GET':
# Return whatever data we currently have cached
return flask.render_template('index.html', response = data)
if __name__ == '__main__':
data = Backend() # Need to make sure we grab data before we start the server so we never return None to the user
Thread(target=main_loop).start() #Loop and grab new data at every loop
APP.debug=True
APP.run(port=62000)
DISCLAIMER: I've used Flask and threading before for a few projects, but I am by no means an expert on it or web development, at all. Test this code before using it for anything important (or better yet, find someone who knows that they're doing before using it for anything important)
Edit: data will have to be a global, sorry about that - hence the disclaimer