Develop Reference

Python Flask Application not redirecting

I'm trying to create a list of clickable links to external URLs. My HTML file looks like this
<ul>
{% for post in posts %}
<li><a href={{url_for('go_to_reddit', url=post.url)}}>{{ post.title }}</a></li>
{% endfor %}
</ul>
and the python file like this
#app.route('/redirect/"<url>"')
def go_to_reddit(url):
print("Redirecting to ", url)
return redirect(url)
I keep getting the following error whenever I press on one of the links
Not Found
The requested URL was not found on the server. If you entered the URL manually please check your spelling and try again.
I know it's not reaching the redirect function even though it hits the URL since my console doesn't print anything.

I don't think you can have quotes in a route parameter i.e. #app.route('/redirect/"<url>"') should be #app.route('/redirect/<url>') . Put another way, try dropping the quote around <url>
If you can't have quotes in a url parameter and you need to pass a complete url, then you should consider passing it as a query parameter e.g. /redirect/?link=post.url. Then in your handler for the route, you will do url = request.values.get('link', None)

Django Running Python Function to Update Page w/o Refresh

I am making a website that tracks population statistics. The site needs to update about every 5 seconds with the latest information.
Here is the relevant code for displaying the pandas df on the page (in a file titled "home.html"):
{% block content %}
<h1>Population Tracker</h1>
{% for index, label,data in pop_df.itertuples %}
<div class = "row__data">
<p>{{label}}: {{data}}</p>
</div>
{% endfor %}
{% endblock content %}
Here is the code for my scraper (in a separate file called "scraper.py")
class Scraper():
def __init__(self):
self.URL = "https://countrymeters.info/en/Japan"
def scrape(self):
"Scrapes the population data once"
page = requests.get(self.URL)
soup = BeautifulSoup(page.content,'html.parser')
data_div = soup.find_all('div',class_="data_div")[0]
table = data_div.findAll("table")[0]
tr = table.findAll('tr')
labels = []
numbers = []
for n, i in enumerate(tr):
number = i.findAll('td',{"class":"counter"})[0].text # numbers
label = i.findAll('td',{"class":"data_name"})[0].text # labels
labels.append(label)
numbers.append(number)
pop_df = pd.DataFrame(
{
'Labels':labels,
'Data': numbers
}
)
return pop_df
In my views.py file, here is what I have done:
from django.shortcuts import render
from bsoup_tracker.scraper import Scraper
scraper = Scraper()
df = scraper.scrape()
def home(request):
context = {
'pop_df':df
}
return render(request,'tracker/home.html',context)
Basically, I would like to be able to call the render onto my home.html page every 5 seconds to reupdate the page, without needing refreshes. I have tried to look elsewhere and see that Ajax could help; however I do not know where to begin.

Instead of using Django to render the page, create API and call every after 5 minutes and after getting the results, refresh the HTML content using JavaScript.
If you need more information please let me know.

AJAX stands for "asynchronous JavaScript and XML" so as you thought that would be the way to go if you need to fetch data from your backend and refresh the interface.
The base component to do so in the XmlHttpRequest object in vanilla JavaScript. However, I strongly advice using a library like jQuery, to me it's really easier to use. With vanilla JS, jQuery or any other library you choose, you can modify DOM to expose data you got from your backend. The major drawback is that you will probably end up with not so clean code which will get harder and harder to maintain.
Nowadays the most common solution would be to use djangorestframework (not mandatory, you can also use django's JsonResponse) to create an API along with a nodeJS framework like React or VueJS to create your interface using API's data. That way you will have a lot more control on your interface.
Finally, if you need to have some sort of live website (pulling data and refreshing interface every 5 seconds seems like a poor design pattern to me), you should use websockets for your frontend and ASGI in backend (instead of WSGI). Django-channel is a nice package to do so, but just Google "django websockets" and you will find a lot of documentation.

How do I output plots from matplotlib to an html template in flask on a ubuntu server?

I am trying to set up a side project on DigitalOcean, and I am using the git framework from https://blog.miguelgrinberg.com/post/the-flask-mega-tutorial-part-xvii-deployment-on-linux to get started.
Within this framework, I have added code within one of the flask routes (/explore) in which I generate a plot with matplotlib, and I want to return this plot as an object when I render the template as the return function of this route. I don't need to save the plot if it can be sent to the template without doing so (e.g with io.BytesIO()), but I have been unable to get the syntax correct to use this approach and get the plot to render in the resulting template.
While my attempts with io.BytesIO() have been unsuccessful, if it would help to output the results with that approach, please let me know how to best utilize it, and I will attempt to run this code with the suggested changes and report the results.
Thank you in advance!
I have tried to save the file and send it to the template, as well as sending the file data via BytesIO(), but neither approach has worked for me.
Below is my attempt to save the file to the static directory and send the image to the template, but a solution that works in this environment with io.BytesIO() or similar without saving the file would be even better.
Here is the code that I added to the explore route in /app/main/routes.py to save the plot image to the static directory and return the path to the template:
new_graph_name = url_for('static', filename='tmp_png.png')
plt.savefig(new_graph_name)
return render_template('index.html', url=new_graph_name)
Here is the code that I added to the index.html template:
{% if url %}
<img src={{ url }} alt="Chart" height="42" width="42" />
{% endif %}

In terms of saving the plot and then displaying it, could you try something similar to the code the below? This has worked for me recently.
In routes.py:
#app.route("/")
def index():
new_graph_name = 'tmp_png'
plt.savefig('static/images/' + new_graph_name)
return render_template("index.html", new_graph_name=new_graph_name)
In index.html:
<img src="{{ url_for('static', filename='images/' + new_graph_name + '.png') }}"
With Bytes.IO I think I've tried something like this before:
In routes.py:
import io
from io import BytesIO
import base64
img = io.BytesIO()
fig.savefig(img)
img.seek(0)
buffer = b''.join(img)
b2 = base64.b64encode(buffer)
barplot=b2.decode('utf-8')
I cannot remember how I displayed it in the .html template but could it just be a matter of passing it as a variable?

Why might this link not be working...?

I'm rendering a bunch of posts on a page where a user can browse listings and click on one of them and be sent to a 'singles page' for more information on whatever product they clicked. This method works for every link EXCEPT for the first one.
Anytime I click on the very first link of the page, I get a Not Found: The requested URL was not found on the server. If you entered the URL manually please check your spelling and try again. error.
The logic I have in place for the HMTL/jinja is (everything is closed off properly, I'm cutting some unnecessary code for the sake of brevity):
{% set i = 0 %}
{% for row in data %}
{% set i = i + 1 %}
<a href="/iLike/{{ i }}">
<li>content</li>
</a>
and my python code:
#app.route('/iLike/<int:num>', methods=['GET','POST'])
def single2(num):
try:
loc = session.get('loc')
transType = session.get('transType')
data = singlesQuery()
return render_template('single.html', loc=loc,transType=transType,data=data[num-1])
except Exception as e:
return (str(e))

There is no need to build URLs manually. The best way it to use flask's built-in function url_for:
{{url_for('single2', num=i)}}
There is also no need for calculating the i manually, becaue there is built-in loop.index and loop.index0:
{% for row in data %}
<a href="{{url_for('single2', num=loop.index)}}">
I believe this should always create a valid URL.

Jenkins job to fetch data from database and display in jenkins

On a higher level -
Using Jenkins (by creating a Job) is it possible to fetch data from database and display extracted data in that job or attach as artifact for that job ?
One options which was considered -
Create a job with build step being 'execution of Python script'. This scripts connects to database and fetch data.
c = sqlite3.connect('db_path.db')
cur = c.cursor()
cur.execute("SELECT column_name from table_name LIMIT 11")
test = cur.fetchall()
And dump the results in html format using a template or something like that
<table>
{% for row in test %}
<tr>
<th> {{ row[0] }}</th>
</tr>
{% endfor %}
</table>
But the question here is, how do I display this html file in Jenkins job or attach as artifact, considering this job will be used multiple times for various jobs.
I could use HTML Publisher Plugin, but the limitation is that the in my case the html name cannot be static. And I don't think this will work in my case and I don't know how to link html to python fetchall().
Can anyone please help me with this. Is there any other/better option to perform this activity ?
Question 1: How could I establish connection from fetchAll() to the html page ?
Question 2: I saw in HTML Publisher Plugin that index.html is the html page used to publish. Will this page be a standard template ? or everytime I run a job with different configurations will this page be different ?
I don't know how to do it !!! :(

You can archive the html report for each run even as it runs concurrently. If the job is dynamic you can also add a description for the job so each run will make sense to you using the Description plugin. In the job configuration you can define how many instances you'd like to keep, how many with artifacts, etc.
In general your process sounds ok for what you want to do, where is the limitation?
To fill data into HTML in Python you can use Mako.Templates.
Quick example:
from mako.template import Template
from mako.lookup import TemplateLookup
from mako.runtime import Context
def fill_template(template, data):
lookup = TemplateLookup(directories=["templates"])
mytemplate = Template(filename=template, lookup=lookup)
buf = io.StringIO()
ctx = Context(buf, data=data)
mytemplate.render_context(ctx)
htm = buf.getvalue()
print(htm)
file = open("Report/index.html", "wb")
file.write(bytes(htm.encode('utf-8')))
file.close()
return 0
fdata = {'test':data}
fill_template("templates/digest.html", fdata)
Template file:
<table style="border-bottom:solid black 1pt; cellspacing:5%">
% for test in sorted(data['test']):
<tr>
.... Do something with test data (depends on how you store it) ...
</tr>
% endfor
</table>