On a higher level -
Using Jenkins (by creating a Job) is it possible to fetch data from database and display extracted data in that job or attach as artifact for that job ?
One options which was considered -
Create a job with build step being 'execution of Python script'. This scripts connects to database and fetch data.
c = sqlite3.connect('db_path.db')
cur = c.cursor()
cur.execute("SELECT column_name from table_name LIMIT 11")
test = cur.fetchall()
And dump the results in html format using a template or something like that
<table>
{% for row in test %}
<tr>
<th> {{ row[0] }}</th>
</tr>
{% endfor %}
</table>
But the question here is, how do I display this html file in Jenkins job or attach as artifact, considering this job will be used multiple times for various jobs.
I could use HTML Publisher Plugin, but the limitation is that the in my case the html name cannot be static. And I don't think this will work in my case and I don't know how to link html to python fetchall().
Can anyone please help me with this. Is there any other/better option to perform this activity ?
Question 1: How could I establish connection from fetchAll() to the html page ?
Question 2: I saw in HTML Publisher Plugin that index.html is the html page used to publish. Will this page be a standard template ? or everytime I run a job with different configurations will this page be different ?
I don't know how to do it !!! :(
You can archive the html report for each run even as it runs concurrently. If the job is dynamic you can also add a description for the job so each run will make sense to you using the Description plugin. In the job configuration you can define how many instances you'd like to keep, how many with artifacts, etc.
In general your process sounds ok for what you want to do, where is the limitation?
To fill data into HTML in Python you can use Mako.Templates.
Quick example:
from mako.template import Template
from mako.lookup import TemplateLookup
from mako.runtime import Context
def fill_template(template, data):
lookup = TemplateLookup(directories=["templates"])
mytemplate = Template(filename=template, lookup=lookup)
buf = io.StringIO()
ctx = Context(buf, data=data)
mytemplate.render_context(ctx)
htm = buf.getvalue()
print(htm)
file = open("Report/index.html", "wb")
file.write(bytes(htm.encode('utf-8')))
file.close()
return 0
fdata = {'test':data}
fill_template("templates/digest.html", fdata)
Template file:
<table style="border-bottom:solid black 1pt; cellspacing:5%">
% for test in sorted(data['test']):
<tr>
.... Do something with test data (depends on how you store it) ...
</tr>
% endfor
</table>
Related
I am writing a tool to record and monitor downtime on a range of equipment.
I have my file structure as below:
File Structure
Sites is just a subfolder containing individual HTMLS for where the equipment is located.
Currently, flask runs webapp.py which contains:
>from . import app
>#app = (__init__.app)
>from . import views
>from . import ReportingTool
views.py has all of my #app.route's in it, up until the [site].html files. From there, on the [site].html file I ask for input from the user. I haven't started writing code to record the user input in any meaningful way, just want to get the data to a python script and commit them to variables. To this end, in the html file I have
<body>
<div class="menu">
<form method="post" enctype="multipart\form-data" action="{{ url_for('downTime') }}">
<fieldset class="datafieldset">
This then requests different data from the user in the form of multiple field sets as seen here: fieldsets
as you see in the code snippet above I set the action to be url_for('downTime'), downTime is a function in my python file ReportingTool.py. this throws out an error, "werkzeug.routing.exceptions.BuildError: Could not build url for endpoint 'downTime'. Did you mean 'supportguide' instead?" traceback
Is there something I need to add or specify on the html document to enable this page (and the other [site].html pages to call functions from the ReportingTool.py file? the #app.route that calls the [site].html file is this and that is called with a redirected from here I've only got it setup like that becuase I wanted the name for the site to appear in the address bar.
Thanks in advance.
I am not sure on steps to fix as I am kind of throwing myself in the deep end to learn basic coding by creating an application for my workplace to replace an excel spreadsheet I created.
You are not reaching the downTime function in the ReportingTool.py file. I suggest trying add_url_rule in your views.py by adding the /reported endpoint referencing the downTime function in ReportingTool.py. Something like this;
app.add_url_rule('/reported', 'ReportingTool.downTime', view_func=ReportingTool.downTime, methods=METHODS)
This answer is based on the responds for this question. You are trying to reach a function in a different file from your main view file. Assuming you are calling the page with the form from a function in the views.py file.
Solved with info from Kakedis' input, and the links they provided.
I added:
app.add_url_rule('/reported', 'ReportingTool.downTime', view_func=ReportingTool.downTime, methods=METHODS)
to webbapp.py, then:
#app.route('/reported')
def downTime():
try:
DTref = request.form['refDT']
except:
DTref = "No Reference"
print(DTref)
print("reported")
return(render_template("/UserRip.html"))
to ReportingTool.py
This now prints the above to console to confirm it's pulling the correct func and brings the user back to the starting page.
I am making a website that tracks population statistics. The site needs to update about every 5 seconds with the latest information.
Here is the relevant code for displaying the pandas df on the page (in a file titled "home.html"):
{% block content %}
<h1>Population Tracker</h1>
{% for index, label,data in pop_df.itertuples %}
<div class = "row__data">
<p>{{label}}: {{data}}</p>
</div>
{% endfor %}
{% endblock content %}
Here is the code for my scraper (in a separate file called "scraper.py")
class Scraper():
def __init__(self):
self.URL = "https://countrymeters.info/en/Japan"
def scrape(self):
"Scrapes the population data once"
page = requests.get(self.URL)
soup = BeautifulSoup(page.content,'html.parser')
data_div = soup.find_all('div',class_="data_div")[0]
table = data_div.findAll("table")[0]
tr = table.findAll('tr')
labels = []
numbers = []
for n, i in enumerate(tr):
number = i.findAll('td',{"class":"counter"})[0].text # numbers
label = i.findAll('td',{"class":"data_name"})[0].text # labels
labels.append(label)
numbers.append(number)
pop_df = pd.DataFrame(
{
'Labels':labels,
'Data': numbers
}
)
return pop_df
In my views.py file, here is what I have done:
from django.shortcuts import render
from bsoup_tracker.scraper import Scraper
scraper = Scraper()
df = scraper.scrape()
def home(request):
context = {
'pop_df':df
}
return render(request,'tracker/home.html',context)
Basically, I would like to be able to call the render onto my home.html page every 5 seconds to reupdate the page, without needing refreshes. I have tried to look elsewhere and see that Ajax could help; however I do not know where to begin.
Instead of using Django to render the page, create API and call every after 5 minutes and after getting the results, refresh the HTML content using JavaScript.
If you need more information please let me know.
AJAX stands for "asynchronous JavaScript and XML" so as you thought that would be the way to go if you need to fetch data from your backend and refresh the interface.
The base component to do so in the XmlHttpRequest object in vanilla JavaScript. However, I strongly advice using a library like jQuery, to me it's really easier to use. With vanilla JS, jQuery or any other library you choose, you can modify DOM to expose data you got from your backend. The major drawback is that you will probably end up with not so clean code which will get harder and harder to maintain.
Nowadays the most common solution would be to use djangorestframework (not mandatory, you can also use django's JsonResponse) to create an API along with a nodeJS framework like React or VueJS to create your interface using API's data. That way you will have a lot more control on your interface.
Finally, if you need to have some sort of live website (pulling data and refreshing interface every 5 seconds seems like a poor design pattern to me), you should use websockets for your frontend and ASGI in backend (instead of WSGI). Django-channel is a nice package to do so, but just Google "django websockets" and you will find a lot of documentation.
This question already has answers here:
Sending data from HTML form to a Python script in Flask
(2 answers)
Get the data received in a Flask request
(23 answers)
Closed 2 years ago.
So here's what i'm trying to do:
Static Website in HTML(just a simple table) Screenshot Here
Admin Page to update text's on that website by html for (I'm not sure if it's good idea) Another Screenshot
I was trying in Python Flask, but I have no idea how to storage data from HTML Form into Python variables. All I did is to use {{ }} in my index.html(first ss) and then I can edit content straight from Python Code, but it's not what I want.
What i want?
I want to make a form or smth like that which allows user to easy update text on index(first ss).
User has to choose which cell he wants to update and write a new content, then submit and that's all.
Is it Python Flask good idea to do it? (I'm pretty new in Python)
At the moment I'm using Wordpress+PostGrid plugin to do something like that, but editing the grid(first ss) is there so hard.
I'm stubborn that's why I want to do it on my self.
Any advises about method which I should use or language will be very helpful.
Thanks!
Since you do NOT want to render a template of your table website, but change the HTML file itself, I have the following proposition for you:
Set up a Flask application to receive your request, e.g. on an update route (/update).
Have your admin website use a form that posts to your update route. Make sure to give your select and input tags the name attribute. For example:
<form action="{{url_for('update')}}" method="post">
<select name="element">
<option value="imptitle">imptitle</option>
<option value="zaktitle">zaktitle</option>
</select>
<input type="text" name="new_value">
<input type="submit" value="Update">
</form>
In your Flask app you can extract the information from the request by using request.form, using the same names that you specified in your select and input tags, e.g.:
#app.route('/update', methods=['POST'])
def update():
element = request.form['element']
new_value = request.form['new_value']
Then replace the element's value within your table HTML: open your table HTML file, read its content, search the element and replace its value using regular expression (you can read up on how to do that here), overwrite the contents of the file with updated content. For example:
with open('index.html', 'r') as f:
content = f.read()
# new_content = ... (do your search and replace here)
with open('index.html', 'w') as f:
f.write(new_content)
This question already has answers here:
Passing HTML to template using Flask/Jinja2
(7 answers)
Closed 5 years ago.
I am seeing a strange issue when I attempt to write some data to a HTML table, using Flask. When the page loads, it is printing the data into a single long string as below:
<table class="table table-striped table-hover"><tr><th> Task </th><th> duration </th><th> points </th></tr><tr><td>50 Pullups</td><td>15</td><td>0.5000</tr><tr><td>5K Run</td><td>45</td><td>1.0000</tr><tr><td>Abs of Steel</td><td>30</td><td>1.0000</tr><tr><td>Data Science Study</td><td>60</td><td>1.0000</tr><tr><td>Drums</td><td>30</td><td>1.0000</tr><tr><td>Kegel</td><td>10</td><td>1.0000</tr><tr><td>Metta Bhavna</td><td>25</td><td>1.0000</tr><tr><td>Mindfulness</td><td>30</td><td>1.0000</tr><tr><td>Physio</td><td>10</td><td>1.0000</tr><tr><td>Singing</td><td>30</td><td>1.0000</tr><tr><td>Skipping</td><td>15</td><td>0.0000</tr><tr><td>Sprint</td><td>20</td><td>1.0000</tr><tr><td>Typing</td><td>10</td><td>1.0000</tr><tr><td>Yoga</td><td>15</td><td>1.0000</tr></table>
However, obviously it would be great to have it render as a table. Note that when I copy that HTML code into the page source, it renders fine (so no issue with the HTML in and of itself.)
I have set up my environment as below: NB fetch_results is a script querying a mysql db and returning the results in a string format.
import os
from flask import Flask, render_template
#app.route('/')
def table_maker():
from myfunctions import fetch_results
table = fetch_results()
return render_template('home.html', table = table)
The page source contains:
<div class="container">
{{ table }}
</div>
Help much appreciated.
Use {{ table|safe }}.
safe is one of the Jinja2 built in filter you can use that to solve your problem.
The safe filter explicitly marks a string as "safe", i.e., it should not be automatically-escaped if auto-escaping is enabled.
Using web.py, I'm creating a handful of result sets by using
web.database.query()
And I'd like to display the query results on an html template.
Does web.py have any built-in facility for doing this? I didn't see any going through the doc and code samples.
I'd also appreciate any pointers to other modules that will coerce the sql result set into something that can be displayed by a jquery or google data grid. I started working on a google data grid converter but I had to jump through some hoops to handle the various datatypes. I figure this has been done tons of times before - I'd just like to know where I should be looking.
Check out the docs for web.py's built-in templating system. There are also cookbook examples for hooking in 3rd party template engines like Mako, Cheetah and Jinja2.
In order to convert the data to be displayed by Javascript you could use one of the JSON modules for Python: the standard json module (2.6+), simplejson, or python-cjson.
Here is a quick example of using the web.template module to render a simple HTML page showing some database query results.
Updated The select method doesn't return the column names as a separate attribute. It returns an iterator where each row is a dictionary. Therefore, to get the column headers you need to grab then from the first row. I've updated the example to show how this can be done:
import web
TEMPLATE = '''$def with (rows, cols)
<html><body>
<h2>Results:</h2>
<table>
<tr>
$for col in cols:
<th>$col</th>
</tr>
$for row in rows:
<tr>
$for col in cols:
<td>$row[col]</td>
</tr>
</table></body></html>'''
class query:
def GET(self, arg):
res = db.select('player', what='id,name')
cols = []
rows = res.list()
if rows:
cols = rows[0].keys()
return tmpl(rows, cols)
db = web.database(dbn='mysql')
tmpl = web.template.Template(TEMPLATE)
urls = ('/(.*)', 'query')
if __name__ == '__main__':
app = web.application(urls, globals())
app.run()
Output (collapsed to save space):
<html><body>
<h2>Results:</h2>
<table>
<tr><th>id</th><th>name</th></tr>
<tr><td>1</td> <td>Joe</td></tr>
<tr><td>2</td><td>Gary</td></tr>
<tr> <td>3</td><td>Fred</td></tr>
</table></body></html>