Develop Reference

I am getting a `KeyError` when I receive a request to my API

I am building a dictionary app with Flask where users can add new words, I am trying to request the word from the word input , I am having issues with the POST request, the error I am receiving on my terminal is this:
line 50, in add_word
word = req['word']
keyError:'word'
and this is how I wrote the code in my app.py file:
#app.route('/word', methods= ['POST'])
def add_word():
req = request.get_json()
word = req['word']
meaning = req['meaning']
conn = mysql.get_db()
cur = conn.cursor()
cur.execute('insert into word(word, meaning) VALUES (%s, %s)',(word, meaning))
conn.commit()
cur.close()
return json.dumps("success")
here is the json in my JavaScript file, I am posting to my flask app:
$('#word-form').submit(function() {
let word = $('word').val();
let meaning = $('meaning').val();
$.ajax({
url: '/word',
type: 'POST',
dataType: 'json',
data : JSON.stringify({
'word': word,
'meaning': meaning
}),
contentType: 'application/json, charset = UTF-8',
success: function(data) {
location.reload();
},
error: function(err) {
console.log(err);
}
})
here is the Html page:
<div class="div col-md-2 sidenav">
All words
Add New
<div>
<form action="javascript:0" id="word-form">
<div class="form-group">
<label for="word">Word:</label>
<input type="text"
class="form-control"
name="word"
id="word"
placeholder="Type in the word here:"
required>
</div>
<div class="form-group">
<label for="Meaning">Meaning:</label>
<textarea class="form-control" id="meaning"
placeholder="enter the meaning here: " required></textarea>
</div>
<button type="submit" class="btn btn-primary btn-block btn-lg" id="submit">Submit</button>
<button type="button" class="btn btn-warning btn-block btn-lg" id="cancel">Cancel</button>
</form>
</div>
</div>
<div class="div col-md-10 main">
<table style="border: 2px;">
<thead>
<tr>
<th>SN</th>
<th>Word</th>
<th>Meaning</th>
<th></th>
<th></th>
</tr>
</thead>
<tbody>
{% for word in words %}
<tr>
<td>{{ loop.index }}</td>
<td>{{ word['word'] }}</td>
<td>{{ word['meaning'] }}</td>
<td><button class="btn btn-sm btn-success btn-block edit" id="{{word['id']}}">Edit</button></td>
<td><button class="btn btn-sm btn-danger btn-block delete" id="{{word['id']}}">Delete</button></td>
</tr>
{% else %}
<tr>
<td colspan="3">The dictionary has no words at the moment, please come bay later</td>
</tr>
{% endfor %}
</tbody>
</table>
</div>

Things seem to be in a confused state in the client code, and potentially the application architecture in general.
There are two general approaches to designing web apps that impacts how you create routes and build requests. One approach is the AJAX-based single page app architecture that loads one HTML skeleton, then uses JS (jQuery here) to make AJAX requests to a JSON API and injects the response data into the page using JS instead of page refreshes and navigations. Since you don't have a client-side router, this doesn't qualify as a SPA, but it's worth understanding to provide context on your design.
On the other hand, you can use HTML form submissions (<form action="/path/to/resource" method="POST">) and render_template to display new pages with a browser refresh for all form submissions.
The code here is somewhere in the middle, which is potentially fine (you can use AJAX for certain events and form submissions but mostly rely on full-navigation templates for routes). But it's important to be clear on the request-response workflow you're adopting so the design makes sense and can be debugged.
Here are few oddities/inconsistencies in your current design:
return json.dumps("success") is not really JSON as it seems like you want it to be--use jsonify and a dictionary or list, e.g. jsonify({"status": "success"}; it's customary for JSON routes to return JSON responses if they aren't rendering templates or redirecting.
The client ignores the JSON response and calls location.reload. If you're just planning on reloading and you have no special client processing to do, there's not much point in using AJAX or JSON here--just submit the form to the backend and redirect to the template or static HTML page you want to show next. No client-side JS involved. Redirect to an error page or render a template with errors shown on the form on error.
Links with href="#" are poor practice. Better to use buttons if you're adding JS to these handlers and you don't want them to trigger navigation. This is semantically more appropriate and doesn't hijack the URL.
<form action="javascript:0" id="word-form"> looks like it's trying to prevent the form submission, but all this does is replace the page content with the character "0". I can't imagine how this is useful or desirable. Submitting a form to a JSON route can produce the error you're seeing--another sign of confusion about which architecture you're following. Use event.preventDefault() (add the event parameter to the callback to .submit()) to prevent the form submission from refreshing the page.
After you've prevented the page refresh, you can debug the AJAX request.
When a route is complaining about missing keys, consider that objects with keys pointing to undefined disappear when serialized as JSON (undefined is not a thing in JSON):
const word = undefined;
const foo = 42;
const bar = "baz";
console.log({word, foo, bar}); /* =>
{
"word": undefined,
"foo": 42,
"bar": "baz"
}
*/
console.log(JSON.stringify({
word,
foo,
bar,
})); // => {"foo":42,"bar":"baz"}
If you add a console.log to see if your values are there (or print the JSON on the backend route before indexing into it), these values aren't defined:
let word = $('word').val();
let meaning = $('meaning').val();
console.log(word, meaning); // <-- undefined, undefined
Why? The reason is that these selectors are missing the # symbol prefix to denote an id. Without it, jQuery looks for <word></word> and <meaning></meaning> HTML elements that don't exist.
Change these lines to:
const word = $('#word').val();
const meaning = $('#meaning').val();
and now your request body should be ready to send.
Next problem: $.ajax's dataType key specifies the response type, not the request type. Use dataType: "json" to specify the appropriate request header to trigger the Flask handler to JSON parse the request body.
After these changes, things should work, with the caveat that it might be time for a rethink of your overall design, or at least a redesign of this route workflow.
A word of advice: work slowly and test all of your assuptions at each step. The code here shows many errors that are hard to debug because they're stacked on top of each other. Isolate and validate each behavior in your app. For example, when adding the jQuery submit handler and collecting the form values, print them to make sure they're actually there as you expected.
In case you're stuck, here's minimal, complete, runnable code you can reference.
app.py:
from flask import (
Flask, jsonify, render_template, request, url_for
)
app = Flask(__name__)
#app.route("/")
def index():
return render_template("index.html")
#app.post("/words/")
def words():
payload = request.get_json()
word = payload.get("word")
meaning = payload.get("meaning")
if word is None or meaning is None:
return (jsonify({
"error": "missing required keys `word` or `meaning`"
}), 400)
# handle db operation and report failure as above
return jsonify({"status": "success"})
if __name__ == "__main__":
app.run(debug=True)
templates/index.html:
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8" />
<script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/3.6.0/jquery.min.js"></script>
</head>
<body>
<form id="word-form">
<div>
<label for="word">Word:</label>
<input
name="word"
id="word"
placeholder="Type in the word here:"
required
>
</div>
<div>
<label for="meaning">Meaning:</label>
<textarea
name="meaning"
id="meaning"
placeholder="enter the meaning here: "
required
></textarea>
</div>
<button type="submit">Submit</button>
</form>
<script>
$('#word-form').submit(function (e) {
e.preventDefault();
const word = $('#word').val();
const meaning = $('#meaning').val();
console.log(word, meaning);
$.ajax({
url: "{{url_for('words')}}",
type: "POST",
dataType: "json",
contentType: "application/json",
data: JSON.stringify({word, meaning}),
success: function (data) {
console.log(data);
},
error: function (err) {
console.error(err);
}
});
});
</script>
</body>
</html>
See also: How to get POSTed JSON in Flask?
Here are a few additional notes that are somewhat tangential to the main issue but have to be mentioned.
You have <label for="Meaning"> but no name="meaning" element to shift focus to when clicked.
It's another antipattern to put ids on everything promiscuously. Only add ids to elements when they must have one because you're using it for something specific. Prefer classes, especially for styling.
On the backend, the code here is unsafe:
req = request.get_json()
word = req['word']
meaning = req['meaning']
If your client gives a bad request with missing values, you should detect that and return a 400/422 response (or similar) rather than crashing.
For example (from the above code snippet):
req = request.get_json()
word = req.get("word")
meaning = req.get("meaning")
if word is None or meaning is None:
return (jsonify({
"error": "missing required keys `word` or `meaning`"
}), 400)
Similarly, don't assume the database operation will succeed. Always check for errors and return an appropriate response to the client.
Resources are usually plural, not singular: words, users, posts.

How to show displayed db.query search results on same page in python

I have a html page (searchCustomer.html), which has a input field, where we input the string to search in the database, the records are fetched successfully, after we click the search button and are displayed in html . I want to show the details of the record clicked from the html , when the first column (which is href) is clicked in the same page (searchCustomer.html)
<form action="/getsearchresults" method="post">
<!-- <form action="/getsearchresults" method= methods=['GET', 'POST'] > -->
<Text text="Customer Name" id="text0"/>
<Input width="200px" name="custName" required placeholder="Please enter the customer name"/>
<button type="submit">Search Customer</button>
<button id="#btnID">TEst Customer</button>
</form>
<table border="1">
<tr><td><strong>Customer Id</strong></td><td><strong>Name</strong></td></tr>
{% for name1 in customers %}
<tr><td>{{name1.id}}</td><td>{{name1.name}}
</td></tr>
{% endfor %}
</table>
on the click of the href,I want to display the details of that record in the same page

Here is a possible solution
from flask import request, jsonify
#app.route('/DispCustomerInfo')
def get_customer_info():
name = request.args.get('query', False)
customer = {}
if name:
#1 Query database
#2 Assign result to customer
return jsonify(customer)
On the front side use Ajax to get customer data.
<td><a class="customername" href="#" data-name="{{ name1.name }}">{{name1.id}}</a></td>
$( document ).ready(function() {
$(".customername").on('click', function(){
var name = $(this).data('name');
$.get("/getReport", {'query': name}).done(function (data) {
// Use jquery to append customer data to a DOM element.
});
});
});

Django: Get a table into Django view

I have Django model that contains data ("Name" and "Status") for 10 people. I display this model in a DataTable with checkboxes and if I check a box it changes the Class into selected, e.g. <tr role="row" class="odd selected"
The user makes a selection from the DataTable and press a button ("Send selection by email"). The selection will go into a Django View. I will write code that filter out those contains a selected, obtain its PK and send an email with info.
However, I have problem in the first step, to get the DataTable object into the view.
A similar approach would have been to use a Multiselect, but here I want the look of a table and not a form box.
My attempted solution is to put form tags around the DataTable and then read the table to filter out all that are selected.
Problem I currently have is that I cannot get the DataTable into the views.py.
So my question is: How do I access the table in Django View?
I have tried both POST and GET within below, but request object returns as None
$('#datatable').DataTable({
responsive: true,
columnDefs: [{
orderable: false,
className: 'select-checkbox',
targets: 0
}],
select: {
style: 'multi',
selector: 'td:first-child',
// selectRow: true
},
order: [
[1, 'asc']
],
language: {
searchPlaceholder: 'Search...',
sSearch: '',
lengthMenu: '_MENU_ items/page',
}
});
<!DOCTYPE html>
<html lang="en">
<head>
<title>Bootstrap Example</title>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/4.3.1/css/bootstrap.min.css">
<script src="https://ajax.googleapis.com/ajax/libs/jquery/3.4.1/jquery.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/popper.js/1.14.7/umd/popper.min.js"></script>
<script src="https://maxcdn.bootstrapcdn.com/bootstrap/4.3.1/js/bootstrap.min.js"></script>
<script src="https://cdn.datatables.net/1.10.20/js/jquery.dataTables.min.js"></script>
<script src="https://cdn.datatables.net/select/1.3.1/js/dataTables.select.min.js"></script>
</head>
<body>
<div class="container">
<form method="post" action="{% url 'product-send2' %}">
{% csrf_token %}
<div class="table-wrapper">
<table class="form-control table display responsive" id='datatable' style="width:100%;">
<thead>
<tr>
<th>Send</th>
<th>Name</th>
<th>Status</th>
</tr>
</thead>
<tbody>
{% for object in object_list %}
<tr>
<td></td>
<td>{{ object.name }}</td>
<td>{{ object.status }}</td>
<tr>
{% endfor %}
</tbody>
<tfoot>
</tfoot>
</table>
</div>
<button type="submit" class="btn btn-primary">Submit</button>
</form>
</div>
</body>
</html>
In the Urls
path('product/send/', send_product2, name='product-send2')
and in the views
def send_product2(request):
recipients = request.POST.get('datatable')
print(recipients)
return HttpResponseRedirect('/product/list')
I have tried variants such as
recipients = request.GET.get('datatable')
and
recipients = request.POST.get('datatable', False)
I cannot get hold of the "object" and don't really know how to debug it.
In the end, it should work as a multiselect for the form (that will be sent by email).
I hope I can help with some hints how to progress.
Thanks!
When I print(request.__dict__) it looks like below
: <SimpleLazyObject: <function AuthenticationMiddleware.process_request.<locals>.<lambda> at 0x7f907ab5f680>>, '_messages': <django.contrib.messages.storage.fallback.FallbackStorage object at 0x7f907ab82b10>, '_body': b'csrfmiddlewaretoken=xxx_length=10', '_post': <QueryDict: {'csrfmiddlewaretoken': ['xxxxx'], 'datatable_length': ['10']}>, '_files': <MultiValueDict: {}>, 'csrf_processing_done': True}

you can not submit an entire table in a form, especially based on id tag. From the code you posted it looks like you configured your datatable without event handlers.
From: https://editor.datatables.net/examples/inline-editing/submitButton.html
dom: "Bfrtip",
ajax: "../php/staff.php",
columns: [
{
data: null,
defaultContent: '',
className: 'select-checkbox',
orderable: false
},
{ data: "first_name" },
{ data: "last_name" },
{ data: "position" },
{ data: "office" },
{ data: "start_date" },
{ data: "salary", render: $.fn.dataTable.render.number( ',', '.', 0, '$' ) }
],
order: [ 1, 'asc' ],
select: {
style: 'os',
selector: 'td:first-child'
},
buttons: [
{ extend: "create", editor: editor },
{ extend: "edit", editor: editor },
{ extend: "remove", editor: editor }
]
and then added a submit button that actually tries to pull all the elements with name tag in the form dom sub-tree and sends those to the back-end.
In order to achieve what you want you need to create a custom event handler in the front-end that is hooked to the checkbox(or use functions integrated in the DataTable plugin) then generate a form on the fly with javascript where you add the relevant information(the pk of the selected record/s I assume). Submitting the newly generated form will result in a valid POST request that you can handle in the view.
Given all above, I would recommend to avoid using such mechanics in general. POST/GET that are not REST based lead to the regeneration of the entire page which is a bad thing to do when dealing with large interactive tables.

How to login in a website with phython requests?

The registration of the website is for free: http://software.broadinstitute.org/gsea/login.jsp
I wrote this code to login the website based on some tutorials:
import requests
url = "http://software.broadinstitute.org/gsea/login.jsp"
# Fill in your details here to be posted to the login form.
payload = {
'j_username': 'xxx#gmail.com',
'j_password': 'password'
}
# Use 'with' to ensure the session context is closed after use.
s = request.session()
p = s.post(url, data=payload)
# print the html returned or something more intelligent to see if it's a successful login page.
print p, p.url, p.status_code
print 'is redirected: ', p.is_redirect
r = s.get("https://software.broadinstitute.org/gsea/msigdb/download_file.jsp?filePath=/resources/msigdb/6.2/msigdb_v6.2.xml")
# print r.text
print r, r.url, r.status_code
print 'is redirected: ', r.is_redirect
with open("lol.xml", "wb") as handle:
handle.write(r.content)
I am not sure if I have to fill in the password, because it is hidden?
The post command return me back a 200 OK, but I am still not logged in:
<Response [200]> http://software.broadinstitute.org/gsea/login.jsp 200
is redirected: False
<Response [200]> https://software.broadinstitute.org/gsea/login.jsp 200
is redirected: False
Ok, possible error sources are the payload with wrong dictionary keys.
The html code looks like this:
<form id="loginForm" name="loginForm" action="j_spring_security_check" method="POST">
<table border="0" class="bodyfont" cellpadding="5" cellspacing="5">
<tbody><tr>
<td colspan="2" align="left">Items marked with <font color="red">*</font> are required.</td>
</tr>
<tr>
<td colspan="2"> </td>
</tr>
<tr>
<td><h3>Email: <font color="red">*</font> </h3></td>
<td><input id="email" type="text" name="j_username" value="">
<input id="password" type="hidden" name="j_password" value="password"></td>
</tr>
<tr>
<td> </td>
<td><input type="button" name="login" value="login" style="margin-top:10px;" onclick="validateForm()"></td>
</tr>
</tbody></table>
</form>
Do I miss something? Why is not it logged in?

As i mentioned in my comment it is a really good first step to look at the logs in the Network tab of Chrome when you want to login somewhere with requests. Your code doesn't work because you simply use the wrong url for your post-request! You also have some typos in your code e.g.: request.session() instead of requests.session().
import requests
login_url = "http://software.broadinstitute.org/gsea/j_spring_security_check"
url = "http://software.broadinstitute.org/gsea/index.jsp"
payload = {
'j_username': 'a4702585#nwytg.net',
'j_password': 'password'
}
with requests.Session() as session:
login = session.post(login_url, data=payload)
req = session.get(url)
In general i also doubt that looking at the response status code is a good way of figuring out if the login worked or not.
open a session
sent the post request including the payload (to the right url)
make a get request to simulate the redirect which happens automatically in the browser (again something you learn easily by using chrome network tab)
You can replace the url with any subdomain you want obviously...

get_template_attribute not displaying data in html

i came across get_template_attribute in the flask api and thought of it as a great way to achieve partial page rendering.
so the goal was
create a 2 layer menu-page layout
menu should be static
on click of
menu, page should be shown dynamically without refreshing the entire
page
so i did something like so
LandingPage.html
<html>
<head>
<style>
html,body{
height:100%;
}
</style>
<script type="text/javascript">
function finalConfirmationToStartJob() {
alert('in here')
$.ajax({
type : 'GET',
url : 'home2',
data : ''
});
};
</script></head>
<body>
<table border=1 height=100% width=100%>
<tr><td colspan='2'/></tr>
<tr>
<td width=15%>
<ul>
<li>ABC</li>
<ul>
<li>ABC1</li>
<li>ABC2</li>
<li>ABC3</li>
</ul>
<li>DEF</li>
<li>GHI</li>
<li>JKL</li>
</ul>
</td>
<td>
{% macro hello(name) %}Hello {{ name }}!{% endmacro %}
how are you darlings!!! {{hello()}}
</td>
</tr>
<tr><td colspan='2'/></tr>
</table>
</body>
</html>
Controller
#app.route("/home")
def goto_home():
return render_template('/LandingPage.html')
#app.route("/home2")
def try_out_macro():
print 'hellooo elloo'
hello = get_template_attribute('/LandingPage.html', 'hello')
return hello('World')
However when i do click on the action link, even though the controller actually returns perfect data , the data is not displayed on the screen.
When i hit the url /home2 i do get perfect data, however on click on action link nothing happens on the html
Can someone please help me, i am not sure what i am doing wrong
PS : i got it to work using innr html and div combination, however i would still like to know why the above doesnt work.