Develop Reference

send metatags through Google Cloud Signed URL with Django

I'm using signed url to get or post/put video fileq on my google storage. My modules refers on signed_url V4 and it's working quite well.
I wanted to add some metadata tags to my requests in order to manage more efficiently the charges related to GCS. But since I added those headers, the requests failed returning a cors policy error : (I have shortened the signature in the block above to make it more readable)
Access to XMLHttpRequest at 'https://test-dev-slotll.storage.googleapis.com/uploads/dave/FR/eaa678c9/2020/9/785f/f45d3d82-785f_full.mp4?X-
Goog-Algorithm=GOOG4-RSA-SHA256&X-Goog-Credential=dev-storage%40brilliant-tower-264412.iam.gserviceaccount.com%2F20200926%2Fauto%2Fstorage%2Fgoog4_request&X-Goog-Date=20200926T093102Z&X-Goog-Expires=3600&X-Goog-SignedHeaders=host%3Bx-goog-meta-account&x-goog-signature=6fbb27e[...]bd0891d21' from origin 'http://localhost:8000' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource.
and the detailed message:
<Error>
<Code>MalformedSecurityHeader</Code>
<Message>Your request has a malformed header.</Message>
<ParameterName>x-goog-meta-account</ParameterName>
<Details>Header was included in signedheaders, but not in the request.</Details>
</Error>
The cors are conigured to allow almost everything on my bucket :
[{"maxAgeSeconds": 3600, "method": ["*"], "origin": ["*"], "responseHeader": ["*"]}]
and here is the Python/Django function
def _get_signed_url(self, http_method="GET"):
"""' create a signed url with google storage
create a 'GET' request by default,
add method='w' or 'put' to create a 'PUT' request
get_signed_url('w')->PUT / get_signed_url()->GET
"""
if http_method.lower() in ["w", "put", "post"]:
http_method = "PUT"
else:
http_method = "GET"
signed_url = generate_signed_url(
settings.GS_BUCKET_NAME,
self.file.name,
subresource=None,
expiration=3600,
http_method=http_method,
query_parameters=None,
headers={'x-goog-meta-language':'french','x-goog-meta-account':'david',},
)
return signed_url
As I wrote it above, method get_signed_url() is copied from signed_url V4
if i replace headers = {'x-goog-meta-language':'french','x-goog-meta-account':'david',},
by hearders = {} or headers = None (as it was previously, it works fine
last, when I click on the link given by the signed-url, I got an error message:
The signed url as well as blob uploading or downloading are working fine without the headers for months but I do not see why the server is responding that the header meta tags are malformed...
I will appreciate any help
thanks !

I was getting the same error message when I was performing a GET request on the pre-signed urls from GCP. Adding the content-type: "application/octet-stream" solved it for me.
ajaxCall_getFile = $.ajax({
xhr: function() {
var xhr = new window.XMLHttpRequest();
return xhr;
},
type: 'GET',
url: <PRE-SIGNED URL>,
contentType: "application/octet-stream",
processData: false,
success: function(file_data){
alert("file downloaded");
},
error: function(err) {
alert('Download failed, please try again.');
}
});

As #Sideshowbarker suggests it in his comments, the problem came from the client.
The signed url was used to send files to storage through ajax but no custom header were added to the ajax.
By specifying the headers in the ajax, the PUT request of a signed url with custom metadata works well.
function ajaxSendToStorage(uuid, url) {
// Sending ONE file to storage
var file = getFileById(uuid);
$.ajax({
method: "PUT",
contentType: file.type,
processData: false,
dataType: "xml",
crossDomain: true,
data: file,
url: url,
headers: {"x-goog-meta-account":"david","x-goog-meta-language": "fr"},
success: function() {},
complete: function() {},
});
}

Fetching data after a certain time interval(10 sec) from a continuously increasing database like mysql using flask

I want to create an api using python and flask that fetches data in regular time interval(10 sec) from a continuously increasing database where data is continuously coming and stored.I don't want to fetch the old data which were already fetched.

Say you currently have an API endpoint that returns all the database stored data:
#app.route('/data', methods=['post'])
def data():
all_the_data = Data.query.order_by(Data.created.desc()).all()
return jsonify(results=all_the_data)
So your ajax call currently doing something like:
$.ajax({
type: "POST",
url: "/data",
dataType: "json",
success: function(data) {
console.log(data);
update_graph(data);
}
});
You just need a way for the system to filter what's going out, back to the client-- so we instead of querying all the data, we can filter based on a reference:
#app.route('/data', methods=['post'])
def data():
client_data = request.json
reference = client_data.get('reference')
if reference:
# we have a reference, so lets filter the query:
query_data = Data.query.filter(Data.id>reference).order_by(Data.created.desc()).all()
else:
# no reference, so send the lot!
query_data = Data.query.order_by(Data.created.desc()).all()
return jsonify(results=query_data)
Then your ajax request needs to get the last reference from the last query it did-- and supply that to the API endpoint:
$.ajax({
type: "POST",
url: "/data",
data: JSON.stringify({ reference: 999 }),
contentType: "application/json; charset=utf-8",
dataType: "json",
success: function(data) {
console.log(data)
update_graph(data["results"]);
}
});
So you just need to work out how to get that reference value from the last set of values you recieved (the API could send that back as another key, or you could poll your current set within javascript, etc).

URL sent to Django using AJAX is different when printed from Django

I have a JQuery event that, when a link is clicked, an AJAX POST request is sent to a Django function. There I print the recieved url and do other stuff with it. However, when it is printed (by Django) some characters in the url are changed.
The specific url this happened with was :
https://www.catholicleague.org/05press_releases/quarter%204/051110_put_on_notice.htm
Which was printed as :
https://www.catholicleague.org/05press_releases/quarter+4/051110_put_on_notice.htm
Where %20 was changed to +
Here is the AJAX and Django code:
$("a").on("click", function(e){
e.preventDefault();
if(e.target.href){
let clicked_source = e.target.href;
let csrf_tok = parent.document.getElementById("csrf_tok").value;
$.ajax({
url: "/change_origin/",
data: JSON.stringify({"clicked_source":clicked_source}),
type: "POST",
beforeSend: function (xhr, settings) { xhr.setRequestHeader("X-CSRFToken", csrf_tok );},
success: function(response) {
console.log(response.msg)
},
error:function(error) { console.log(error); }
});
}
});
def change_origin(request):
if request.method == 'POST':
received = ast.literal_eval(request.body.decode())
print(received)
clicked_source_url = received['clicked_source']
return JsonResponse({'msg': "ok"})
Where decode is used as the JSON object is received in Python as a byte-like object. And ast is used to turn the string representation of the object to an actual object (or dict) for access.
I need either:
1) A way to just send a string from Ajax to Django
2) A better way to deal with the received object as I believe using .decode() might be the one causing this issue.
EDIT: The link is the second link in the "origin" part of this article
https://www.snopes.com/fact-check/an-unmerried-woman/

Unable to receive data from AJAX call [Flask]

What I Am Trying To Do: Receive data from AJAX to Flask. Eventually, I would like to send token data (Which will come from stripe) to the flask side
The Problem: I can't print any data to the console. So I'm assuming data is not being passed through.
I am unable to receive data from my Ajax call. I have been searching for some time now and I haven't found a fix. I've seen multiple different solutions for others but none of them worked for me. I'm am trying to implement a custom Stripe payment flow.
What I plan to do eventually is to pass in all the data I need (in the token) through the 'data' parameter in JSON format. Here are the different sides of code
index.html
var handler = StripeCheckout.configure({
key: 'test_key',
image: 'image_url',
locale: 'auto',
token: function(token) {
$.ajax({
url: '/charge',
data: {
'token': '(data im trying to access/print in app.py)'
},
type: 'POST',
success: function(response) {
console.log(response);
},
error: function(error) {
console.log("ERROR");
console.log(error);
},
dataType: "json",
contentType: "application/json"
});
}
});
app.py
from flask import Flask, request
import json
#app.route('/charge', methods=['POST'])
def charge():
# Grab token information
token = request.form['token']
# The line below never prints to console
print(token)
# This test print statement below never prints to console
print("This print statement never runs")
Nothing prints to the console. I've wasted so much time on this so any leads or pointers would be greatly appreciated!
UPDATES
I did some updates suggested by #Daniel Roseman but nothing at all prints to the console.

Try the following code
In javascript:
var data = {token: token}
$.ajax({
url: '/charge',
data: JSON.stringify(data),
type: 'POST',
success: function (response) {
console.log(response);
},
error: function (error) {
console.log("ERROR");
console.log(error);
},
dataType: "json",
contentType: 'application/json;charset=UTF-8',
});
In controller [charge method]:
from flask import Flask
from flask import render_template, request, jsonify
#app.route('/charge', methods=['POST'])
def charge():
# Grab token information
token = request.json['token']
# This is the expected token from json
print(token)
# This test print statement below now prints to console
print("This print statement now runs surely")
# return JSON
return jsonify(
code=0,
msg="Success"
)

You're not posting JSON.
Instead of accessing request.get_json(), you can just access the individual elements from the request.form dictionary. In this case:
token = request.form['token']

Receiving a 2-D list as request parameters in Python Flask

I am attempting to receive a 2-D list in my flask application. The json object sent is as follows:
function someFunction() {
var two_d_arr = [[a,1],[b,2],[c,3]];
var data = {
'arr' : two_d_arr
};
$.ajax({
url: 'http://localhost:5025/',
dataType: 'jsonp',
data: data,
success: function(response) {
alert(response);
}
});
}
I have tried various times to retrieve these tuples as a list and am failing. The way I would want to see them is a list as follows:
lst = [[a,1],[b,2],[c,3]]
When I print request.data I get the following output (formatted for easy reading):
ImmutableMultiDict([(
'callback', u'jQuery1113017347401613369584_1450454196704'),
('liveCells[2][]', u'c'), ('liveCells[2][]', u'3'),
('liveCells[1][]', u'b'), ('liveCells[1][]', u'2'),
('liveCells[0][]', u'a'), ('liveCells[0][]', u'1')
])
I have tried various ways to extract just the list (request.form.getlist, request.args.getList) etc.

You are not encoding the request to JSON. You are only specifying what you expect to receive from the server (because you expect JSONP jQuery is including a callback parameter).
Encode to JSON on the client side:
$.ajax({
url: 'http://localhost:5025/',
data: JSON.stringify(data),
success: function(response) {
alert(response);
}
});
I omitted the dataType argument, only set one if jQuery can't autodetect the response type.
On the server side, read the response with request.get_json().