I've solved my issue, but I'd like to know what was going wrong so I can address it in the future. I'm having issues decoding incoming JSON for use in my Flask application.
The code that sends it in Angular:
$http.post("/login", JSON.stringify($scope.loginForm))
.success(function(data, status, headers, config) {
console.log(data);
})
.error(function(data, status, headers, config) {
console.log("Submitting form failed!");
});
Important to note that the request type is set to application/json earlier up, with
$http.defaults.headers.post["Content-Type"] = "application/json";
The code that receives it within Flask:
data = request.get_json()
email_address = data.get("email_address")
password = data.get("password")
Attempting to load it this way returns an error 400, but any other way leads to some very strange issues. For example:
return json.dumps(request.get_json())
Will log {"password": "password", "email_address": "email#email.com"} in the console, but attempting to do this:
data = request.get_json()
email_address = data.get("email_address")
password = data.get("password")
With no difference whatsoever between this and the first block of code except that I'm not forcing it, I receive the exception "ValueError: need more than 1 value to unpack". Which implies that there aren't two values to unpack.
HOWEVER, they both work individually. If I do the above request and omit either of the data.get() lines above, the other will work.
What about my setup causes my JSON object to disintegrate the first time it's accessed?
I got around this by using request.json instead of request.get_json() but as request.json is being deprecated it's fairly important I know how to solve this in the future. Any pointers would be appreciated!
You can omit JSON.stringify and pass object directly to $http.post() method because angular will serialize it to JSON automatically it formData is object. So I assume that JSON.stringify will force angular to send is as x-www-form-urlencoded instead of application/json media type.
See default transformations section: angular $http service documentation
Related
Please help me with this, been searching for hours now. Here is a snippet of my code:
headers = {
'consent':'true',
'date_of_birth':'2000-08-05',
'email':email,
'password':password,
'username':username
}
response = req.post('https://discord.com/api/v9/auth/register', data=headers)
print(response.text)
email and password and username are correctly defined
it then returns me this error
{"message": "Invalid Form Body", "code": 50035}.
What can I do to fix this error?
I tried changing data to json and it came back with a different error. I want it to execute the api.
Here's an issue with this code snippet
The consent field seems to be missing in the payload of the post request, this field might be mandatory in discord API endpoint, so you should include it.
The data argument should not be used when passing headers to the requests.post() method. Instead, you should use the headers argument.
Here is the modified version of the code:
headers = {
'consent':'true',
'date_of_birth':'2000-08-05',
'email':email,
'password':password,
'username':username
}
response = req.post('https://discord.com/api/v9/auth/register', headers=headers)
print(response.text)
Please keep in mind that this code may work only if the endpoint accepts the payload you're providing in the headers, also keep in mind that this code is not handling the errors that might happen like network problems, invalid email/password, or other issues. You should add some error handling in the code to handle such cases.
Also, this will only work if you've implemented the logic for getting the email, password, username variables, you need to initialize the variables email, password, username with valid values before using them in the headers.
You might also need to check the discord API documentation to confirm the endpoint URL and its requirements to use the API endpoint.
I'm trying to use the salesforce bulk api 2.0 to upsert some data, and they only accept csv data. In this documentation, for step 2, they say create the csv file. Then in step 4, I need to upload the csv data. I have code that doesn't throw any errors, but the record is not processed, which makes me think I am doing something wrong.
So I have the following as my csv_string:
csv_string = "Id, Name__c\n,\"Doe, John\""
Here is how I am currently sending the data
headers = {'Content-Type': 'text/csv', 'Accept': 'application/json'}
data = {'file': csv_string}
async with self.session.put(upload_url, data = data, headers = headers) as response:
r = await response.text()
print(r)
According to the documentation, I am supposed to get " response that includes the job ID, with a job state of Open." but =it just prints an empty line.
Then when I do step 16: Check the job status and results, it successfully returns JobComplete and response.text() returns the following: "sf__Id","sf__Created",file=Id%2C+Name__c%0A%2C+%22Doe%2C+John%22 which is basically a url encoded version of my csv_string. There is no change to the data in salesforce, so the upsert fails. The fact that an empty line is printed out makes me believe that I am not passing the csv in correctly.
I've tried using aiohttp's FormData, but that changes the data type to multi-part encoded which is not accepted. I've also tried passing data = csv_string which makes salesforce return an error. I was thinking maybe I need to pass it in as binary data, for example, when you open a file using open("file_name", "rb"), but I don't know how to convert this existing string to binary data. Can someone give an example of how to pass csv data in a request using aiohttp? Or maybe tell me how to convert this string to binary data to I can try passing it in this way?
Thanks #identigral. This was one of the issues.
One major thing that helped me debug was going to setup->bulk data load jobs. If you click on a specific job, and hover over the "state message", it will give you the reason why a job failed. Although salesforce has an api for getting the failed job record here, which supposedly is supposed to return an error message, it did not work, which is why I felt kind of stuck, and led me to believe I wasn't passing in the csv correctly.
So I had a few errors:
Like identigral pointed out, I used "CLRF" as the line ending because I thought I was on windows, but since I type out the string myself in the code, I had to use "LF". I believe if I read in a csv file that I create using Excel, I would probably have to use "CLRF", although I haven't tested it yet.
Salesforce doesn't like the space in front of "Name__c", so although I had a field with that name on my object, it said "field Name__c" not found.
The documentation I linked said that after uploading the csv, "You should get a response that includes the job ID still in the Open state." However, that is not the case. The PUT request to upload the csv will have an empty request body and only return status 201 if the request was successful. This is found here: link
I realized this was the correct way as in this documentation, it gives an example of passing in data of type text/plain by doing data='Привет, Мир!', so I figured text/csv should be the same.
So the final code to send the csv that ended up working is as follows: (self.session is an instance of aiohttp.ClientSession() and I had already included the bearer token in the default headers when initializing the session):
csv_string = "Id,Name__c\n,\"Doe,John\""
headers = {'Content-Type': 'text/csv', 'Accept': 'application/json'}
async with self.session.put(upload_url, data = csv_string, headers = headers) as response:
assert response.status == 201 #data was successfully received.
The following was how I defined my when creating the job (replace MyObject__c with the API name of the object from salesforce):
body = {'object': 'MyObject__c',
'contentType': 'CSV',
'operation': 'upsert',
"lineEnding": "LF",
"externalIdFieldName": "Id" }
I looked many questions similar to my title but I have not found any that had same problem as me yet.
I did requests.post to post JSON to API restful server. Below is the snippet
import requests
def upload_data():
url = "http://127.0.0.1:8088/"
data = {"value":"abc123"}
response = requests.post(url, data=data)
print response.status_code, response.reason, response.text
upload_data()
And for the server side
from flask_restful import Api, Resource
from flask import request
class MyAPI(Resource):
def get():
pass
def post(self):
value = request.data['value']
response_object = {
'value':value
}
return response_object, 201
I was hoping to get the POST function to work by showing the result of 201 Created with
{
'value':'abc123'
}
But whenever I run the script, it gives me error saying that
value = request.data["value"]
TypeError: string indices must be integers, not str
I am sorry if this is a bad question but if anyone could show me what I have been missing in this script, I really appreciate it. Thank you.
That's because request data hasn't been parsed into a python dictionary. Were you perhaps thinking of
data = json.loads(request.data)
However please note that you are not actually posting a JSON body to your flask server. You are posting multipart formdata. So you may probably be looking for the answer posted by luoluo.
One the other hand if you really wanted to deal with json, The correct way to send json looks something like this:
requests.post(url, json=data)
And then the loads as suggested.
The request.data is a string, while request.values is a MultiDict.
You need update your code to :
value = request.values.get('value')
instead of
value = request.data['value']
According to the doc
args
A MultiDict with the parsed contents of the query string. (The part in the URL after the question mark).
form
A MultiDict with the parsed form data from POST or PUT requests. Please keep in mind that file uploads will not end up here, but instead in the files attribute.
values
A CombinedMultiDict with the contents of both form and args.
data
Contains the incoming request data as string in case it came with a mimetype Flask does not handle.
I would like to POST data from my iOS app to my Django server — encoding it all in the URL is just impractical. I want to do this without a form, simply sending JSON data up and getting JSON data out. When I try, I’m running into this error:
Forbidden (CSRF token missing or incorrect.)
For development, I’m currently getting around this by declaring my view method to be #csrf_exempt, but I’m worried that’s unsafe for production. Is it if I’m also declaring the method #login_required?
Assuming it is unsafe, how can I solve this? I understand the approach using {% csrf_token %} in HTML templates -- I’ve used that in other Django apps -- but again, I’m not actually using a form, nor do I wish to. I’m simply encoding my request data in a JSON dictionary, and getting a JSON dictionary back. Here’s the relevant part of the client side:
NSData *data = [NSJSONSerialization dataWithJSONObject:dictWithMyData options:NSJSONWritingPrettyPrinted error:&error];
if (!data) {
if (error)
NSLog(#"NSJSONSerialization error = %#", [error localizedDescription]);
return NO;
}
NSMutableURLRequest *request = [NSMutableURLRequest requestWithURL:[NSURL URLWithString:myServerMethodUrl] cachePolicy:NSURLRequestReloadIgnoringLocalAndRemoteCacheData timeoutInterval:timeout];
[request setHTTPMethod:#"POST"];
[request setValue:#"application/json" forHTTPHeaderField:#"Content-Type"];
[request setHTTPBody:data];
NSURLResponse *urlResponse = nil;
NSData *urlData = [NSURLConnection sendSynchronousRequest:request returningResponse:&urlResponse error:&error];
if (error) {
NSLog(#"NSURLConnection error = %#", [error localizedDescription]);
return NO;
}
And here’s the relevant server part:
#login_required
def my_server_method(request):
data = simplejson.loads(request.body)
R = {}
# Do stuff with the data here and generate a response with JSON data in R
....
return HttpResponse(json.dumps(R), content_type="application/json")
From what I’ve learned of the CSRF mechanism, the CSRF token is valid for the entire session. So I thought I could make this work by: a) upon authentication, passing the CSRF token to my iOS client, and then b) including the token when I post my data. Here’s how I tried that.
Upon authentication in the server, I find the CSRF token with this and send it to the client:
session_csrf_token = django.middleware.csrf.get_token(request)
And I try to pass it back by modifying my client code above like so. First I augment the data I send with my session_csrf_token (stored as _csrfToken on the client upon login):
NSMutableDictionary *dictCSRF = [NSMutableDictionary dictionaryWithDictionary:dictWithMyData];
[dictCSRF setObject:_csrfToken forKey:#"csrfmiddlewaretoken"];
NSData *data = [NSJSONSerialization dataWithJSONObject:dictCSRF options:NSJSONWritingPrettyPrinted error:error];
Then I augment the request header like so:
[request setValue:_csrfToken forHTTPHeaderField:#"X-CSRF-Token"];
But it’s not working. I get the same "CSRF token missing or incorrect" error.
Any ideas how to get this to work?
I'm trying to interface with a RESTful API that accepts application/x-protobuf objects in post.
.proto example object:
message register
{
required string name = 1;
required int32 id = 2;
}
using requests I'm able to POST this message to the server and receive a 200.
e.g.
register = protobuf_pb2.register()
register.name = "foo"
register.id = 1
response = requests.request("POST", url, data=register.SerializeToString(), headers=headers)
when I look at this payload in SerializeToString() format it shows similar to
b'\n\t\n\x03foo\x10\x01'
As a sanity check, I can then create a new message object and .ParseFromString() on that back into the correct message format.
A second uri on the server allows you to send a GET request to retrieve that same data. So I used the following:
register = protobuf_pb2.register()
response = requests.request("GET", url, headers=headers).content
register.ParseFromString(response)
The response returns
b'foo:1\n'
And ParseFromString returns the exception
DecodeError('Error parsing message',)
Am I missing a step somewhere?
Should the .content payload be encoded somehow before being parsed?
Many thanks.
The object you are getting back from the server is binary data, UTF-8 encoded, so Python interprets it as a bytes object (in Python 3, bytes and str are different types).
m = b'foo:1\n'
type(m)
Out[1]: bytes
m.decode('utf-8')
Out[2]: 'foo:1\n'
ParseFromString however expects a string, so it throws a DecodeError; just use the decode() with the appropriate encoding.
In the response headers you should be able to find the encoding used: pretty much everyone uses UTF-8, but in "real" Production code, you should check that too: in the Content-Type (or you can even request a specific one with Accept-Charset but you may get a 406, if the server cannot fulfil it).