Develop Reference

Python+Flask : how to get text from request with newlines?

I am not sure how it works, but it worked in one of my projects and doesn't works in a new one.
Obviously, I am missing something.
I want a simple REST server with only one POST.
It should get text from POST request. Text contains newlines.
This is my text.txt:
hello
how are you
This is how I test it:
curl -i -s -X POST -d #test.txt http://*ip*:*port*/api/gogogo
This is part of my python script with flask app:
#app.route('/gogogo', methods=['POST'])
def translate():
request_str = request.get_data().decode('utf-8').strip()
request_lines = request_str.split('\n')
print(request_lines)
return "yeah"
But "print(request_lines)" prints "hellohow are you".
Any ideas?
I also tried to use instead of '\n', didn't help. And even if it did - my requests use '\n', so i have to support this format.

Your problem is curl. It removes new line when you send it with -d.
You have to use --data-binary
curl -X POST --data-binary #test.txt http://localhost:5000/gogogo
BTW: you can test requests also with page https://httpbin.org which sends back as JSON all data which it get in request from you - headers, body, post data, url arguments.
It will also send back string without \n if you use -d instead of --data-binary
curl -X POST -d #test.txt https://httpbin.org/post
curl -X POST --data-binary #test.txt https://httpbin.org/post
BTW: Now it may send it with \r\n instead of \n but maybe it depend on system and how it keep new line in file. Better use .splitlines() instead of .split('\n')

try this
request_str = request.get_data().decode('utf-8').strip()
request_lines = request_str.splitlines(True)
for line in request_lines:
print(line)

Cannot access the request json_body when using Chalice

I'm attempting to make a curl request to my python api that is using the AWS package Chalice.
When I try to access the app.current_request.json_body a JSON Parse error is thrown. Cannot figure out why this is happening. My JSON is formatted properly as far as I can tell.
Here is the curl request:
(echo -n '{"data": "test"}') |
curl -H "Content-Type: application/json" -d #- $URL
Here is the python Chalice code:
app = Chalice(app_name='predictor')
#app.route('/', methods=['POST'], content_types=['application/json'])
def index():
try:
body = app.current_request.json_body
except Exception as e:
return {'error': str(e)}
When I invoke the route using the above curl request I get the following error:
{"error": "BadRequestError: Error Parsing JSON"}
Note: When I remove the .json_body from the app.current_request. I no longer get the error.
Any thoughts?

The documentation indeed indicates that the problem is Content-Type:
The default behavior of a view function supports a request body of application/json. When a request is made with a Content-Type of application/json, the app.current_request.json_body attribute is automatically set for you. This value is the parsed JSON body.
You can also configure a view function to support other content types. You can do this by specifying the content_types parameter value to your app.route function. This parameter is a list of acceptable content types.
It suggests that changing the Content-Type might make json_body work, but I didn't manage to have any success with it.
However using app.current_request.raw_body.decode() instead of app.current_request.json_body solves the problem.

web2py convert Storage object to string

I have some RESTful API calls in web2py which take in JSON file. I make the calls through the command line using curl like:
curl -H "Content-Type: application/json" --data #mydata.json https://mywebsite/doWork
In my web2py controller I have it setup to read the POST request. When I examine request.vars by returning it, it contains the full JSON data, but request.vars is of type: class 'gluon.storage.Storage'.
I need the data in string format in the controller. However, if I do str(request.vars), web2py appends a Storage tag in front of my data.
#request.restful()
def doWork():
def POST(*args, **vars):
return(request.vars)
return locals()
Assuming the mydata.json file is something simple like:
{
"key": "value"
}
The request to POST will return {"key": "value"} as a gluon.storage.Storage object.
If I change the line return(request.vars) to return(str(request.vars)) the output is now: < Storage {u'key': u'value'}>
How can I convert this request.vars to a string without it appending the Storage tag to the front, and changing the double quotes to single quotes?

If you want to convert vars back to JSON, you can use the Python json library or just use response.json:
return response.json(vars)
Alternatively, you can add the .json extension to the request and simply return vars directly:
return vars
with the following request:
curl -H "Content-Type: application/json" --data #mydata.json https://mywebsite/doWork.json
In that case, web2py will automatically convert the response to JSON.

Converting url encode data from curl to json object in python using requests

What is the best way to convert the below curl post into python request using the requests module:
curl -X POST https://api.google.com/gmail --data-urlencode json='{"user": [{"message":"abc123", "subject":"helloworld"}]}'
I tried using python requests as below, but it didn't work:
payload = {"user": [{"message":"abc123", "subject":"helloworld"}]}
url = https://api.google.com/gmail
requests.post(url, data=json.dumps(payload),auth=(user, password))
Can anybody help.

As the comment mentioned, you should put your url variable string in quotes "" first.
Otherwise, your question is not clear. What errors are being thrown and/or behavior is happening?
New cURL method in Python

compare python requests with curl

I am interfacing with an API using requests and requests_oauthlib.
I successfully authenticate and access all the GET methods of the API, but get error 500 with POST methods. For example:
r = oauth.post("https://api.timelyapp.com/1.0/1/clients",
data={"client":{"name":"newclient", "color":"c697c0" }},
allow_redirects=False, headers={"Content-Type": "application/json"})
The issue is that I tested the same exact call with curl and it works correctly, here the curl code:
curl -v -X POST -H "Content-Type: application/json" -H "Authorization: Bearer XXXXXXXXXXXX" --data '{"client": { "name": "newclient", "color":"c697c0" }}' "https://api.timelyapp.com/1.0/1/clients"
how can I dig deeper in requests to compare its call with curl?
UPDATE:
Also, noticed that if I do not specify content type:
r = oauth.post("https://api.timelyapp.com/1.0/1/clients",
data={"client":{"name":"newclient", "color":"c697c0" }},
allow_redirects=True)
I get instead a 302 with redirection to the site homepage, where I fetch the content of the page. In any case the new client is not added.

You might want to try this instead:
data=json.dumps(payload)
From python-requests doc:
There are many times that you want to send data that is not
form-encoded. If you pass in a string instead of a dict, that data
will be posted directly.