I have some simple code, like this:
import json
from bottle import route, request,run
#route('/process_json',methods='POST')
def data_process():
data = json.loads(request.data)
username = data['username']
password = data['password']
run(host='localhost', port=8080, debug=True)
I would like to send a data in json format, like this:
$ curl -i -H "Accept: application/json" -H "Content-Type: application/json" -X POST -d
'{"username":"fooba","password":"1234"}' url
However, I have this problem:
File "/usr/lib/python2.7/dist-packages/bottle.py", line 862,
in _handle return route.call(**args)
File "/usr/lib/python2.7/dist-packages/bottle.py", line 1729,
in wrapper rv = callback(*a, **ka)
File "testtest.py", line 5,
in data_process data = json.loads(request.data)
File "/usr/lib/python2.7/dist-packages/bottle.py", line 1391,
in getattr raise AttributeError('Attribute %r not defined.' % name)
AttributeError: Attribute 'data' not defined
I also tried add at the beginning like here(http://bottlepy.org/docs/dev/tutorial.html#html-form-handling):
return '''
Username:
Password:
'''
But it also doesn't work.
It is good, you have provided the bottle application code. It was broken.
Following modification works:
import json
from bottle import route, request, run
#route('/process_json', method="POST")
def data_process():
data = json.load(request.body)
print "data", data
username = data['username']
password = data['password']
run(host='localhost', port=8080, debug=True)
Testing with curl
$ curl -i -H "Accept: application/json" -H "Content-Type: application/json" -X POST -d '{"username": "fooba", "password": "1234"}' http://localhost:8080/process_json
HTTP/1.0 200 OK
Date: Mon, 14 Jul 2014 16:18:25 GMT
Server: WSGIServer/0.1 Python/2.7.6
Content-Length: 0
Content-Type: text/html; charset=UTF-8
Testing with HTTPie
$ http POST http://localhost:8080/process_json username=jan password=pswd
HTTP/1.0 200 OK
Content-Length: 0
Content-Type: text/html; charset=UTF-8
Date: Mon, 14 Jul 2014 16:15:16 GMT
Server: WSGIServer/0.1 Python/2.7.6
Actually, I was playing with your example to find out, how it would look like in http command. It is really much simpler and using --verbose we may check, it really forms valid JSON payload:
$ http --verbose POST http://localhost:8080/process_json Accept:application/json
Content_type:application/json username=jan password=pswd
POST /process_json HTTP/1.1
Accept: application/json
Accept-Encoding: gzip, deflate
Authorization: Basic amFuOnZsY2luc2t5
Content-Length: 39
Content-Type: application/json; charset=utf-8
Content_type: application/json
Host: localhost:8080
User-Agent: HTTPie/0.8.0
{
"password": "pswd",
"username": "jan"
}
HTTP/1.0 200 OK
Content-Length: 0
Content-Type: text/html; charset=UTF-8
Date: Mon, 14 Jul 2014 16:21:02 GMT
Server: WSGIServer/0.1 Python/2.7.6
The shortest form is:
$ http :8080/process_json username=jan password=pswd
HTTP/1.0 200 OK
Content-Length: 0
Content-Type: text/html; charset=UTF-8
Date: Mon, 14 Jul 2014 16:25:12 GMT
Server: WSGIServer/0.1 Python/2.7.6
as http://localhost:8008 can be shortened to :8080 (the same applies to curl)
and if there is a payload, default method is POST.
Related
I spotted a weird warning in logs:
[WARNING] urllib3.connectionpool:467: Failed to parse headers (url=https://REDACTED): [MissingHeaderBodySeparatorDefect()], unparsed data: 'trol,Content-Type\r\n\r\n'
Traceback (most recent call last):
File "/usr/local/lib/python3.8/dist-packages/urllib3/connectionpool.py", line 465, in _make_request
assert_header_parsing(httplib_response.msg)
File "/usr/local/lib/python3.8/dist-packages/urllib3/util/response.py", line 91, in assert_header_parsing
raise HeaderParsingError(defects=defects, unparsed_data=unparsed_data)
urllib3.exceptions.HeaderParsingError: [MissingHeaderBodySeparatorDefect()], unparsed data: 'trol,Content-Type\r\n\r\n'
This is from calling a standard requests.post() on a web service I fully control (a Python app behind nginx).
When I turn on debuglevel=1 in http.client.HTTPResponse I see this:
reply: 'HTTP/1.1 200 OK\r\n'
header: Server: nginx/1.18.0 (Ubuntu)
header: Date: Tue, 30 Nov 2021 22:14:04 GMT
header: Content-Type: application/json
header: Transfer-Encoding: chunked
header: Connection: keep-alive
header: Vary: Accept-Encoding
header: Access-Control-Allow-Origin: *
header: Access-Control-Allow-Credentials: true
header: Access-Control-Allow-Methods: GET, POST, OPTIONS
header: Access-Control-Allow-Headers: DNT,X-Mx-ReqToken,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Con
Note the last header ending abruptly in ,If-Modified-Since,Cache-Con.
Clearly, requests==2.26.0 (via urllib3==1.26.7 via http.client) cuts the last header in half for some reason during parsing, and then later complains it has "left over" data with the remaining trol,Content-Type\r\n\r\n.
In this case the warning is not critical, because the header is not really needed. But it's scary this is happening, because… what else is being cut / misparsed?
The same endpoint works fine from e.g. curl:
$ curl -i -XPOST https://REDACTED
HTTP/1.1 200 OK
Server: nginx/1.18.0 (Ubuntu)
Date: Sat, 04 Dec 2021 20:08:59 GMT
Content-Type: application/json
Content-Length: 53
Connection: keep-alive
Vary: Accept-Encoding
Access-Control-Allow-Origin: *
Access-Control-Allow-Credentials: true
Access-Control-Allow-Methods: GET, POST, OPTIONS
Access-Control-Allow-Headers: DNT,X-Mx-ReqToken,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Con
trol,Content-Type
…JSON response…
Any idea what could be wrong? Many thanks.
Your webserver, or its configuration, looks broken. Have a look at what is generating that CORS Access-Control-Allow-Headers header because it is not permitted to contain a line break.
I am writing an controller endpoint in Python and I want to modify response code, content-text etc. How can I achieve it.
#http.route('/get/data/',methods=['GET','POST'],type="http",auth='public',csrf=False)
def fetchSms(self,**kwargs):
mydata = {"date":"2018-10-13T00:46:17.25Z"}
return simplejson.dumps(mydata)
I have to return mydata to client after setting content-type="application/json" and response response code should be 200
Current Output:
HTTP/1.1 200 OK
Server: nginx/1.12.2
Date: Mon, 15 Oct 2018 16:57:49 GMT
Content-Type: text/html; charset=utf-8
Content-Length: 192
Connection: keep-alive
Set-Cookie: session_id=43edf5hfgh436sdgdga9d7618deb74af7; Expires=Sun, 13-Jan-2019 16:57:49 GMT; Max-Age=7776000; Path=/
I'm trying to write a python script that would help me install a theme remotely. Unfortunately, the upload part doesn't play nice, trying to do it with requests' POST helpers.
The HTTP headers of a successful upload look like this:
http://127.0.0.1/wordpress/wp-admin/update.php?action=upload-theme
POST /wordpress/wp-admin/update.php?action=upload-theme HTTP/1.1
Host: 127.0.0.1
User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:53.0) Gecko/20100101 Firefox/53.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Content-Type: multipart/form-data; boundary=---------------------------2455316848522
Content-Length: 2580849
Referer: http://127.0.0.1/wordpress/wp-admin/theme-install.php
Cookie: wordpress_5bd7a9c61cda6e66fc921a05bc80ee93=admin%7C1497659497%7C4a1VklpOs93uqpjylWqckQs80PccH1QMbZqn15lovQu%7Cee7366eea9b5bc9a9d492a664a04cb0916b97b0d211e892875cec86cf43e2f9d; wordpress_test_cookie=WP+Cookie+check; wordpress_logged_in_5bd7a9c61cda6e66fc921a05bc80ee93=admin%7C1497659497%7C4a1VklpOs93uqpjylWqckQs80PccH1QMbZqn15lovQu%7C9949f19ef5d900daf1b859c0bb4e2129cf86d6a970718a1b63e3b9e56dc5e710; wp-settings-1=libraryContent%3Dbrowse; wp-settings-time-1=1497486698
Connection: keep-alive
Upgrade-Insecure-Requests: 1
-----------------------------2455316848522: undefined
Content-Disposition: form-data; name="_wpnonce"
b1467671e0
-----------------------------2455316848522
Content-Disposition: form-data; name="_wp_http_referer"
/wordpress/wp-admin/theme-install.php
-----------------------------2455316848522
Content-Disposition: form-data; name="themezip"; filename="oedipus_theme.zip"
Content-Type: application/octet-stream
PK
HTTP/1.1 200 OK
Date: Thu, 15 Jun 2017 01:33:25 GMT
Server: Apache/2.4.25 (Win32) OpenSSL/1.0.2j PHP/7.1.1
X-Powered-By: PHP/7.1.1
Expires: Wed, 11 Jan 1984 05:00:00 GMT
Cache-Control: no-cache, must-revalidate, max-age=0
X-Frame-Options: SAMEORIGIN
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
Transfer-Encoding: chunked
Content-Type: text/html; charset=UTF-8
----------------------------------------------------------
To create a simple session for WP, in order to use later for uploads:
global wp_session
def wpCreateSession(uname, upassword, site_link):
"""
:param uname: Username for the login.
:param upaswword: Password for the login.
:param site_link: Site to login on.
:return: Returns a sessions for the said website.
"""
global wp_session
wp_session = requests.session()
wp_session.post(site_link, data={'log' : uname, 'pwd' : upassword})
To upload the said file to WP, using the wp_session global:
def wpUploadTheme(file_name):
global wp_session
try:
with open(file_name, 'rb') as up_file:
r = wp_session.post('http://127.0.0.1/wordpress/wp-admin/update.php', files = {file_name: up_file})
print "Got after try."
finally:
up_file.close()
And this last bit is where it doesn't work, the upload is not successful and I get returned to WordPress' basic 404.
I have also tried requests_toolbelt MultiPart_Encoder to no avail.
Question: 'requests' POST file fails when trying to upload
Check your files dict, your dict is invalid
files = {file_name: up_file}
Maybe you need a full blown files dict, for instance:
files = {'themezip': ('oedipus_theme.zip',
open('oedipus_theme.zip', 'rb'),
'application/octet-stream', {'Expires': '0'})}
From docs.python-requests.org
files = {'file': open('test.jpg', 'rb')}
requests.post(url, files=files)
From SO Answer Upload Image using POST form data in Python-requests
I'm having some difficulties parsing multipart form data when using the django-rest-framework. I've set up a minimal view to just echo back the request data:
from rest_framework.views import APIView
from rest_framework.response import Response
from rest_framework.parsers import MultiPartParser, FormParser
class FileUpload(APIView):
parser_classes = (MultiPartParser, FormParser, )
def post(self, request, format=None, *args, **kwargs):
return Response({'raw': request.data, 'data': request._request.POST,
'files': str(request._request.FILES)})
I am expecting that raw (slightly badly named I admit) contains effectively the same data as request._request.POST and request._request.FILES.
If I POST to the view with Content-Type= application/x-www-form-urlencoded this works as expected:
$ http -v --form POST http://localhost:8000/upload/api/ course=3 name=name
POST /upload/api/ HTTP/1.1
Accept: */*
Accept-Encoding: gzip, deflate
Connection: keep-alive
Content-Length: 18
Content-Type: application/x-www-form-urlencoded; charset=utf-8
Host: localhost:8000
User-Agent: HTTPie/0.9.2
course=3&name=name
HTTP/1.0 200 OK
Allow: POST, OPTIONS
Content-Type: application/json
Date: Thu, 17 Dec 2015 16:52:37 GMT
Server: WSGIServer/0.2 CPython/3.4.3
Vary: Accept, Cookie
X-Frame-Options: SAMEORIGIN
{
"data": {
"course": "3",
"name": "name"
},
"files": "<MultiValueDict: {}>",
"raw": {
"course": "3",
"name": "name"
}
}
However if I post with Content-Type=multipart/form-data I get the following:
$ http -v --form POST http://localhost:8000/upload/api/ file#~/Projects/lms/manage.py course=3 name=name
POST /upload/api/ HTTP/1.1
Accept: */*
Accept-Encoding: gzip, deflate
Connection: keep-alive
Content-Length: 577
Content-Type: multipart/form-data; boundary=634ec7c7e89a487b89c1c07c0d24744c
Host: localhost:8000
User-Agent: HTTPie/0.9.2
--634ec7c7e89a487b89c1c07c0d24744c
Content-Disposition: form-data; name="course"
3
--634ec7c7e89a487b89c1c07c0d24744c
Content-Disposition: form-data; name="name"
name
--634ec7c7e89a487b89c1c07c0d24744c
Content-Disposition: form-data; name="file"; filename="manage.py"
#!/usr/bin/env python
import os
import sys
if __name__ == "__main__":
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "config.settings.local")
from django.core.management import execute_from_command_line
execute_from_command_line(sys.argv)
--634ec7c7e89a487b89c1c07c0d24744c--
HTTP/1.0 200 OK
Allow: POST, OPTIONS
Content-Type: application/json
Date: Thu, 17 Dec 2015 16:55:44 GMT
Server: WSGIServer/0.2 CPython/3.4.3
Vary: Accept, Cookie
X-Frame-Options: SAMEORIGIN
{
"data": {
"course": "3",
"name": "name"
},
"files": "<MultiValueDict: {'file': [<InMemoryUploadedFile: manage.py ()>]}>",
"raw": {}
}
Am I missing something here? I am using HTTPIE to generate the requests here but the same behaviour exists with curl so I'm pretty sure that is not the problem. I am using djangorestframework==3.3.0 and Django==1.8.4
EDIT:
It seems that PUTing to the url (with an otherwise identical request) achieves the desired result:
$ http -v --form PUT http://localhost:8000/upload/api/ file#~/Projects/lms/manage.py course=3 name=name
PUT /upload/api/ HTTP/1.1
Accept: */*
Accept-Encoding: gzip, deflate
Connection: keep-alive
Content-Length: 577
Content-Type: multipart/form-data; boundary=98feb59a8abe4bfb95a7321f536ed800
Host: localhost:8000
User-Agent: HTTPie/0.9.2
--98feb59a8abe4bfb95a7321f536ed800
Content-Disposition: form-data; name="course"
3
--98feb59a8abe4bfb95a7321f536ed800
Content-Disposition: form-data; name="name"
name
--98feb59a8abe4bfb95a7321f536ed800
Content-Disposition: form-data; name="file"; filename="manage.py"
#!/usr/bin/env python
import os
import sys
if __name__ == "__main__":
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "config.settings.local")
from django.core.management import execute_from_command_line
execute_from_command_line(sys.argv)
--98feb59a8abe4bfb95a7321f536ed800--
HTTP/1.0 200 OK
Allow: POST, PUT, OPTIONS
Content-Type: application/json
Date: Thu, 17 Dec 2015 18:10:34 GMT
Server: WSGIServer/0.2 CPython/3.4.3
Vary: Accept, Cookie
X-Frame-Options: SAMEORIGIN
{
"data": {},
"files": "<MultiValueDict: {}>",
"raw": "<QueryDict: {'name': ['name'], 'course': ['3'], 'file': [<InMemoryUploadedFile: manage.py ()>]}>"
}
So I could just use PUT. That is however not ideal as the client does not control what the file is named or where it is located on the server. POST is more appropriate in that sense. In any sense, I don't see why PUT works when POST doesn't.
It is a known issue for the version you're using. Upgrading django rest framework to the latest version will solve the problem. However, you can PUT the request as a workaround.
I am trying to perform a simple PUT with JSON data in the tastypie-Django Python application. But I'm seeing a 401 response error when I call it through code, but no error when I execute a cURL command from the terminal. I have the code:
data_to_update = json.dumps({ "NAME" : username,
"type": mytype
})
headers = { "Content-type": "application/json",
"Authorization: ApiKey": '{0}:{1}'.format(username, key)
}
conn = httplib.HTTPConnection('localhost:8000')
conn.set_debuglevel(1)
conn.request('PUT', '/api/v1/table/1/', data_to_update, headers=headers)
response = conn.getresponse()
print response.status, response.reason
conn.close()
and I see the output:
send: u'PUT /api/v1/table/10/ HTTP/1.1\r\nHost: localhost:8000\r\nAccept-Encoding: identity\r\nContent-Length: 148\r\nContent-type: application/json\r\nAuthorization: ApiKey: api:79910a14-a82c-41f9-bb79-458247e6b31a\r\n\r\n{"username": "johnny", "type": "admin_user", "resource_uri": "/api/v1/table/10/"}'
reply: 'HTTP/1.0 401 UNAUTHORIZED\r\n'
header: Date: Fri, 15 Aug 2014 20:07:36 GMT
header: Server: WSGIServer/0.1 Python/2.7.5
header: X-Frame-Options: SAMEORIGIN
header: Content-Type: text/html; charset=utf-8
401 UNAUTHORIZED
But when I run the cURL command via terminal:
curl -X PUT -v -H "Content-Type: application/json" -H "Authorization: ApiKey api:79910a14-a82c-41f9-bb79-458247e6b31a" --data '{"username": "johnny", "type": admin_user, "resource_uri": "/api/v1/table/1/"}' http://localhost:8000/api/v1/table/10/
And it works. (It results in 204 NO CONTENT, which is expected). However, I can't find any differences between these two calls and their data. Does anyone have any ideas or pointers where I'm going wrong here?
"Authorization: ApiKey": '{0}:{1}'.format(username, key)
That's not a valid header. It should look like this:
"Authorization": 'ApiKey {0}:{1}'.format(username, key)
With your code you were sending this (in my opinion a decent library should raise an exception since a header name cannot contain a colon):
Authorization: ApiKey: username:key
instead of this:
Authorization: ApiKey username:key