I am trying to send a POST request with Python to upload a file. I'm converting the following sample code to Python but I'm not familiar with how to set this up.
POST /path/to/upload/script HTTP/1.0
Connection: Keep-Alive
User-Agent: My Client App v1.0
Host:
https://bulksell.ebay.com/ws/eBayISAPI.dll?FileExchangeUpload
Content-type: multipart/form-data;
boundary=THIS_STRING_SEPARATES
Content-Length: 256
--THIS_STRING_SEPARATES
Content-Disposition: form-data; name="token"
12345678987654321
--THIS_STRING_SEPARATES
Content-Disposition: form-data; name="file";
filename="listings.csv"
Content-Type: text/csv
... contents of listings.csv ...
--THIS_STRING_SEPARATES
I think I need to set the headers as follows:
headers = {
"Connection": "Keep-Alive",
"User-Agent": "My Client App v1.0",
"Host": "https://bulksell.ebay.com/ws/eBayISAPI.dll?FileExchangeUpload"
"Content-type": "multipart/form-data;"
"Content-Length": "256",
"Host": "https://bulksell.ebay.com/ws/eBayISAPI.dll?FileExchangeUpload",
...
}
Do I need to include these --THIS_STRING_SEPERATES strings?
How do I include my token here? The example just sends it alone.
What would be the correct way to format this for a request.post?
Thank you.
Related
I am trying to send an image from my flutter mobile app to my ComputerVision flask app.
Everything is also working normally on my local.
After deploying my Flask app using Azure Web service, everything seems to be working fine, using the browser works, POSTMAN works, but whenever I am posting from the Flutter app, I always get this 500 error code.
Peering into the SUKU console, it seems that there is a problem with the PIL package?
2022-11-03T04:22:27.232127482Z: [ERROR] File "/tmp/8dabcd6400b99b6/antenv/lib/python3.9/site-packages/PIL/TiffImagePlugin.py", line 878, in tobytes
2022-11-03T04:22:27.232130983Z: [ERROR] data = ifd.tobytes(offset)
2022-11-03T04:22:27.232134283Z: [ERROR] File "/tmp/8dabcd6400b99b6/antenv/lib/python3.9/site-packages/PIL/TiffImagePlugin.py", line 887, in tobytes
2022-11-03T04:22:27.232137883Z: [ERROR] "<table: %d bytes>" % len(data) if len(data) >= 16 else str(values)
2022-11-03T04:22:27.232142983Z: [ERROR] TypeError: object of type 'int' has no len()
This is the code I'm using to send the POST request (this works fine for the Flask app in local though)
var stream = http.ByteStream(DelegatingStream.typed(imageFile.openRead()));
// get file length
var length = await imageFile.length();
// string to uri
var uri = Uri.parse(ai_url);
// create multipart request
var request = http.MultipartRequest("POST", uri);
// multipart that takes file
var multipartFile = http.MultipartFile('file', stream, length,
filename: basename(imageFile.path));
request.files.add(multipartFile);
Map<String, String> headers = {
"Content-Type": "multipart/form-data",
"Accept-Encoding": "gzip, deflate, br",
"Accept": "*/*",
"Accept": "application/json",
"Connection": "keep-alive"
};
// add file to multipart
request.headers.addAll(headers);
// send
var response = await request.send();
return await response.stream.bytesToString();
And this is the POSTMAN logs (which is working for both deployed and local FLask app)
POST / HTTP/1.1
User-Agent: PostmanRuntime/7.29.0
Accept: */*
Postman-Token: 6e9a5808-7b9a-44a9-a6e3-58509552da11
Host: [...].azurewebsites.net
Accept-Encoding: gzip, deflate, br
Connection: keep-alive
Content-Type: multipart/form-data; boundary=--------------------------040251306687332340508582
Content-Length: 87777
----------------------------040251306687332340508582
Content-Disposition: form-data; name="file"; filename="img.jpg"
<img.jpg>
----------------------------040251306687332340508582--
HTTP/1.1 200 OK
Content-Length: 400
Content-Type: text/html; charset=utf-8
Date: Thu, 03 Nov 2022 04:21:17 GMT
Server: gunicorn
[Response...]
Can anyone help me?
i am trying to send sql query to my wordpress database using adminer script but the problem im missing somthing needed to be sent as body or headers in my opinion ( if i'm wrong please connect me )
Request raw
POST /REV/adminer-4.7.5-en.php?server=localhost&username=adepfran_wp975&db=adepfran_wp975&sql=select%20*%20from%20wplj_users HTTP/1.1
Host: mywebsite
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Referer: https://mywebsite/REV/adminer-4.7.5-en.php?server=localhost&username=adepfran_wp975&db=adepfran_wp975&sql=
Content-Type: multipart/form-data; boundary=---------------------------1328964205768204682490124619
Content-Length: 425
Cookie: adminer_sid=00e0c898e031284904f8e51b591c1dee; adminer_key=320bc6e9870ffdf2f54982cb2292de87
Connection: close
Upgrade-Insecure-Requests: 1
-----------------------------1328964205768204682490124619
Content-Disposition: form-data; name="query"
select * from wplj_users
-----------------------------1328964205768204682490124619
Content-Disposition: form-data; name="limit"
-----------------------------1328964205768204682490124619
Content-Disposition: form-data; name="token"
401937:659783
-----------------------------1328964205768204682490124619--
Headers raw
-----------------------------1328964205768204682490124619
Content-Disposition: form-data; name="query"
select * from wplj_users
-----------------------------1328964205768204682490124619
Content-Disposition: form-data; name="limit"
-----------------------------1328964205768204682490124619
Content-Disposition: form-data; name="token"
401937:659783
-----------------------------1328964205768204682490124619--
also i intercepted the requests using Burp Suite to clarify further
Request raw
Request parameters
Request Headers
my actual code
ses = requests.Session()
data = {"server": "localhost",
"username": wpuser,
"db": wpdb,
"sql": "SELECT * from wplj_users"}
url="https://mywebsite/REV/adminer-4.7.5-en.php?server=localhost&username=adepfran_wp975&db=adepfran_wp975&sql=SELECT%20*%20from%20wplj_users"
request = ses.post(url,data=data )
the request without limit,query,token (Content-Disposition) does not return the wanted response , how can i pass them ?
It seems you have to send it as files=
For test I used https://httpbin.org which send back all what you get in requests so I can display it and compare with expected data
In files I used (None, "SELECT * from wplj_users") so this None will remove filename="query"
import requests
params = {
'server': 'localhost',
'username': 'adepfran_wp975',
'db': 'adepfran_wp975',
'sql': 'SELECT * from wplj_users',
}
data = {
"query": (None, "SELECT * from wplj_users"),
"limit": (None, ""),
"token": (None, "401937:659783"),
}
headers = {
'User-Agent': 'Mozilla/5.0',
#'Referer': 'https://mywebsite/REV/adminer-4.7.5-en.php?server=localhost&username=adepfran_wp975&db=adepfran_wp975&sql='
# requests.Session() should care of cookies so this header shouldn't be needed
#'Cookie': 'adminer_sid=00e0c898e031284904f8e51b591c1dee; adminer_key=320bc6e9870ffdf2f54982cb2292de87'
}
url = "https://httpbin.org/post"
#url = "https://mywebsite/REV/adminer-4.7.5-en.php"
s = requests.Session()
#r = s.get(url) # to get fresh cookies
r = s.post(url, params=params, headers=headers, files=data)
print('\n=== url ===\n')
print(r.request.url)
print('\n=== headers ===\n')
for key, val in r.request.headers.items():
print('{}: {}'.format(key, val))
print('\n=== body ===\n')
print(r.request.body.decode())
Results
=== url ===
https://httpbin.org/post?server=localhost&username=adepfran_wp975&db=adepfran_wp975&sql=SELECT+%2A+from+wplj_users
=== headers ===
User-Agent: Mozilla/5.0
Accept-Encoding: gzip, deflate
Accept: */*
Connection: keep-alive
Content-Length: 331
Content-Type: multipart/form-data; boundary=79f18e4306b943ea92a49bae21b51b9c
=== body ===
--79f18e4306b943ea92a49bae21b51b9c
Content-Disposition: form-data; name="query"
SELECT * from wplj_users
--79f18e4306b943ea92a49bae21b51b9c
Content-Disposition: form-data; name="limit"
--79f18e4306b943ea92a49bae21b51b9c
Content-Disposition: form-data; name="token"
401937:659783
--79f18e4306b943ea92a49bae21b51b9c--
I've got problem while trying to post the file to the server. I'm trying to make file upload script to server, this server is very 'Sensitive to correctness post request'
I debugged page that is sending the file to server and browser send this (TextView):
POST http://example.com/post HTTP/1.1
Host: example.com
Connection: keep-alive
Content-Length: 20625
Accept: application/json, text/javascript, */*; q=0.01
Origin: http://example.com
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.104 Safari/537.36
Content-Type: multipart/form-data; boundary=----WebKitFormBoundarykGHBkXoER9gNuVna
Referer: http://example.com/foo
Accept-Encoding: gzip, deflate
Accept-Language: pl-PL,pl;q=0.8,en-US;q=0.6,en;q=0.4,pt;q=0.2
------WebKitFormBoundarykGHBkXoER9gNuVna
Content-Disposition: form-data; name="files[]"; filename="file.zip"
Content-Type: application/octet-stream
...raw file data...
------WebKitFormBoundarykGHBkXoER9gNuVna--
However, my script is sending this (TextView):
POST http://example.com/post HTTP/1.1
Host: example.com
Connection: keep-alive
Accept-Encoding: gzip, deflate
Accept: */*
User-Agent: python-requests/2.18.1
Content-Length: 20604
--f8c266cf436941019c5a80c7d4779a57
Content-Disposition: form-data; name="files[]"; filename="file.zip"
Content-Type: application/zip
...raw file data...
--f8c266cf436941019c5a80c7d4779a57--
With causes error on server, additional note: this error started when I changed files=files to data=files
Current Code:
files = MultipartEncoder({'files[]': (filename, open(local_path,'rb'), mimetype)})
UploadFile = requests.post(self.UploadURL, data=files, allow_redirects=False)
Working code:
files = {'files[]': (filename, open(local_path,'rb'), mimetype)}
UploadFile = requests.post(self.UploadURL, files=files, allow_redirects=False)
I'm using MultipartEncoder to allow sending huge files.
I see that biggest mismatch is "boundary", but why this 'boundary' is generating in working code but in Current code not?
How to fix that?
You are not setting the Content-Type header, the MultipartEncoder provides it for you:
files = MultipartEncoder({'files[]': (filename, open(local_path,'rb'), mimetype)})
UploadFile = requests.post(
self.UploadURL, data=files, allow_redirects=False,
headers={'Content-Type': files.content_type})
The header must come from the multi-part encoding, because it is responsible for picking the boundary used to deliniate the various MIME parts in the multipart response. In your upload that's:
--f8c266cf436941019c5a80c7d4779a57
but it is generated at random each time your code runs. The header provided would look like:
Content-Type: multipart/form-data; boundary=--f8c266cf436941019c5a80c7d4779a57
I'm having some difficulties parsing multipart form data when using the django-rest-framework. I've set up a minimal view to just echo back the request data:
from rest_framework.views import APIView
from rest_framework.response import Response
from rest_framework.parsers import MultiPartParser, FormParser
class FileUpload(APIView):
parser_classes = (MultiPartParser, FormParser, )
def post(self, request, format=None, *args, **kwargs):
return Response({'raw': request.data, 'data': request._request.POST,
'files': str(request._request.FILES)})
I am expecting that raw (slightly badly named I admit) contains effectively the same data as request._request.POST and request._request.FILES.
If I POST to the view with Content-Type= application/x-www-form-urlencoded this works as expected:
$ http -v --form POST http://localhost:8000/upload/api/ course=3 name=name
POST /upload/api/ HTTP/1.1
Accept: */*
Accept-Encoding: gzip, deflate
Connection: keep-alive
Content-Length: 18
Content-Type: application/x-www-form-urlencoded; charset=utf-8
Host: localhost:8000
User-Agent: HTTPie/0.9.2
course=3&name=name
HTTP/1.0 200 OK
Allow: POST, OPTIONS
Content-Type: application/json
Date: Thu, 17 Dec 2015 16:52:37 GMT
Server: WSGIServer/0.2 CPython/3.4.3
Vary: Accept, Cookie
X-Frame-Options: SAMEORIGIN
{
"data": {
"course": "3",
"name": "name"
},
"files": "<MultiValueDict: {}>",
"raw": {
"course": "3",
"name": "name"
}
}
However if I post with Content-Type=multipart/form-data I get the following:
$ http -v --form POST http://localhost:8000/upload/api/ file#~/Projects/lms/manage.py course=3 name=name
POST /upload/api/ HTTP/1.1
Accept: */*
Accept-Encoding: gzip, deflate
Connection: keep-alive
Content-Length: 577
Content-Type: multipart/form-data; boundary=634ec7c7e89a487b89c1c07c0d24744c
Host: localhost:8000
User-Agent: HTTPie/0.9.2
--634ec7c7e89a487b89c1c07c0d24744c
Content-Disposition: form-data; name="course"
3
--634ec7c7e89a487b89c1c07c0d24744c
Content-Disposition: form-data; name="name"
name
--634ec7c7e89a487b89c1c07c0d24744c
Content-Disposition: form-data; name="file"; filename="manage.py"
#!/usr/bin/env python
import os
import sys
if __name__ == "__main__":
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "config.settings.local")
from django.core.management import execute_from_command_line
execute_from_command_line(sys.argv)
--634ec7c7e89a487b89c1c07c0d24744c--
HTTP/1.0 200 OK
Allow: POST, OPTIONS
Content-Type: application/json
Date: Thu, 17 Dec 2015 16:55:44 GMT
Server: WSGIServer/0.2 CPython/3.4.3
Vary: Accept, Cookie
X-Frame-Options: SAMEORIGIN
{
"data": {
"course": "3",
"name": "name"
},
"files": "<MultiValueDict: {'file': [<InMemoryUploadedFile: manage.py ()>]}>",
"raw": {}
}
Am I missing something here? I am using HTTPIE to generate the requests here but the same behaviour exists with curl so I'm pretty sure that is not the problem. I am using djangorestframework==3.3.0 and Django==1.8.4
EDIT:
It seems that PUTing to the url (with an otherwise identical request) achieves the desired result:
$ http -v --form PUT http://localhost:8000/upload/api/ file#~/Projects/lms/manage.py course=3 name=name
PUT /upload/api/ HTTP/1.1
Accept: */*
Accept-Encoding: gzip, deflate
Connection: keep-alive
Content-Length: 577
Content-Type: multipart/form-data; boundary=98feb59a8abe4bfb95a7321f536ed800
Host: localhost:8000
User-Agent: HTTPie/0.9.2
--98feb59a8abe4bfb95a7321f536ed800
Content-Disposition: form-data; name="course"
3
--98feb59a8abe4bfb95a7321f536ed800
Content-Disposition: form-data; name="name"
name
--98feb59a8abe4bfb95a7321f536ed800
Content-Disposition: form-data; name="file"; filename="manage.py"
#!/usr/bin/env python
import os
import sys
if __name__ == "__main__":
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "config.settings.local")
from django.core.management import execute_from_command_line
execute_from_command_line(sys.argv)
--98feb59a8abe4bfb95a7321f536ed800--
HTTP/1.0 200 OK
Allow: POST, PUT, OPTIONS
Content-Type: application/json
Date: Thu, 17 Dec 2015 18:10:34 GMT
Server: WSGIServer/0.2 CPython/3.4.3
Vary: Accept, Cookie
X-Frame-Options: SAMEORIGIN
{
"data": {},
"files": "<MultiValueDict: {}>",
"raw": "<QueryDict: {'name': ['name'], 'course': ['3'], 'file': [<InMemoryUploadedFile: manage.py ()>]}>"
}
So I could just use PUT. That is however not ideal as the client does not control what the file is named or where it is located on the server. POST is more appropriate in that sense. In any sense, I don't see why PUT works when POST doesn't.
It is a known issue for the version you're using. Upgrading django rest framework to the latest version will solve the problem. However, you can PUT the request as a workaround.
I am trying to write an automatic subtitle finder in Python 3.x using The SubDB (http://thesubdb.com/api/). I'm now working on an upload feature. However, I cannot get it to work. I keep getting a 415 error (Unsupported Media Type). On the SubDB website, a 'request sample' is given:
POST /?action=upload HTTP/1.1
Host: api.thesubdb.com
User-Agent: SubDB/1.0 (Pyrrot/0.1; http://github.com/jrhames/pyrrot-cli)
Content-Length: 60047
Content-Type: multipart/form-data; boundary=xYzZY
- - --xYzZY
Content-Disposition: form-data; name="hash"
edc1981d6459c6111fe36205b4aff6c2
- - --xYzZY
Content-Disposition: form-data; name="file"; filename="subtitle.srt"
Content-Type: application/octet-stream
[PAYLOAD]
But I do not know how to interpret this and I couldn't find an answer online. This is my current code:
def uploadSubtitle(hash, path):
params = {'action': "upload", 'hash': hash}
response = requests.post(
url=base_url.format(urllib.parse.urlencode(params)),
data=open(path,'r').read(),
headers = {
'User-Agent': user_agent,
'Content-Length': 51200,
'Content-Type': "multipart/form-data; boundary=xYzZY",
'Host': "api.thesubdb.com"
}
)
return response.status_code
Any advice would be greatly appreciated!
I was having the same issues.
You may want to see this https://github.com/ivandrofly/SubDBSharp
Try following:
POST /?action=upload HTTP/1.1
Host: api.thesubdb.com
User-Agent: SubDB/1.0 (Pyrrot/0.1; http://github.com/jrhames/pyrrot-cli)
Content-Length: [Subtitle content length including boundary length]
Content-Type: multipart/form-data; boundary=xYzZY
--xYzZY
Content-Disposition: form-data; name="hash"
edc1981d6459c6111fe36205b4aff6c2
--xYzZY
Content-Disposition: form-data; name="file"; filename="subtitle.srt"
Content-Type: application/octet-stream
[SUBTITLE CONTENT]
--xYzZY
# Content-Length:
Content length should be from --xYzZY to last --xYzZY see above.
Note: Boundary started --xYzZY and ends after [SUBTITLE CONTENT]