Develop Reference

zip file upload with Python Requests Not working

Hi am attempting to upload a file to a server when making a post with requests but everytime I get errors. If do the same with cURL it goes through but Im not familiar with cURL or uploading files really, mostly get requests, so I have no clue what its doing differently. I am on windows and am running the below. This is being uploaded to mcafee epo so Im not sure if their api just super picky or what the difference is but every python example ive tried for uploading a file via requests module has failed for me.
url = "https://server.url.com:1234/remote/repository.checkInPackage.do?&allowUnsignedPackages=True&option=Normal&branch=Evaluation"
user = "domain\user"
password = "mypass"
filepath = "C:\\my\\folder\\with\\afile.zip"
with open(filepath, "rb") as f:
file_dict = {"file": f}
response = requests.post(url, auth=(user, password), files=file_dict)
I usually get a error as follows:
'Error 0 :\r\njava.lang.reflect.InvocationTargetException\r\n'
if I use cURL it works though
curl.exe -k -s -u "domain\username:mypass" "https://server.url.com:1234/remote/repository.checkInPackage.do?&allowUnsignedPackages=True&option=Normal&branch=Evaluation" -F file=#"C:\my\folder\with\afile.zip"
I cant really see the difference though and am wondering what is being done differently on the backend for cURL or what I could be doing wrong when using python.

How to implement a CURL -o in Python using request or aoihttp

I need to do a get to a URL with certain Headers and Parameters in my python code and then write to an excel file. I am trying the below code but getting the below error when I try to open the excel file. Also given below is a redacted version of a CURL I use to test this API endpoint with success that I cannot replicate in my python code
Python Code:
import requests
rheaders = {
"Content-type": "application/json",
"Accept": "application/json",
"Authorization": "Token XXX",
}
url = "https://api.YYY"
r = requests.get(url, headers=rheaders)
with open("test.xlsx" , mode="wb") as output:
output.write(r.content)
Error: Excel cannot open the file 'test.xlsx' because the file format or file extension is not valid. Verify that the file has not been corrupted and that the file extension matches the format of the file.
The assosciated CURL that runs fine in osx terminal and writes the output file without errors
curl -H "Accept: application/json; indent=4" -H 'Content-Type: application/json' -H 'Authorization: Token XXX' https://api.YYY --output ~/Downloads/output.xlsx
I have tried replacing the content-type header with this:
Content-Type: application/vnd.openxmlformats-officedocument.spreadsheetml.sheet
No luck. Any suggestions? The above is my attempt with python library requests. I have also tried using aiohttp

The issue is that Excel cannot understand the format of the data that is being written into the file, because Python by itself does not support the Excel format. Try using a library such as xlrd to write to it. Alternatively, you could write to a .csv file instead, which wouldn't require any extra libraries and would still be readable by Excel

Python requests zip upload makes zipfile unreadable in Windows

I'm trying to upload a zipfile to a Server using Python requests. The upload works fine. However the uploaded file cannot be opened using Windows Explorer or ark. I suppose there's some problem with mime-type or content-Length.
Oddly, uploading the file using curl, does not seem to cause the same problem.
Here is my python code for the request:
s = requests.Session()
headers = {'Content-Type': 'application/zip'}
zip = open('file.zip', 'rb')
files = {'file': ('file.zip', zip, 'application/zip')}
fc = {'Content-Disposition': 'attachment; filename=file.zip'}
headers.update(fc)
r = requests.Request('POST', url, files=files, headers=headers, auth=(user, password))
prepared = r.prepare()
resp = s.send(prepared)
This is the curl code, which works flawlessly:
curl -X POST \
-ik \
-u user:password \
--data-binary '#file.zip' \
-H 'Content-Type: application/zip' \
-H "Content-Disposition: attachment; filename=file.zip" \
url
Uploading the file works in both, the Server also seems to recognize the content-type. However the file is rendered invalid when re-downloading. The zifile is readable before sending via requests or after sending with normal curl, using --data-binary.
Opening the downloaded zifile with unip or file-roller works either way.
EDIT:
I was uploading two files successively. Oddly the error was fixed when uploading the exact same files in reverse order.
This has NOT been a python problem. When trying with standard curl
I must have accidentally reversed the order, which is why it has been working.
I can not explain this behavior nor do I have a fix for it.
In conclusion: Uploading the bigger file first did the trick.
All of the above seems to be applicable in curl, pycurl and python requests, so I assume it's some kind of bug in one of the curl libraries.

python 3 tornados file body with headers

I'm tying to write some simple app on python3 and tornado for server, and requests for client, and I'm getting some headers in 'self.request.body', which I can't dispose of. For instance, for file 'blahblahblah', I get:
--cb5f6ba84bdf42d382dfd3204f6307c7\r\nContent-Disposition: form-data; name="file"; filename="1.bin"\r\n\r\nblahblahblah\n\r\n--cb5f6ba84bdf42d382dfd3204f6307c7--\r\n
Files are sent by
f = {'file': open(FILE, 'rb')}
requests.post(URL_UPLOAD, files=f)
and received by
class UploadHandler(tornado.web.RequestHandler):
def post(self, filename):
with open(Dir + filename, 'wb') as f:
f.write(self.request.body)
My full code can be seen here
When I send the file by curl with curl -X POST -d $(cat ./1.bin) http://localhost:8080/upload/1.bin I get the correct file, but without \n.
There must be something I missed. Please can someone help me with that? Thank You.

There are two ways to upload files: simply using the file as the request body (usually, but not necessarily, with the HTTP PUT method), or using a multipart wrapper (usually with the HTTP POST method). If you upload the file from an HTML form, it will usually use the multipart wrapper. Your requests example is using a multipart wrapper and the curl one is not; your server is not expecting the wrapper.
To use a multipart wrapper: in requests, pass files= as you've done here. With curl, see this answer: Using curl to upload POST data with files. On the server, use self.request.files instead of self.request.body: http://www.tornadoweb.org/en/stable/httpserver.html#tornado.httpserver.HTTPRequest.files
To not use the multipart wrapper, use data=open(FILE, 'rb').read() from requests, and keep the other two components the same.
It is possible to support both styles simultaneously on the server: use self.requests.files when self.request.headers['Content-Type'] == 'multipart/form-data' and self.request.body otherwise.

transpose curl PUT to requests.put

I would like to transpose the curl command (which upload a local file to rackspace)
curl -X PUT -T screenies/hello.jpg -D - \
-H "X-Auth-Token: fc81aaa6-98a1-9ab0-94ba-aba9a89aa9ae" \
https://storage101.dfw1.clouddrive.com/v1/CF_xer7_343/images/hello.jpg
to python requests. So far I have:
url = 'http://storage.clouddrive.com/v1/CF_xer7_343/images/hello.jpg'
headers = {'X-Auth-Token': 'fc81aaa6-98a1-9ab0-94ba-aba9a89aa9ae'}
request = requests.put(url, headers=headers, data={})
where do I specify I want to upload screenies/hello.jpg?
I understand -T in curl represents 'to FTP server', but I have searched the requests's github but cannot find mention of FTP.

No, -T just means 'upload this file', which can be used with FTP but is not limited to that.
You can just upload the file data as the data parameter:
with open('screenies/hello.jpg', 'rb') as image:
request = requests.put(url, headers=headers, data=image)
where data will read and upload the image data for you.