Develop Reference

POST request from Flutter to deployed Azure Flask application not working

I am trying to send an image from my flutter mobile app to my ComputerVision flask app.
Everything is also working normally on my local.
After deploying my Flask app using Azure Web service, everything seems to be working fine, using the browser works, POSTMAN works, but whenever I am posting from the Flutter app, I always get this 500 error code.
Peering into the SUKU console, it seems that there is a problem with the PIL package?
2022-11-03T04:22:27.232127482Z: [ERROR] File "/tmp/8dabcd6400b99b6/antenv/lib/python3.9/site-packages/PIL/TiffImagePlugin.py", line 878, in tobytes
2022-11-03T04:22:27.232130983Z: [ERROR] data = ifd.tobytes(offset)
2022-11-03T04:22:27.232134283Z: [ERROR] File "/tmp/8dabcd6400b99b6/antenv/lib/python3.9/site-packages/PIL/TiffImagePlugin.py", line 887, in tobytes
2022-11-03T04:22:27.232137883Z: [ERROR] "<table: %d bytes>" % len(data) if len(data) >= 16 else str(values)
2022-11-03T04:22:27.232142983Z: [ERROR] TypeError: object of type 'int' has no len()
This is the code I'm using to send the POST request (this works fine for the Flask app in local though)
var stream = http.ByteStream(DelegatingStream.typed(imageFile.openRead()));
// get file length
var length = await imageFile.length();
// string to uri
var uri = Uri.parse(ai_url);
// create multipart request
var request = http.MultipartRequest("POST", uri);
// multipart that takes file
var multipartFile = http.MultipartFile('file', stream, length,
filename: basename(imageFile.path));
request.files.add(multipartFile);
Map<String, String> headers = {
"Content-Type": "multipart/form-data",
"Accept-Encoding": "gzip, deflate, br",
"Accept": "*/*",
"Accept": "application/json",
"Connection": "keep-alive"
};
// add file to multipart
request.headers.addAll(headers);
// send
var response = await request.send();
return await response.stream.bytesToString();
And this is the POSTMAN logs (which is working for both deployed and local FLask app)
POST / HTTP/1.1
User-Agent: PostmanRuntime/7.29.0
Accept: */*
Postman-Token: 6e9a5808-7b9a-44a9-a6e3-58509552da11
Host: [...].azurewebsites.net
Accept-Encoding: gzip, deflate, br
Connection: keep-alive
Content-Type: multipart/form-data; boundary=--------------------------040251306687332340508582
Content-Length: 87777
----------------------------040251306687332340508582
Content-Disposition: form-data; name="file"; filename="img.jpg"
<img.jpg>
----------------------------040251306687332340508582--
HTTP/1.1 200 OK
Content-Length: 400
Content-Type: text/html; charset=utf-8
Date: Thu, 03 Nov 2022 04:21:17 GMT
Server: gunicorn
[Response...]
Can anyone help me?

my proxy server in python

 I'm a fresh man to computer network, and i'm tring to make a proxy server of my own.
 But when I send the request i received from the client to the server, i can't get the response from the server. My code get an exception here :
try:
# connect
serverSock.connect((hostName, 80))
# get the client's request
fp = open("requestCache.txt", "r")
message = fp.read()
fp.close()
# send to the target server
serverSock.send(message)
response = serverSock.recv(4096)
# send to the client
tcpCliSock.send(response)
except:
print('connect failed!')
serverSock.close()
the following is the request received from the client
GET /www.baidu.com HTTP/1.1
Host: localhost:3009
Connection: keep-alive
Cache-Control: max-age=0
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.167 Safari/537.36
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
Accept-Encoding: gzip, deflate, br
Accept-Language: zh-CN,zh;q=0.9

You generally want to avoid enclosing huge chunks of code in try...except blocks unless you understand exactly what will happen when an exception is raised. I usually keep the try...except blocks as minimal as possible and catch as specific errors as possible:
try:
serverSock.connect((hostName, 80))
except OSError as e:
# handle e
You're actually catching and throwing away a very useful error:
TypeError Traceback (most recent call last)
<ipython-input-13-78a255a190f8> in <module>()
10
11 # send to the target server
---> 12 serverSock.send(message)
13 response = serverSock.recv(4096)
14
TypeError: a bytes-like object is required, not 'str'
Your message is a string, but sockets deal with bytes. To fix it, read the file's contents as bytes instead ('rb' mode instead of just 'r'):
# connect
serverSock.connect((hostName, 80))
# get the client's request
with open("requestCache.txt", "rb") as handle:
message = handle.read()
# send to the target server
serverSock.send(message)
response = serverSock.recv(4096)
# send to the client
tcpCliSock.send(response)

http POST message body not received

Im currently creating a python socket http server, and I'm working on my GET and POST requests. I got my GET implementation working fine, but the body element of the POST requests won't show up.
Code snippet:
self.host = ''
self.port = 8080
self.listener = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.listener.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
self.listener.bind((self.host, self.port))
self.listener.listen(1)
while True:
client_connection, client_address = self.listener.accept()
request = client_connection.recv(2048)
print request
This code yields the http header after processing the post request from the webpage:
POST /test.txt HTTP/1.1
Host: localhost:8080
Content-Type: application/x-www-form-urlencoded
Origin: http://localhost:8080
Content-Length: 21
Connection: keep-alive
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_5) AppleWebKit/601.6.17 (KHTML, like Gecko) Version/9.1.1 Safari/601.6.17
Referer: http://localhost:8080/
Accept-Language: nb-no
Accept-Encoding: gzip, deflate
But there is no body, so the question is why am i not receiving the http body when i know it is sent?
Thanks!

while True:
client_connection, client_address = self.listener.accept()
request = client_connection.recv(2048)
print request
recv does not read exactly 2048 bytes but it reads up to 2048 bytes. If some data arrive recv will return with the data even if more data might follow. My guess is that in your case the client is first sending the HTTP header and then the body. If NAGLE algorithms is off at the client side (common) it is likely that your first recv will only get the header and that you would need another recv for the body. This would explain what happens in your case: you get the header but not the body since you don't do another recv.
But even that would be a too simple implementation which will go wrong sooner or later. To make it correctly you should implement the HTTP protocol correctly: first read the HTTP header which might need multiple recv if the header is large. Then you should parse the header, figure out the size of the body (Content-length header) and read the remaining bytes.

Python Requests module - proxy not working

I'm trying to connect to website with python requests, but not with my real IP. So, I found some proxy on the internet and wrote this code:
import requests
proksi = {
'http': 'http://5.45.64.97:3128'
}
x = requests.get('http://www.whatsmybrowser.org/', proxies = proksi)
print(x.text)
When I get output, proxy simple don't work. Site returns my real IP Address. What I did wrong? Thanks.

The answer is quite simple. Although it is a proxy service, it doesn't guarantee 100% anonymity. When you send the HTTP GET request via the proxy server, the request sent by your program to the proxy server is:
GET http://www.whatsmybrowser.org/ HTTP/1.1
Host: www.whatsmybrowser.org
Connection: keep-alive
Accept-Encoding: gzip, deflate
Accept: */*
User-Agent: python-requests/2.10.0
Now, when the proxy server sends this request to the actual destination, it sends:
GET http://www.whatsmybrowser.org/ HTTP/1.1
Host: www.whatsmybrowser.org
Accept-Encoding: gzip, deflate
Accept: */*
User-Agent: python-requests/2.10.0
Via: 1.1 naxserver (squid/3.1.8)
X-Forwarded-For: 122.126.64.43
Cache-Control: max-age=18000
Connection: keep-alive
As you can see, it throws your IP (in my case, 122.126.64.43) in the HTTP header: X-Forwarded-For and hence the website knows that the request was sent on behalf of 122.126.64.43
Read more about this header at: https://www.rfc-editor.org/rfc/rfc7239
If you want to host your own squid proxy server and want to disable setting X-Forwarded-For header, read: http://www.squid-cache.org/Doc/config/forwarded_for/

how to debug requests library?

I'm trying to convert my python script from issuing a curl command via os.system() to using requests. I thought I'd use pycurl, but this question convinced me otherwise. The problem is I'm getting an error returned from the server that I can see when using r.text (from this answer) but I need more information. Is there a better way to debug what's happening?
for what it's worth I think the issue revoles around converting my --data flag from curl/pycurl to requests. I've created a dictionary of the params i was passing to --data before. My guess is that one of those isn't valid but how can I get more info to know for sure?
example:
headers2 = {"Accept":"*/*", \
"Content-Type":"application/x-www-form-urlencoded", \
"User-Agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/27.0.1453.116 Safari/537.36", \
"Origin":"https://somedomain.com", \
"X-Requested-With":"XMLHttpRequest", \
"Connection":"keep-alive", \
"Accept-Language":"en-US,en;q=0.8", \
"Referer":"https://somedomain.com/release_cr_new.html?releaseid=%s&v=2&m=a&prev_release_id=%s" % (current_release_id, previous_release_id), \
"Host":"somedomain.com", \
"Accept-Encoding":"gzip,deflate,sdch", \
"Cookie":'cookie_val'}
for bug_id in ids:
print bug_id
data = {'dump_json':'1','releaseid':current_release_id, 'v':'2','m':'a','prev_release_id': previous_release_id,'bug_ids': bug_id, 'set_cols':'sqa_status&sqa_updates%5B0%5D%5Bbugid%5D=' + bug_id + '&sqa_updates%5B0%5D%5Bsqa_status%5D=6'}
print 'current_release_id' , data['releaseid']
print 'previous_release_id', data['prev_release_id']
r = requests.post(post_url, data=json.dumps(data), headers=headers2)
print r.text
The output I'm getting is a pretty generic html message that I've seen before when I've queried the server in the wrong way. So I know I'm reaching the right server at least.
I'm not really expecting any output. This should just post to the server and update a field in the DB.

Anatomy of an http response
Example (loading this page)
HTTP/1.1 200 OK
Cache-Control: public, max-age=60
Content-Type: text/html; charset=utf-8
Content-Encoding: gzip
Expires: Fri, 27 Sep 2013 19:22:41 GMT
Last-Modified: Fri, 27 Sep 2013 19:21:41 GMT
Vary: *
X-Frame-Options: SAMEORIGIN
Date: Fri, 27 Sep 2013 19:21:41 GMT
Content-Length: 12706
<!DOCTYPE html>
<html>
... truncated rest of body ...
The first line is the status line and consists of the status code and status text.
Headers are key/value pairs. Headers are ended with an empty new line. The empty line denotes there are no more headers and the start of the payload / body follows.
body consumes the rest of the message.
The following explains how to extract the 3 parts:
Status Line
Use the following to get the status line sent back from the server
>>> bad_r = requests.get('http://httpbin.org/status/404')
>>> bad_r.status_code
404
>>> bad_r.raise_for_status()
Traceback (most recent call last):
File "requests/models.py", line 832, in raise_for_status
raise http_error
requests.exceptions.HTTPError: 404 Client Error
(source)
Headers:
r = requests.get('http://en.wikipedia.org/wiki/Monty_Python')
# response headers:
r.headers
# request headers:
r.request.headers
Body
Use r.text.
Post Request Encoding
The 'content-type' you send to the server in the request should match the content-type you're actually sending. In your case, you are sending json but telling the server you're sending form data (which is the default if you do not specify).
From the headers you show above:
"Content-Type":"application/x-www-form-urlencoded",
But your request.post call sets data=json.dumps(data) which is JSON. The headers should say:
"Content-type": "application/json",

The value returned from the request object contains the request information under .request.
Example:
r = requests.request("POST", url, ...)
print("Request headers:", r.request.headers)
print("Request body:", r.request.body)
print("Response status code:", r.status_code)
print("Response text:", r.text.encode('utf8'))