Develop Reference

Python PUT call fails while curl call doesn't

Best wishes (first things first!)
I want to enable/disable a PoE port on my UniFi switch. For this I aim using Python 3.9.1 (first time) with the following code:
import requests
import json
import sys
import urllib3
urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)
gateway = {"ip": "MYSERVER.COM", "port": "8443"}
headers = {"Accept": "application/json", "Content-Type": "application/json"}
login_url = f"https://{gateway['ip']}:{gateway['port']}/api/login"
login_data = {
"username": "MYUSERNAME",
"password": "MYPASSWORD"
}
session = requests.Session()
login_response = session.post(login_url, headers=headers, data=json.dumps(login_data), verify=False)
if (login_response.status_code == 200):
api_url_portoverrides = 'api/s/default/rest/device/MYDEVICEID'
poe_url = f"https://{gateway['ip']}:{gateway['port']}/{api_url_portoverrides}"
# build json for port overrides
json_poe_state_on = '{"port_overrides": [{"port_idx": 6, "portconf_id": "MYPROFILE1"}]}'
json_poe_state_off = '{"port_overrides": [{"port_idx": 6, "portconf_id": "MYPROFILE2"}]}'
post_response = session.put(poe_url, headers=headers, data=json.dumps(json_poe_state_off))
print('Response HTTP Request {request}'.format(request=post_response.request ))
else:
print("Login failed")
The login works (I get the 2 security cookies and tried them in Paw (a macOS REST API client) to see if these were ok)) but the second call, the. PUT, returns OK but noting happens.
Before I've done this in Python, I tried all my calls in Paw first and there it works. I tried everything in bash with curl and there it works too. So I am a bit at a loss here.
Anyone has an idea?
Thank you for your time!
Best regards!
Peter

Solved it! By looking into what was posted with Wireshark I saw that the payload was different. The culprit was the json.dumps function which encoded the string by putting a backslash in front of each double quote.
It works now!

Read timed out. error while sending a POST request to a node.js API

I have a node.js API as below to which I send a POST request from python as below,the issue am facing is if I remove the
headers={"Content-Type": "application/json"} the POST goes thorugh,if not i get a Read timed out. error, can anyone provide guidance on how to fix this timeout error ?
node.js endpoint
app.post("/api/bats_push",(req, res) => {
//console.log("Calling bats_push...")
const d = {
method: req.method,
headers: req.headers,
query: req.query,
body: ''
}
req.on('data', (c) => {
//console.log(c)
d.body = d.body + c
});
req.on('end', () => {
DATA.push(d);
res.end('Saved BATS job details');
//res.status(200).json({
//message: "Saved BATS job details",
//posts: req.body
//});
});
});
Python POST
try:
json"},timeout=10.0)
r = requests.post(webhook_url,data=json_data.encode("utf8"),verify=False,headers={"Content-Type": "application/json"})
print "posted"
print(r.status_code, r.reason)
print r.url
print r.text
except Exception as e:
print (e)
Error:-
InsecureRequestWarning)
HTTPSConnectionPool(host='company.com', port=443): Read timed out. (read timeout=10.0)

I seems that you are using express.js. I believe that your problem is, that body is actually already parsed. You can check it by reading req.body. The situation is caused because express.js already read whole body (due to the content type) and trying to read body again will cause timeout (event data and event end are not emitted).
There are several ways how to fix it.
disable express.js body parser - or reconfigure it to ignore json
remove reading body code and use directly req.body
app.post("/api/bats_push",(req, res) => {
//console.log("Calling bats_push...")
const d = {
method: req.method,
headers: req.headers,
query: req.query,
body: req.body
}
DATA.push(d);
res.end('Saved BATS job details');
});

according to requests why not use this:
#replace this
r = requests.post(webhook_url,data=json_data.encode("utf8"),verify=False,headers={"Content-Type": "application/json"})
#by this...assuming that 'data' is a dict
r = requests.post(webhook, json=data, verify=False)

Looks like it could be something related to your SSL.
Check if you get the same error sending a request to your localhost with the server running.

From your question,the key word is:
if I remove the headers={"Content-Type": "application/json"} the POST goes thorugh.
The reason may clear:it is a wrong way use about the header.
Simpler say: the node.js app check the header before into the logic code.
if do not send the header by ourself,the requests part use the default headers below:
{
'User-Agent': 'python-requests/2.22.0',
'Accept-Encoding': 'gzip, deflate',
'Accept': '*/*',
'Connection': 'keep-alive',
}
through the requests's code can print the default headers when post.
you just use the header
{"Content-Type": "application/json"}
may resulting in the node.js app think the reqeust not legitimate(the word I do not know how to explain in English).
If the node.js app is developed by yourself, you can try to find the frame's check after create a tcp connection and before the logic code,the way is to read the source code.
If the node.js app is not developed by yourself,try to change the header mixing the default header to find which header key checked by the node.js app.
But in my thought,It is import ,about the node.js app's interface description:
we just use by interface description engouth,just need to know the error from the api header's check, which the description should show but not show to us?
Hope to help you.

POST request using urllib2 doesn't correctly send data (401 error)

I am trying to make a POST request in Python 2, using urllib2. My code is currently as follows;
url = 'http://' + server_url + '/playlists/upload?'
data = urllib.urlencode(OrderedDict([("sectionID", section_id), ("path", current_playlist), ("X-Plex-Token", plex_token)]))
req = urllib2.Request(url, data)
response = urllib2.urlopen(req)
d = response.read()
print(d)
'url' and 'data' return correctly formatted with the variables, I know this because I can copy their output into Postman for checking and the POST works fine (see example url below)
http://192.168.1.96:32400/playlists/upload?sectionID=11&path=D%3A%5CMedia%5CPPP%5Ctmp%5Cplex%5CAmbient.m3u&X-Plex-Token=XXXXXXXXX
When I run my Python code I get a 401 error returned, presumably meaning the X-Plex-Token parameter was not correctly sent, hence I am not allowed access.
Can anyone tell me where I'm going wrong? Help is greatly appreciated.

Have you tried removing the question mark and not using OrderedDict (no idea why you would need that) ?
url = 'http://' + server_url + '/playlists/upload'
data = urllib.urlencode({"sectionID":section_id), "path":current_playlist,"X-Plex-Token":plex_token})
req = urllib2.Request(url, data)
response = urllib2.urlopen(req)
d = response.read()
print(d)
Of course you should be using requests instead anyway:
import requests
r = requests.post('http://{}/playlists/upload'.format(server_url), data = {"sectionID":section_id), "path":current_playlist,"X-Plex-Token":plex_token})
print r.url
print r.text
print r.json

I've ended up switching to Python 3, as I didn't realise that the requests module was included by default. Still no idea why the above wasn't working, but maybe something to do with the lack of headers
headers = {'cache-control': "no-cache"}
edit:
This is what I'm using now, as mentioned above I probably don't need OrderedDict.
import requests
url = 'http://' + server_url + '/playlists/upload'
headers = {'cache-control': "no-cache"}
querystring = urllib.parse.urlencode(OrderedDict([("sectionID", section_id), ("path", current_playlist), ("X-Plex-Token", plex_token)]))
response = requests.request("POST", url, data = "", headers = headers, params = querystring)
print(response.text)

How to refer a local file for Azure emotion api

I am trying to refer a local jpg file for using in Azure Emotion API.
To do this, I refer my file through "file:///" like below.
body = "{'url': 'file:///Users/jonghkim/dev_jhk/Research/Crowdfunding/Face_Analysis/me.jpg'}"
But the response says "Invalid image URL." How could I fix it?
{"error":{"code":"InvalidUrl","message":"Invalid image URL."}}
Whole code looks like below.
########### Python 2.7 #############
import httplib, urllib, base64
headers = {
# Request headers. Replace the placeholder key below with your subscription key.
'Content-Type': 'application/json',
'Ocp-Apim-Subscription-Key': '***********************',
}
params = urllib.urlencode({
})
# Replace the example URL below with the URL of the image you want to analyze.
body = "{'url': 'file:///Users/jonghkim/dev_jhk/Research/Crowdfunding/Face_Analysis/me.jpg'}"
try:
conn = httplib.HTTPSConnection('westus.api.cognitive.microsoft.com')
conn.request("POST", "/emotion/v1.0/recognize?%s" % params, body, headers)
response = conn.getresponse()
data = response.read()
print(data)
conn.close()
except Exception as e:
print("[Errno {0}] {1}".format(e.errno, e.strerror))

I solved this problem. The true reason was two fold. At first, when we refer local file, we should use 'Content-Type': 'application/octet-stream' in a header.
The second problem is that the image should satisfy the condition of Azure (learn.microsoft.com/ko-kr/azure/cognitive-services/emotion/fa‌​q).
Full code is here:
########### Python 2.7 #############
import httplib, urllib, base64
headers = {
# Request headers. Replace the placeholder key below with your subscription key.
'Content-Type': 'application/octet-stream',
'Ocp-Apim-Subscription-Key': '**************************',
}
params = urllib.urlencode({
})
# Replace the example URL below with the URL of the image you want to analyze.
body = open('test.jpg','rb').read()
conn = httplib.HTTPSConnection('westus.api.cognitive.microsoft.com')
conn.request("POST", "/emotion/v1.0/recognize?%s" % params, body, headers)
response = conn.getresponse()
data = response.read()
print(data)
conn.close()

You're executing the emotion API within Cognitive Services - just take a look at the URI. This code is not being executed locally. It's a service. Being run somewhere else.
So, when the service gets the URL (via url in the body), it then needs to reach out to that resource, which is impossible to do if the resource is on your computer. And file:// is going to be an invalid scheme because the service won't be reading from its own file system.
You'll need to have your resource in an accessible place (e.g. a public or SAS-signed blob, an image link from a website, etc).

HTTPS POST request Python, returning .csv

I want to make a post request to a HTTPS-site that should respond with a .csv file. I have this Python code:
try:
#conn = httplib.HTTPSConnection(host="www.site.com", port=443)
=> Gives an BadStatusLine: ' ' error
conn = httplib.HTTPConnection("www.site.com");
params = urllib.urlencode({'val1':'123','val2':'abc','val3':'1b3'})
conn.request("POST", "/nps/servlet/exportdatadownload", params)
content = conn.getresponse()
print content.reason, content.status
print content.read()
conn.close()
except:
import sys
print sys.exc_info()[:2]
Output:
Found 302
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<HTML><HEAD>
<TITLE>302 Found</TITLE>
</HEAD><BODY>
<H1>Found</H1>
The document has moved here.<P>
<HR>
<ADDRESS>Oracle-Application-Server-10g/10.1.3.5.0 Oracle-HTTP-Server Server at mp-www1.mrco.be Port 7778</ADDRESS>
</BODY></HTML>
What am I doing wrong? How do I 'catch' the returning .csv file?
I tried the POST request with an Chrome Extentions (Advanced Rest Client, and that's working...)

The HTML output with the 302 error is because you're connecting to the site using HTTP instead of HTTPS, which is in your code here:
conn = httplib.HTTPConnection("www.site.com");
Presumably you're doing that because of some other error with the commented out section of your code.
If I were trying to do this I would use Requests, which comes with very clear documentation. With Requests the code would be something like:
import requests
url = "https://www.example.com/nps/servlet/exportdatadownload"
payload = { "val1": "123", "val2": "abc", "val3": "1b3" }
r = requests.post(url, data=payload, verify=True)
The CSV file should be in r.content, which can be written to a file.