I am using Web2Py to create a simple app which sends Push notifications through UrbanAirship. For some reason, I am getting a 400 response when I try to send it through my code. It UA API works fine using REST client. This is my code:
url = 'https://go.urbanairship.com/api/push/'
passman = urllib2.HTTPPasswordMgrWithDefaultRealm()
# this creates a password manager
passman.add_password(None, url, username, password)
# because we have put None at the start it will always
# use this username/password combination for urls
# for which `theurl` is a super-url
authhandler = urllib2.HTTPBasicAuthHandler(passman)
# create the AuthHandler
opener = urllib2.build_opener(authhandler)
urllib2.install_opener(opener)
# All calls to urllib2.urlopen will now use our handler
# Make sure not to include the protocol in with the URL, or
# HTTPPasswordMgrWithDefaultRealm will be very confused.
# You must (of course) use it when fetching the page though.
values = {"device_tokens": ["<DEVICE TOKEN>"], "aps": {"alert": "Hello!"}}
data = urllib.urlencode(values)
headers = {'Content-Type': 'application/json'}
req = urllib2.Request(url, data, headers)
try:
response = urllib2.urlopen(req)
return response
except IOError, e:
if e.code == 200:
return "Push sent!"
else:
return 'The server couldn\'t fulfill the request. Error: %d' % e.code
As far as I can understand, the problem is in the format of data being sent. Where am I going wrong?
The urllib.urlencode function is for making a URL-encoded parameter body (Content-Type: application/x-www-form-urlencoded). For JSON, which is apparently what you want, use json.dumps instead.
Related
I am writing a Python3 script to trigger an Axis camera via their HTTP API. The string they need in the url to trigger the Digital Input (with authorization) is:
"http://{ip}/axis-cgi/param.cgi?action=update&IOPort.I0.Input.Trig={open/closed}"
If I put this in a browser it works AKA response 200 - OK. When I run this on my Linux machine using the urllib request.
#encode user,pass
values = { 'username': username,'password': password }
data = urllib.urlencode(values)
#Axis param.cgi action
action = "update"
trig = "closed"
cgi_args = {"action": action, "IOPort.I0.Input.Trig": trig}
cgi_data = urllib.urlencode(cgi_args)
url = "http://{ip}/axis-cgi/param.cgi?{data}".format(ip=ip, data=cgi_data)
req = urllib2.Request(url, data, headers={'Content-type': 'text/html'})
print(req.get_full_url())
response = urllib2.urlopen(req)
result = response.read()
print (result)
The output is:
http://192.168.50.191/axis-cgi/param.cgi?action=update&IOPort.I0.Input.Trig=closed
action must be specified
I know that I am authenticated otherwise I get Unauthorized response from server.
As you can see for debugging I print the req.get_full_url() to make sure I've built the url string correctly, however the server responds with action must be specified which is what I get in the browser when I just have http://192.168.50.191/axis-cgi/param.cgi? in the address bar. So the action portion of the URL appears to not be making it to the server.
I've tried:
Using %3F as the ? character, but I get 404 error
Embedding the data in the data parameter does not work either
Anything I am missing here?
used pycurl with digest authorization for a GET request with auth:
def configCurl():
username = "user"
password = "pass"
auth_mode = pycurl.HTTPAUTH_DIGEST
curl = pycurl.Curl()
curl.setopt(pycurl.HTTPAUTH, auth_mode)
curl.setopt(pycurl.USERPWD, "{}:{}".format(username, password))
curl.setopt(pycurl.WRITEFUNCTION, lambda x: None)
return curl
curl = configCurl()
curl.setopt(curl.URL, url)
curl.perform()
curl.close()
I am getting HTTP POST error when I am trying to connect to a Service Now Instance for Change Request Automation using Python. Here is the script I am using with Python 3.4.4
# SNOW CR AUTOMATION SCRIPT
import requests
import json
# put the ip address or dns of your SNOW API in this url
url = 'http://<>/change_request.do?JSONv2&sysparm_action=insert'
data= {
'short_description': '<value>',
'priority': '<value>',
'reason': '<value>',
'u_reason_for_change': '<value>',
'u_business_driver': '<value>',
'u_plan_of_record_id': '<value>'
}
print ("Data Inserted :")
print (data)
#Content type must be included in the header
header = {"Authorization":"Basic V1NfRVRPX1ROOkBiY2RlNTQzMjE=","Content- Type":"application/json"}
#Performs a POST on the specified url.
response = requests.request('POST', url, auth=("<value>","<value>"), json=data, headers=header)
print ( " Header is : ")
print (response.headers)
print (" ")
print ( "HTTP Response is :" )
print (response)
print (" ")
print ("***********************")
print (" Output : ")
print ( response.text)
I am getting an error as below while running the above script.
Output :
{"reason":null,"error":"Request JSON object for insert cannot be null."}
I am not sure why this error is thrown. Can anybody please help on this ?
This is a working example I tested on my instance. I am using REST Table API to insert a change request. It's not true that it can not be http. It's whatever protocol your instance allows to connect, say from browser.
#Need to install requests package for python
#easy_install requests
import requests
# Set the request parameters
url = '<yourinstance base url>/api/now/table/change_request'
user = <username>
pwd = <password>
# Set proper headers
headers = {"Content-Type":"application/json","Accept":"application/json"}
# Do the HTTP request
response = requests.post(url, auth=(user, pwd), headers=headers ,data="{\"short_description\":\"test in python\"}")
# Check for HTTP codes other than 201
if response.status_code != 201:
print('Status:', response.status_code, 'Headers:', response.headers, 'Error Response:',response.json())
exit()
# Decode the JSON response into a dictionary and use the data
data = response.json()
print(data)
I think you should use SSL, so no http!
First error I see in your script is how you pass your payload, you need to transform your dictionary into a JSON Object/String. And you don't need to authenticate twice, you have the basic http authentication handled by requests.post so no need for it in the header.
With this script it should work:
import json
import requests
url = 'https://instancename.service-now.com/change_request.do?JSONv2'
user = 'admin'
pwd = 'admin'
# Set proper headers
headers = {"Content-Type":"application/json","Accept":"application/json"}
payload = {
'sysparm_action': 'insert',
'short_description': 'test_jsonv2',
'priority': '1'
}
# Do the HTTP request
response = requests.post(url, auth=(user, pwd), headers=headers, data=json.dumps(payload))
# Check for HTTP codes other than 200
if response.status_code != 200:
print('Status:', response.status_code, 'Headers:', response.headers, 'Error Response:',response.json())
exit()
# Decode the JSON response into a dictionary and use the data
data = response.json()
print(data)
i created a script that will get the users friend list (GET request) and i was successful. Now i am attempting to make a script that will follow a particular user (POST request) and i've been unsuccessful.
here is my oauth function (where the problem lies):
def augment_POST(url,**kwargs) :
secrets = hidden.oauth()
consumer = oauth2.Consumer(secrets['consumer_key'], secrets['consumer_secret'])
token = oauth2.Token(secrets['token_key'],secrets['token_secret'])
oauth_request = oauth2.Request.from_consumer_and_token(consumer, token= token, http_method='POST', http_url=url, parameters=kwargs)
oauth_request.to_postdata() # this returns post data, where should i put it?
oauth_request.sign_request(oauth2.SignatureMethod_HMAC_SHA1(), consumer, token)
return oauth_request.to_url()
my augment_GET function is the exact same thing except http_mehtod='GET'
for clarity:
def follow_user(id):
seedurl="https://api.twitter.com/1.1/friendships/create.json"
print 'Attempting to follow: %d' % (id,)
url = augment_POST(seedurl,user_id=id)
connection = urllib.urlopen(url)
data = connection.read()
headers = connection.info().dict
any help will be greatly appreciated.
First it seems you need to import urllib2 to make a POST request.
You have to send the POST data that you get from the to_postdata method
using the data argument of urlopen:
def augment_POST(url, **kwargs) :
secrets = hidden.oauth()
consumer = oauth2.Consumer(secrets['consumer_key'],
secrets['consumer_secret'])
token = oauth2.Token(secrets['token_key'],
secrets['token_secret'])
oauth_request = oauth2.Request.from_consumer_and_token(
consumer,
token= token,
http_method='POST',
http_url=url,
parameters=kwargs
)
oauth_request.sign_request(oauth2.SignatureMethod_HMAC_SHA1(),
consumer, token)
# this is the data that must be sent with you POST request
return oauth_request.to_postdata()
def follow_user(id):
url = "https://api.twitter.com/1.1/friendships/create.json"
print 'Attempting to follow: %d' % id
postdata = augment(url, method='GET', user_id=id)
# Send the POST request with the data argument
# The url is the same as the data is sent in the body of the request
connection = urllib2.urlopen(url, data=postdata)
data = connection.read()
headers = connection.info().dict
I would recommend to use the requests_oauthlib module which makes all this really easy:
from requests_oauthlib import OAuth1Session
tokens = hidden.oauth()
client = OAuth1Session(tokens['consumer_key'],
tokens['consumer_secret'],
tokens['token_key'],
tokens['token_secret'])
def follow_user(id):
url = "https://api.twitter.com/1.1/friendships/create.json"
print 'Attempting to follow: %d' % id
# for GET requests use client.get and the `params` argument
# instead of the `data` argument
response = client.post(url, data={'user_id': id})
data = response.text
# or even `data = response.json()` to decode the data
headers = response.headers
I am trying access a REST API.
I can get it working in Curl/REST Client (the UI tool), with preemptive authentication enabled.
But, using urllib2, it doesn't seem to support this by default and I can't find a way to turn it on.
Thanks :)
Here's a simple Preemptive HTTP basic auth handler, based on the code from urllib2.HTTPBasicAuthHandler. It can be used in the exact same manner, except an Authorization header will be added to every request with a matching URL. Note that this handler should be used with a HTTPPasswordMgrWithDefaultRealm. That's because there is no realm coming back in a WWW-Authenticate challenge since you're being preemptive.
class PreemptiveBasicAuthHandler(urllib2.BaseHandler):
def __init__(self, password_mgr=None):
if password_mgr is None:
password_mgr = urllib2.HTTPPasswordMgrWithDefaultRealm()
self.passwd = password_mgr
self.add_password = self.passwd.add_password
def http_request(self,req):
uri = req.get_full_url()
user, pw = self.passwd.find_user_password(None,uri)
#logging.debug('ADDING REQUEST HEADER for uri (%s): %s:%s',uri,user,pw)
if pw is None: return req
raw = "%s:%s" % (user, pw)
auth = 'Basic %s' % base64.b64encode(raw).strip()
req.add_unredirected_header('Authorization', auth)
return req
similar to #thom-nichols's answer; but subclassing HTTPBasicAuthHandler also handling HTTPS requests.
import urllib2
import base64
class PreemptiveBasicAuthHandler(urllib2.HTTPBasicAuthHandler):
'''Preemptive basic auth.
Instead of waiting for a 403 to then retry with the credentials,
send the credentials if the url is handled by the password manager.
Note: please use realm=None when calling add_password.'''
def http_request(self, req):
url = req.get_full_url()
realm = None
# this is very similar to the code from retry_http_basic_auth()
# but returns a request object.
user, pw = self.passwd.find_user_password(realm, url)
if pw:
raw = "%s:%s" % (user, pw)
auth = 'Basic %s' % base64.b64encode(raw).strip()
req.add_unredirected_header(self.auth_header, auth)
return req
https_request = http_request
here is an example for dealing with a jenkins server which does not send you 401 http errors (retry with auth). I'm using urllib2.install_opener to make things easy.
jenkins_url = "https://jenkins.example.com"
username = "johndoe"
api_token = "some-cryptic-value"
auth_handler = PreemptiveBasicAuthHandler()
auth_handler.add_password(
realm=None, # default realm.
uri=jenkins_url,
user=username,
passwd=api_token)
opener = urllib2.build_opener(auth_handler)
urllib2.install_opener(opener)
Depending on what kind of authentication is required, you can send the Authorization headers manually by adding them to your request before you send out a body.
Update: based on Lee's comment I decided to condense my code to a really simple script and run it from the command line:
import urllib2
import sys
username = sys.argv[1]
password = sys.argv[2]
url = sys.argv[3]
print("calling %s with %s:%s\n" % (url, username, password))
passman = urllib2.HTTPPasswordMgrWithDefaultRealm()
passman.add_password(None, url, username, password)
urllib2.install_opener(urllib2.build_opener(urllib2.HTTPBasicAuthHandler(passman)))
req = urllib2.Request(url)
f = urllib2.urlopen(req)
data = f.read()
print(data)
Unfortunately it still won't generate the Authorization header (per Wireshark) :(
I'm having a problem sending basic AUTH over urllib2. I took a look at this article, and followed the example. My code:
passman = urllib2.HTTPPasswordMgrWithDefaultRealm()
passman.add_password(None, "api.foursquare.com", username, password)
urllib2.install_opener(urllib2.build_opener(urllib2.HTTPBasicAuthHandler(passman)))
req = urllib2.Request("http://api.foursquare.com/v1/user")
f = urllib2.urlopen(req)
data = f.read()
I'm seeing the following on the Wire via wireshark:
GET /v1/user HTTP/1.1
Host: api.foursquare.com
Connection: close
Accept-Encoding: gzip
User-Agent: Python-urllib/2.5
You can see the Authorization is not sent, vs. when I send a request via curl: curl -u user:password http://api.foursquare.com/v1/user
GET /v1/user HTTP/1.1
Authorization: Basic =SNIP=
User-Agent: curl/7.19.4 (universal-apple-darwin10.0) libcurl/7.19.4 OpenSSL/0.9.8k zlib/1.2.3
Host: api.foursquare.com
Accept: */*
For some reason my code seems to not send the authentication - anyone see what I'm missing?
thanks
-simon
The problem could be that the Python libraries, per HTTP-Standard, first send an unauthenticated request, and then only if it's answered with a 401 retry, are the correct credentials sent. If the Foursquare servers don't do "totally standard authentication" then the libraries won't work.
Try using headers to do authentication:
import urllib2, base64
request = urllib2.Request("http://api.foursquare.com/v1/user")
base64string = base64.b64encode('%s:%s' % (username, password))
request.add_header("Authorization", "Basic %s" % base64string)
result = urllib2.urlopen(request)
Had the same problem as you and found the solution from this thread: http://forums.shopify.com/categories/9/posts/27662
(copy-paste/adapted from https://stackoverflow.com/a/24048772/1733117).
First you can subclass urllib2.BaseHandler or urllib2.HTTPBasicAuthHandler, and implement http_request so that each request has the appropriate Authorization header.
import urllib2
import base64
class PreemptiveBasicAuthHandler(urllib2.HTTPBasicAuthHandler):
'''Preemptive basic auth.
Instead of waiting for a 403 to then retry with the credentials,
send the credentials if the url is handled by the password manager.
Note: please use realm=None when calling add_password.'''
def http_request(self, req):
url = req.get_full_url()
realm = None
# this is very similar to the code from retry_http_basic_auth()
# but returns a request object.
user, pw = self.passwd.find_user_password(realm, url)
if pw:
raw = "%s:%s" % (user, pw)
auth = 'Basic %s' % base64.b64encode(raw).strip()
req.add_unredirected_header(self.auth_header, auth)
return req
https_request = http_request
Then if you are lazy like me, install the handler globally
api_url = "http://api.foursquare.com/"
api_username = "johndoe"
api_password = "some-cryptic-value"
auth_handler = PreemptiveBasicAuthHandler()
auth_handler.add_password(
realm=None, # default realm.
uri=api_url,
user=api_username,
passwd=api_password)
opener = urllib2.build_opener(auth_handler)
urllib2.install_opener(opener)
Here's what I'm using to deal with a similar problem I encountered while trying to access MailChimp's API. This does the same thing, just formatted nicer.
import urllib2
import base64
chimpConfig = {
"headers" : {
"Content-Type": "application/json",
"Authorization": "Basic " + base64.encodestring("hayden:MYSECRETAPIKEY").replace('\n', '')
},
"url": 'https://us12.api.mailchimp.com/3.0/'}
#perform authentication
datas = None
request = urllib2.Request(chimpConfig["url"], datas, chimpConfig["headers"])
result = urllib2.urlopen(request)
The second parameter must be a URI, not a domain name. i.e.
passman = urllib2.HTTPPasswordMgrWithDefaultRealm()
passman.add_password(None, "http://api.foursquare.com/", username, password)
I would suggest that the current solution is to use my package urllib2_prior_auth which solves this pretty nicely (I work on inclusion to the standard lib.