Develop Reference

How to Itreate through Json

I need [0] to increase everytime and fetch the data when index change. from 0 to 13
import requests as r
import json
url = "https://services6.arcgis.com/bKYAIlQgwHslVRaK/arcgis/rest/services/CasesByRegion_ViewLayer/FeatureServer/0/query?where=1%3D1&outFields=*&outSR=4326&f=json"
response = urlopen(url)
Data= json.load(response )
for index in Data:
list = Data['features'][0]['attributes']
[0]+1
print(list)

Here is another simple approach without using urllib:
import requests as r
import json
url = "https://jsonplaceholder.typicode.com/todos/1"
response = r.get(url)
data = response.json()
print(data)

requests.get().json() delivers the complete dict from the response payload:
import requests as r
response = r.get(url)
Data = response.json()
Your json.load() doesn't work as expected because response is a dictionary from the requests module, containing some HTTP stuff like status code, reason, encoding. For API calls, this is not what you want (HTTP errors should be handled with exceptions). What you want is response.json() or response.text.
Also, you imported requests but didn't use it? I don't know about urlopen(). Use requests.get().

How to set params in Python requests library

I have the following code using urllib in Python 2.7 and its working. I'm trying to do the same request using the requests library but I cant get it to work.
import urllib
import urllib2
import json
req = urllib2.Request(url='https://testone.limequery.com/index.php/admin/remotecontrol',\
data='{\"method\":\"get_session_key\",\"params\":[\"username\",\"password\"],\"id\":1}')
req.add_header('content-type', 'application/json')
req.add_header('connection', 'Keep-Alive')
f = urllib2.urlopen(req)
myretun = f.read()
j=json.loads(myretun)
print(j['result'])
Using requests library( Doesn't work)
import requests
import json
d= {"method":"get_session_key","params":["username","password"],"id":"1"}
headers = {'content-type' :'application/json','connection': 'Keep-Alive'}
req2 = requests.get(url='https://testone.limequery.com/index.php/admin/remotecontrol',data=d,headers=headers)
json_data = json.loads(req2.text)
print(json data['result'])
I'm getting an error JSONDecodeError: Expecting value: line 1 column 1 (char 0) How can I make the code work with the requests library?

First, you're sending the wrong type of request. You're sending a GET request, but you need to send a POST, with requests.post.
Second, passing a dict as data will form-encode the data rather than JSON-encoding it. If you want to use JSON in your request body, use the json argument, not data:
requests.post(url=..., json=d)

Reference Link: http://docs.python-requests.org/en/master/api/
You can use requests module of python like so
import requests
Req = requests.request(
method = "GET", # or "POST", "PUT", "DELETE", "PATCH" etcetera
url = "http(s)://*",
params = {"key": "value"}, # IF GET Request (Optional)
data = {"key": "value"}, # IF POST Request (Optional)
headers = {"header_name": "header_value"} # (Optional)
)
print Req.content
You can surround the code with try::catch block like below to catch any exception thrown by requests module
try:
# requests.request(** Arguments)
except requests.exceptions.RequestException as e:
print e
For full argument list, please check reference link.

Python requests library how to pass Authorization header with single token

I have a request URI and a token. If I use:
curl -s "<MY_URI>" -H "Authorization: TOK:<MY_TOKEN>"
etc., I get a 200 and view the corresponding JSON data.
So, I installed requests and when I attempt to access this resource I get a 403 probably because I do not know the correct syntax to pass that token. Can anyone help me figure it out?
This is what I have:
import sys,socket
import requests
r = requests.get('<MY_URI>','<MY_TOKEN>')
r. status_code
I already tried:
r = requests.get('<MY_URI>',auth=('<MY_TOKEN>'))
r = requests.get('<MY_URI>',auth=('TOK','<MY_TOKEN>'))
r = requests.get('<MY_URI>',headers=('Authorization: TOK:<MY_TOKEN>'))
But none of these work.

In python:
('<MY_TOKEN>')
is equivalent to
'<MY_TOKEN>'
And requests interprets
('TOK', '<MY_TOKEN>')
As you wanting requests to use Basic Authentication and craft an authorization header like so:
'VE9LOjxNWV9UT0tFTj4K'
Which is the base64 representation of 'TOK:<MY_TOKEN>'
To pass your own header you pass in a dictionary like so:
r = requests.get('<MY_URI>', headers={'Authorization': 'TOK:<MY_TOKEN>'})

I was looking for something similar and came across this. It looks like in the first option you mentioned
r = requests.get('<MY_URI>', auth=('<MY_TOKEN>'))
"auth" takes two parameters: username and password, so the actual statement should be
r=requests.get('<MY_URI>', auth=('<YOUR_USERNAME>', '<YOUR_PASSWORD>'))
In my case, there was no password, so I left the second parameter in auth field empty as shown below:
r=requests.get('<MY_URI', auth=('MY_USERNAME', ''))
Hope this helps somebody :)

This worked for me:
access_token = #yourAccessTokenHere#
result = requests.post(url,
headers={'Content-Type':'application/json',
'Authorization': 'Bearer {}'.format(access_token)})

You can also set headers for the entire session:
TOKEN = 'abcd0123'
HEADERS = {'Authorization': 'token {}'.format(TOKEN)}
with requests.Session() as s:
s.headers.update(HEADERS)
resp = s.get('http://example.com/')

I found it here, it's working for me with Linkedin:
https://auth0.com/docs/flows/guides/auth-code/call-api-auth-code
The code I used with Linkedin login is:
ref = 'https://api.linkedin.com/v2/me'
headers = {"content-type": "application/json; charset=UTF-8",'Authorization':'Bearer {}'.format(access_token)}
Linkedin_user_info = requests.get(ref1, headers=headers).json()

Requests natively supports basic auth only with user-pass params, not with tokens.
You could, if you wanted, add the following class to have requests support token based basic authentication:
import requests
from base64 import b64encode
class BasicAuthToken(requests.auth.AuthBase):
def __init__(self, token):
self.token = token
def __call__(self, r):
authstr = 'Basic ' + b64encode(('token:' + self.token).encode('utf-8')).decode('utf-8')
r.headers['Authorization'] = authstr
return r
Then, to use it run the following request :
r = requests.get(url, auth=BasicAuthToken(api_token))
An alternative would be to formulate a custom header instead, just as was suggested by other users here.

You can try something like this
r = requests.get(ENDPOINT, params=params, headers={'Authorization': 'Basic %s' % API_KEY})

This worked for me:
r = requests.get('http://127.0.0.1:8000/api/ray/musics/', headers={'Authorization': 'Token 22ec0cc4207ebead1f51dea06ff149342082b190'})
My code uses user generated token.

You have a request needing an authorization maybe you have a result 401.
Suppose your request is like this :
REQ ='https://api.asite.com/something/else/else'
You have your token :
TOKEN = 'fliuzabuvdgfnsuczkncsq12454632'
build your header like this :
HEADER = {'Authorization': f'{TOKEN}'}
and use it like this :
req.get(REQ, headers=HEADER)
display your result like this :
req.get(COACH, headers=HEADER).json()

Python request with authentication (access_token)

I am trying to use an API query in Python. From the command line I can use curl like so:
curl --header "Authorization:access_token myToken" https://website.example/id
This gives some JSON output. myToken is a hexadecimal variable that remains constant throughout.
I would like to make this call from python so that I can loop through different ids and analyze the output. Before authentication was needed I had done that with urllib2. I have also taken a look at the requests module but couldn't figure out how to authenticate with it.

The requests package has a very nice API for HTTP requests, adding a custom header works like this (source: official docs):
>>> import requests
>>> response = requests.get(
... 'https://website.example/id', headers={'Authorization': 'access_token myToken'})
If you don't want to use an external dependency, the same thing using urllib2 of the standard library looks like this (source: the missing manual):
>>> import urllib2
>>> response = urllib2.urlopen(
... urllib2.Request('https://website.example/id', headers={'Authorization': 'access_token myToken'})

I had the same problem when trying to use a token with Github.
The only syntax that has worked for me with Python 3 is:
import requests
myToken = '<token>'
myUrl = '<website>'
head = {'Authorization': 'token {}'.format(myToken)}
response = requests.get(myUrl, headers=head)

>>> import requests
>>> response = requests.get('https://website.com/id', headers={'Authorization': 'access_token myToken'})
If the above doesnt work , try this:
>>> import requests
>>> response = requests.get('https://api.buildkite.com/v2/organizations/orgName/pipelines/pipelineName/builds/1230', headers={ 'Authorization': 'Bearer <your_token>' })
>>> print response.json()

import requests
BASE_URL = 'http://localhost:8080/v3/getPlan'
token = "eyJhbGciOiJSUzI1NiIsImtpZCI6ImR"
headers = {'Authorization': "Bearer {}".format(token)}
auth_response = requests.get(BASE_URL, headers=headers)
print(auth_response.json())
Output :
{
"plans": [
{
"field": false,
"description": "plan 12",
"enabled": true
}
]
}

A lot of good answers already, but I didn't see this option yet:
If you're using requests, you could also specify a custom authentication class, similar to HTTPBasicAuth. For example:
from requests.auth import AuthBase
class TokenAuth(AuthBase):
def __init__(self, token, auth_scheme='Bearer'):
self.token = token
self.auth_scheme = auth_scheme
def __call__(self, request):
request.headers['Authorization'] = f'{self.auth_scheme} {self.token}'
return request
This could be used as follows (using the custom auth_scheme from the example):
response = requests.get(
url='https://example.com',
auth=TokenAuth(token='abcde', auth_scheme='access_token'),
)
This may look like a more complicated way to set the Request.headers attribute, but it can be advantageous if you want to support multiple types of authentication. Note this allows us to use the auth argument instead of the headers argument.

Have you tried the uncurl package (https://github.com/spulec/uncurl)? You can install it via pip, pip install uncurl. Your curl request returns:
>>> uncurl "curl --header \"Authorization:access_token myToken\" https://website.com/id"
requests.get("https://website.com/id",
headers={
"Authorization": "access_token myToken"
},
cookies={},
)

I'll add a bit hint: it seems what you pass as the key value of a header depends on your authorization type, in my case that was PRIVATE-TOKEN
header = {'PRIVATE-TOKEN': 'my_token'}
response = requests.get(myUrl, headers=header)

One of the option used in python to retrieve below:
import requests
token="abcd" < retrieved based>
headers = {'Authorization': "Bearer {}".format(token)}
response = requests.get(
'https://<url api>',
headers=headers,
verify="root ca certificate"
)
print(response.content)
If you get hostname mismatch error then additional SANs need to be configured in the server with the hostnames.
Hope this helps.

Is there any way to do HTTP PUT in python

I need to upload some data to a server using HTTP PUT in python. From my brief reading of the urllib2 docs, it only does HTTP POST. Is there any way to do an HTTP PUT in python?

I've used a variety of python HTTP libs in the past, and I've settled on requests as my favourite. Existing libs had pretty useable interfaces, but code can end up being a few lines too long for simple operations. A basic PUT in requests looks like:
payload = {'username': 'bob', 'email': 'bob#bob.com'}
>>> r = requests.put("http://somedomain.org/endpoint", data=payload)
You can then check the response status code with:
r.status_code
or the response with:
r.content
Requests has a lot synactic sugar and shortcuts that'll make your life easier.

import urllib2
opener = urllib2.build_opener(urllib2.HTTPHandler)
request = urllib2.Request('http://example.org', data='your_put_data')
request.add_header('Content-Type', 'your/contenttype')
request.get_method = lambda: 'PUT'
url = opener.open(request)

Httplib seems like a cleaner choice.
import httplib
connection = httplib.HTTPConnection('1.2.3.4:1234')
body_content = 'BODY CONTENT GOES HERE'
connection.request('PUT', '/url/path/to/put/to', body_content)
result = connection.getresponse()
# Now result.status and result.reason contains interesting stuff

You can use the requests library, it simplifies things a lot in comparison to taking the urllib2 approach. First install it from pip:
pip install requests
More on installing requests.
Then setup the put request:
import requests
import json
url = 'https://api.github.com/some/endpoint'
payload = {'some': 'data'}
# Create your header as required
headers = {"content-type": "application/json", "Authorization": "<auth-key>" }
r = requests.put(url, data=json.dumps(payload), headers=headers)
See the quickstart for requests library. I think this is a lot simpler than urllib2 but does require this additional package to be installed and imported.

This was made better in python3 and documented in the stdlib documentation
The urllib.request.Request class gained a method=... parameter in python3.
Some sample usage:
req = urllib.request.Request('https://example.com/', data=b'DATA!', method='PUT')
urllib.request.urlopen(req)

You should have a look at the httplib module. It should let you make whatever sort of HTTP request you want.

I needed to solve this problem too a while back so that I could act as a client for a RESTful API. I settled on httplib2 because it allowed me to send PUT and DELETE in addition to GET and POST. Httplib2 is not part of the standard library but you can easily get it from the cheese shop.

I also recommend httplib2 by Joe Gregario. I use this regularly instead of httplib in the standard lib.

Have you taken a look at put.py? I've used it in the past. You can also just hack up your own request with urllib.

You can of course roll your own with the existing standard libraries at any level from sockets up to tweaking urllib.
http://pycurl.sourceforge.net/
"PyCurl is a Python interface to libcurl."
"libcurl is a free and easy-to-use client-side URL transfer library, ... supports ... HTTP PUT"
"The main drawback with PycURL is that it is a relative thin layer over libcurl without any of those nice Pythonic class hierarchies. This means it has a somewhat steep learning curve unless you are already familiar with libcurl's C API. "

If you want to stay within the standard library, you can subclass urllib2.Request:
import urllib2
class RequestWithMethod(urllib2.Request):
def __init__(self, *args, **kwargs):
self._method = kwargs.pop('method', None)
urllib2.Request.__init__(self, *args, **kwargs)
def get_method(self):
return self._method if self._method else super(RequestWithMethod, self).get_method()
def put_request(url, data):
opener = urllib2.build_opener(urllib2.HTTPHandler)
request = RequestWithMethod(url, method='PUT', data=data)
return opener.open(request)

You can use requests.request
import requests
url = "https://www.example/com/some/url/"
payload="{\"param1\": 1, \"param1\": 2}"
headers = {
'Authorization': '....',
'Content-Type': 'application/json'
}
response = requests.request("PUT", url, headers=headers, data=payload)
print(response.text)

A more proper way of doing this with requests would be:
import requests
payload = {'username': 'bob', 'email': 'bob#bob.com'}
try:
response = requests.put(url="http://somedomain.org/endpoint", data=payload)
response.raise_for_status()
except requests.exceptions.RequestException as e:
print(e)
raise
This raises an exception if there is an error in the HTTP PUT request.

Using urllib3
To do that, you will need to manually encode query parameters in the URL.
>>> import urllib3
>>> http = urllib3.PoolManager()
>>> from urllib.parse import urlencode
>>> encoded_args = urlencode({"name":"Zion","salary":"1123","age":"23"})
>>> url = 'http://dummy.restapiexample.com/api/v1/update/15410' + encoded_args
>>> r = http.request('PUT', url)
>>> import json
>>> json.loads(r.data.decode('utf-8'))
{'status': 'success', 'data': [], 'message': 'Successfully! Record has been updated.'}
Using requests
>>> import requests
>>> r = requests.put('https://httpbin.org/put', data = {'key':'value'})
>>> r.status_code
200