Develop Reference

How to Itreate through Json

I need [0] to increase everytime and fetch the data when index change. from 0 to 13
import requests as r
import json
url = "https://services6.arcgis.com/bKYAIlQgwHslVRaK/arcgis/rest/services/CasesByRegion_ViewLayer/FeatureServer/0/query?where=1%3D1&outFields=*&outSR=4326&f=json"
response = urlopen(url)
Data= json.load(response )
for index in Data:
list = Data['features'][0]['attributes']
[0]+1
print(list)

Here is another simple approach without using urllib:
import requests as r
import json
url = "https://jsonplaceholder.typicode.com/todos/1"
response = r.get(url)
data = response.json()
print(data)

requests.get().json() delivers the complete dict from the response payload:
import requests as r
response = r.get(url)
Data = response.json()
Your json.load() doesn't work as expected because response is a dictionary from the requests module, containing some HTTP stuff like status code, reason, encoding. For API calls, this is not what you want (HTTP errors should be handled with exceptions). What you want is response.json() or response.text.
Also, you imported requests but didn't use it? I don't know about urlopen(). Use requests.get().

X-WWW Put Request List inside of Dict

Attempting to write a put request in Python that keeps getting denied - the issue seems to be that the request isn't accepting the list as value for the dict.
Any suggestions on how I can get this to be accepted?
import requests
key = 'Bearer abc123'
url = 'www.my_url.com'
headers = {'Content-Type':'application/x-www-form-urlencoded',
'Accept':'application/json',
'Authorization':key}
data = {}
data['first_name']='John'
data['last_name']='Doe'
data['employee_job[roles]'] = [{'name':'CEO'}]
r = requests.put(url,data=data,headers=headers)

if the server accept put request,then the problem maybe json format
you could try this,put the data as json directly,not the form encoded.
need more information to detect the problem,can you provide the api document?
import requests
key = 'Bearer abc123'
url = 'www.my_url.com'
headers = {
'Accept':'application/json',
'Authorization':key}
data = {}
data['first_name']='John'
data['last_name']='Doe'
data['employee_job'] = {'roles':[{'name':'CEO'}]}
r = requests.put(url,json=data,headers=headers)

I need to perform a GET Request on a URL using python requests library and fetch only the last 100 lines

Summary:
Currently i am doing a GET Request on a {.log} URL which is having around 7000+ lines.
I need to GET the Response, validate for a particular message in the response and if its not present, i need to do a GET Request again on the same URL.
This iteration on the GET is very time consuming and most of the time results in a stuck state
Expectation:
I need a way out wherein i do a GET Request operation and fetch only last 100 lines as a response rather than fetching all the 7000+ lines every time.
URL = "http://sdd.log"
Code
def get_log(self):
logging.info("Sending a get request to retrieve pronghorn log")
resp = requests.request("GET", "http://ssdg.log")
logging.info("Printing the callback url response")
#logging.info(resp)
#logging.info(resp.text)
return resp.text

You cannot simply download only the last 100 lines of an HTTP request. You can however simply get the last 100 lines of the resulting response by using
data = resp.text.split('\n')
last_lines = '\n'.join(data[-100:])
return last_lines

So, if your server accepts range requests then you can use code like this to get the last 4096 bytes
import requests
from io import BytesIO
url = 'https://file-examples.com/wp-content/uploads/2017/10/file_example_JPG_100kB.jpg'
resp = requests.request("HEAD", url)
unit = resp.headers['Accept-Ranges']
print(resp.headers['Content-Length'])
print(unit)
headers = {'Range': f'{unit}=-4096'}
print(headers)
resp = requests.request("GET", url, headers=headers)
b = BytesIO()
for chunk in resp.iter_content(chunk_size=128):
b.write(chunk)
print(b.tell())
b.seek(0)
data = b.read()
print(f"len(data): {len(data)}")

Passing file and data to requests, error at every turn

Python 3.6.7, Requests 2.21.0
I have an issue that gives me a new error at every solution.
What I want: To send a file with data in a POST command using the requests library.
url_upload = BASE_URL + "upload?action=save"
data = {'data':{'name':'test.txt','contenttype':'text/plain', 'size':37}}
files = {'file': open('/home/user/test.txt', 'rb')}
req = session.post(url=url_upload, files=files, data=data)
The end server is using Spring (I assume) and the response text contains this error:
"net.sf.json.JSONException: A JSONObject text must begin with \'{\' at character 1 of name"
So, I try
data = json.dumps(data)
But, of course requests doesn't want that:
ValueError: Data must not be a string.
If I add the headers:
headers = {'Content-type': 'multipart/form-data'}
org.apache.commons.fileupload.FileUploadException: the request was rejected because no multipart boundary was found
Help would be appreciated.

What I needed to do was:
req = session.post(url=url_upload, files=files, data={'data': json.dumps(data)})
That way I'm giving the function variable 'data' the form-data variable name 'data' which contains the variable that has the key 'data'...
http://www.trekmate.org.uk/wp-content/uploads/2015/02/Data-star-trek-the-next-generation-31159191-1024-768.png

Using Python to test HTTP APIs

I'm fairly new to Python programming and I don't know all the libraries needed for the following.
I would like to use Python to test some HTTP APIs. Mainly I want to use OAuth and make a few JSON calls. The APIs in question can be found on: https://developers.trustpilot.com/authentication and the generate product review link (I can only use one link)
I want to authenticate myself and then generate a product review link in one step. So far I've been using the Advanced REST client (ARC) to make these calls individually. I could also use .arc files if you think it's easier.
The idea would be make these calls successively in one go. So it would be something along the lines:
1) Make the authentication call.
The HTTP Method looks like this:
https://api.trustpilot.com/v1/oauth/oauth-business-users-for-applications/accesstoken
Method Post:
Header
Authorization: Basic Base64encode(APIkey:Secret)
Content-Type: application/x-www-form-urlencoded
Payload:
grant_type=password&username=user#mail.com&password=SomePass
Translate this bit into Python basically.
1.a) Add a header to the call
Header Authorization: base64encode hash Content-Type: application/x-www-form-urlencoded
1.b) Add a payload to the call
Payload: grant_type=password&username
4) Receive the token from call made in step 1) (Result is format)
"access token": Auth_token
5) Take the token and use it in creating a product review.
5.a) Add the token in the header
Header: Authorization: Bearer Auth_token
6.a) Add a JSON payload to the call made in step 5.
Here's the code I have so far:
Import requests
header = {'Authorization: Basic NnNrQUprTWRHTU5VSXJGYXBVRGxack1oT01oTUFRZHI6QTFvOGJjRUNDdUxBTmVqUQ==}','Content-Type: application/x-www-form-urlencoded'}
payload = {'grant_type=password&username=email#address.com&password=SomePassword'}
r = requests.post('https://api.trustpilot.com/v1/oauth/oauth-business-users-for-applications/accesstoken', headers=header, params=payload )
Ideally I want to create the requests.post(url, header, payload) and then return what the server answers in JSON format. I think that print r.text would do the last part.
So this is the code I have writtent (that works now):
import requests
import getpass
import json
from requests.auth import HTTPBasicAuth
header = {'grant_type':'password' , 'username':'mail#maildomain.com', 'password':'YourPassword'}
username= "YOURAPIKEY" #APIKey
password= "YOURSECRET" #Secret
res = requests.post(
'URL/v1/oauth/oauth-business-users-for-applications/accesstoken',
auth=HTTPBasicAuth(username, password), # basic authentication
data=header)
#print(res.content) #See content of the call result.
data = res.json() # get response as parsed json (will return a dict)
auth_token = data.get('access_token')

requests can do all what you ask without any work from your part.
See the doc for authentication, parameters, json output, json input
Make the authentication call.
import requests
import getpass
from requests.auth import HTTPBasicAuth
username = raw_input('Username: ')
password = getpass.getpass('Password: ')
res = requests.post(
'https://api.trustpilot.com/v1/oauth/oauth-business-users-for-applications/accesstoken',
auth=HTTPBasicAuth(username, password), # basic authentication
params={ # url parameters
'grant_type': 'password',
'username': 'email#address.com',
'password': 'SomePassword'
})
Receive the token from call made in step 1) (Result is format)
# res = requests.post.....
data = res.json() # get response as parsed json (will return a dict)
auth_token = data.get('access token')
Take the token and use it in creating a product review.
request.post(
'.../product_review',
headers={
'Authorization': 'Bearer ' + auth_token
},
json={'my': 'payload'}) # send data as json