Extracting specific JSON values in python - python

JSON return from spotify api. Example:
{
"tracks": {
"href": "https://api.spotify.com/v1/search?query=Stero+Hearts&type=track&offset=0&limit=1",
"items": [
{
"album": {
"album_type": "album",
"artists": [
{
"external_urls": {
"spotify": "https://open.spotify.com/artist/4IJczjB0fJ04gs4uvP0Fli"
},
"href": "https://api.spotify.com/v1/artists/4IJczjB0fJ04gs4uvP0Fli",
"id": "4IJczjB0fJ04gs4uvP0Fli",
"name": "Gym Class Heroes",
"type": "artist",
"uri": "spotify:artist:4IJczjB0fJ04gs4uvP0Fli"
}
]
}
}
]
}
}
Broken Code
import requests, json
spotifytrack = input("Name of Song?\\n")
link = "https://api.spotify.com/v1/search?q=" + spotifytrack + "&type=track&limit=1"
token = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
header = {
"Authorization": "Bearer {}".format(token),
"Content-Type": "application/json",
"Accept": "application/json",
}
auth_response = requests.get(link, headers=header)
pretty_response = json.dumps(auth_response.json(), indent=4)
data_by_user = {}
for d in auth_response:
data_by_user[d["artist"]] = d
print(data_by_user["uri"])
"""
def find_track_from_json(auth_response, artist):
return [p for p in auth_response if p["artist"] == artist][0]["uri"]
urii = find_track_from_json(auth_response, "uri")
print(urii)
x = load.json(auth_response.json())
print("Here is the data whic we have imported\n")
print(pretty_response)
print(x["name"])
print(x["uri"])
print(x["spotify"])
"""
Errors noticed:
File "spotify.py", line 19, in <module>
data_by_user[d["artist"]] = d
TypeError: byte indices must be integers or slices, not str
The aim is to convert word search to link in a cli application.
I tried load.json which i saw in some website and also tried def.
I expected the program to find out the artist name and uri from the json and print it in the cli interface.

You are iterating over the encoded json string:
auth_response = requests.get(link, headers=header)
for d in auth_response:
Python is complaining that you aren't providing a numerical index, which is correct as auth_response is just a string!
You should call json.loads to decode the string, and then you can iterate over it.
auth_response = requests.get(link, headers=header)
decoded_auth_response = json.loads(auth_response)
data_by_user = {}
for d in decoded_auth_response:
data_by_user[d["artist"]] = d
As you haven't provided the full json output from the API call I'm not sure what data is actually in decoded_auth_response, and you haven't described what your expected output would look like, so you may need to do some more work to find the correct data in each iteration.

The result from requests.get() is a requests.Response object. As far as I can see you want to iterate over the response body which is JSON. The requests.Response object has a .json() method which returns a dict from the response JSON.
Looking at the response you would probably want to iterate over resp_json['tracks']['items'] which is a list.
So to summarize your code should look something like this:
auth_response = requests.get(link, headers=header)
items = auth_response.json()['tracks']['items']
for d in items:
print(d)

Related

Rebrandly API in python

I'm using this Rebrandly API in python and I'm new to python so I'm struggling with it. I want to have all the links in a file.txt line by line converted into shortlink. I tried various methods but fails. Are there any possible solution for my problem?
file.txt :
https://link1.com
https://link2.com
https://link3.com
Here's the code :
url = "https://api.rebrandly.com/v1/links/new?destination=https%3A%2F%2Ftest.com"
headers = {
"Accept": "application/json",
"apikey": "removed"
}
response = requests.get(url, headers=headers)
print(response.text)
I've divided the things you need to do into three steps.
Step 1: Read the links that you would like to short.
with open('input_links.txt') as input_fp:
lines = input_fp.readlines() # lines = ["https://link1.com", "https://link2.com", ...]
Step 2: Have a method to shorten the links from the API call.
# Inspired from Rebrandly docs.
# Link: https://developers.rebrandly.com/docs.
import requests
import json
def shorten_link(destination):
"""
Returns a short link if successful. Otherwise, an empty string.
"""
headers = {
"Content-type": "application/json",
"apikey": "YOUR_API_KEY",
"workspace": "YOUR_WORKSPACE_ID"
}
body = {
"domain": { "fullName": "rebrand.ly" }
"destination": destination,
}
r = requests.post(
"https://api.rebrandly.com/v1/links",
data = json.dumps(body),
headers=headers
)
if r.status_code == requests.codes.ok:
link = r.json()
return link["shortUrl"]
return ""
Step 3: Save the results into a file.
with open('output_links.txt', 'w') as output_fp:
for link in links:
short_link = shorten_link(link)
output_fp.write(f"{short_link}\n")

REST APIs: How to POST several entries from a (JSON) file in a python loop?

I am new in python and REST world.
My Python script
import json
import requests
with open(r"create-multiple-Users.json", "r") as payload:
data = json.load(payload)
json_data = json.dumps(data, indent=2)
headers = {'content-type': 'application/json; charset=utf-8'}
for i in range(len(data)):
r = requests.post('http://localhost:3000/users',
data=json_data, headers=headers)
Mock API server: https://github.com/typicode/json-server .
Entry file: "info.json" with Endpoint: /users that has one user initially.
{
"users": [
{
"id": 1,
"name": "John",
"job": "Wong"
}
]
}
Issue:
POSTing from a file with only one user works perfectly. The new user is appended to info.json as expected as an object.
But when trying to POST let's say 3 users from file "create-multiple-Users.json" below, then the users are appended to the "info.json" as lists of objects 3 times (i.e. the number of objects/iterations)
[
{
"id": 10,
"name": "Janet",
"job": "Weaver"
},
{
"id": 12,
"name": "Kwonn",
"job": "Wingtsel"
},
{
"id": 13,
"name": "Eve",
"job": "Holt"
}
]
I would expect the users to be appended one by one as separate objects.
Maybe I am too oversimplifying the looping?
Any help is highly appreciated.
PS: Sorry I couldn't get the multiple-users file formatted ;(
A simple change in your for iteration would help:
import json
import requests
with open(r"create-multiple-Users.json", "r") as payload:
data = json.load(payload)
json_data = json.dumps(data, indent=2)
headers = {'content-type': 'application/json; charset=utf-8'}
for row in data: # Change this to iterate the json list
r = requests.post('http://localhost:3000/users',
data=row, headers=headers) # Send row that is a single object
I found the solution by using the hint thanks to "enriqueojedalara"
import json
import requests
with open(r"create-multiple-Users.json", "r") as payload:
data = json.load(payload) #<class 'list'>
headers = {'content-type': 'application/json; charset=utf-8'}
print("Total number of objects: ", len(data))
for i in range(len(data)):
data_new = json.dumps(data[i])
r = requests.post('http://localhost:3000/users', data=data_new, headers=headers)
print("Item#", i, "added", " -> ", data_new)

Getting a specific value of JSON data

I'm getting a JSON data from RESTCONF HTTPS request, using the following code
https_request = 'https://' + host + '/restconf/data/' + operation
headers = {'Content-type': 'application/yang-data+json', 'Accept': 'application/yang-data+json'}
r = requests.get(https_request, auth=(user, password), headers=headers, verify=False)
print r.json()
The data I got is the following:
{
"Cisco-IOS-XE-segment-routing:ipv4": {
"prefixes": [
{
"ipprefix": "1.1.1.1/32",
"index": {
"range-start": 333,
"range": 1
}
}
]
}
}
Basically, I want to return the field's "range-start" value which is 333. I tried the following but it did not work.
for element in r:
id = element['range-start']
print(id)
Is there anyway to get that value?
From Python Console:
>>> import json
... data = json.loads('{"Cisco-IOS-XE-segment-routing:ipv4": {"prefixes": [{"ipprefix": "1.1.1.1/32", "index": {"range-start": 333, "range": 1}}]}}')
... print(data['Cisco-IOS-XE-segment-routing:ipv4']['prefixes'][0]['index']['range-start'])
333
>>>
You need to start at the beginning of the JSON and work your way to the key you want. To do this you need to start at Cisco-IOS-XE-segment-routing:ipv4.
prefixes = r.json()["Cisco-IOS-XE-segment-routing:ipv4"]["prefixes"]
id = prefixes[0]["index"]["range-start"]
If there are multiple prefixes you can loop over them and access each range-start.
Since you are looping over elements, I would suggest this approach using a helper function:
def get_id(element):
prefixes = r.json()["Cisco-IOS-XE-segment-routing:ipv4"]["prefixes"]
id = prefixes[0]["index"]["range-start"]
return id
Then you can do, as in your question:
for element in r:
id = get_id(element)
print(id)

JSONDecodeError: Expecting value: line 1 column 1 (char 0) error

I am facing this error while making request to fetch json from api.
I can get json data using the "/v1/articles' path.
conn = http.client.HTTPSConnection("api.xxxx.com.tr")
headers = {
'accept': "application/json",
'apikey': "cd6b6c96799847698d87dec9f9a731d6"
}
filter = "daily"
conn.request("GET", "/v1/articles", headers=headers)
reader = codecs.getreader("utf-8")
res = conn.getresponse()
data = json.load(reader(res))
json.dumps(data)
return data
But i am having JSONDecodeError if i set filter. Code:
conn = http.client.HTTPSConnection("api.xxxx.com.tr")
headers = {
'accept': "application/json",
'apikey': "cd6b6c96799847698d87dec9f9a731d6"
}
conn.request("GET", "/v1/articles?$filter=Path eq '/daily/'", headers=headers)
reader = codecs.getreader("utf-8")
res = conn.getresponse()
data = json.load(reader(res))
json.dumps(data)
return data
I tried same filter using Postman with no error and i can get Json data.
Returned Json data from Postman:
[
{
"Id": "40778196",
"ContentType": "Article",
"CreatedDate": "2018-03-20T08:28:05.385Z",
"Description": "İspanya'da 2016 yılında çalınan lüks otomobil, şasi numarası değiştirilerek Bulgaristan üzerinden getirildiği Türkiye'de bulundu.",
"Files": [
{
"FileUrl": "http://i.xxxx.com/i/xxxx/98/620x0/5ab0c6a9c9de3d18a866eb54.jpg",
"Metadata": {
"Title": "",
"Description": ""
}
}
],
"ModifiedDate": "2018-03-20T08:32:12.001Z",
"Path": "/gundem/",
"StartDate": "2018-03-20T08:32:12.001Z",
"Tags": [
"ispanya",
"Araç",
"Hırsız",
"Dolandırıcı"
],
"Title": "İspanya'da çalınan lüks araç Türkiye'de bulundu!",
"Url": "http://www.xxxx.com.tr/gundem/ispanyada-calinan-luks-arac-turkiyede-bulundu-40778196"
}
]
I can not figure out the problem. It would be great if anyone help me about this issue. Thank you.
I finally figured out the problem! Using the requests library have solved my problem now I can filter the api request.
data = requests.get('https://api.xxxxx.com.tr/v1/articles', headers =
headers, params={"$filter":"Path eq '/xxxxxx/'"}).json()
I am leaving this answer here for anyone else who can need this solution in the future.
Thanks for all your suggestions.
The problem is in the following line
data = json.load(reader(res))
when your response is not a json string, JSONDecodeError occurs. so, add an additional logic to see if the response is None or a json string. First thing, print the reader(res) and see what the return is

"Errors": ["Cannot parse input stream due to I/O error as JSON document: Parse error: expected '}' but saw ',' [ chars read = *****]

I'm trying to create defects programmatically. I am getting a couple errors and having trouble getting any further. Here, essentially, is the code:
import requests, json
rally_auth = ('**uid', '***pwd')
rally_auth_url = 'https://rally1.rallydev.com/slm/webservice/v2.0/security/authorize'
rally_defect = 'https://rally1.rallydev.com/slm/webservice/v2.0/hierarchicalrequirement'
workspace_ref = 'https://rally1.rallydev.com/slm/webservice/v2.0/workspace/123***'
fe_project_ref = 'https://rally1.rallydev.com/slm/webservice/v2.0/project/134***'
user_ref = 'https://rally1.rallydev.com/slm/webservice/v2.0/user/106***'
l2_ref = 'https://rally1.rallydev.com/slm/webservice/v2.0/portfolioitem/l2roadmapitem/166***'
headers = {"Accept": "application/json", "Content-Type": "application/json", "ZSESSIONID" : "_iv********"}
s = requests.Session()
token = '_iv**********'
url = rally_defect + '/create?key=' + token
payload = {
'Workspace' : workspace_ref,
'Name': 'Tesing',
'Description': 'Testing',
'Project': fe_project_ref,
'StoryType': "New Feature",
'PortfolioItem' : l2_ref,
'Owner' : user_ref,
'ScheduleState':'Defined',
}
r = s.put(url, data=json.dumps(payload), headers=headers)
print r.text
print r.status_code
{"CreateResult": {"_rallyAPIMajor": "2", "_rallyAPIMinor": "0", "Errors": ["Cannot parse input stream due to I/O error as JSON document: Parse error: expected '}' but saw ',' [ chars read = *****], "Warnings": []}}
You will need to provide the Artifact type in your JSON. Below is an update to your code that should work for you. I am also assuming 'StoryType' is a Custom String Field. You will need to update the name to 'c_StoryType' to add a value to a Custom Field.
I also removed some of the extra lines. Since you are using an API Key and setting it as the ZSessionID in the Headers, you will not need the Security Token to create the Artifact.
import requests, json
rally_defect = 'https://rally1.rallydev.com/slm/webservice/v2.0/hierarchicalrequirement'
workspace_ref = 'https://rally1.rallydev.com/slm/webservice/v2.0/workspace/123***'
fe_project_ref = 'https://rally1.rallydev.com/slm/webservice/v2.0/project/134***'
user_ref = 'https://rally1.rallydev.com/slm/webservice/v2.0/user/106***'
l2_ref = 'https://rally1.rallydev.com/slm/webservice/v2.0/portfolioitem/l2roadmapitem/166***'
headers = {"Accept": "application/json", "Content-Type": "application/json", "ZSESSIONID" : "_iv********"}
s = requests.Session()
url = rally_defect + '/create'
payload = {
"HierarchicalRequirement" : {
"Workspace" : workspace_ref,
"Name" : "Tesing",
"Description" : "Testing",
"Project" : fe_project_ref,
"c_StoryType" : "New Feature",
"PortfolioItem" : l2_ref,
"Owner" : user_ref,
"ScheduleState" : "Defined"
}
}
r = s.put(url, data=json.dumps(payload), headers=headers)
print r.text
print r.status_code

Categories

Resources