I keep receiving key error 'Notes'. Notes is in a nested response. How do I solve this? I have included a sample of the json.
Traceback added by request.
Traceback (most recent call last):
File "/Users/xxxxxx/Documents/code/trade show w notes", line 16, in <module>
target = "%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s" % (contact["company_name"], contact["location"], contact["summary"], contact["job_title"], contact["name"], contact["job_industry"], contact["email"], contact["first_name"], contact["last_name"], contact["notes"])
KeyError: 'notes'
\
"data": [
{
"team_id": 53806,
"name": "Nicholas Bancroft Cooke",
"first_name": "Nicholas",
],
"email": null,
"metadata": null,
"qualification": [
{
"qualification_id": 17573056,
"qualification": "connected",
"notes": null,
\\
page = 1
url = "https://teams-api.grip.events/1/team/53806/event/123721/member/236388/contact/inbound_lead/reviewed?page=1"
headers = {
'authorization': 'Bearer eaf3bd4b-6861-4ca2-a86e-3a96c73deac0',
}
data = ["company_name", "job_title", "name", "job_industry", "summary", "notes", "location", "first_name", "last_name", "email"]
with open("list31.txt", "w", encoding='utf-8') as f: #added " encoding='utf-8' "
for page in range(1, 1000):
response = requests.get(url=url, headers=headers).json()
contacts = response["data"]
for contact in contacts:
target = "%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s" % (contact["company_name"], contact["location"], contact["summary"], contact["job_title"], contact["name"], contact["job_industry"], contact["email"], contact["first_name"], contact["last_name"], contact["notes"])
f.write(target + "\n")
print(target)
If you are referring to the notes key which is under qualification.
Then it should be: contact["qualification"][0]["notes"]
Here contact["qualification"] is a list. if by any chance contact["qualification"] is an empty list it will raise an IndexError. you need to handle it with a try-catch or if-clause
Related
I am trying to create a post request to send a message on the site. But for some reason it doesn't work. I am on the right page, but for some reason the message is still not being sent
print(f'{login}', lastChat['userSms'])
Jurl = 'https://funpay.com/chat/?node=userId'
seska = session.get(Jurl )
soup = BeautifulSoup(seska.text, 'lxml')
tag1 = soup.find('div', id='live-counters').get('data-orders')
tag3 = soup.find('div', class_='chat chat-float').get('data-bookmarks-tag')
tag2 = soup.find("div", class_='chat chat-float').get('data-tag')
tag4 = soup.find('div', class_='param-item chat-panel hidden').get('data-tag')
session.get(Jurl)
dataMes = soup.findAll('div', class_='message')
print(dataMes[-1])
objects = '[{"type":"orders_counters","id":"5214431","tag":"' + tag1 + '","data":false},' \
'{"type":"chat_node","id":"users-5054677-5214431","tag":"' + tag2 + '",' \
'"data":{"node":"users-5054677-5214431","last_message":' + lastMes + ',"content":"s"}},' \
'{"type":"chat_bookmarks","id":"5214431","tag":"' + tag3 + '","data":false},' \
'{"type":"c-p-u","id":"5054677","tag":"' + tag4 + '","data":false}]'
request = '{"action":"chat_message",' \
'"data":{"node":"users-5054677-5214431",' \
'"last_message":' + lastMes + ',"content":"s"}}'
datas = {
"objects": objects,
"request": request,
"csrf_token": csrf
}
print(datas)
session.post(Jurl, data=datas)
Creating the JSON with concatenated strings is error-prone and most likely it is failing because the JSON won't parse.
For example, if variable lastMes is a string value then it must be quoted.
request = '{"action":"chat_message",' \
'"data":{"node":"users-5054677-5214431",' \
'"last_message":"' + lastMes + '","content":"s"}}'
# ^ ^
Safer to create a dictionary of the values then use json.dumps() to convert the objects to correctly formatted JSON.
import json
tag1 = tag2 = tag3 = tag4 = lastMes = "XXX"
csrf = "csrf"
objects = [{"type": "orders_counters", "id": "5214431", "tag": tag1, "data": False},
{"type": "chat_node", "id": "users-5054677-5214431", "tag": tag2,
"data": {"node": "users-5054677-5214431",
"last_message": lastMes, "content": "s"}},
{"type": "chat_bookmarks", "id": "5214431", "tag": tag3, "data": False},
{"type": "c-p-u", "id": "5054677", "tag": tag4, "data": False}]
request = {"action": "chat_message",
"data": {"node": "users-5054677-5214431",
"last_message": lastMes,
"content": "s"}}
datas = {
"objects": json.dumps(objects),
"request": json.dumps(request),
"csrf_token": csrf
}
Then make the HTTP POST and check r.status_code.
r = session.post(Jurl, data=datas)
print(r.status_code)
However, if the web service requires a JSON payload (with Content-type header=application/json) not form-encoded data then you need to use the json= parameter.
datas = {
"objects": objects,
"request": request,
"csrf_token": csrf
}
r = session.post(Jurl, json=datas)
I have this json file:
[
{
"#timestamp": "",
"userID": "",
"destinationUserName": "",
"message": ": 12,050",
"name": "Purge Client Events"
},
{
"#timestamp": "",
"userID": "",
"destinationUserName": "",
"message": "",
"name": ""
},
{
"#timestamp": "",
"userID": "",
"destinationUserName": "",
"message": "",
"name": ""
},
{
"#timestamp": "",
"userID": "",
"name": "",
"sourceUserName": "",
"deviceAction": ""
}
]
I am looking for a solution in which I can loop over all the file, and count the unique values for UserID and return that value printed.
I found different solution but non of them worked for me and I am completely stuck.
So far this is my code, its just a formatter that convert the file into a json format.
After that I tried to check the length of the file and loop over it appending unique elements.
import json
to_queue = []
def structure_json():
with open("file.json", "r+") as f:
old = f.read()
f.seek(0) # rewind
# save to the old string after replace
new = old.replace('}{', '},{')
f.write(new)
tmps = '[' + str(new) + ']'
json_string = json.loads(tmps)
for key in json_string:
to_queue.append(key)
f.close
with open('update_2.json', 'w') as file:
json.dump(json_string, file, indent=2)
size=len(file["UserID"])
uniqueNames = [];
for i in range(0,size,1):
if(file["UserID"] not in uniqueNames):
uniqueNames.append(file["UserID"]);
print(uniqueNames)
structure_json()
print(to_queue)
But I get the following error:
Traceback (most recent call last):
File "format.py", line 24, in <module>
structure_json()
File "format.py", line 17, in structure_json
size=len(file["UserID"])
TypeError: '_io.TextIOWrapper' object is not subscriptable
Please any help will be much appreciated. Thank you so much for any help, and if you need any more info just let me know
Open the file and load the content. Then you can iterate over list of dicts and crate set of all values for key userID. Note, if any missing key it will yield None and will affect the count (+1).
import json
with open('your_file.json') as f:
data = json.load(f)
users = set(item.get('userID') for item in data)
print(len(users))
print(users)
I am trying to get my list of contacts from my WIX website using their API endpoint url and the requests module in python. I am totally stuck.
Here's my code so far:
import requests
auth_key = "my auth key"
r = requests.get("https://www.wixapis.com/crm/v1/contacts", headers={"Authorization": auth_key})
print(r.status_code)
dict = r.json()
contacts_list = dict["contacts"]
for i in contacts_list:
for key in i:
print(key, ':', i[key])
Here is what I get:
200
id : long id string 1
emails : [{'tag': 'UNTAGGED', 'email': 'sampleemail1#yahoo.com'}]
phones : []
addresses : [{'tag': 'UNTAGGED', 'countryCode': 'US'}]
metadata : {'createdAt': '2020-07-08T22:41:07.135Z', 'updatedAt': '2020-07-08T22:42:19.327Z'}
source : {'sourceType': 'SITE_MEMBERS'}
id : long id string 2
emails : [{'tag': 'UNTAGGED', 'email': 'sampleemail2#yahoo.com'}]
phones : []
addresses : []
metadata : {'createdAt': '2020-07-03T00:51:21.127Z', 'updatedAt': '2020-07-04T03:26:16.370Z'}
source : {'sourceType': 'SITE_MEMBERS'}
Process finished with exit code 0
Each line is a string. I need each row of the csv to be a new contact (There are two sample contacts). The columns should be the keys. I plan to use the csv module to writerow(Fields), where fields is a list of string (keys) such as Fields = [id, emails, phones, addresses, metadata, source]
All I really need is the emails in a single column of a csv though. Is there a way to maybe just get the email for each contact?
A CSV file with one column is basically just a text file with one item per line, but you can use the csv module to do it if you really want, as shown below.
I commented-out the 'python-requests' stuff and used some sample input for testing.
test_data = {
"contacts": [
{
"id": "long id string 1",
"emails": [
{
"tag": "UNTAGGED",
"email": "sampleemail1#yahoo.com"
}
],
"phones": [],
"addresses": [
{
"tag": "UNTAGGED",
"countryCode": "US"
}
],
"metadata": {
"createdAt": "2020-07-08T22:41:07.135Z",
"updatedAt": "2020-07-08T22:42:19.327Z"
},
"source": {
"sourceType": "SITE_MEMBERS"
}
},
{
"id": "long id string 2",
"emails": [
{
"tag": "UNTAGGED",
"email": "sampleemail2#yahoo.com"
}
],
"phones": [],
"addresses": [],
"metadata": {
"createdAt": "2020-07-03T00:51:21.127Z",
"updatedAt": "2020-07-04T03:26:16.370Z"
},
"source": {
"sourceType": "SITE_MEMBERS"
}
}
]
}
import csv
import json
import requests
auth_key = "my auth key"
output_filename = 'whatever.csv'
#r = requests.get("https://www.wixapis.com/crm/v1/contacts", headers={"Authorization": auth_key})
#print(r.status_code)
#json_obj = r.json()
json_obj = test_data # FOR TESTING PURPOSES
contacts_list = json_obj["contacts"]
with open(output_filename, 'w', newline='') as outp:
writer = csv.writer(outp)
writer.writerow(['email']) # Write csv header.
for contact in contacts_list:
email = contact['emails'][0]['email'] # Get the first one.
writer.writerow([email])
print('email csv file written')
Contents of whatever.csv file afterwards:
email
sampleemail1#yahoo.com
sampleemail2#yahoo.com
Update:
As pointed by #martineau, I just saw you can array in few values, you need to cater it. You may make them string with [].join() in the for loop
you can write it to csv like this using csv package.
import csv, json, sys
auth_key = "my auth key"
r = requests.get("https://www.wixapis.com/crm/v1/contacts", headers={"Authorization": auth_key})
print(r.status_code)
dict = r.json()
contacts_list = dict["contacts"]
output = csv.writer(sys.stdout)
#insert header(keys)
output.writerow(data[0].keys())
for i in contacts_list:
output.writerow(i.values())
At the end you can print and verify output
I am creating a Python script to parse the JSON response from https://vulners.com/api/v3/search/stats/
I have the following code in my .py:
import json
import requests
response = requests.get('https://vulners.com/api/v3/search/stats/')
vuln_set = json.loads(response.text)
vuln_type = vuln_set['data']['type_results']
vuln_bulletinfamily = vuln_set['data']['type_results'][vuln_type]['bulletinFamily']
vuln_name = vuln_set['data']['type_results'][vuln_type]['displayName']
print("Type: " + vuln_type)
print("Bulletin Family: " + vuln_bulletinfamily)
print("Name: " + vuln_name)
I need to get the vuln_type aswell as the child information (vuln_bulletinfamily & vuln_name)
An excerpt from the JSON response:
"data": {
"type_results": {
"aix": {
"lastUpdated": [],
"bulletinFamily": "unix",
"displayName": "IBM AIX",
"lastrun": "2017-09-14T14:04:56",
"count": 110,
"workTime": "0:00:10.983795"
},
"akamaiblog": {
"lastUpdated": [],
"bulletinFamily": "blog",
"displayName": "Akamai Blog",
"lastrun": "2017-09-14T10:38:52",
"count": 1463,
"workTime": "0:00:00.358691"
},
"amazon": {
"lastUpdated": [],
"bulletinFamily": "unix",
"displayName": "Amazon Linux AMI",
"lastrun": "2017-09-14T14:17:40",
"count": 889,
"workTime": "0:00:01.839594"
},
I am getting an error of TypeError: unhashable type: 'dict'
Traceback:
Traceback (most recent call last):
File "test.py", line 9, in <module>
vuln_bulletinfamily = vuln_set['data']['type_results'][vuln_type]['bulletinFamily']
TypeError: unhashable type: 'dict'
In the traceback line, the next line and the first print line, you are trying to access a dict type_results and vuln_type with a key that is also a dictionary.
You need to loop through the keys, like:-
import json
import requests
response = requests.get('https://vulners.com/api/v3/search/stats/')
vuln_set = json.loads(response.text)
vuln_type = vuln_set['data']['type_results']
for k in vuln_type :
vuln_bulletinfamily = vuln_set['data']['type_results'][k]['bulletinFamily']
vuln_name = vuln_set['data']['type_results'][k]['displayName']
print("Type: " + k)
print("Bulletin Family: " + vuln_bulletinfamily)
print("Name: " + vuln_name)
vuln_set = json.loads(response.text) vs response.json()
I try to get all lattitudes and longtitudes from this json.
Code:
import urllib.parse
import requests
raw_json = 'http://live.ksmobile.net/live/getreplayvideos?userid='
print()
userid = 735890904669618176
#userid = input('UserID: ')
url = raw_json + urllib.parse.urlencode({'userid=': userid}) + '&page_size=1000'
print(url)
json_data = requests.get(url).json()
print()
for coordinates in json_data['data']['video_info']:
print(coordinates['lat'], coordinates['lnt'])
print()
Error:
/usr/bin/python3.6 /media/anon/3D0B8DD536C9574F/PythonProjects/getLocation/getCoordinates
http://live.ksmobile.net/live/getreplayvideos?userid=userid%3D=735890904669618176&page_size=1000
Traceback (most recent call last):
File "/media/anon/3D0B8DD536C9574F/PythonProjects/getLocation/getCoordinates", line 17, in <module>
for coordinates in json_data['data']['video_info']:
TypeError: list indices must be integers or slices, not str
Process finished with exit code 1
Where do I go wrong?
In advance, thanks for your help and time.
I just post some of the json to show what it looks like.
The json looks like this:
{
"status": "200",
"msg": "",
"data": {
"time": "1499275646",
"video_info": [
{
"vid": "14992026438883533757",
"watchnumber": "38",
"topicid": "0",
"topic": "",
"vtime": "1499202678",
"title": "happy 4th of july",
"userid": "735890904669618176",
"online": "0",
"addr": "",
"isaddr": "2",
"lnt": "-80.1282576",
"lat": "26.2810628",
"area": "A_US",
"countryCode": "US",
"chatSystem": "1",
},
Full json: https://pastebin.com/qJywTqa1
Your URL construction is incorrect. The URL you have built (as shown in the output of your script) is:
http://live.ksmobile.net/live/getreplayvideos?userid=userid%3D=735890904669618176&page_size=1000
Where you actually want this:
http://live.ksmobile.net/live/getreplayvideos?userid=735890904669618176&page_size=1000
So your were actually getting this JSON in your response:
{
"status": "200",
"msg": "",
"data": []
}
Which is why you were seeing that error.
Here is the corrected script:
import urllib.parse
import requests
raw_json = 'http://live.ksmobile.net/live/getreplayvideos?'
print()
userid = 735890904669618176
#userid = input('UserID: ')
url = raw_json + urllib.parse.urlencode({'userid': userid}) + '&page_size=1000'
print(url)
json_data = requests.get(url).json()
print()
for coordinates in json_data['data']['video_info']:
print(coordinates['lat'], coordinates['lnt'])
print()
According to your posted json, you have problem in this statement-
print(coordinates['lat'], coordinates['lnt'])
Here coordinates is a list having only one item which is dictionary. So your statement should be-
print(coordinates[0]['lat'], coordinates[0]['lnt'])