I have used tweepy user timeline API to extract information for some users. Here is the link to the file.
The content of the file is in string format. I tried to load the same using JSON.loads(<string>) but it is showing the error
ValueError: Expecting property name: line 1 column 2 (char 1)
I need to make the string to work as a dict/json so that I can iterate on the keys.
Can we see the full code of how you implemented the API call, but also try converting it into a collection first. try
from collection import Counter
Counter(data)
Related
When using python and steam api trying to get a certain value using data = profile['gameextrainfo'] profile has the value of the API which is.
d = {"response":
{"players":[
{"steamid":"76561199446676130",
"communityvisibilitystate":3,
"profilestate":1,
"personaname":"S7 WatchDog",
"profileurl":"https://steamcommunity.com/profiles/76561199446676130/",
"avatar":"https://avatars.akamai.steamstatic.com/415bd0e2ddd5d8e99309eec6d7a2566cbb09022d.jpg","avatarmedium":"https://avatars.akamai.steamstatic.com/415bd0e2ddd5d8e99309eec6d7a2566cbb09022d_medium.jpg","avatarfull":"https://avatars.akamai.steamstatic.com/415bd0e2ddd5d8e99309eec6d7a2566cbb09022d_full.jpg",
"avatarhash":"415bd0e2ddd5d8e99309eec6d7a2566cbb09022d",
"personastate":1,
"primaryclanid":"103582791429521408",
"timecreated":1671522419,
"personastateflags":0,
"gameextrainfo":"Counter-Strike: Global Offensive",
"gameid":"730"}]
}
}
I cannot seem to filter out any key. I've tried all of them and python just fails to find them. Any ideas
Tried all keys. Tried using requests python module
You could use the json module to parse the response and access the individual data points. For example:
import json
data = json.loads(profile)
gameextrainfo = data['response']['players'][0]['gameextrainfo']
print(gameextrainfo)
Im trying to retrieve data from a database named RethinkDB, they output JSON when called with r.db("Databasename").table("tablename").insert([{ "id or primary key": line}]).run(), when doing so it outputs [{'id': 'ValueInRowOfid\n'}] and I want to parse that to just the value eg. "ValueInRowOfid". Ive tried with JSON in Python, but I always end up with the typeerror: list indices must be integers or slices, not str, and Ive been told that it is because the Database outputs invalid JSON format. My question is how can a JSON format be invalid (I cant see what is invalid with the output) and also what would be the best way to parse it so that the value "ValueInRowOfid" is left in a Operator eg. Value = ("ValueInRowOfid").
This part imports the modules used and connects to RethinkDB:
import json
from rethinkdb import RethinkDB
r = RethinkDB()
r.connect( "localhost", 28015).repl()
This part is getting the output/value and my trial at parsing it:
getvalue = r.db("Databasename").table("tablename").sample(1).run() # gets a single row/value from the table
print(getvalue) # If I print that, it will show as [{'id': 'ValueInRowOfid\n'}]
dumper = json.dumps(getvalue) # I cant use `json.loads(dumper)` as JSON object must be str. Which the output of the database isnt (The output is a list)
parsevalue = json.loads(dumper) # After `json.dumps(getvalue)` I can now load it, but I cant use the loaded JSON.
print(parsevalue["id"]) # When doing this it now says that the list is a str and it needs to be an integers or slices. Quite frustrating for me as it is opposing it self eg. It first wants str and now it cant use str
print(parsevalue{'id'}) # I also tried to shuffle it around as seen here, but still the same result
I know this is janky and is very hard to comprehend this level of stupidity that I might be on. As I dont know if it is the most simple problem or something that just isnt possible (Which it should or else I cant use my data in the database.)
Thank you for reading this through and not jumping straight into the comments and say that I have to read the JSON documentation, because I have and I havent found a single piece that could help me.
I tried reading the documentation and watching tutorials about JSON and JSON parsing. I also looked for others whom have had the same problems as me and couldnt find.
It looks like it's returning a dictionary ({}) inside a list ([]) of one element.
Try:
getvalue = r.db("Databasename").table("tablename").sample(1).run()
print(getvalue[0]['id'])
I'm going crazy trying to get data through an API call using request and pandas. It looks like it's nested data, but I cant get the data i need.
https://xorosoft.docs.apiary.io/#reference/sales-orders/get-sales-orders
above is the api documentation. I'm just trying to keep it simple and get the itemnumber and qtyremainingtoship, but i cant even figure out how to access the nested data. I'm trying to use DataFrame to get it, but am just lost. any help would be appreciated. i keep getting stuck at the 'Data' level.
type(json['Data'])
df = pd.DataFrame(['Data'])
df.explode('SoEstimateHeader')
df.explode('SoEstimateHeader')
Cell In [64], line 1
df.explode([0:])
^
SyntaxError: invalid syntax
I used the link to grab a sample response from the API documentation page you provided. From the code you provided it looks like you are already able to get the data and I'm assuming the you have it as a dictionary type already.
From what I can tell I don't think you should be using pandas, unless its some downstream requirement in the task you are doing. But to get the ItemNumber & QtyRemainingToShip you can use the code below.
# get the interesting part of the data out of the api response
data_list = json['Data']
#the data_list is only one element long, so grab the first element which is of type dictionary
data = data_list[0]
# the dictionary has two keys at the top level
so_estimate_header = data['SoEstimateHeader']
# similar to the data list the value associated with "SoEstimateItemLineArr" is of type list and has 1 element in it, so we grab the first & only element.
so_estimate_item_line_arr = data['SoEstimateItemLineArr'][0]
# now we can grab the pieces of information we're interested in out of the dictionary
qtyremainingtoship = so_estimate_item_line_arr["QtyRemainingToShip"]
itemnumber = so_estimate_item_line_arr["ItemNumber"]
print("QtyRemainingToShip: ", qtyremainingtoship)
print("ItemNumber: ", itemnumber)
Output
QtyRemainingToShip: 1
ItemNumber: BC
Side Note
As a side note I wouldn't name any variables json because thats also the name of a popular library in python for parsing json, so that will be confusing to future readers and will clash with the name if you end up having to import the json library.
I'm a beginner programmer trying to work with the Yelp API and I've been able to pull the information I need but I can't figure out how to only export a single part of the address into my csv file. This is the code I am working with:
**Convert the JSON string to a dictionary*
business_data = response.json()
c = csv.writer(open('testing.csv', 'a'), lineterminator ='\n')
for biz in business_data['businesses']:
c.writerow([biz['name'], biz['location'], biz['phone'], biz['url']])
In the last line of code, in the for loop, I want to be able to target a specific element of the 'location' like this:
#c.writerow([biz['name'], biz['address1'], biz['city'], biz['state'], biz['zip_code'], biz['phone'], biz['url']])
on the Yelp website it shows that I can target these specifics but I just can't seem to figure out how with c.writerow()
Yelp shows that I can target these like this:
businesses[x].location.address1
businesses[x].location.address2
businesses[x].location.city
From the response, biz['location'] is a python dictionary meaning it consists of key, value pairs.
You can validate this by printing type(biz['location']). To answer your question, all you need is to call the dict key & write the value into the file.
c.writerow([biz['name'], biz['location']['address1'], biz['location']['city'], biz['location']['state'], biz['location']['zip_code'] biz['phone'], biz['url']])
I'm newbie in python and trying to parse data in my application using these lines of codes
json_str = request.body.decode('utf-8')
py_str = json.loads(json_str)
But I'm getting this error on json.loads
Expecting value: line 1 column 1 (char 0)
this is json formatted data that I send from angular app (Updated)
Object { ClientTypeId: 6, ClientName: "asdasd", ClientId: 0, PhoneNo: "123", FaxNo: "123", NTN: "1238", GSTNumber: "1982", OfficialAddress: "sads", MailingAddress: "asdasd", RegStartDate: "17-Aug-2016", 15 more… }
these are the values that I get in json_str
ClientTypeId=5&ClientName=asdasd&ClientId=0&PhoneNo=123&FaxNo=123&NTN=123&GSTNumber=12&OfficialAddress=adkjh&MailingAddress=adjh&RegStartDate=09-Aug-2016&RegEndDate=16-Aug-2016&Status=1&CanCreateUser=true&UserQuotaFor=11&UserQuotaType=9&MaxUsers=132123&ApplyUserCharges=true&ApplyReportCharges=true&EmailInvoice=true&BillingType=1&UserCharges=132&ReportCharges=123&MonthlyCharges=123&BillingDate=16-Aug-2016&UserSessionId=324
I don't know what's wrong in it.. can anyone mention what's the mistake is??
Your data is not JSON-formatted, not even the one you included in your updated answer. Your data is a JavaScript-object, not an encoded string. Please note the "N" in JSON: Notation -- it is a format inspired from how data is written in JavaScript code, but runtime JavaScript data is not represented in JSON. The "JSON" you pasted is how your browser represents the object to you, it is not proper JSON (that would be {"ClientTypeId": 6, ...} -- note the quotes around the property name).
When sending this data to the server, you have to encode it. You think you are sending it JSON-encoded, but you aren't. You are sending it "web form encoded" (data of type application/x-www-form-urlencoded).
Now either you have to learn how to send the data in JSON format from Angular, or use the correct parsing routine in Python: urllib.parse.parse_qs. Depending on the library you are using, there might be a convenience method to access the data as well, as this is a common use case.