Extracting data using python from the steam api formatted in JSON - python

I want to get the name and percent data from the json string...
{'achievementpercentages': {'achievements': [{'name': 'camp_bonus_01_stalingrad_rail_german_engineering', 'percent': 42}, {'name': 'count_camp_1', 'percent': 41.5}
Ive been trying to do this using something like...
for achievementpercentages in repos:
print(achievementpercentages['name'])
print(achievementpercentages['percent'])
but this returns the error...
TypeError: string indices must be integers

That's not a valid json string but that doesn't mean you can't use it. The issue you're having is explained by the error. Your loop isn't saying what you want to do based on the structure of your dictionary. If your repo variable is filled with a list of many rows(dicts) that look like your example line then your code needs to be:
for row in repo:
for achievement in row['achievementpercentages']['achievements']:
print achievement['name']
print achievement['percent']
It's a little unclear what repos is... if repos is your entire string you pasted it needs to be:
for achievemnt in repo['achievementpercentages']['achievements']:
print achievement['name']
print achievement['percent']`

Related

JSON Parsing with python from Rethink database [Python]

Im trying to retrieve data from a database named RethinkDB, they output JSON when called with r.db("Databasename").table("tablename").insert([{ "id or primary key": line}]).run(), when doing so it outputs [{'id': 'ValueInRowOfid\n'}] and I want to parse that to just the value eg. "ValueInRowOfid". Ive tried with JSON in Python, but I always end up with the typeerror: list indices must be integers or slices, not str, and Ive been told that it is because the Database outputs invalid JSON format. My question is how can a JSON format be invalid (I cant see what is invalid with the output) and also what would be the best way to parse it so that the value "ValueInRowOfid" is left in a Operator eg. Value = ("ValueInRowOfid").
This part imports the modules used and connects to RethinkDB:
import json
from rethinkdb import RethinkDB
r = RethinkDB()
r.connect( "localhost", 28015).repl()
This part is getting the output/value and my trial at parsing it:
getvalue = r.db("Databasename").table("tablename").sample(1).run() # gets a single row/value from the table
print(getvalue) # If I print that, it will show as [{'id': 'ValueInRowOfid\n'}]
dumper = json.dumps(getvalue) # I cant use `json.loads(dumper)` as JSON object must be str. Which the output of the database isnt (The output is a list)
parsevalue = json.loads(dumper) # After `json.dumps(getvalue)` I can now load it, but I cant use the loaded JSON.
print(parsevalue["id"]) # When doing this it now says that the list is a str and it needs to be an integers or slices. Quite frustrating for me as it is opposing it self eg. It first wants str and now it cant use str
print(parsevalue{'id'}) # I also tried to shuffle it around as seen here, but still the same result
I know this is janky and is very hard to comprehend this level of stupidity that I might be on. As I dont know if it is the most simple problem or something that just isnt possible (Which it should or else I cant use my data in the database.)
Thank you for reading this through and not jumping straight into the comments and say that I have to read the JSON documentation, because I have and I havent found a single piece that could help me.
I tried reading the documentation and watching tutorials about JSON and JSON parsing. I also looked for others whom have had the same problems as me and couldnt find.
It looks like it's returning a dictionary ({}) inside a list ([]) of one element.
Try:
getvalue = r.db("Databasename").table("tablename").sample(1).run()
print(getvalue[0]['id'])

Convert Json format String to Link{"link":"https://i.imgur.com/zfxsqlk.png"}

I try to convert this String to only the link: {"link":"https://i.imgur.com/zfxsqlk.png"}
I'm trying to create a discord bot, which sends random pictures from the API https://some-random-api.ml/img/red_panda.
With imageURL = json.loads(requests.get(redpandaurl).content) I get the json String, but what do I have to do that I only get the Link like this https://i.imgur.com/zfxsqlk.png
Sorry if my question is confusingly written, I'm new to programming and don't really know how to describe this problem.
You can simply do this:
image_url = requests.get(your_api_url).json()["link"]
Directly use requests.json(), no need to load the string with json.loads and other manual stuff.
What you get from json.loads() is a Python dict. You can access values in the dict by specifying their keys.
In your case, there is only one key-value pair in the dict: "link" is the key and "https://i.imgur.com/zfxsqlk.png" is the value. You can get the link and store it in the value by appending ["link"] to your line of code:
imageURL = json.loads(requests.get(redpandaurl).content)["link"]

Python: Parsing JSON data from API get - referring to the dictionary key?

I'm pretty new to Python so I'm only just starting to work with API's. I have retrieved the data I need from an API and it returns in the following format:
{u'id': u'000123', u'payload': u"{'account_title': u’sam b’, 'phone_num': ‘1234567890’, 'security_pin': u'000000', 'remote_word': u’secret123’, 'email_address': ‘email#gmail.com’, 'password': u’password123’}”}
So when I print my variable, it returns the above...
If I just want part of what it returns, how do I go about writing this? I was able to return id without issue, and if I specified 'payload' it returned everything after that. It seems like account_title, phone_num, security_pin, remote_word, email_address and password are all nested inside of 'payload'
Would would the best way be to have a variable, when printed, return just the email_address for example?
Thanks!
Welcome to Python! Sounds like you're getting right into it. It would be best to begin reading fundamentals, specifically about the Dictionary Data Structure
The Dictionary, or dict is what you are referencing in your question. It's a key-value store that is generally[1] un-ordered. The dict is a great way to represent JSON data.
Now you are asking how to extract information from a dictionary. Well, you seem to have it working out thus far! Let's use your example:
d = {u'id': u'000123', u'payload': u"{'account_title': u’sam b’, 'phone_num': ‘1234567890’, 'security_pin': u'000000', 'remote_word': u’secret123’, 'email_address': ‘email#gmail.com’, 'password': u’password123’}"}
Now if we write d['id'], we'll get the id (which is 000123)
If we write d['payload'], we'll get the dictionary within this larger dictionary. Cool part about dicts, they can be nested like this! As many times as you need.
d['payload']
"{'account_title': u’sam b’, 'phone_num': ‘1234567890’, 'security_pin': u'000000', 'remote_word': u’secret123’, 'email_address': ‘email#gmail.com’, 'password': u’password123’}"
Then as per your question, if you wanted to get email, it's the same syntax and you're just nesting the accessor. Like so:
d['payload']['email_address']
Hope that helps!
For the longest time, dicts were un-ordered in Python. In versions 3.6 and up, things began changing. This answer provides great detail on that. Otherwise, prior to that, using collections.OrderedDict was the only way to get a dict ordered by insertion-order

python dictionary from url

I am trying to gather weather data from the national weather service and read it into a python script. They offer a JSON return, but they also offer another return which isn't formatted JSON but has more variables (which I want). This set of data looks like it is formatted as a python dictionary. It looks like this:
stations={
KAPC:
{
'id':'KAPC',
'stnid':'92',
'name':'Napa, Napa County Airport',
'elev':'33',
'latitude':'38.20750',
'longitude':'-122.27944',
'distance':'',
'provider':'NWS/FAA',
'link':'http://www.wrh.noaa.gov/mesowest/getobext.php?sid=KAPC',
'Date':'24 Feb 8:54 am',
'Temp':'39',
'TempC':'4',
'Dewp':'29',
'Relh':'67',
'Wind':'NE#6',
'Direction':'50&#176',
'Winds':'6',
'WindChill':'35',
'Windd':'50',
'SLP':'1027.1',
'Altimeter':'30.36',
'Weather':'',
'Visibility':'10.00',
'Wx':'',
'Clouds':'CLR',
[...]
So, to me, it looks like its got a defined variable stations equal to a dictionary of dictionaries containing the stations and their variables. My question is how do I access this data. Right now I am trying:
import urllib
response = urrllib.urlopen(url)
r = response.read()
If I try to use the JSON module, it clearly fails because this isn't json. And if I just try to read the file, it comes back with a long string of characters. Any suggestions on how to extract this data? If possible, I would just like to get the dictionary as it exists in the url return, ie stations={...} Thanks!
See, As far I infer from the question, I assume that you have data in the form of text which in not a valid JSON data, So given we have a text like: line = "stations={'KAPC':{'id':'KAPC', 'stnid':'92', 'name':'Napa, Napa County Airport'}}" (say), then we can extract the dictionary by splitting it at the = symbol and then use the eval() method which initializes the dictionary variable with the required data.
dictionary_text = line.split("=")[1]
python_dictionary = eval(dictionary_text)
print python_dictionary
>>> {'KAPC': {'id': 'KAPC', 'name': 'Napa, Napa County Airport', 'stnid': '92'}}
The python_dictionary now behaves like a Python Dictionary with key, value pairs , and you can access any attribute using python_dictionary["KAPC"]["id"]

Decoding json data to Python dictionary

I am currently trying to create a dictionary from a json formatted server response:
{"id": null,{"version": "1.1","result": "9QtirjtH9b","error": null}}
Therefore I am using json.loads(). But I always get the following error:
ValueError: Expecting property name: line 1 column 12 (char 12)
I know that this means that there is an error in the json syntax and I found some threads (like this one) here at stackoverflow, but they did not include an answer that solved my problem.
However, I was not sure if the null value within the json response causes the error, so I had a closer look at the json.org Reference Manual and it seems to be a valid syntax. Any ideas?
It's not valid. The outer object needs a property name for the second element; raw values are not valid in an object.
{"id": null, "somename":{"version": "1.1","result": "9QtirjtH9b","error": null}}
The problem here is the lack of a key for the nested object, not the null. You'd need to find a way to fix that syntax or parse it yourself.
If we make a few assumptions about the syntax, you should be able to use a regular expression to fix the JSON data before decoding:
import re
from itertools import count
def _gen_id(match, count=count()):
return '{1}"generated_id_{0}":{2}'.format(next(count), *match.groups())
_no_key = re.compile(r'(,)({)')
def fix_json(json_data):
return _no_key.sub(_gen_id, json_data)
This assumes that any ,{ combo indicates the location of a missing key, and generates one to insert there. That is a reasonable assumption to make, but may break things if you have string data with exactly that sequence.
Demo:
>>> json_data = '{"id": null,{"version": "1.1","result": "9QtirjtH9b","error": null}}'
>>> fix_json(json_data)
'{"id": null,"generated_id_0":{"version": "1.1","result": "9QtirjtH9b","error": null}}'
>>> json.loads(fix_json(json_data))
{u'id': None, u'generated_id_1': {u'version': u'1.1', u'result': u'9QtirjtH9b', u'error': None}}

Categories

Resources