Develop Reference

JSON Parsing with python from Rethink database [Python]

Im trying to retrieve data from a database named RethinkDB, they output JSON when called with r.db("Databasename").table("tablename").insert([{ "id or primary key": line}]).run(), when doing so it outputs [{'id': 'ValueInRowOfid\n'}] and I want to parse that to just the value eg. "ValueInRowOfid". Ive tried with JSON in Python, but I always end up with the typeerror: list indices must be integers or slices, not str, and Ive been told that it is because the Database outputs invalid JSON format. My question is how can a JSON format be invalid (I cant see what is invalid with the output) and also what would be the best way to parse it so that the value "ValueInRowOfid" is left in a Operator eg. Value = ("ValueInRowOfid").
This part imports the modules used and connects to RethinkDB:
import json
from rethinkdb import RethinkDB
r = RethinkDB()
r.connect( "localhost", 28015).repl()
This part is getting the output/value and my trial at parsing it:
getvalue = r.db("Databasename").table("tablename").sample(1).run() # gets a single row/value from the table
print(getvalue) # If I print that, it will show as [{'id': 'ValueInRowOfid\n'}]
dumper = json.dumps(getvalue) # I cant use `json.loads(dumper)` as JSON object must be str. Which the output of the database isnt (The output is a list)
parsevalue = json.loads(dumper) # After `json.dumps(getvalue)` I can now load it, but I cant use the loaded JSON.
print(parsevalue["id"]) # When doing this it now says that the list is a str and it needs to be an integers or slices. Quite frustrating for me as it is opposing it self eg. It first wants str and now it cant use str
print(parsevalue{'id'}) # I also tried to shuffle it around as seen here, but still the same result
I know this is janky and is very hard to comprehend this level of stupidity that I might be on. As I dont know if it is the most simple problem or something that just isnt possible (Which it should or else I cant use my data in the database.)
Thank you for reading this through and not jumping straight into the comments and say that I have to read the JSON documentation, because I have and I havent found a single piece that could help me.
I tried reading the documentation and watching tutorials about JSON and JSON parsing. I also looked for others whom have had the same problems as me and couldnt find.

It looks like it's returning a dictionary ({}) inside a list ([]) of one element.
Try:
getvalue = r.db("Databasename").table("tablename").sample(1).run()
print(getvalue[0]['id'])

Convert Json format String to Link{"link":"https://i.imgur.com/zfxsqlk.png"}

I try to convert this String to only the link: {"link":"https://i.imgur.com/zfxsqlk.png"}
I'm trying to create a discord bot, which sends random pictures from the API https://some-random-api.ml/img/red_panda.
With imageURL = json.loads(requests.get(redpandaurl).content) I get the json String, but what do I have to do that I only get the Link like this https://i.imgur.com/zfxsqlk.png
Sorry if my question is confusingly written, I'm new to programming and don't really know how to describe this problem.

You can simply do this:
image_url = requests.get(your_api_url).json()["link"]
Directly use requests.json(), no need to load the string with json.loads and other manual stuff.

What you get from json.loads() is a Python dict. You can access values in the dict by specifying their keys.
In your case, there is only one key-value pair in the dict: "link" is the key and "https://i.imgur.com/zfxsqlk.png" is the value. You can get the link and store it in the value by appending ["link"] to your line of code:
imageURL = json.loads(requests.get(redpandaurl).content)["link"]

How do I pull out a certain segment from a string

I'm using an API that is giving me and output formatted as
['{"quote":{"symbol":"AAPL"', '"companyName":"Apple Inc."', '"primaryExchange":"Nasdaq Global Select"', '"sector":"Technology"', '"calculationPrice":"close"', '"open":367.88', '"openTime":1593696600532', '"close":364.11', '"closeTime":1593720000277', '"high":370.47', '"low":363.64', '"latestPrice":364.11'}]
...(it keeps going like this with many more categories.)
I am attempting to pull out only the latest price. What would be the best way to do that?
This is what I have but I get a bunch of errors.
string = (data.decode("utf-8"))
data_v = string.split(',')
for word in data_v[latestPrice]:
if word == ["latestPrice"]:
print(word)
print(data_v)

Judging by the output this is JSON. To parse this easily use the JSON module (see https://docs.python.org/3/library/json.html ).
If I'm correct you got this output from Yahoo Finance, if this indeed the case don't fetch and parse it manually but use the yfinance module (see https://pypi.org/project/yfinance/ )

You will have to use JSON module to parse this JSON string. You can convert it into dictionary then. I have indented the JSON code for ease of understanding. You can use the following approach,
import json
text_to_parse = """
{"quote":
{
"symbol":"AAPL",
"companyName":"Apple Inc.",
"primaryExchange":"Nasdaq Global Select",
"sector":"Technology",
"calculationPrice":"close",
"open":367.88,
"openTime":1593696600532,
"close":364.11,
"closeTime":1593720000277,
"high":370.47,
"low":363.64,
"latestPrice":364.11
}
}
"""
parsed_dict = json.loads(text_to_parse)
print(parsed_dict["quote"]["latestPrice"])
When the program is run, it outputs 364.11

How to separate data in a Restful API?

I am working on a program that reads the content of a Restful API from ImportIO. The connection works, and data is returned, but it's a jumbled mess. I'm trying to clean it to only return Asins.
I have tried using the split keyword and delimiter to no success.
stuff = requests.get('https://data.import.io/extractor***')
stuff.content
I get the content, but I want to extract only Asins.
results

While .content gives you access to the raw bytes of the response payload, you will often want to convert them into a string using a character encoding such as UTF-8. the response will do that for you when you access .text.
response.txt
Because the decoding of bytes to str requires an encoding scheme, requests will try to guess the encoding based on the response’s headers if you do not specify one. You can provide an explicit encoding by setting .encoding before accessing .text:
If you take a look at the response, you’ll see that it is actually serialized JSON content. To get a dictionary, you could take the str you retrieved from .text and deserialize it using json.loads(). However, a simpler way to accomplish this task is to use .json():
response.json()
The type of the return value of .json() is a dictionary, so you can access values in the object by key.
You can do a lot with status codes and message bodies. But, if you need more information, like metadata about the response itself, you’ll need to look at the response’s headers.
For More Info: https://realpython.com/python-requests/

What format is the return information in? Typically Restful API's will return the data as json, you will likely have luck parsing the it as a json object.
https://realpython.com/python-requests/#content
stuff_dictionary = stuff.json()
With that, you can load the content is returned as a dictionary and you will have a much easier time.
EDIT:
Since I don't have the full URL to test, I can't give an exact answer. Given the content type is CSV, using a pandas DataFrame is pretty easy. With a quick StackOverflow search, I found the following answer: https://stackoverflow.com/a/43312861/11530367
So I tried the following in the terminal and got a dataframe from it
from io import StringIO
import pandas as pd
pd.read_csv(StringIO("HI\r\ntest\r\n"))
So you should be able to perform the following
from io import StringIO
import pandas as pd
df = pd.read_csv(StringIO(stuff.content))
If that doesn't work, consider dropping the first three bytes you have in your response: b'\xef\xbb\xf'. Check the answer from Mark Tolonen to get parse this.
After that, selecting the ASIN (your second column) from your dataframe should be easy.
asins = df.loc[:, 'ASIN']
asins_arr = asins.array

The response is the byte string of CSV content encoded in UTF-8. The first three escaped byte codes are a UTF-8-encoded BOM signature. So stuff.content.decode('utf-8-sig') should decode it. stuff.text may also work if the encoding was returned correctly in the response headers.

Python grabbing JSON from POST method

I have an Android appthat originally posted some strings in json format to a python cgi script, which all worked fine. The problem is when the json object contains lists, then python (Using simplejson) when it gets them is still treating them as a big string
Here is a text dump of the json once it reaches python before I parse it:
{"Prob1":"[1, 2, 3]","Name":"aaa","action":1,"Prob2":"[20, 20, 20]","Tasks":"[1 task, 2 task, 3 task]","Description":""}
if we look at the "Tasks" key, the list after is clearly a single string with the elements all treated as one string (i.e. no quotes around each element). it's the same for prob1 and prob2. action, Name etc are all fine. I'm not sure if this is what python is expecting but I'm guessing not?
Just in case the android data was to blame i added quotes around each element of the arraylist like this:
Tasks.add('"'+row.get(1).toString()+'"'); instead of Tasks.add(row.get(1).toString());
On the webserver it's now received as
{"Prob1":"[1, 2, 3]","Name":"aaa","action":1,"Prob2":"[20, 20, 20]","Tasks":"[\"1 task\", \"2 task\", \"3 task\"]","Description":""}
but i still get the same problem; when i iterate through "Tasks" in a loop it's looping through each individual character as if the whole thing were a string :/
Since I don't know what the json structure should look like before it gets to Python I'm wondering whether it's a probem with the Android sending the data or my python interpreting it.. though from the looks of that script I've been guessing it's been the sending.
In the Android App I'm sending one big JSONObject containing "Tasks" and the associated arraylist as one of the key value pairs... is this correct? or should JSONArray be involved anywhere?
Thanks for any help everyone, I'm new to the whole JSON thing as well as to Android/Java (And only really a novice at Python too..). I can post additional code if anyone needs it, I just didn't want to lengthen the post too much
EDIT:
when I add
json_data=json_data.replace(r'"[','[')
json_data=json_data.replace(r']"',']')
json_data=json_data.replace(r'\"','"')
to the python it WORKS!!!! but that strikes me as a bit nasty and just papering over a crack..

Tasks is just a big string. To be a valid list, it would have to be ["1 task", "2 task", "3 task"]
Same goes for Prob1 and Prob2. To be a valid list, the brackets should not be enclosed in quotes.