Develop Reference

How to check each key separately from a list in a loop without creating multiple loops. Which may have a KeyError etc

I wrote a code that takes 9 keys from API.
The authors, isbn_one, isbn_two, thumbinail, page_count fields may not always be retrievable, and if any of them are missing, I would like it to be None. Unfortunately, if, or even nested, doesn't work. Because that leads to a lot of loops. I also tried try and except KeyError etc. because each key has a different error and it is not known which to assign none to. Here is an example of logic when a photo is missing:
th = result['volumeInfo'].get('imageLinks')
if th is not None:
book_exists_thumbinail = {
'thumbinail': result['volumeInfo']['imageLinks']['thumbnail']
}
dnew = {**book_data, **book_exists_thumbinail}
book_import.append(dnew)
else:
book_exists_thumbinail_n = {
'thumbinail': None
}
dnew_none = {**book_data, **book_exists_thumbinail_n}
book_import.append(dnew_none)
When I use logic, you know when one condition is met, e.g. for thumbinail, the rest is not even checked.
When I use try and except, it's similar. There's also an ISBN in the keys, but there's a list in the dictionary over there, and I need to use something like this:
isbn_zer = result['volumeInfo']['industryIdentifiers']
dic = collections.defaultdict(list)
for d in isbn_zer:
for k, v in d.items():
dic[k].append(v)
Output data: [{'type': 'ISBN_10', 'identifier': '8320717507'}, {'type': 'ISBN_13', 'identifier': '9788320717501'}]
I don't know what to use anymore to check each key separately and in the case of its absence or lack of one ISBN (identifier) assign the value None. I have already tried many ideas.
The rest of the code:
book_import = []
if request.method == 'POST':
filter_ch = BookFilterForm(request.POST)
if filter_ch.is_valid():
cd = filter_ch.cleaned_data
filter_choice = cd['choose_v']
filter_search = cd['search']
search_url = "https://www.googleapis.com/books/v1/volumes?"
params = {
'q': '{}{}'.format(filter_choice, filter_search),
'key': settings.BOOK_DATA_API_KEY,
'maxResults': 2,
'printType': 'books'
}
r = requests.get(search_url, params=params)
results = r.json()['items']
for result in results:
book_data = {
'title': result['volumeInfo']['title'],
'authors': result['volumeInfo']['authors'][0],
'publish_date': result['volumeInfo']['publishedDate'],
'isbn_one': result['volumeInfo']['industryIdentifiers'][0]['identifier'],
'isbn_two': result['volumeInfo']['industryIdentifiers'][1]['identifier'],
'page_count': result['volumeInfo']['pageCount'],
'thumbnail': result['volumeInfo']['imageLinks']['thumbnail'],
'country': result['saleInfo']['country']
}
book_import.append(book_data)
else:
filter_ch = BookFilterForm()
return render(request, "BookApp/book_import.html", {'book_import': book_import,
'filter_ch': filter_ch})```

How to take from key value and take from this value another value

How can i get needed value, because i send post request to other site and cant edit answer from site.
I have this dict from responded content:
{'username': 'DeadFinder', 'subscriptions': [{'subscription': 'default', 'expiry': '1635683460'}], 'ip': 'not at this life'}
How you can see in this dict there is a key subscriptions, i'm need value expiry(this is timestamp) but how can i get this value if when i'm trying to call this value i'm not see any results (code not gives needed value), maybe any variants how to get this value? I'm not finded anything like this.
Maybe my small part of code can smally help you but i doubt.
data1 = {f"hwid":"", "type":"login", "username": {username}, "pass": {password},
"sessionid":f"{response_cut2}", "name":"test_app", "ownerid":"5OLbm5S3fS"}
url1 = "nope"
response1 = requests.post(url1, data1)
data = response1.json()
#get = data.get('expiry')
file_write = open("test.txt", "w")
file_write.write(str(data))
file_write.close()
for key in data.keys():
if key == 'info':
print (data[key])

Are you trying to achieve this as result ?
data = {'username': 'DeadFinder', 'subscriptions': [{'subscription': 'default', 'expiry': '1635683460'}], 'ip': 'not at this life'}
print(data['subscriptions'][0]['expiry'])
# first get 'subscriptions' which returns an array,
# so use [0] to get this dict {'subscription': 'default', 'expiry': '1635683460'}
# then get 'expiry'
EDIT : In case subscriptions has multiple values then use for loop
subscriptions = data['subscriptions']
for subscription in subscriptions:
print(subscription['expiry'])
Output
1635683460

Iterating through separately-JSON-encoded strings found inside a document

I am having trouble iterating though json, containing nested json strings (with escaped quotes) in itself.
(My apologies in advance, I am sort of new and probably missing some important info...)
Actually I have several questions:
1) How can I iterate (as I tried to do below with nested for loops) through the elements beneath "section-content" of the section "nodes" (!not of the section "element-names"!)? My problem seems to be, that section-content is a string with escaped quotes, which represents a separate json string in itself.
2) Is the JSON example provided even valid json? I tried several validators, which all seem to fail when the escaped quotes come into play.
3) Is there a smarter method of accessing specific elements, instead of just iterating through the whole tree?
I am thinking of something that specifies key/value pairs like:
my_json_obj['sections']['section-id' = 'nodes']['section-content']['occ_id' = '051MZjd97jUdYfSEOG}k10']
Code:
import json
import requests
import pprint
client = requests.session()
header = {'X-CSRF-Token': 'Fetch', 'Accept': 'application/json', 'Content-Type': 'application/json'}
response = client.get('http://xxxxxx.xxx/ProcessManagement/BranchContentSet(BranchId=\'051MZjd97jUdYfX7{dREAm\',SiteId=\'\',SystemRole=\'D\')/$value',auth=('TestUser', 'TestPass'),headers=header)
my_json_obj = response.json()
sections = my_json_obj['sections']
for mysection in sections:
print(mysection['section-id'])
if mysection['section-id'] == 'NODES':
nodes = mysection['section-content'] #nodes seems to be string
for mynode in nodes:
print(mynode) #prints string character by character
JSON example:
{
"smud-data-version": "0.1",
"sections": [
{
"section-id": "ELEMENT-NAMES",
"section-content-version": "",
"section-content": "{\"D\":[
{\"occ_id\":\"051MZjd97kcBgtZiEI0IvW\",\"lang\":\"E\",\"name\":\"0TD1 manuell\"},
{\"occ_id\":\"051MZjd97kcBgtZiEH}IvW\",\"lang\":\"E\",\"name\":\"Documentation\"}
]}"
},
{
"section-id": "NODES",
"section-content-version": "1.0",
"section-content": "[
{\"occ_id\":\"051MZjd97jUdYfSEOG}k10\",\"obj_type\":\"ROOT\",\"reference\":\"\",\"deleted\":\"\",\"attributes\":[]},
{\"occ_id\":\"051MZjd97jUdYfSEOH0k10\",\"obj_type\":\"ROOTGRP\",\"reference\":\"\",\"deleted\":\"\",\"attributes\":[]},
{\"occ_id\":\"051MZjd97jcAnKoe03JRRm\",\"obj_type\":\"SCN\",\"reference\":\"\",\"deleted\":\"\",\"attributes\":[
{\"attr_type\":\"NODE_CHANGED_AT\",\"lang\":\"\",\"values\":[\"20190213095843\"]},
{\"attr_type\":\"NODE_CHANGED_BY\",\"lang\":\"\",\"values\":[\"TestUser\"]},
{\"attr_type\":\"TCASSIGNMENTTYPE\",\"lang\":\"\",\"values\":[\"A\"]},
{\"attr_type\":\"DESCRIPTION\",\"lang\":\"E\",\"values\":[\"Scenario\"]}
]}
]"
}
]
}
Actual output:
ELEMENT-NAMES
NODES
[
{
"
o
c
c
_
i
d
"

Hopefully you can convince the folks who are generating this data to fix their server. That said, to work around the issues might look like:
# instead of using requests.json(), remove literal newlines and then decode ourselves
# ...because the original data has newline literals in positions where they aren't allowed.
my_json_obj = json.loads(response.text.replace('\n', ''))
for section in my_json_obj['sections']:
if section['section-id'] != 'NODES': continue
# doing another json.loads() here so you treat content as an array, not a string
for node in json.loads(section['section-content']):
__import__('pprint').pprint(node)
...properly emits as output:
{u'attributes': [],
u'deleted': u'',
u'obj_type': u'ROOT',
u'occ_id': u'051MZjd97jUdYfSEOG}k10',
u'reference': u''}
{u'attributes': [],
u'deleted': u'',
u'obj_type': u'ROOTGRP',
u'occ_id': u'051MZjd97jUdYfSEOH0k10',
u'reference': u''}
{u'attributes': [{u'attr_type': u'NODE_CHANGED_AT',
u'lang': u'',
u'values': [u'20190213095843']},
{u'attr_type': u'NODE_CHANGED_BY',
u'lang': u'',
u'values': [u'TestUser']},
{u'attr_type': u'TCASSIGNMENTTYPE',
u'lang': u'',
u'values': [u'A']},
{u'attr_type': u'DESCRIPTION',
u'lang': u'E',
u'values': [u'Scenario']}],
u'deleted': u'',
u'obj_type': u'SCN',
u'occ_id': u'051MZjd97jcAnKoe03JRRm',
u'reference': u''}```

Using the data for 'requests' module in python

I am able to load in some weather data from the requests module for python with the following code:
from pprint import pprint
import requests
r = requests.get('http://api.openweathermap.org/data/2.5/weather?q=London')
pprint(r.json())
But how do I actually use the data it produces? I cannot for the life of me find the relevant documentation or tutorial on how to do this. this is the output of the pprint:
{u'base': u'cmc stations',
u'clouds': {u'all': 0},
u'cod': 200,
u'coord': {u'lat': 42.98, u'lon': -81.23},
u'dt': 1397676977,
u'id': 6058560,
u'main': {u'humidity': 25,
u'pressure': 1024,
u'temp': 278,
u'temp_max': 283.15,
u'temp_min': 275.37},
u'name': u'London',
u'rain': {u'1h': 0.25},
u'snow': {u'3h': 0},
u'sys': {u'country': u'CA',
u'message': 0.0467,
u'sunrise': 1397644810,
u'sunset': 1397693338},
u'weather': [{u'description': u'light rain'
u'icon': u'10d',
u'id': 500,
u'main': u'Rain'}],
u'wind': {u'deg': 168.001, u'speed': 3.52}}
How could I address an item within the list? For example to print just the temp on it's own and maybe to use it as a variable. E.g.:
temp = *not sure what to put here*
print temp

Now that you have the results:
results = r.json()
just access it like any other Python dict:
main = results['main'] # Get the 'main' key's value out of results
temp = main['temp'] # Get the 'temp' key's value out of main
print temp
or more tersely (and the way you'd almost always write this in real life):
print results['main']['temp']

How to select records with multiple ids from the MYSQL database

I am using python and MYSQL for fetching data , i know this is a basic question, but i am unable to find it.
I had a list of ids generated from some dictionary in python like example
dic = {'1': u'', '3': u'', '2': u'', '4': u''}
lis_ids = dic.keys()
query = "SELECT * FROM File_upload where id IN %s" % [int(i) for i in a]
print query
Result
"SELECT * FROM File_upload where id IN [1, 3, 2, 4]"
The above result will display syntax error because actually it should be as below
"SELECT * FROM File_upload where id IN (1, 3, 2, 4)"
How we can convert the list in to tuple in python, because i am getting the ids from a list.
Edited Code
Actually this is the second scenario which i got
The dict i have is
dic = {'submit': u'', '1': u'', '3': u'', '2': u'', '4': u'', 'groups': {}}
lis_ids = dic.keys()
Actually we will get submit and groups also in to the list, but i want only integer keys as ids
so also i want to implement this functionality like if not len(key) > 1 take the key
So can anyone able to implement all this functionality in one line as below ?

Try this:
dic = {'submit': u'', '1': u'', '3': u'', '2': u'', '4': u'', 'groups': {}}
list_ids = [elem for elem in dic.keys() if elem.isdigit()]
query = "SELECT * FROM File_upload where id IN (%s)" % ','.join(list_ids)
print query

Try this:
query = "SELECT * FROM File_upload where id IN %s" % str(tuple(int(i) for i in a))
(The str() call is required, unfortunately; you'll get a TypeError without it.)
If you want to just take the keys that are convertible to integers, try this:
def only_numbers(seq):
for i in seq:
try:
yield int(i)
except ValueError:
pass
query = "SELECT * FROM File_upload where id IN %s" % str(tuple(only_numbers(dic.keys())))
It's not as simple, because you have to try to convert each key to an integer and handle errors by doing nothing, so we need a generator function to handle that part of it.