Develop Reference

How to handle dictionary keys that do not exist in a loop?

I tried to parse json and use for loop in Python.
obj is json and as you know it is like dictionary in list in dictionary in dictionary...and so on.
So I parse json and make it as a for loop like this:
all = obj['products']
for a in obj:
recommendSellerScore.append(a['recommendSellerScore'])
However, the problem is 'recommendSellerScore' which is key does not exist in some lists.
What I want as a result is to print 'None' where recommendSellerScore does not exist, and print values where recommendSellerScore does exist.
for a in all:
if a['recommendSellerScore'] in all:
recommendSellerScore.append(a['recommendSellerScore'])
else:
continue
print('None')
I ran this code above, but got an Error because 'recommendSellerScore' does not exist.
for a in all:
if ['recommendSellerScore'] in all:
recommendSellerScore.append(a['recommendSellerScore'])
else:
continue
print('None')
This one worked, however, since 'recommendSellerScore' exists in list in dictionary, it was not iterable. (I guess)
Please fix my code, and any comments or advice will be appreciated!

Here is fixed version of your code (I took out the continue statement and the [] around the string):
for a in all:
if "recommendSellerScore" in a:
recommendSellerScore.append(a["recommendSellerScore"])
else:
print("None")

Maybe something like the code below works for you?
for a in all:
if a.get(['recommendSellerScore']) is not None:
recommendSellerScore.append(a['recommendSellerScore'])
else:
print("None")
# No need for continue here.

You can use get method of dictionaries to try to access the dictionary value, if the key is not present you can provide a default value, if you don't specify one, it will output None, for example:
key = 'recommendSellerScore'
a = [{key: 1}, {}]
recommendSellerScore = []
for d in a:
score = d.get(key)
if score is not None:
print(score)
recommendSellerScore.append(score)
else:
print('None')
Edit: as pointed by #kederrac if your data can have {key: None} values and you are interested in keeping those None values then it's best to ask if the key is truly present in the dictionary, for example:
key = 'recommendSellerScore'
a = [{key: 1}, {key: None}]
recommendSellerScore = []
for d in a:
if key in d.keys():
print(d[key])
recommendSellerScore.append(d[key])
else:
print('None')
It will print:
>>> 1
>>> None

Best way to write for loop while being able to access items in container Python

This is relatively straightforward but more pointed question on how to write the most concise if/any/for loop that accomplishes what I need. I'm rather new to Python, and much more used to the C++ way of doing things.
I need to do something like the following:
for item in my_dict:
if some_var == item.some_attribute:
if item.another_attribute exists:
variable = "A"
else variable = "B"
print "Duplicate item exists in my list of type: {}".format(variable)
I've condensed it into something like this:
if any(some_var == item.some_attribute for item in my_dict):
variable = "A" if item.another_attribute else "B"
print "Duplicate item exists in my list of type: {}".format(variable)
However, when doing this, I'm getting an "unresolved reference" error for item. Any ideas as to how I can write a concise loop, equivalence check, and presence check like I've described above in a way that allows me to access and perform methods on item?
Thank you!
EDIT: thank you so much for the answer #pjhaugh, that's exactly what I was looking for. Works beautifully! Thank you everyone for your helpful comments as well.

What's happening here is that item does not leak out of the generator expression scope (this is a good thing). You could obtain item by getting the next thing yielded by a generator, or catching that generator yielding nothing.
try:
item = next(item for item in my_dict if some_var == item.some_attribute)
variable = "A" if item.another_attribute else "B"
except StopIteration:
# there is no such item in my_dict

You can extract the objects yielded by an iterator in a for loop like this
for item in (i for i in my_dict if some_var == item.some_attribute):
variable = "A" if item.another_attribute is not None else "B"
print("Duplicate item exists in my list of type: {}".format(variable))
break
else: # no break encountered
... # not found
Currently you're iterating over the keys of the dictionary. If you want the items (pairs in c++ lingo), you need my_dict.items().

As pointed out in my comment - the any(some_var == item.some_attribute for item in my_dict) builds its own local scope. You cannot use item outside of it.
You can however loop over your dict like so:
Create minimal verifyable complete example data:
class K:
def __init__(self):
self.some_attribute = None
self.another_attribute = None
def __repr__(self):
return "{} - {}".format(self.some_attribute,self.another_attribute)
def __str__(self):
return "{} - {}".format(self.some_attribute,self.another_attribute)
k1 = K()
k1.some_attribute = 99
k2 = K()
k2.some_attribute = 33
k3 = K()
k3.some_attribute = 33
k3.another_attribute = 1
my_dict = { 1: k1, 2:k2, 3:k3}
some_var = 33
You can use a for loop like so:
variable = None
for item in my_dict.items():
k,v = item
if v.some_attribute != some_var:
print "No dupe: {}".format(v)
continue
if v.some_attribute == some_var and v.another_attribute is not None:
variable = "A"
else:
variable = "B"
print "Duplicate item exists in my list of type: {}".format(variable)
print v
Output:
No dupe: 99 - None
Duplicate item exists in my list of type: B
33 - None
Duplicate item exists in my list of type: A
33 - 1

Test if dictionary key exists, is not None and isn't blank

I have code that works but I'm wondering if there is a more pythonic way to do this. I have a dictionary and I want to see if:
a key exists
that value isn't None (NULL from SQL in this case)
that value isn't simply quote quote (blank?)
that value doesn't solely consist of spaces
So in my code the keys of "a", "b", and "c" would succeed, which is correct.
import re
mydict = {
"a":"alpha",
"b":0,
"c":False,
"d":None,
"e":"",
"g":" ",
}
#a,b,c should succeed
for k in mydict.keys():
if k in mydict and mydict[k] is not None and not re.search("^\s*$", str(mydict[k])):
print(k)
else:
print("I am incomplete and sad")
What I have above works, but that seems like an awfully long set of conditions. Maybe this simply is the right solution but I'm wondering if there is a more pythonic "exists and has stuff" or better way to do this?
UPDATE
Thank you all for wonderful answers and thoughtful comments. With some of the points and tips, I've updated the question a little bit as there some conditions I didn't have which should also succeed. I have also changed the example to a loop (just easier to test right?).

Try to fetch the value and store it in a variable, then use object "truthyness" to go further on with the value
v = mydict.get("a")
if v and v.strip():
if "a" is not in the dict, get returns None and fails the first condition
if "a" is in the dict but yields None or empty string, test fails, if "a" yields a blank string, strip() returns falsy string and it fails too.
let's test this:
for k in "abcde":
v = mydict.get(k)
if v and v.strip():
print(k,"I am here and have stuff")
else:
print(k,"I am incomplete and sad")
results:
a I am here and have stuff
b I am incomplete and sad # key isn't in dict
c I am incomplete and sad # c is None
d I am incomplete and sad # d is empty string
e I am incomplete and sad # e is only blanks
if your values can contain False, 0 or other "falsy" non-strings, you'll have to test for string, in that case replace:
if v and v.strip():
by
if v is not None and (not isinstance(v,str) or v.strip()):
so condition matches if not None and either not a string (everything matches) or if a string, the string isn't blank.

The get method for checking if a key exists is more efficient that iterating through the keys. It checks to see if the key exists without iteration using an O(1) complexity as apposed to O(n). My preferred method would look something like this:
if mydict.get("a") is not None and str(mydict.get("a")).replace(" ", "") != '':
# Do some work

You can use a list comprehension with str.strip to account for whitespace in strings.
Using if v is natural in Python to cover False-like objects, e.g. None, False, 0, etc. So note this only works if 0 is not an acceptable value.
res = [k for k, v in mydict.items() if (v.strip() if isinstance(v, str) else v)]
['a']

Here's a simple one-liner to check:
The key exists
The key is not None
The key is not ""
bool(myDict.get("some_key"))
As for checking if the value contains only spaces, you would need to be more careful as None doesn't have a strip() method.
Something like this as an example:
try:
exists = bool(myDict.get('some_key').strip())
except AttributeError:
exists = False

Well I have 2 suggestions to offer you, especially if your main issue is the length of the conditions.
The first one is for the check if the key is in the dict. You don't need to use "a" in mydict.keys() you can just use "a" in mydict.
The second suggestion to make the condition smaller is to break down into smaller conditions stored as booleans, and check these in your final condition:
import re
mydict = {
"a":"alpha",
"c":None,
"d":"",
"e":" ",
}
inKeys = True if "a" in mydict else False
isNotNone = True if mydict["a"] is not None else False
isValidKey = True if not re.search("^\s*$", mydict["a"]) else False
if inKeys and isNotNone and isValidKey:
print("I am here and have stuff")
else:
print("I am incomplete and sad")

it check exactly for NoneType not only None
from types import NoneType # dont forget to import this
mydict = {
"a":"alpha",
"b":0,
"c":False,
"d":None,
"e":"",
"g":" ",
}
#a,b,c should succeed
for k in mydict:
if type(mydict[k]) != NoneType:
if type(mydict[k]) != str or type(mydict[k]) == str and mydict[k].strip():
print(k)
else:
print("I am incomplete and sad")
else:
print("I am incomplete and sad")

cond is a generator function responsible for generating conditions to apply in a short-circuiting manner using the all function. Given d = cond(), next(d) will check if a exists in the dict, and so on until there is no condition to apply, in that case all(d) will evaluate to True.
mydict = {
"a":"alpha",
"c":None,
"d":"",
"e":" ",
}
def cond ():
yield 'a' in mydict
yield mydict ['a']
yield mydict ['a'].strip ()
if all (cond ()):
print("I am here and have stuff")
else:
print("I am incomplete and sad")

Add new item to Dictionary dynamically using a variable

I'm currently reading values from file and spliting them as parameter and value e.g. #id=7 becomes param = #id, value = 7. I would like to use the param variable as a new key in the dictionary. However, it is not working as expected. I'm using the following code.
list1 = {}
with open('C:/Temp/file1.txt') as f:
lines = f.read().splitlines()
for line in lines:
middle = line.find("=")
param = line[:middle]
value = line[middle+1:]
list1[param] = value
In this code, the dictionary key and value becomes 7.
Thanks in advance.

You have to define your dictionary (d is a nice name). You can do it this way:
with open('C:/Temp/file1.txt') as f:#
d = dict(line.strip().split('=', 1) for line in f)
for k,v in d.iteritems():
print("param = {0}, value = {1}".format(k,v))

If you are defining list1 as a dict list1 = {} then your print statement is incorrect.
print("param = " + list1[param] + ", value = " + value)
both list1[param] and value would be 7. since list1[param] would give you the value of it's contents and not it's key.
Try looking at the dictionary afterwards by printing it.
print(list1)

I know this wasn't what you were asking for, but I would suggest to look at ConfigParser. This is a standard way to use configuration files.

Finding matching keys in two large dictionaries and doing it fast

I am trying to find corresponding keys in two different dictionaries. Each has about 600k entries.
Say for example:
myRDP = { 'Actinobacter': 'GATCGA...TCA', 'subtilus sp.': 'ATCGATT...ACT' }
myNames = { 'Actinobacter': '8924342' }
I want to print out the value for Actinobacter (8924342) since it matches a value in myRDP.
The following code works, but is very slow:
for key in myRDP:
for jey in myNames:
if key == jey:
print key, myNames[key]
I've tried the following but it always results in a KeyError:
for key in myRDP:
print myNames[key]
Is there perhaps a function implemented in C for doing this? I've googled around but nothing seems to work.
Thanks.

Use sets, because they have a built-in intersection method which ought to be quick:
myRDP = { 'Actinobacter': 'GATCGA...TCA', 'subtilus sp.': 'ATCGATT...ACT' }
myNames = { 'Actinobacter': '8924342' }
rdpSet = set(myRDP)
namesSet = set(myNames)
for name in rdpSet.intersection(namesSet):
print name, myNames[name]
# Prints: Actinobacter 8924342

You could do this:
for key in myRDP:
if key in myNames:
print key, myNames[key]
Your first attempt was slow because you were comparing every key in myRDP with every key in myNames. In algorithmic jargon, if myRDP has n elements and myNames has m elements, then that algorithm would take O(n×m) operations. For 600k elements each, this is 360,000,000,000 comparisons!
But testing whether a particular element is a key of a dictionary is fast -- in fact, this is one of the defining characteristics of dictionaries. In algorithmic terms, the key in dict test is O(1), or constant-time. So my algorithm will take O(n) time, which is one 600,000th of the time.

in python 3 you can just do
myNames.keys() & myRDP.keys()

for key in myRDP:
name = myNames.get(key, None)
if name:
print key, name
dict.get returns the default value you give it (in this case, None) if the key doesn't exist.

You could start by finding the common keys and then iterating over them. Set operations should be fast because they are implemented in C, at least in modern versions of Python.
common_keys = set(myRDP).intersection(myNames)
for key in common_keys:
print key, myNames[key]

Best and easiest way would be simply perform common set operations(Python 3).
a = {"a": 1, "b":2, "c":3, "d":4}
b = {"t1": 1, "b":2, "e":5, "c":3}
res = a.items() & b.items() # {('b', 2), ('c', 3)} For common Key and Value
res = {i[0]:i[1] for i in res} # In dict format
common_keys = a.keys() & b.keys() # {'b', 'c'}
Cheers!

Use the get method instead:
for key in myRDP:
value = myNames.get(key)
if value != None:
print key, "=", value

You can simply write this code and it will save the common key in a list.
common = [i for i in myRDP.keys() if i in myNames.keys()]

Copy both dictionaries into one dictionary/array. This makes sense as you have 1:1 related values. Then you need only one search, no comparison loop, and can access the related value directly.
Example Resulting Dictionary/Array:
[Name][Value1][Value2]
[Actinobacter][GATCGA...TCA][8924342]
[XYZbacter][BCABCA...ABC][43594344]
...

Here is my code for doing intersections, unions, differences, and other set operations on dictionaries:
class DictDiffer(object):
"""
Calculate the difference between two dictionaries as:
(1) items added
(2) items removed
(3) keys same in both but changed values
(4) keys same in both and unchanged values
"""
def __init__(self, current_dict, past_dict):
self.current_dict, self.past_dict = current_dict, past_dict
self.set_current, self.set_past = set(current_dict.keys()), set(past_dict.keys())
self.intersect = self.set_current.intersection(self.set_past)
def added(self):
return self.set_current - self.intersect
def removed(self):
return self.set_past - self.intersect
def changed(self):
return set(o for o in self.intersect if self.past_dict[o] != self.current_dict[o])
def unchanged(self):
return set(o for o in self.intersect if self.past_dict[o] == self.current_dict[o])
if __name__ == '__main__':
import unittest
class TestDictDifferNoChanged(unittest.TestCase):
def setUp(self):
self.past = dict((k, 2*k) for k in range(5))
self.current = dict((k, 2*k) for k in range(3,8))
self.d = DictDiffer(self.current, self.past)
def testAdded(self):
self.assertEqual(self.d.added(), set((5,6,7)))
def testRemoved(self):
self.assertEqual(self.d.removed(), set((0,1,2)))
def testChanged(self):
self.assertEqual(self.d.changed(), set())
def testUnchanged(self):
self.assertEqual(self.d.unchanged(), set((3,4)))
class TestDictDifferNoCUnchanged(unittest.TestCase):
def setUp(self):
self.past = dict((k, 2*k) for k in range(5))
self.current = dict((k, 2*k+1) for k in range(3,8))
self.d = DictDiffer(self.current, self.past)
def testAdded(self):
self.assertEqual(self.d.added(), set((5,6,7)))
def testRemoved(self):
self.assertEqual(self.d.removed(), set((0,1,2)))
def testChanged(self):
self.assertEqual(self.d.changed(), set((3,4)))
def testUnchanged(self):
self.assertEqual(self.d.unchanged(), set())
unittest.main()

def combine_two_json(json_request, json_request2):
intersect = {}
for item in json_request.keys():
if item in json_request2.keys():
intersect[item]=json_request2.get(item)
return intersect