nested loops causing unwanted duplicate list entries in python - python

My little program uses the Riot API (game), where I put players into either the 'ally team' or the 'enemy team'. Since the data comes from JSON, there are lots of lists and dicts are involved, and my issues probably stems from there, though I have not been able to find out where.
Here is the part that causes the issue:
first_game_test = game_list[0]
summ_team_ID = first_game_test["teamId"]
summoners_in_game = first_game_test["fellowPlayers"]
ally_team = []
enemy_team = []
for i in range(len(summoners_in_game)):
for name, value in summoners_in_game[i].iteritems():
if summoners_in_game[i]["teamId"] == summ_team_ID:
#if summoners_in_game[i] not in ally_team:
summoner_name = idtosummoner.idToSummonerName(summoners_in_game[i]['summonerId'])
summoner_champ = champion_id.champIdGet(summoners_in_game[i]['championId'])
ally_team.append({summoner_name: summoner_champ})
else:
#if summoners_in_game[i] not in enemy_team:
enemy_team.append(summoners_in_game[i])
The idtosummoner and champion_id modules have been checked multiple times; I'm quite certain that the issue does not stem from there.
As you can see, I used a simple duplicate check fix (commented out). It started to mess with further coding, however: the summoner_name, and summoner_champ variables cause an error at the 3th or 4th index (I haven't added the lines to else yet, since I want to fix the issue first).
The console output shows the following:
PS C:\Users\ptnfolder> python matchhistory.py
Nussen
Nussen
Nussen
kimbb
Traceback (most recent call last):
File "matchhistory.py", line 67, in <module>
matchHistory("thebirdistheword")
File "matchhistory.py", line 39, in matchHistory
print idtosummoner.idToSummonerName(summoners_in_game[i].get('summonerId'))
File "C:\Users\ptnfolder\idtosummoner.py", line 10, in idToSummonerName
champ_name_dict = json_data[str(summID)]
KeyError: '29716673'
The strange part is that the KeyError actually should resolve to 'kimbb' - since the for loop somehow triples every entry -; it works once, and then the program crashes.

You are looping over the keys and values of the dictionaries in a list:
for i in range(len(summoners_in_game)):
for name, value in summoners_in_game[i].iteritems():
so for each key-value pair, you execute your loop body. In your loop body, you test a specific key:
if summoners_in_game[i]["teamId"] == summ_team_ID:
so for each key in the dictionary, you test if the value for the 'teamId' key matches summ_team_ID.
This executes as many times as there are keys in the dictionary, but you only want to test one of the keys.
Just remove the loop over the key-value pairs:
for i in range(len(summoners_in_game)):
if summoners_in_game[i]["teamId"] == summ_team_ID:
summoner_name = idtosummoner.idToSummonerName(summoners_in_game[i]['summonerId'])
summoner_champ = champion_id.champIdGet(summoners_in_game[i]['championId'])
ally_team.append({summoner_name: summoner_champ})
else:
enemy_team.append(summoners_in_game[i])
Rather than use indices generated by range(), you could just loop over the list directly, and not have to keep indexing:
for team in summoners_in_game:
if team["teamId"] == summ_team_ID:
summoner_name = idtosummoner.idToSummonerName(team['summonerId'])
summoner_champ = champion_id.champIdGet(team['championId'])
ally_team.append({summoner_name: summoner_champ})
else:
enemy_team.append(team)

Related

Iterate over Python list with clear code - rewriting functions

I've followed a tutorial to write a Flask REST API and have a special request about a Python code.
The offered code is following:
# data list is where my objects are stored
def put_one(name):
list_by_id = [list for list in data_list if list['name'] == name]
list_by_id[0]['name'] = [new_name]
print({'list_by_id' : list_by_id[0]})
It works, which is nice, and even though I understand what line 2 is doing, I would like to rewrite it in a way that it's clear how the function iterates over the different lists. I already have an approach but it returns Key Error: 0
def put(name):
list_by_id = []
list = []
for list in data_list:
if(list['name'] == name):
list_by_id = list
list_by_id[0]['name'] = request.json['name']
return jsonify({'list_by_id' : list_by_id[0]})
My goal with this is also to be able to put other elements, that don't necessarily have the type 'name'. If I get to rewrite the function in an other way I'll be more likely to adapt it to my needs.
I've looked for tools to convert one way of coding into the other and answers in forums before coming here and couldn't find it.
It may not be beatiful code, but it gets the job done:
def put(value):
for i in range(len(data_list)):
key_list = list(data_list[i].keys())
if data_list[i][key_list[0]] == value:
print(f"old value: {key_list[0], data_list[i][key_list[0]]}")
data_list[i][key_list[0]] = request.json[test_key]
print(f"new value: {key_list[0], data_list[i][key_list[0]]}")
break
Now it doesn't matter what the key value is, with this iteration the method will only change the value when it finds in the data_list. Before the code breaked at every iteration cause the keys were different and they played a role.

Running a loop where each iteration returns a dict. How best to combine them into one?

I've found several SO posts on similar questions but I'm maybe overthinking my problem.
I'm running a loop. Each iteration returns a dict with the same keys and their own values. I'd like to combine them into a new master dict.
On each loop iteration I can save the results to a list
store_response = [] # will store the results of each iteration here
myloop:
code here...
store_response.append(iresponse.copy())
Or I can do:
store_response = {} # will store the results of each iteration here
myloop:
code here...
store_response[page_token] = iresponse # store this iteration and call it whatever string page_token currently is
So I can return either a list of dicts or dict of dicts.
My question is, how can I combine them into just one dict?
Tried several for loops but keep hitting errors e.g.:
for d in store_response:
for key, value in d.iteritems():
test[key].append(value)
Traceback (most recent call last):
File "<input>", line 2, in <module>
AttributeError: 'dict' object has no attribute 'iteritems'
Here is how the variable looks in PyCharms variables pane, currently in list form but I could make it a dict:
How can I take each dict within store response and create a single master dict?
You could try a pattern like:
from collections import defaultdict
store_response = defaultdict(list)
for _ in loop:
# Assuming the loop provides the key and value
store_response[key].append(value)
This will result in a dict with one key that collapses all values for that key as a list (in your use case since your dictionaries only have one key - this solution works for arbitrarily many keys like 'reports').
You are using Python 3, and in Python 3 iteritems has been removed use items instead.
for d in store_response:
for key, value in d.items():
test.setdefault(key, [])
test[key].append(value)

Python shelve having items that aren't listed

I've been saving a bunch of dictionaries to file using Python's shelve module (with Python 3.4 on OSX 10.9.5). Each key is a string of an int (e.g., "84554"), and each value is a dictionary of dictionaries of a few smallish strings.
No keys are used twice, and I know the total superset of all possible keys. I am adding these key-value pairs to the shelf via threads and which keys/values are added changes each time I run it (which is expected).
The problem I've been having is that the number of keys iterable/visible with shelve's shelf.keys() and the number of unique keys for which key in shelf.keys() are different.
Here's my code. I first initialize things and load ids, which is the list of all possible keys.
import shelve
from custom_code import *
MAIN_PATH = "/Users/myname/project_path/src/"
ids = list(set(load_list(MAIN_PATH + "id_list.pkl")))
c = c2 = 0
good_keys = []
bad_keys = []
I then open the shelf, counting all the number of keys that I iterate through with db.keys(), adding the "good" keys to a list.
db = shelve.open(MAIN_PATH + "first_3")
for k in db.keys():
c2+=1
good_keys+=[k]
Then, I check each possible key to see if it's in the shelf, checking to see if it exists in the shelf, and doing the same thing as above.
for j in set(ids):
if j in db.keys():
c+=1
bad_keys+=[j]
The two counters, c and c2, should be the same, but doing:
print("With `db.keys()`: {0}, with verifying from the list: {1}".format(c2, c))
yields:
With `db.keys()`: 628, with verifying from the list: 669
I then look at keys that were in bad_keys but not good_keys (i.e., collected from db.keys()) and pick an example.
odd_men_out = list( set(bad_keys).difference( set(good_keys) ) )
bad_key = odd_men_out[0]
print(bad_key) # '84554'
I then check the following:
print(bad_key in db.keys()) # True
print(bad_key in db) # True
print(db[bad_key]) # A dictionary of dictionaries that wraps ~12ish lines
print(bad_key in list(db.keys())) # False
Note that last check. Does anybody know what gives? I thought shelves was supposed to be easy, but it's been giving me complete hell.
Perhaps unrelatedly (but perhaps not), when I let an even greater number of entries accumulate in the shelf and try to do something like for k in db.keys() or list(db.keys()), I get the following error:
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/_collections_abc.py", line 482, in __iter__
yield from self._mapping
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/shelve.py", line 95, in __iter__
for k in self.dict.keys():
SystemError: Negative size passed to PyBytes_FromStringAndSize
But can still access the data by trying all possible keys. Evidently that's because I'm not using gdbm?
When I tried to save some numpy arrays with more than 1000 elements in my shelf it would only save some and completely skip others, without generating an error.
Apparently this is an issue when using Shelve in Mac OSX (here are some bug reports (https://bugs.python.org/issue33074 , https://bugs.python.org/issue30388).
The only easy solution I found was to use Pickle instead of Shelve.

Python: My directory is not giving individual value output

I have created a code that imports data via .xlrd in two directories in Python.
Code:
import xlrd
#category.clear()
#term.clear()
book = xlrd.open_workbook("C:\Users\Koen\Google Drive\etc...etc..")
sheet = book.sheet_by_index(0)
num_rows = sheet.nrows
for i in range(1,num_rows,1):
category = {i:( sheet.cell_value(i, 0))}
term = {i:( sheet.cell_value(i, 1))}
When I open one of the two directories (category or term), it will present me with a list of values.
print(category[i])
So far, so good.
However, when I try to open an individual value
print(category["2"])
, it will consistently give me an error>>
Traceback (most recent call last):
File "testfile", line 15, in <module>
print(category["2"])
KeyError: '2'
The key's are indeed numbered (as determined by i).
I've already tried to []{}""'', etc etc. Nothing works.
As I need those values later on in the code, I would like to know what the cause of the key-error is.
Thanks in advance for taking a look!
First off, you are reassigning category and term in every iteration of the for loop, this way the dictionary will always have one key at each iteration, finishing with the last index, so if our sheet have 100 lines, the dict will only have the key 99. To overcome this, you need to define the dictionary outside the loop and assign the keys inside the loop, like following:
category = {}
term = {}
for i in range(1, num_rows, 1):
category[i] = (sheet.cell_value(i, 0))
term[i] = (sheet.cell_value(i, 1))
And second, the way you are defining the keys using the for i in range(1, num_rows, 1):, they are integers, so you have to access the dictionary keys like so category[1]. To use string keys you need to cast them with category[str(i)] for example.
I hope have clarifying the problem.

Error while iterating list to put those into a key value pairs in python

I am new to python . I have a python list old_ecim_mims_list like below :-
['ReqSyncPort_v2_5_0', 'ECIM_SwM_v2_1_0_2_2', 'ECIM_SwM_v3_0_0_2_3', 'ResPowerDistribution_v1_0_0', 'ECIM_SwM_v4_2_0_3_2', 'ResPowerDistribution_v3_4_1', 'LratBb_v1_8025_0']
Now my requirement is here to iterate it and put it into a map like below key value pairs structure :-
ReqSyncPort=ReqSyncPort_v2_5_0
ECIM_SwM=ECIM_SwM_v2_1_0_2_2,ECIM_SwM_v3_0_0_2_3,ECIM_SwM_v4_2_0_3_2
ResPowerDistribution=ResPowerDistribution_v1_0_0,ResPowerDistribution_v3_4_1
LratBb=LratBb_v1_8025_0
I have done a sample program for this but I am getting error while executing :-
old_ecim_mims_map={} ;
for index , item in enumerate(old_ecim_mims_list) :
print(index , item ) ;
split_str=item.split("_v");
#print(split_str[0]);
if split_str[0] in old_ecim_mims_map :
new_prop_map[split_str[0]].append(item);
#old_ecim_mims_map.update({split_str[0]:item }) ;
else :
old_ecim_mims_map[split_str[0]]=item ;
Error :-
Traceback (most recent call last):
File "F:/DeltaProject/com/dash/abinash/DeltaOperation/Createdelta.py", line 50, in <module>
new_prop_map[split_str[0]].append(item);
AttributeError: 'str' object has no attribute 'append'
Suggest me where I am doing wrong .Searched lots of concepts , but those did not help me that much .Any help will be appreciated .
Your code fails because you add a string as value in dictionary (map), instead of enclosing it in [] to make an array in last line (old_ecim_mims_map[split_str[0]]=item). Next time you come across same key, you try to append to string, not to array.
What you have to do (and managed to do) is first check whether a certain key is already in a map. If it is, then you can append to the list old_ecim_mims_dict[key]. If there is no such key, a KeyError will be raised and then you have to create new list and put el inside it.
old_ecim_mims_list = ['ReqSyncPort_v2_5_0', 'ECIM_SwM_v2_1_0_2_2', 'ECIM_SwM_v3_0_0_2_3', 'ResPowerDistribution_v1_0_0', 'ECIM_SwM_v4_2_0_3_2', 'ResPowerDistribution_v3_4_1', 'LratBb_v1_8025_0']
old_ecim_mims_map = {}
for el in old_ecim_mims_list:
key, _ = el.split('_v')
try:
old_ecim_mims_map[key].append(el)
except KeyError:
old_ecim_mims_map[key] = [el]
This code is much cleaner. If you want to rewrite your code, just change last line to
old_ecim_mims_map[split_str[0]]=[item]
Edit: As suggested in the comments, although I do not prefer this, it can be done by checking whether key is in map:
old_ecim_mims_list = ['ReqSyncPort_v2_5_0', 'ECIM_SwM_v2_1_0_2_2', 'ECIM_SwM_v3_0_0_2_3', 'ResPowerDistribution_v1_0_0', 'ECIM_SwM_v4_2_0_3_2', 'ResPowerDistribution_v3_4_1', 'LratBb_v1_8025_0']
old_ecim_mims_map = {}
for el in old_ecim_mims_list:
key, _ = el.split('_v')
if key in old_ecim_mims_map: # The same as if key in old_ecim_mims_map.keys()
old_ecim_mims_map[key].append(el)
else:
old_ecim_mims_map[key] = [el]

Categories

Resources