Develop Reference

for loop through a dictionary - python3

I am just trying to make sense of the piece of code below:
names = [{'name': alice}, {'name':bob}, {'name': david}]
with_comma= ', '.join(name['name'] for name in names[:-1])
What is name['name'] there? How does it extract names from the corresponding dictionaries?
Additionally, what I learned from textbooks is we are supposed to write what will happen after every iteration of for loop "AFTER" the for name in names[:-1], and where is it?
I hope I have written clearly. Thanks for the help.

You need to break down the code in smaller parts and understand the smaller parts to understand the whole.
names is a list containing three dictionaries, each of the dictionaries only contains a single key 'name' with a value. Those values are apparently defined in some other part of the code that we're not given, as these values appear to have been assigned to the variables alice, bob and david.
names[:-1] means 'all the elements of names, except the last', so it's effectively [{'name': alice}, {'name': bob}].
name for name in names[:-1] is a generator, which yields the elements of names[:-1] one at a time. So, it yields {'name': alice} first and then {'name': bob}, and then it completes since there is nothing more to yield.
name['name'] tries to index the variable name with the key 'name'. When name has the value {'name': alice}, name['name'] is whatever the value of alice was when the dictionary was defined.
The .join() method of a string takes an iterable as an argument and puts every element from the iterable together in a new string, with a copy of the original string stuck inbetween each element.
So, this piece of code:
alice = 'Alice'
bob = 'Bob'
david = 'Whatever'
names = [{'name': alice}, {'name': bob}, {'name': david}]
with_comma= ', '.join(name['name'] for name in names[:-1])
Results in with_comma having the value 'Alice, Bob'.
If this is code from a Python training though, I would recommend taking another class or course, because it's not doing a very good job of teaching you Python, throwing in complications like these without explaining the basics.

Trying to Create a List of Dictionaries From Web-Scraped Content

I'm new to Python but have worked in R for a while but am mostly used to working with data frames. I web-scraped the following for which the output is four lists included below: but with values taken out to create a minimal reproducible example.
I am trying to put these into a list of dictionaries such that the data is set up in the following way.
Each dictionary, rankings[i], should have these keys:
rankings[i]['name']: The name of the restaurant, a string.
rankings[i]['stars']: The star rating, as a string, e.g., '4.5', '4.0'
rankings[i]['numrevs']: The number of reviews, as an integer.
rankings[i]['price']: The price range, as dollar signs, e.g., '$', '$$', '$$$', or '$$$$'.
I get so confused by dictionaries within lists and sequences of sequences in general, so if you have any great resources, please link them here! I've been reading through Think Python.
This is what I have, but it ends up returning one dictionary with lists of values, which is not what I need.
Thanks much!
def Convert(tup, di):
for w, x,y,z in tup:
di.setdefault(w, []).extend((w,x,y,z))
return di
yelp = list(zip(name, stars, numrevs, price))
dictionary = {}
output = Convert(yelp, dictionary)
#data
name = ['Mary Mac’s Tea Room', 'Busy Bee Cafe', 'Richards’ Southern Fried']
stars = ['4.0 ', '4.0 ', '4.0']
numrevs = ['80', '49', '549']
price = ['$$', '$$', '$$']
Update:
This will give me a dictionary of a single restaurant:
def Convert(tup, di):
dictionary = {}
for name, stars, numrevs, price in tup:
yelp = list(zip(name, stars, numrevs, price))
name, stars, numrevs, price = tup[0]
entry = {"name" : name,
"stars" : stars,
"numrevs": numrevs,
"print" : price}
return entry
output = Convert(yelp, dictionary)
output
This is my attempt to iterate over the all restaurants and add them to a list. It looks like I am only getting the final restaurant and everything else is being written over. Perhaps I need to do something like
def Convert(tup, di):
dictionary = {}
for name, stars, numrevs, price in tup:
yelp = list(zip(name, stars, numrevs, price))
for i in name, stars, numrevs, price:
l = []
entry = {"name" : name,
"stars" : stars,
"numrevs": numrevs,
"price" : price}
l.append(entry)
return l
output = Convert(yelp, dictionary)
output

There is nothing in your function that attempts to build a list of dicts in the format you want. You make each value with the extend method. This adds to a list; it does not contrsuct a dict.
Start from the inside: build a single dict in the format you want, from only the first set of elements. Also, use meaningful variable names -- do you expect everyone else to learn your one-letter mappings, so you can save a few seconds of typing?
name, stars, numrevs, price = tup[0]
entry = {"name" : name,
"stars" : stars,
"numrevs": numrevs,
"print" : price}
That is how you make a single dict for your list of dicts.
That should get you started. Now you need to (1) learn to add entry to a list; (2) iterate through all of the quadruples in tup (another meaningless name to replace) so that you end up adding each entry to the final list, which you return.
Update for second question:
It keeps overwriting because you explicitly tell it to do so. Every time through the loop, you reset l to an empty list. Stop that! There are plenty of places that show you how to accumulate iteration results in a list. Start by lifting the initialization to before the loop.
l = []
for ...

Creating dictionary from list of lists

I am working on an online course exercise (practice problem before the final test).
The test involves working with a big csv file (not downloadable) and answering questions about the dataset. You're expected to write code to get the answers.
The data set is a list of all documented baby names each year, along with
#how often each name was used for boys and for girls.
A sample list of the first 10 lines is also given:
Isabella,42567,Girl
Sophia,42261,Girl
Jacob,42164,Boy
and so on.
Questions you're asked include things like 'how many names in the data set', 'how many boys' names beginning with z' etc.
I can get all the data into a list of lists:
[['Isabella', '42567', 'Girl'], ['Sophia', '42261', 'Girl'], ['Jacob', '42164', 'Boy']]
My plan was to convert into a dictionary, as that would probably be easier for answering some of the other questions. The list of lists is saved to the variable 'data':
names = {}
for d in data:
names[d[0]] = d[1:]
print(names)
{'Isabella': ['42567', 'Girl'], 'Sophia': ['42261', 'Girl'], 'Jacob': ['42164', 'Boy']}
Works perfectly.
Here's where it gets weird. If instead of opening the sample file with 10 lines, I open the real csv file, with around 16,000 lines. everything works perfectly right up to the very last bit.
I get the complete list of lists, but when I go to create the dictionary, it breaks - here I'm just showing the first three items, but the full 16000 lines are all wrong in a similar way):
names = {}
for d in data:
names[d[0]] = d[1:]
print(names)
{'Isabella': ['56', 'Boy'], 'Sophia': ['48', 'Boy'], 'Jacob': ['49', 'Girl']
I know the data is there and correct, since I can read it directly:
for d in data:
print(d[0], d[1], d[2])
Isabella 42567 Girl
Sophia 42261 Girl
Jacob 42164 Boy
Why would this dictionary work fine with the cvs file with 10 lines, but completely break with the full file? I can't find any

Follow the comments to create two dicts, or a single dictionary with tuple keys. Using tuples as keys is fine if you keep your variables inside python, but you might get into trouble when exporting to json for example.
Try a dictionary comprehension with list unpacking
names = {(name, sex): freq for name, freq, sex in data}
Or a for loop as you started
names = dict()
for name, freq, sex in data:
names[(name, freq)] = freq

I'd go with something like
results = {}
for d in data:
name, amount, gender = d.split(',')
results[name] = data.get(name, {})
results[name].update({ gender: amount })
this way you'll get results in smth like
{
'Isabella': {'Girl': '42567', 'Boy': '67'},
'Sophia': {'Girl': '42261'},
'Jacob': {'Boy': '42164'}
}
However duplicated values will override previous, so you need to take that into account if there are some and it also assumes that the whole file matches format you've provided

Iterating through a list of strings and appending a json returned element for each term in the list

Warning this is very very complex. (I'm an idiot and should probably take a break), Its a confusing title but I'll do my best to explain what I'm running into,
I have this code:
test_terms = ['spartan', 'forerunner', 'didact']
test_list = []
for term in test_terms:
search_term = term
search_service = PyMsCognitiveWebSearch('API_Key', search_term,
custom_params={'title': 'name', 'domain': 'url', 'mkt': 'en-usa', 'description': 'url'})
result = search_service.search(limit=3, format='json')
test_list.append(result[term].json)
print(test_list)
This works when its outside of the loop and I run simply this:
search_term = 'spartan'
search_service = PyMsCognitiveWebSearch('API_Key', search_term,
custom_params={'title': 'name', 'domain': 'url', 'mkt': 'en-usa',
'description': 'url'})
result = search_service.search(limit=3, format='json')
test_list.append(result[0].json)
print(test_list)
which gives me:
[{'id': 'https://api.cognitive.microsoft.com/api/v7/#WebPages.0', 'name': 'Obstacle Course Races | Spartan Race', 'url': 'https://www.spartan.com/', 'isFamilyFriendly': True, 'displayUrl': 'https://www.spartan.com', 'snippet': 'Spartan Race is the global leader in obstacle course races, with the right challenge for anyone - from first-time racers to elite athletes.', 'deepLinks': ... etc, for three more results}]
My first instinct wast to do simply for term in range(len(test_terms)):, however this returned a result but instead of the items in the list, I got information on the number 1 through 3.
Next I tried for item, term enumerate(test_terms):
but I got this error:
TypeError: list indices must be integers or slices, not str
Which makes sense, however I'm not sure exactly how to proceed given that the index is a "string" and I've tried enumerating it in two different ways, removing result[term].json and changing it to result.json throws an object error too.
Any help would be great! Also the package specs if it helps.
UPDATE:
result is a web object that is essentially this (via the documentation):
packaged_results = [WebResult(single_result_json) for single_result_json in json_results.get("webPages", {}).get("value", [])]
Link the source code
But its a giant nested list, I wrote it out on a white board, it returns 3 lists consisting of 9 elements, of mixed types, one list and one dictionary. Literally my nightmare.

The way you're using 'term' in your for loop makes 'term' a string (one of ['spartan', 'forerunner', 'didact']). You're looking for an integer to use in "result[term]", so, stick with enumerate, but remember using enumerate the first part of the tuple is the "enumeration" and the second part is the item.
for i, term in enumerate(test_terms):
...
test_list.append(result[i].json)

How do I replace lines in a file using data contained elsewhere in the same file?

Let's say I have a file called 'Food' listing the names of some food, and their prices. Some of these items are raw ingredients, and others are made from different amounts of these- for example i might manually list the price of eggs as 1 and find that the omelette has a default price of 10, but then find that an omelette will only need 5 eggs, so i would need the program to read the price of eggs, find the line containing the omelette, and replace it with "omelette: " + str(5*eggs). I may also need to add extra ingredients/ items of food e.g. a pile of omelettes which is made from 5 omelettes. the basic goal would be to make it possible to just edit the value of eggs, and the value of omelette and pileofomelettes to update. I've started the code simply by creating a list of the lines contained within the file.
with open("Food.txt") as g:
foodlist=g.readlines()
The file 'Food.txt' would be in the following format:
eggs: 5
omelette: 20
pileofomelettes: 120
etc...
and after the code runs it should look like
eggs: 5
omelette: 25
pileofomelettes: 125
I would code the relations manually since they would be so unlikely to ever change (and even if they did it would be fairly easy for me to go in and change the coefficients)
and would be read by python in its list format as something like
'['egg 2\n', 'flour 1\n', 'butter 1\n', 'sugar 3\n', 'almond 5\n', 'cherry 8\n']'
I have searched for search/replace algorithms that can search for a specific phrase and replace it with another specific phrase, but i don't know how i'd apply it if the line was subject to change (the user could change the raw ingredient values if he wanted to update all of the values related to it). One solution i can think of involves converting them into a dictionary format, with them all listed as a string-integer value pair, so that i could just replace the integer part of the pair based on the integer values stored within other string-integer pairs, but, being inexperienced, I don't know how i'd convert the list (or the raw file itself, even better) into a dictionary.
Any advice on how to carry out steps of this program would be greatly appreciated :)
EDIT- in the actual application of the program, it doesn't matter what order the items are listed in in the final file, so if i listed out all the raw ingredients in 1 place and all of the composite items in another (With a large space in between them if more raw items need to be added) then i could just re-write the entire second half of the file in an arbitrary order with no problem- so long as the line position of the raw ingredients remains the same.

Okay, I would suggest make a relations text file which you can parse in case you think the relations can later change, or just so that your code is easier to read and mutable. This can be then parsed to find the required relations between raw ingredients and complexes. Let it be "relations.txt" , and of the type:
omelette: 5 x eggs + 1 x onions
pileofomelettes: 6 x omelette
Here, you can put arbitrary number of ingredients of the type:
complex: number1 x ingredient1 + number2 x ingredient2 + ...
and so on.
And your food.txt contains prices of all ingredients and complexes:
eggs: 2
onions: 1
omelette: 11.0
pileofomelettes: 60
Now we can see that the value for pileofomlettes is intentionally not mapped here correctly. So, we will run the code below, and also you can change numbers and see the results.
#!/usr/bin/python
''' This program takes in a relations file and a food text files as inputs
and can be used to update the food text file based on changes in either of these'''
relations = {}
foodDict = {}
# Mapping ingredients to each other in the relations dictionary
with open("relations.txt") as r:
relationlist=r.read().splitlines()
for relation in relationlist:
item, equatedTo = relation.split(': ')
ingredientsAndCoefficients = equatedTo.split(' + ')
listIngredients = []
for ingredient in ingredientsAndCoefficients:
coefficient, item2 = ingredient.split(' x ')
# A list of sets of amount and type of ingredient
listIngredients.append((float(coefficient),item2))
relations.update({item:listIngredients})
# Creating a food dictionary with values from food.txt and mapping to the relations dictionary
with open("food.txt") as g:
foodlist=g.read().splitlines()
for item in foodlist:
food,value = item.split(': ')
foodDict.update({food:value})
for food in relations.keys():
# (Raw) Ingredients with no left hand side value in relations.txt will not change here.
value = 0.
for item2 in range(len(relations[food])):
# Calculating new value for complex here.
value += relations[food][item2][0]* float(foodDict[relations[food][item2][1]])
foodDict.update({food: value })
# Altering the food.txt with the new dictionary values
with open("food.txt",'w') as g:
for key in sorted(foodDict.keys()):
g.write(key + ': ' + str (foodDict[key])+ '\n')
print key + ': ' + str(foodDict[key])
And it comes out be:
eggs: 2
onions: 1
omelette: 11.0
pileofomelettes: 66.0
You can change the price of eggs to 5 in the food.txt file, and
eggs: 5
onions: 1
omelette: 26.0
pileofomelettes: 156.0

How does your program know the components of each item? I suggest that you keep two files: one with the cost of atomic items (eggs) and another with recipes (omelette <= 5 eggs).
Read both files. Store the atomic costs, remembering how many of these items you have, atomic_count. Extend this table from the recipes file, one line at a time. If the recipe you're reading consists entirely of items with known costs, then compute the cost and add that item to the "known" list. Otherwise, append the recipe to a "later" list and continue.
When you reach the end of both input files, you will have a list of known costs, and a few other recipes that depended on items farther down the recipe file. Now cycle through this "unknown" list until (a) it's empty; (b) you don't have anything with all the ingredients known. If case (b), you have something wrong with your input: either an ingredient with no definition, or a circular dependency. Print the remaining recipes list and debug your input files.
In case (a), you are now ready to print your Food.txt list. Go through your "known" list and write out one item or recipe at a time. When you get to item [atomic_count], write out a second file, a new recipe list. This is your old recipe list, but in a useful top-down order. In the future, you won't have any "unknown" recipes after the first pass.
For future changes ... don't bother. You have only 173 items, and the list sounds unlikely to grow past 500. When you change or add an item, just hand-edit the file and rerun the program. That will be faster than the string-replacement algorithm you're trying to write.
In summary, I suggest that you do just the initial computation problem, which is quite a bit simpler than adding the string update. Don't do incremental updates; redo the whole list from scratch. For such a small list, the computer will do this faster than you can write and debug the extra coding.

I'm still not really sure what you are asking but this is what I came up with...
from collections import OrderedDict
food_map = {'omelette': {'eggs': 5, 'price': None}, 'pileofomelettes': {'eggs': 25, 'price': None}, 'eggs': {'price': 5}}
with open('food.txt', 'rb') as f:
data = f.read().splitlines()
data = OrderedDict([(x[0], int(x[1])) for x in [x.split(': ') for x in data]])
for key, val in data.items():
if key == 'eggs':
continue
food_rel = food_map.get(key, {})
val = food_rel.get('eggs', 1) * food_map.get('eggs', {}).get('price', 1)
data[key] = val
with open('out.txt', 'wb') as f:
data = '\n'.join(['{0}: {1}'.format(key, val) for key, val in data.items()])
f.write(data)