I have a statistics file like this:
dict-count.txt
apple 15
orange 12
mango 10
apple 1
banana 14
mango 4
I need to count the number of each element and create a dictionary like this: {'orange': 12, 'mango': 14, 'apple': 16, 'banana': 14}. I do the following to achieve this:
from __future__ import with_statement
with open('dict-count.txt') as f:
lines = f.readlines()
output = {}
for line in lines:
key, val = line.split('\t')
output[key] = output.get(key, 0) + int(val)
print output
I am particularly concerned about this part:
key, val = line.split('\t')
output[key] = output.get(key, 0) + int(val)
Is there a better way to do this? Or this is the only way?
Thanks.
For a small file, you can use .readlines(), but that will slurp the entire contents of the file into memory in one go. You can write this using the file object f as an iterator; when you iterate it, you get one line of input at a time.
So, the easiest way to write this is to use a defaultdict as #Amber already showed, but my version doesn't build a list of input lines; it just builds the dictionary as it goes.
I used terse variable names, like d for the dict instead of output.
from __future__ import with_statement
from collections import defaultdict
from operator import itemgetter
d = defaultdict(int)
with open('dict-count.txt') as f:
for line in f:
k, v = line.split()
d[k] += int(v)
lst = d.items()
# sort twice: once for alphabetical order, then for frequency (descending).
# Because the Python sort is "stable", we will end up with descending
# frequency, but alphabetical order for any frequency values that are equal.
lst.sort(key=itemgetter(0))
lst.sort(key=itemgetter(1), reverse=True)
for key, value in lst:
print("%10s| %d" % (key, value))
Use a defaultdict:
from __future__ import with_statement
from collections import defaultdict
output = defaultdict(int)
with open('dict-count.txt') as f:
for line in f:
key, val = line.split('\t')
output[key] += int(val)
print output
Related
For some reason my code refuses to convert to uppercase and I cant figure out why. Im trying to then write the dictionary to a file with the uppercase dictionary values being inputted into a sort of template file.
#!/usr/bin/env python3
import fileinput
from collections import Counter
#take every word from a file and put into dictionary
newDict = {}
dict2 = {}
with open('words.txt', 'r') as f:
for line in f:
k,v = line.strip().split(' ')
newDict[k.strip()] = v.strip()
print(newDict)
choice = input('Enter 1 for all uppercase keys or 2 for all lowercase, 3 for capitalized case or 0 for unchanged \n')
print("Your choice was " + choice)
if choice == 1:
for k,v in newDict.items():
newDict.update({k.upper(): v.upper()})
if choice == 2:
for k,v in newDict.items():
dict2.update({k.lower(): v})
#find keys and replace with word
print(newDict)
with open("tester.txt", "rt") as fin:
with open("outwords.txt", "wt") as fout:
for line in fin:
fout.write(line.replace('{PETNAME}', str(newDict['PETNAME:'])))
fout.write(line.replace('{ACTIVITY}', str(newDict['ACTIVITY:'])))
myfile = open("outwords.txt")
txt = myfile.read()
print(txt)
myfile.close()
In python 3 you cannot do that:
for k,v in newDict.items():
newDict.update({k.upper(): v.upper()})
because it changes the dictionary while iterating over it and python doesn't allow that (It doesn't happen with python 2 because items() used to return a copy of the elements as a list). Besides, even if it worked, it would keep the old keys (also: it's very slow to create a dictionary at each iteration...)
Instead, rebuild your dict in a dict comprehension:
newDict = {k.upper():v.upper() for k,v in newDict.items()}
You should not change dictionary items as you iterate over them. The docs state:
Iterating views while adding or deleting entries in the dictionary may
raise a RuntimeError or fail to iterate over all entries.
One way to update your dictionary as required is to pop values and reassign in a for loop. For example:
d = {'abc': 'xyz', 'def': 'uvw', 'ghi': 'rst'}
for k, v in d.items():
d[k.upper()] = d.pop(k).upper()
print(d)
{'ABC': 'XYZ', 'DEF': 'UVW', 'GHI': 'RST'}
An alternative is a dictionary comprehension, as shown by #Jean-FrançoisFabre.
I have a text file and its content is something like this:
A:3
B:5
C:7
A:8
C:6
I need to print:
A numbers: 3, 8
B numbers: 5
C numbers: 7, 6
I'm a beginner so if you could give some help I would appreciate it. I have made a dictionary but that's pretty much all I know.
You could use an approach that keeps the values in a dictionary:
d = {} # create an empty dictionary
for line in open(filename): # opens the file
k, v = line.split(':') # unpack each line in the char before : and after
if k in d: # add the values to the dictionary
d[k].append(v)
else:
d[k] = [v]
This gives you a dictionary containing your file in a format that you can utilize to get the desired output:
for key, values in sorted(d.items()):
print(key, 'numbers:' ', '.join(values))
The sorted is required because dictionaries are unordered.
Note that using collections.defaultdict instead of a normal dict could simplify the approach somewhat. The:
d = {}
...
if k in d: # add the values to the dictionary
d[k].append(v)
else:
d[k] = [v]
could then be replaced by:
from collections import defaultdict
d = defaultdict(list)
...
d[k].append(v)
Short version (Which should sort in alphabetic order)
d = {}
lines = [line.rstrip('\n') for line in open('filename.txt')]
[d.setdefault(line[0], []).append(line[2]) for line in lines]
[print(key, 'numbers:', ', '.join(values)) for key,values in sorted(d.items())]
Or if you want to maintain the order as they appear in file (file order)
from collections import OrderedDict
d = OrderedDict() # Empty dict
lines = [line.rstrip('\n') for line in open('filename.txt')] # Get the lines
[d.setdefault(line[0], []).append(line[2]) for line in lines] # Add lines to dictionary
[print(key, 'numbers:', ', '.join(values)) for key,values in d.items()] # Print lines
Tested with Python 3.5.
You can treat your file as csv (comma separated value) so you can use the csv module to parse the file in one line. Then use defaultdict with input in the costructor the class list to say that to create it when the key not exists. Then use OrderedDict class because standard dictionary don't keeps the order of your keys.
import csv
from collection import defaultdict, OrderedDict
values = list(csv.reader(open('your_file_name'), delimiter=":")) #[['A', '3'], ['B', '5'], ['C', '7'], ['A', '8'], ['C', '6']]
dct_values = defaultdict(list)
for k, v in values:
dct_values[k].append(v)
dct_values = OrderedDict(sorted(dct_values.items()))
Then you can simply print iterating the dictionary.
A very easy way to group by key is by external library, if you are interested try PyFunctional
I am trying to come up with a neat way of doing this in python.
I have a list of pairs of alphabets and numbers that look like this :
[(a,1),(a,2),(a,3),(b,10),(b,100),(c,99),(d,-1),(d,-2)]
What I want to do is to create a new list for each alphabet and append all the numerical values to it.
So, output should look like:
alist = [1,2,3]
blist = [10,100]
clist = [99]
dlist = [-1,-2]
Is there a neat way of doing this in Python?
from collections import defaultdict
data = [('a',1),('a',2),('a',3),('b',10),('b',100),('c',99),('d',-1),('d',-2)]
if __name__ == '__main__':
result = defaultdict(list)
for alphabet, number in data:
result[alphabet].append(number)
or without collections module:
if __name__ == '__main__':
result = {}
for alphabet, number in data:
if alphabet not in result:
result[alphabet] = [number, ]
continue
result[alphabet].append(number)
But i think, that first solution more effective and clear.
If you want to avoid using a defaultdict but are comfortable using itertools, you can do it with a one-liner
from itertools import groupby
data = [('a',1),('a',2),('a',3),('b',10),('b',100),('c',99),('d',-1),('d',-2)]
grouped = dict((key, list(pair[1] for pair in values)) for (key, values) in groupby(data, lambda pair: pair[0]))
# gives {'b': [10, 100], 'a': [1, 2, 3], 'c': [99], 'd': [-1, -2]}
After seeing the responses in the thread and reading the implementation of defaultdict, I implemented my own version of it since I didn't want to use the collections library.
mydict = {}
for alphabet, value in data:
try:
mydict[alphabet].append(value)
except KeyError:
mydict[alphabet] = []
mydict[alphabet].append(value)
You can use defaultdict from the collections module for this:
from collections import defaultdict
l = [('a',1),('a',2),('a',3),('b',10),('b',100),('c',99),('d',-1),('d',-2)]
d = defaultdict(list)
for k,v in l:
d[k].append(v)
for k,v in d.items():
exec(k + "list=" + str(v))
I have a files scores that looks like:
z:100
a:50
c:75
a:-20
c:45
a:10
c:20
z:10
z:-50
I want to return a dictionary with the key being the letter and the value being he sum of the value that letter has in the file. I am unsure how to keep a sum going while using this method.
I have
a = {key: value for (key,value) in [line.strip().split(":") for line in open("scores.txt","r")]}
print a
I'd advise against cramming everything into one line here. You can do something like this (using collections.defaultdict):
from collections import defaultdict
counts = defaultdict(int)
with open('scores.txt', 'r') as f:
for line in f:
name, score = line.strip().split(':')
d[name] += int(score)
Are you sure you want a "one-line" dictionary comprehension?
from itertools import imap, groupby
from operator import methodcaller, itemgetter
a = { k : sum(int(v[1]) for v in v_itr)
for k, v_itr in groupby(sorted(imap(methodcaller("split", ":"),
imap(methodcaller("rstrip"),
open("scores.txt", "r"))),
key=itemgetter(0)), itemgetter(0)) }
(This is not a recommendation to actually use this code; use the loop in arshajii's answer.)
A slightly easier-to-read version that uses a generator expression instead of nested calls to imap.
a = { k : sum(int(v[1]) for v in v_itr)
for k, v_itr in groupby(sorted(line.rstrip().split(":")
for line in open("scores.txt", "r"),
key=itemgetter(0)), itemgetter(0)) }
I have option file in this format:
key value\t\n
N:B:. Some values show tab after it.
I use Code like :
src = open("conf.cfg").readlines()
item = item.split(" ")[0:2]
key = item[0]
value = item[1]
dict_[key] = value
Can I use generator expression to get the same result ??
You could use a dictionary comprehension, for example:
with open("conf.cfg") as f:
dict_ = {key: value
for key, value in (line.strip().split(" ")[:2]
for line in f)}
Maybe this way:
with open('config.txt') as f:
dict_ = {k: v for k, v in (line.split() for line in f.readlines())}
Yes, you can use a generator expression, it would look like this:
with open("conf.cfg") as f:
dict_ = dict(line.split() for line in f)
I am assuming you don't have spaces in the keys or values. If you have spaces in them (and "they are quoted" strings) then it will be easier for you to read your file with the csv module.
Using generator function:
def myGenerator():
with open("conf.cfg") as f:
for line in f.readlines():
yield(line.split())
result = {k:v for k,v in myGenerator()}
print result