python 2.7:iterate dictionary and map values to a file - python
I have a list of dictionaries which I build from .xml file:
list_1=[{'lat': '00.6849879', 'phone': '+3002201600', 'amenity': 'restaurant', 'lon': '00.2855850', 'name': 'Telegraf'},{'lat': '00.6850230', 'addr:housenumber': '6', 'lon': '00.2844493', 'addr:city': 'XXX', 'addr:street': 'YYY.'},{'lat': '00.6860304', 'crossing': 'traffic_signals', 'lon': '00.2861978', 'highway': 'crossing'}]
My aim is to build a text file with values (not keys) in such order:
lat,lon,'addr:street','addr:housenumber','addr:city','amenity','crossing' etc...
00.6849879,00.2855850, , , ,restaurant, ,'\n'00.6850230,00.2844493,YYY,6,XXX, , ,'\n'00.6860304,00.2861978, , , , ,traffic_signals,'\n'
if value not exists there should be empty space.
I tried to loop with for loop:
for i in list_1:
line= i['lat'],i['lon']
print line
Problem occurs if I add value which does not exist in some cases:
for i in list_1:
line= i['lat'],i['lon'],i['phone']
print line
Also tried to loop and use map() function, but results seems not correct:
for i in list_1:
line=map(lambda x1,x2:x1+','+x2+'\n',i['lat'],i['lon'])
print line
Also tried:
for i in list_1:
for k,v in i.items():
if k=='addr:housenumber':
print v
This time I think there might be too many if/else conditions to write.
Seems like solutions is somewhere close. But can't figure out the solution and its optimal way.
I would look to use the csv module, in particular DictWriter. The fieldnames dictate the order in which the dictionary information is written out. Actually writing the header is optional:
import csv
fields = ['lat','lon','addr:street','addr:housenumber','addr:city','amenity','crossing',...]
with open('<file>', 'w') as f:
writer = csv.DictWriter(f, fields)
#writer.writeheader() # If you want a header
writer.writerows(list_1)
If you really didn't want to use csv module then you can simple iterate over the list of the fields you want in the order you want them:
fields = ['lat','lon','addr:street','addr:housenumber','addr:city','amenity','crossing',...]
for row in line_1:
print(','.join(row.get(field, '') for field in fields))
If you can't or don't want to use csv you can do something like
order = ['lat','lon','addr:street','addr:housenumber',
'addr:city','amenity','crossing']
for entry in list_1:
f.write(", ".join([entry.get(x, "") for x in order]) + "\n")
This will create a list with the values from the entry map in the order present in the order list, and default to "" if the value is not present in the map.
If your output is a csv file, I strongly recommend using the csv module because it will also escape values correctly and other csv file specific things that we don't think about right now.
Thanks guys
I found the solution. Maybe it is not so elegant but it works.
I made a list of node keys look for them in another list and get values.
key_list=['lat','lon','addr:street','addr:housenumber','amenity','source','name','operator']
list=[{'lat': '00.6849879', 'phone': '+3002201600', 'amenity': 'restaurant', 'lon': '00.2855850', 'name': 'Telegraf'},{'lat': '00.6850230', 'addr:housenumber': '6', 'lon': '00.2844493', 'addr:city': 'XXX', 'addr:street': 'YYY.'},{'lat': '00.6860304', 'crossing': 'traffic_signals', 'lon': '00.2861978', 'highway': 'crossing'}]
Solution:
final_list=[]
for i in list:
line=str()
for ii in key_list:
if ii in i:
x=ii
line=line+str(i[x])+','
else:
line=line+' '+','
final_list.append(line)
Related
Python: How to convert CSV file to lists of dictionaries without importing reader or external libraries
I need to convert a CSV file into a list of dictionaries without importing CSV or other external libraries for a project I am doing for class. Attempt I am able to get the keys using header line but when I try to extract the values it goes row by row instead of column by column and starts in the wrong place. However when I append it to the list it goes back to starting at the right place. However I am unsure of how to connect the keys to the correct column in the list. CSV file This is the CSV file I am using, I am only using the descriptions portion up to the first comma. I tried using a for 6 loop in order to cycle through each key but it seems to go row by row and I don't know how to change it. If anybody could steer me in the right direction it would be very appreciated. CSV sample - sample is not saving correctly but it has the three headers on top and then the three matching information below and so on. (Code,Name,State)\n (ACAD,Acadia National Park,ME)\n (ARCH,Arches National Park,UT)\n (BADL, Badlands National Park,SD)\n
read your question. I am posting code from what I understood from your question. You should learn to post the code in question. It is a mandatory skill. Always open a file using the "with" block. I made a demo CSV file with two rows of records. The following code fetched all the rows as a list of dictionaries.
def readParksFile(fileName="national_parks.csv"):
with open(fileName) as infile:
column_names = infile.readline()
keys = column_names.split(",")
number_of_columns = len(keys)
list_of_dictionaries = []
data = infile.readlines()
list_of_rows = []
for row in data:
list_of_rows.append(row.split(","))
infile.close()
for item in list_of_rows:
row_as_a_dictionary = {}
for i in range(number_of_columns):
row_as_a_dictionary[keys[i]] = item[i]
list_of_dictionaries.append(row_as_a_dictionary)
for i in range(len(list_of_dictionaries)):
print(list_of_dictionaries[i])
Output:
{'Code': 'cell1', 'Name': 'cell2', 'State': 'cell3', 'Acres': 'cell4', 'Latitude': 'cell5', 'Longitude': 'cell6', 'Date': 'cell7', 'Description\n': 'cell8\n'}
{'Code': 'cell11', 'Name': 'cell12', 'State': 'cell13', 'Acres': 'cell14', 'Latitude': 'cell15', 'Longitude': 'cell16', 'Date': 'cell17', 'Description\n': 'cell18'}
I would create a class with a constructor that has the keys from the first row of the CSV as properties. Then create an empty list to store your dictionaries. Then open the file (that is a built-in library so I assume you can use it) and read it line by line. Store the line as a string and use the split method with a comma as the delimiter and store that list in a variable. Call the constructor of your class for each line to construct your dictionary using the indexes of the list from the split method. Before reading the next line, append the dictionary to your list. This is probably not the easiest way to do it but it doesn't use any external libraries (although as others have mentioned, there is a built-in CSV module).
Code:
#Class with constructor
class Park:
def __init__(self, code, name, state):
self.code = code
self.name = name
self.state = state
#Empty array for storing the dictionaries
parks = []
#Open file
parks_csv = open("parks.csv")
#Skip first line
lines = parks_csv.readlines()[1:]
#Read the rest of the lines
for line in lines:
parkProperties = line.split(",")
newPark = Park(parkProperties[0], parkProperties[1], parkProperties[2])
parks.append(newPark)
#Print park dictionaries
#It would be easier to parse this using the JSON library
#But since you said you can't use any libraries
for park in parks:
print(f'{{code: {park.code}, name: {park.name}, state: {park.state}}}')
#Don't forget to close the file
parks_csv.close()
Output:
{code: ACAD, name: Acadia National Park, state: ME}
{code: ARCH, name: Arches National Park, state: UT}
{code: BADL, name: Badlands National Park, state: SD}
How to create/read nested dictionary from file?
Here is the text file1 content
name = test1,description=None,releaseDate="2020-02-27"
name = test2,description=None,releaseDate="2020-02-28"
name = test3,description=None,releaseDate="2020-02-29"
I want a nested dictionary like this. How to create this?
{ 'test1': {'description':'None','releaseDate':'2020-02-27'},
'test2': {'description':'None','releaseDate':'2020-02-28'},
'test3': {'description':'None','releaseDate':'2020-02-29'}}
After this I want to append these values in the following line of code through "for" loop for a list of projects.
Example: For a project="IJTP2" want to go through each name in the dictionary like below
project.create(name="test1", project="IJTP2", description=None, releaseDate="2020-02-27")
project.create(name="test2", project="IJTP2", description=None, releaseDate="2020-02-28")
project.create(name="test3", project="IJTP2", description=None, releaseDate="2020-02-29")
Now to the next project:
List of projects is stored in another file as below
IJTP1
IJTP2
IJTP3
IJTP4
I just started working on Python and have never worked on the nested dictionaries.
I assume that:
each file line has comma-separated columns
each column has only one = and key on its left, value on its right
only first column is special(name)
Of course, as #Alex Hall mentioned, I recommend JSON or CSV, too.
Anyway, I wrote code for your case.
d = {}
with open('test-200229.txt') as f:
for line in f:
(_, name), *rest = (
tuple(value.strip() for value in column.split('='))
for column in line.split(',')
)
d[name] = dict(rest)
print(d)
output:
{'test1': {'description': 'None', 'releaseDate': '"2020-02-27"'}, 'test2': {'description': 'None', 'releaseDate': '"2020-02-28"'}, 'test3': {'description': 'None', 'releaseDate': '"2020-02-29"'}}
How to extract from dictionaries to only print certain variables python
I had a tsv file like such
Name School Course
Nicole UVA Biology
Jenna GWU CS
from there,
I only want to print the Name and the Course from that dictionary. How would I go about this?
The code below is how I put the original TSV file into the dictionary above.
import csv
data = csv.reader(open('data.tsv'),delimiter='\t')
fields = data.next()
for row in data:
item = dict(zip(fields, row))
print item
So now I got a dictionary like such:
{'Name':'Nicole.', 'School':'UVA.','Course':'Biology'}
{'Name':'Jenna.', 'School':'GWU','Course':'CS'}
{'Name':'Shan', 'School':'Columbia','Course':'Astronomy'}
{'Name':'BILL', 'School':'UMD.','Course':'Algebra'}
I only want to print the Name and the Course from that dictionary. How would I go about this?
I want to add code so that I'm only printing
{'Name':'Jenna.','Course':'CS'}
{'Name':'Shan','Course':'Astronomy'}
{'Name':'BILL','Course':'Algebra'}
Please guide. Thank You
Just delete the key in the loop for row in data: item = dict(zip(fields, row)) del item['School'] print item
The easiest way is to just remove the item['School'] entry before printing
for row in data:
item = dict(zip(fields, row))
del item['School']
print item
But this only works if you know exactly what the dictionary looks like, it has no other entries that you don't want, and it already has a School entry. I would instead recommend that you build a new dictionary out of the old one, only keeping the Name and Course entries
for row in data:
item = dict(zip(fields, row))
item = {k, v for k, v in item.items() if k in ('Name', 'Course')}
print item
Maybe use a DictReader in the first place, and rebuild the row-dict only if key matches a pre-defined list:
import csv
keep = {"Name","Course"}
data = csv.DictReader(open('data.tsv'),delimiter='\t')
for row in data:
row = {k:v for k,v in row.items() if k in keep}
print(row)
result:
{'Course': 'Biology', 'Name': 'Nicole'}
{'Course': 'CS', 'Name': 'Jenna'}
Based on the answer here:
filter items in a python dictionary where keys contain a specific string
print {k:v for k,v in item.iteritems() if "Name" in k or "Course" in k}
You're better off using a library designed for these kinds of tasks (Pandas). A dictionary is great for storing key-value pairs, but it looks like you have spreadsheet-like tabular data, so you should choose a storage type that better reflects the data at hand. You could simply do the following:
import pandas as pd
df = pd.read_csv('myFile.csv', sep = '\t')
print df[['Name','Course']]
You'll find that as you start doing more complicated tasks, it's better to use well written libraries than to cludge something together
replace the your " print(item) " line with the below line.
print(dict(filter(lambda e: e[0]!='School', item)))
OUTPUT:
{'Name':'Jenna.','Course':'CS'}
{'Name':'Shan','Course':'Astronomy'}
{'Name':'BILL','Course':'Algebra'}
Pasing through CSV file to store as dictionary with nested array values. Best approach?
I am trying to take this csv file and parse and store it in a form of a dictionary (sorry if I use the terms incorrectly I am currently learning). The first element is my key and the rest will be values in a form of nested arrays.
targets_value,11.4,10.5,10,10.8,8.3,10.1,10.7,13.1
targets,Cbf1,Sfp1,Ino2,Opi1,Cst6,Stp1,Met31,Ino4
one,"9.6,6.3,7.9,11.4,5.5",N,"8.4,8.1,8.1,8.4,5.9,5.9",5.4,5.1,"8.1,8.3",N,N
two,"7.0,11.4,7.0","4.8,5.3,7.0,8.1,9.0,6.1,4.6,5.0,4.6","6.3,5.9,5.9",N,"4.3,4.8",N,N,N
three,"6.0,9.7,11.4,6.8",N,"11.8,6.3,5.9,5.9,9.5","5.4,8.4","5.1,5.1,4.3,4.8,5.1",N,N,11.8
four,"9.7,11.4,11.4,11.4",4.6,"6.2,7.9,5.9,5.9,6.3","5.6,5.5","4.8,4.8,8.3,5.1,4.3",N,7.9,N
five,7.9,N,"8.1,8.4",N,"4.3,8.3,4.3,4.3",N,N,N
six,"5.7,11.4,9.7,5.5,9.7,9.7","4.4,7.0,7.7,7.5,6.9,4.9,4.6,4.9,4.6","7.9,5.9,5.9,5.9,5.9,6.3",6.7,"5.1,4.8",N,7.9,N
seven,"6.3,11.4","5.2,4.7","6.3,6.0",N,"8.3,4.3,4.8,4.3,5.1","9.8,9.5",N,8.4
eight,"11.4,11.4,5.9","4.4,6.3,6.0,5.6,7.6,7.1,5.1,5.3,5.1,4.9","6.3,6.3,5.9,5.9,6.6,6.6","5.3,5.2,7.0","8.3,4.3,4.3,4.8,4.3,4.3,8.3,4.8,8.3,5.1","9.2,7.4","9.4,9.3,7.9",N
nine,"9.7,9.7,11.4,9.7","5.2,4.6,5.5,6.5,4.5,4.6,5.5","6.3,5.9,5.9,9.5,6.5",N,"4.3,5.1,8.3,8.3,4.3,4.3,4.3,4.8",8.0,8.6,N
ten,"9.7,9.7,9.7,11.4,7.9","5.2,4.6,5.5,6.5,4.5,4.6,5.5","6.3,5.9,5.9,9.5,6.5",5.7,"4.3,4.3,4.3,5.1,8.3,8.3,4.3,4.3,4.3,4.8",8.0,8.6,N
YPL250C_Icy2,"11.4,6.1,11.4",N,"6.3,6.0,6.6,7.0,10.0,6.5,9.5,7.0,10.0",7.1,"4.3,4.3",9.2,"10.7,9.5",N
,,,,,,,,
,,,,,,,,
The issue was that in each line, some columns are a quotes because of multiple values per cell, and some only have a single entry but no quote. And cells that had no value input were inserted with an N. Since there was a mixture of quotes and non quotes, and numbers and non numbers.
Wanted the output to look something like this:
{'eight': ['11.4,11.4,5.9', '4.4,6.3,6.0,5.6,7.6,7.1,5.1,5.3,5.1,4.9', '6.3,6.3,5.9,5.9,6.6,6.6', '5.3,5.2,7.0', '8.3,4.3,4.3,4.8,4.3,4.3,8.3,4.8,8.3,5.1', '9.2,7.4', '9.4,9.3,7.9', 'N'],
'ten': ['9.7,9.7,9.7,11.4,7.9', '5.2,4.6,5.5,6.5,4.5,4.6,5.5', '6.3,5.9,5.9,9.5,6.5', '5.7', '4.3,4.3,4.3,5.1,8.3,8.3,4.3,4.3,4.3,4.8', '8.0', '8.6', 'N'],
'nine': ['9.7,9.7,11.4,9.7', '5.2,4.6,5.5,6.5,4.5,4.6,5.5', '6.3,5.9,5.9,9.5,6.5', 'N', '4.3,5.1,8.3,8.3,4.3,4.3,4.3,4.8', '8.0', '8.6', 'N']
}
I wrote a script to clean it and store it, but was not sure if my script was "too long for no reason". Any tips?
motif_dict = {}
with open(filename, "r") as file:
data = file.readlines()
for line in data:
if ',,,,,,,,' in line:
continue
else:
quoted_holder = re.findall(r'"(\d.*?\d)"' , line)
#reverses the order of the elements contained in the array
quoted_holder = quoted_holder[::-1]
new_line = re.sub(r'"\d.*?\d"', 'h', line).split(',')
for position,element in enumerate(new_line):
if element == 'h':
new_line[position] = quoted_holder.pop()
motif_dict[new_line[0]] = new_line[1:]
There's a csv module which makes working with csv files much easier. In your case, your code becomes
import csv
with open("motif.csv","r",newline="") as fp:
reader = csv.reader(fp)
data = {row[0]: row[1:] for row in reader if row and row[0]}
where the if row and row[0] lets us skip rows which are empty or have an empty first element. This produces (newlines added)
>>> data["eight"]
['11.4,11.4,5.9', '4.4,6.3,6.0,5.6,7.6,7.1,5.1,5.3,5.1,4.9',
'6.3,6.3,5.9,5.9,6.6,6.6', '5.3,5.2,7.0',
'8.3,4.3,4.3,4.8,4.3,4.3,8.3,4.8,8.3,5.1',
'9.2,7.4', '9.4,9.3,7.9', 'N']
>>> data["ten"]
['9.7,9.7,9.7,11.4,7.9', '5.2,4.6,5.5,6.5,4.5,4.6,5.5',
'6.3,5.9,5.9,9.5,6.5', '5.7', '4.3,4.3,4.3,5.1,8.3,8.3,4.3,4.3,4.3,4.8',
'8.0', '8.6', 'N']
In practice, for processing, I think you'd want to replace 'N' with None or some other object as a missing marker, and make every value a list of floats (even if it's only got one element), but that's up to you.
Write list of dictionary values to file [duplicate]
This question already has answers here:
Write values of Python dictionary back to file
(2 answers)
Closed 7 years ago.
I have a list of dictionaries such as:
values = [{'Name': 'John Doe', 'Age': 26, 'ID': '1279abc'},
{'Name': 'Jane Smith', 'Age': 35, 'ID': 'bca9721'}
]
What I'd like to do is print this list of dictionaries to a tab delimited text file to look something like this:
Name Age ID
John Doe 26 1279abc
Jane Smith 35 bca9721
However, I am unable to wrap my head around simply printing the values, as of right now I'm printing the entire dictionary per row via:
for i in values:
f.write(str(i))
f.write("\n")
Perhaps I need to iterate through each dictionary now? I've seen people use something like:
for i, n in iterable:
pass
But I've never understood this. Anyone able to shed some light into this?
EDIT:
Appears that I could use something like this, unless someone has a more pythonic way (Perhaps someone can explain "for i, n in interable"?):
for dic in values:
for entry in dic:
f.write(dic[entry])
This is simple enough to accomplish with a DictWriter. Its purpose is to write column-separated data, but if we specify our delimiter to be that of tabs instead of commas, we can make this work just fine.
from csv import DictWriter
values = [{'Name': 'John Doe', 'Age': 26, 'ID': '1279abc'},
{'Name': 'Jane Smith', 'Age': 35, 'ID': 'bca9721'}]
keys = values[0].keys()
with open("new-file.tsv", "w") as f:
dict_writer = DictWriter(f, keys, delimiter="\t")
dict_writer.writeheader()
for value in values:
dict_writer.writerow(value)
f.write('Name\tAge\tID')
for value in values:
f.write('\t'.join([value.get('Name'), str(value.get('Age')), value.get('ID')]))
You're probably thinking of the items() method. This will return the key and value for each entry in the dictionary. http://www.tutorialspoint.com/python/dictionary_items.htm for k,v in values.items(): pass
# assuming your dictionary is in values
import csv
with open('out.txt', 'w') as fout:
writer = csv.DictWriter(fout, fields=values.keys(). delimiter="\t")
writer.writeheader()
writer.writerow(values.values())