Develop Reference

replacing keys from a dictionary inside a list python

I have a list inside a nested dictionary
body = {'Ready Date': '2020-01-31T12:00:00','Shipment Line List': [{'Description': 'Test', 'Weigth': '5',
'Height': '4.0','Length': '2.0', 'Width': '3.0'}, {'Description': 'Test', 'Weigth': '20', 'Height': '5',
'Length': '30', 'Width': '10']}
I want to iterate over the keys in the nested dictionary and replace "Weigth" with the correct spelling "Weight"
I tried this approach, but I am not getting the expected output
key = {"Weigth":"Weight"}
def find_replace(dict_body, dictionary):
# is the item in the dict?
for item in dict_body:
# iterate by keys
if item in dictionary.keys():
# look up and replace
dict_body = dict_body.replace(item, dictionary[item])
# return updated dict
return dict_body
a = find_replace(body,key)
print(a)

I think a better idea in this particular case is to treat everything as a string, replace and back as a dictionary. Because if you have multiple nested keys, it might be just be easier this way in two lines of code:
from ast import literal_eval
body = literal_eval(str(body).replace("Weigth","Weight"))
This outputs:
{'Ready Date': '2020-01-31T12:00:00',
'Shipment Line List': [{'Description': 'Test',
'Height': '4.0',
'Length': '2.0',
'Weight': '5',
'Width': '3.0'},
{'Description': 'Test',
'Height': '5',
'Length': '30',
'Weight': '20',
'Width': '10'}]}

I want to iterate over the keys in the nested dictionary and replace "Weigth" with the correct spelling "Weight"
something like the below
body = {'Ready Date': '2020-01-31T12:00:00', 'Shipment Line List': [{'Description': 'Test', 'Weigth': '5',
'Height': '4.0', 'Length': '2.0', 'Width': '3.0'},
{'Description': 'Test', 'Weigth': '20',
'Height': '5',
'Length': '30', 'Width': '10'}]}
for entry in body['Shipment Line List']:
entry['Weight'] = entry['Weigth']
del entry['Weigth']
print(body)
output
{'Ready Date': '2020-01-31T12:00:00', 'Shipment Line List': [{'Description': 'Test', 'Height': '4.0', 'Length': '2.0', 'Width': '3.0', 'Weight': '5'}, {'Description': 'Test', 'Height': '5', 'Length': '30', 'Width': '10', 'Weight': '20'}]}

How to convert a JSON list into two different columns in a CSV file?

This is my first question on this spectacular website, I need to know how to export complex information from a JSON to a CSV.
The problem is that I need from the list that I have in the column to have two different values.
I tried a lot of different combinations and I couldn't so one of my last resources are asked to the community.
My code is this:
def output(alerts):
output = list()
for alert in alerts:
applications = alerts['applications']
for app in applications:
categories = app['categories']
for cat in categories:
output_alert = [list(cat.items())[0], app['confidence'], app['icon'],
app['name'], app['version'], app['website'], alerts['language'], alerts['status']]
output.append(output_alert)
df = pd.DataFrame(output, columns=['Categories', 'Confidence', 'Icon', 'Name', 'Version', 'Website',
'Language', 'Status'])
df.to_csv(args.output)
print('Scan completed, you already have your new CSV file')
return
enter image description here
I left you a picture of the CSV file with the problem in column B (I have a list there) but I need actually two columns with each value...
I attached the JSON response that I have from a REST API
{'applications': [{'categories': [{'59': 'JavaScript libraries'}],
'confidence': '100',
'icon': 'Lo-dash.png',
'name': 'Lodash',
'version': '4.17.15',
'website': 'http://www.lodash.com'},
{'categories': [{'12': 'JavaScript frameworks'}],
'confidence': '100',
'icon': 'RequireJS.png',
'name': 'RequireJS',
'version': '2.3.6',
'website': 'http://requirejs.org'},
{'categories': [{'13': 'Issue trackers'}],
'confidence': '100',
'icon': 'Sentry.svg',
'name': 'Sentry',
'version': '4.6.2',
'website': 'https://sentry.io/'},
{'categories': [{'1': 'CMS'},
{'6': 'Ecommerce'},
{'11': 'Blogs'}],
'confidence': '100',
'icon': 'Wix.png',
'name': 'Wix',
'version': None,
'website': 'https://www.wix.com'},
{'categories': [{'59': 'JavaScript libraries'}],
'confidence': '100',
'icon': 'Zepto.png',
'name': 'Zepto',
'version': None,
'website': 'http://zeptojs.com'},
{'categories': [{'19': 'Miscellaneous'}],
'confidence': '100',
'icon': 'webpack.svg',
'name': 'webpack',
'version': None,
'website': 'https://webpack.js.org/'},
{'categories': [{'12': 'JavaScript frameworks'}],
'confidence': '0',
'icon': 'React.png',
'name': 'React',
'version': None,
'website': 'https://reactjs.org'}], 'language': 'es', 'status': 'success'}
[{'59': 'JavaScript libraries'}] this last thing is my big problem! Thank you for your time and help!

You could try using list(cat.keys())[0], list(cat.values())[0] in your output_alert variable to extract key and value separately.

You can use json_normalize to extract your columns without the for-loop, and then create two new columns with the extracted keys and values from categories:
result = pd.json_normalize(
alerts,
record_path=["applications"],
meta=["language", "status"]
).explode("categories")
result["category_labels"] = result.categories.apply(lambda x: list(x.keys())[0])
result["category_values"] = result.categories.apply(lambda x: list(x.values())[0])
The output is:

Python: Extra comma on csv.Dictreader column

I have this read function where it reads a csv file using csv.DictReader. The file.csv is separated by commas and it fully reads. However, this part of my file has a column that contains multiple commas. My question is, how can I make sure that comma is counted as part of a column? I cannot alter my csv file to meet the criteria.
Text File:
ID,Name,University,Street,ZipCode,Country
12,Jon Snow,U of Winterfell,Winterfell #45,60434,Westeros
13,Steve Rogers,NYU,108, Chelsea St.,23333,United States
20,Peter Parker,Yale,34, Tribeca,32444,United States
34,Tyrion Lannister,U of Casterly Rock,Kings Landing #89, 43543,Westeros
The desired output is this:
{'ID': '12', 'Name': 'Jon Snow', 'University': 'U of Winterfell', 'Street': 'Winterfell #45', 'ZipCode': '60434', 'Country': 'Westeros'}
{'ID': '13', 'Name': 'Steve Rogers', 'University': 'NYU', 'Street': '108, Chelsea St.', 'ZipCode': '23333', 'Country': 'United States'}
{'ID': '20', 'Name': 'Peter Parker', 'University': 'Yale', 'Street': '34, Tribeca', 'ZipCode': '32444', 'Country': 'United States'}
{'ID': '34', 'Name': 'Tyrion Lannister', 'University': 'U of Casterly Rock', 'Street': 'Kings Landing #89', 'ZipCode': '43543', 'Country': 'Westeros'}
As you can tell the 'Street' has at least two commas due to the numbers:
13,Steve Rogers,NYU,108, Chelsea St.,23333,United States
20,Peter Parker,Yale,34, Tribeca,32444,United States
Note: Most of the columns being read splits by a str,str BUT under the 'Street' column it is followed by a str, str (there is an extra space after the comma). I hope this makes sense.
The options I tried looking out is using re.split, but I don't know how to implement it on my read file. I was thinking re.split(r'(?!\s),(?!\s)',x[:-1])? How can I make sure the format from my file will count as part of any column? I can't use pandas.
My current output looks like this right now:
{'ID': '12', 'Name': 'Jon Snow', 'University': 'U of Winterfell', 'Street': 'Winterfell #45', 'ZipCode': '60434', 'Country': 'Westeros'}
{'ID': '13', 'Name': 'Steve Rogers', 'University': 'NYU', 'Street': '108', 'ZipCode': 'Chelsea St.', 'Country': '23333', None: ['United States']}
{'ID': '20', 'Name': 'Peter Parker', 'University': 'Yale', 'Street': '34', 'ZipCode': 'Tribeca', 'Country': '32444', None: ['United States']}
{'ID': '34', 'Name': 'Tyrion Lannister', 'University': 'U of Casterly Rock', 'Street': 'Kings Landing #89', 'ZipCode': '43543', 'Country': 'Westeros'}
This is my read function:
import csv
list = []
with open('file.csv', mode='r') as csv_file:
csv_reader = csv.DictReader(csv_file, delimiter=",", skipinitialspace=True)
for col in csv_reader:
list.append(dict(col))
print(dict(col))

You can't use csv if the file isn't valid CSV format.
You need to call re.split() on ordinary lines, not on dictionaries.
list = []
with open('file.csv', mode='r') as csv_file:
keys = csv_file.readline().strip().split(',') # Read header line
for line in csv_file:
line = line.strip()
row = re.split(r'(?!\s),(?!\s)',line)
list.append(dict(zip(keys, row)))

The actual solution for the problem is modifying the script that generates the csv file.
If you have a chance to modify that output you can do 2 things
Use a delimiter other than a comma such as | symbol or ; whatever you believe it doesn't exist in the string.
Or enclose all columns with " so you'll be able to split them by , which are actual separators.
If you don't have a chance to modify the output.
And if you are sure about that multiple commas are only in the street column; then you should use csv.reader instead of DictReader this way you can get the columns by Indexes that you are already sure. for instance row[0] will be ID row[1] will be Name and row[-1] will be Country row[-2] will be ZipCode so row[2:-2] would give you what you need i guess. Indexes can be arranged but the idea is clear I guess.
Hope that helps.
Edit:
import csv
list = []
with open('file.csv', mode='r') as csv_file:
csv_reader = csv.reader(csv_file, delimiter=",", skipinitialspace=True)
# pass the header row
next(csv_reader)
for row in csv_reader:
list.append({"ID": row[0],
"Name": row[1],
"University": row[2],
"Street": ' '.join(row[3:-2]),
"Zipcode": row[-2],
"Country": row[-1]})
print(list)
--
Here is the output (with pprint)
[{'Country': 'Westeros',
'ID': '12',
'Name': 'Jon Snow',
'Street': 'Winterfell #45',
'University': 'U of Winterfell',
'Zipcode': '60434'},
{'Country': 'United States',
'ID': '13',
'Name': 'Steve Rogers',
'Street': '108 Chelsea St.',
'University': 'NYU',
'Zipcode': '23333'},
{'Country': 'United States',
'ID': '20',
'Name': 'Peter Parker',
'Street': '34 Tribeca',
'University': 'Yale',
'Zipcode': '32444'},
{'Country': 'Westeros',
'ID': '34',
'Name': 'Tyrion Lannister',
'Street': 'Kings Landing #89',
'University': 'U of Casterly Rock',
'Zipcode': '43543'}]
-- second edit
edited the index on the street.
Regards.

How to create a list of dictionaries using for loop?

Textfile:
VIP Room, 10, 250
Executive Room,30, 500
Pool Site, 50, 850
Banquet Hall, 200, 1000
Chamber Hall, 500, 2000
Concert Hall, 1000, 3500
My code so far to read the file and create a list:
def readVenueList():
dic={}
venueList=[]
f=open("venue.txt","r")
for line in f:
line = line.split(",")
print(line)
for i in line:
i.split()
dic["name"]=i[0]
dic["num"]=i[1]
dic["cost"]=i[2]
venueList.append(dic)
return(venueList)
How do I create a list of dictionaries with the following output?
venueList = [{'cost': '250', 'name': 'VIP Room', 'num': '10'},
{'cost': '250', 'name': 'Executive Room', 'num': '30'},
# and so on and so forth...
]

You can simply use the csv reader library to handle this.
import csv
headers = ['name', 'num', 'cost']
with open('venue.txt', 'r') as f:
reader = csv.reader(f)
needed_list = [{headers[i]: row[i].strip() for i in range(3)} for row in reader]

It is very similar to earlier answer by #N M
datablob = u"""VIP Room,10,250
Executive Room,30,500
Pool Site,50,850
Banquet Hall,200,1000
Chamber Hall,500,2000
Concert Hall,1000,3500
"""
from csv import reader
from io import StringIO
def readVenueList(fd):
c = reader(fd)
hdr = ["name", "num", "cost"]
for i in c:
d = {}
for el, v in enumerate(i):
d[hdr[el]] = v
yield d
if __name__ == '__main__':
# replace with file object
# file = open("archive1.csv")
file = StringIO(datablob)
print(list(readVenueList(file)))
# Output
[{'name': 'VIP Room', 'num': '10', 'cost': '250'}, {'name':
'Executive Room', 'num': '30', 'cost': '500'}, {'name': 'Pool
Site', 'num': '50', 'cost': '850'}, {'name': 'Banquet Hall',
'num': '200', 'cost': '1000'}, {'name': 'Chamber Hall', 'num':
'500', 'cost': '2000'}, {'name': 'Concert Hall', 'num': '1000',
'cost': '3500'}]

If you don't want to use a CSV reader (though that's probably the best idea), you could also do this using list/dictionary comprehensions
with open('venue.txt', 'r') as f:
lines = (line.split(',') for line in f)
venues = [
{'name': name.strip(), 'number': int(num), 'cost': int(cost)}
for name, num, cost in lines
]

Here's how to modify your code do it properly (and follow the PEP 8 - Style Guide for Python Code recommendations more closely) :
from pprint import pprint
def readVenueList():
venueList = []
with open("venue.txt", "r") as f:
for line in f:
dic = {}
items = [item.strip() for item in line.split(",")]
dic["name"] = items[0]
dic["num"] = items[1]
dic["cost"] = items[2]
venueList.append(dic)
return venueList
venueList = readVenueList()
pprint(venueList)
Output:
[{'cost': '250', 'name': 'VIP Room', 'num': '10'},
{'cost': '500', 'name': 'Executive Room', 'num': '30'},
{'cost': '850', 'name': 'Pool Site', 'num': '50'},
{'cost': '1000', 'name': 'Banquet Hall', 'num': '200'},
{'cost': '2000', 'name': 'Chamber Hall', 'num': '500'},
{'cost': '3500', 'name': 'Concert Hall', 'num': '1000'}]

python 2.x csv writer

When I print these strings
print query, netinfo
I get output below, which is fine. How do i take these strings and put them into a CSV file into a single row?
8.8.8.8 [{'updated': '2012-02-24T00:00:00', 'handle': 'NET-8-0-0-0-1', 'description': 'Level 3 Communications, Inc.', 'tech_emails': 'ipaddressing#level3.com', 'abuse_emails': 'abuse#level3.com', 'postal_code': '80021', 'address': '1025 Eldorado Blvd.', 'cidr': '8.0.0.0/8', 'city': 'Broomfield', 'name': 'LVLT-ORG-8-8', 'created': '1992-12-01T00:00:00', 'country': 'US', 'state': 'CO', 'range': '8.0.0.0 - 8.255.255.255', 'misc_emails': None}, {'updated': '2014-03-14T00:00:00', 'handle': 'NET-8-8-8-0-1', 'description': 'Google Inc.', 'tech_emails': 'arin-contact#google.com', 'abuse_emails': 'arin-contact#google.com', 'postal_code': '94043', 'address': '1600 Amphitheatre Parkway', 'cidr': '8.8.8.0/24', 'city': 'Mountain View', 'name': 'LVLT-GOGL-8-8-8', 'created': '2014-03-14T00:00:00', 'country': 'US', 'state': 'CA', 'range': None, 'misc_emails': None}]
I have tried hobbling this together but it's all jacked up. I could use some help on how to use the csv module.
writer=csv.writer(open('dict.csv', 'ab'))
for key in query:
writer.writerow(query)

You can put your variables in a tuple and write to csv file :
import csv
from operator import itemgetter
with open('ex.csv', 'wb') as csvfile:
spamreader = csv.writer(csvfile, delimiter=' ')
spamreader.writerow((query, netinfo))
Note: if you are in python 3 use following code :
import csv
from operator import itemgetter
with open('ex.csv', 'w',newline='') as csvfile:
spamreader = csv.writer(csvfile, delimiter=' ')
spamreader.writerow((query, netinfo))