I have a program which takes a CSV file and saves the CSV data into a Dictionary and then compares the keys of that dictionary with values fetched from the database.
And If a dictionary key is equal to a database value then I want to save that said key with it's respected value in an empty dictionary for later use.
But I am unable to understand how to save the values.
def LCR(request):
template = "LCR\LCR.html"
dest = [92,923,9234,925]
rates = {} # my main dictionary which gets populated later on
ratelist = {}
csv_file = open(r'.\adoc.csv')
#Example data of the csv file
#Dest , Rate
#980, 0.205
#981, 0.305
#982, 0.015
data_set = csv_file.read().decode("UTF-8")
io_string = io.StringIO(data_set)
next(io_string)
for column in csv.reader(io_string, delimiter=str(u",")):
rates[column[0]] = column[1]
for desNum in dest: #Int gets converted to str
desNum = str(desNum)
# print type(desNum)
for num in desNum:
for venNum in rates:
for VN in venNum:
# print rates
if num[:2] == VN[:2]:
ratelist[venNum] = [rates.values()]
I expect to populate ratelist dictionary with both the key and it's respected value.
for key, value in rates.items():
for VN in key:
if ...:
ratelist[key] = value
Related
I'm importing a CSV to a dictionary, where there are a number of houses labelled (I.E. 1A, 1B,...)
Rows are labelled containing some item such as 'coffee' and etc. In the table is data indicating how much of each item each house hold needs.
Excel screenshot
What I am trying to do it check the values of the key value pairs in the dictionary for anything that isn't blank (containing either 1 or 2), and then take the key value pair and the 'PRODUCT NUMBER' (from the csv) and append those into a new list.
I want to create a shopping list that will contain what item I need, with what quantity, to which household.
the column containing 'week' is not important for this
I import the CSV into python as a dictionary like this:
import csv
import pprint
from typing import List, Dict
input_file_1 = csv.DictReader(open("DATA CWK SHOPPING DATA WEEK 1 FILE B.xlsb.csv"))
table: List[Dict[str, int]] = [] #list
for row in input_file_1:
string_row: Dict[str, int] = {} #dictionary
for column in row:
string_row[column] = row[column]
table.append(string_row)
I found on 'geeksforgeeks' how to access the pair by its value. however when I try this in my dictionary, it only seems to be able to search for the last row.
# creating a new dictionary
my_dict ={"java":100, "python":112, "c":11}
# list out keys and values separately
key_list = list(my_dict.keys())
val_list = list(my_dict.values())
# print key with val 100
position = val_list.index(100)
print(key_list[position])
I also tried to do a for in range loop, but that didn't seem to work either:
for row in table:
if row["PRODUCT NUMBER"] == '1' and row["Week"] == '1':
for i in range(8):
if string_row.values() != ' ':
print(row[i])
Please, if I am unclear anywhere, please let me know and I will clear it up!!
Here is a loop I made that should do what you want.
values = list(table.values())
keys = list(table.keys())
new_table = {}
index = -1
for i in range(values.count("")):
index = values.index("", index +1)
new_table[keys[index]] = values[index]
If you want to remove those values from the original dict you can just add in
d.pop(keys[index]) into the loop
I have a dictionary where the key is a tuple of length 2, and the value is a number, like this:
{('Alf', '2012.xlsx'): 600}
I want to create a sparse matrix, where Alf is the name of a row, 2012.xlsx is the name of a column, and 600 is the value where those two meet. And I want that to happen for all the other values in my dictionary. There may be keys like ('Alf', '2013.xlsx') and ('Elf','2012.xlsx')
The dictionary can be of any size, so I was thinking after creating it, I would loop through it and create a dataframe cell by cell, but I'm struggling to do that.
Here's the code I've written to create this dictionary (ing_dict). I'm open to approaching this problem in a different (better) way.
for filename in os.listdir(inv_folder):
name, ext = os.path.splitext(filename)
if ext == '.xlsx':
if filename==inv_file:
continue
recipe_files.append(filename)
#loop through list of files, load each workbook, and send it to the inventory function
for file in recipe_files:
file_counter += 1
file_path = inv_folder+'\\'+file
wb = load_workbook(file_path,data_only=True)
sheet=wb.active
inventory(sheet,file,file_counter)
def inventory(sheet,file,file_counter):
print('\n',file)
for row in sheet.iter_rows(2,18,1,3):
if row[0].value:
ing_dict[(row[0].value,file)]=row[2].value
Thank you
The following code should do what you want. I added inline comments to explain how I transform your data.
import numpy as np
import pandas as pd
# The expected input data
data = {('Alf', '2012.xlsx'): 600, ('Elf', '2012.xlsx'): 400, ('Alf', '2013.xlsx'): 200, ('Tim', '2014.xlsx'): 150}
row_to_pos = {} # maps a row name to an actual position
data_new = {} # We need to reformat the data structure
# loop through the data
for key, value in data.items():
row=key[0] # e.g., 'Alf'
column=key[1] # e.g., '2012.xlsx'
# if a row name is new, we add it to our mapper
if row not in row_to_pos:
row_to_pos[row] = len(row_to_pos)
# if a column name is new, we add a new entry in `data_new`
if column not in data_new:
data_new[column] = [[],[]]
# store our data, key=column_name, value=a list of two lists
data_new[column][0].append(row_to_pos[key[0]]) # store the position
data_new[column][1].append(value) # store the actual value
# we did not know in the first place how many unique row names we have so we have to loop once more
for key, value in data_new.items():
tmp = np.zeros(len(row_to_pos))
tmp[value[0]] = value[1] # value[0] are the positions, value[1] the corresponding values
data_new[key] = tmp
# create our dataframe
data_new['Name'] = list(row_to_pos.keys())
df = pd.DataFrame(data_new)
df = df.set_index(['Name'])
print(df)
This results in the following output:
2012.xlsx 2013.xlsx 2014.xlsx
Name
Alf 600.0 200.0 0.0
Elf 400.0 0.0 0.0
Tim 0.0 0.0 150.0
I have created a nested list from a larger nested dictionary, and now want to convert that list into a data frame. the list i have created has no keys or values.
I have tried to convert the list into a dictionary using dict() but this does not work.
the list is in this format (names and data changed for anonymity)
['Bigclient', ['All Web Site Data', '129374116'],
'Otherclient', ['All Web Site Data', '164548948'], ['Filtered website data', '142386573'], ['Test', '72551604'].
so i have a parent value 'Bigclient' that then has a child list including the name of the data and an ID number corresponding to that name. Each parent value has different amounts of child pairs. I want to make a data frame that has trhee columns like so
Client_name dataname ID
BigClient All Web 129374116
Other Client All web 164548948
Other Client Filtered 142386573
Other Client Test 7255160
so the clients name (parent value) is used to group the datanames and id's
new =[]
for item in data['items']:
name = item.get('name')
if name:
new.append(name)
webprop = item.get('webProperties')
if webprop:
for profile in webprop:
profile = profile.get('profiles')
if profile:
for idname in profile:
idname = idname.get('name')
for idname1 in profile:
idname1 = idname1.get('id')
if idname:
result = [idname, idname1]
new.append(result)
else:
continue
else:
continue
this is how ive built my list up, however it has no dictionaries.
Here you go:
import pandas as pd
raw_data = ['Bigclient', ['All Web Site Data', '129374116'], 'Otherclient', ['All Web Site Data', '164548948'], ['Filtered website data', '142386573'], ['Test', '72551604']]
# collect dsata
keys_list = []
values_list = [[] for _ in range(2)]
count = -1
for item in raw_data:
if isinstance(item, str):
keys_list.append(item)
count += 1
else:
values_list[count].append(item)
# create data dictionary
data_dict = dict(zip(keys_list, values_list))
# create data frame
raw_df = pd.DataFrame(columns=['Client_name', 'data'])
for key, values in data_dict.items():
for value in values:
raw_df = raw_df.append({'Client_name': key, 'data': value}, ignore_index=True)
# split list data into 2 columns
spilt_data = pd.DataFrame(raw_df['data'].values.tolist(), columns=['dataname','ID'])
# concat data
result = pd.concat([raw_df, spilt_data], axis=1, sort=False)
# drop used column
result = result.drop(['data'], axis=1)
Output:
Client_name dataname ID
0 Bigclient All Web Site Data 129374116
1 Otherclient All Web Site Data 164548948
2 Otherclient Filtered website data 142386573
3 Otherclient Test 72551604
I have a CSV file with column name (in first row) and values (rest of the row). I wanted to create variables to store these values for every row in a loop. So I started off by creating a dictionary with the CSV file and I got a list of the records with a key-value pair. So now I wanted to create variables to store the "value" extracted from the "key" of each item and within a loop for every record. I am not sure if I am setting this correctly.
Here is the dictionary I have.
my_dict = [{'value id':'value1', 'name':'name1','info':'info1'},
{'value id':'value2', 'name':'name2','info':'info2'},
{'value id':'value3', 'name':'name3','info':'info3'},
}]
for i in len(my_dict):
item[value id] = value1
item[name] = name1
item[info] = info1
The value id and name will be unique and are identifiers the list. Ultimately, I wanted to create an item object i.e. item[info] = info1 and I can add other codes to modify the item[info].
try this,
my_dict = [{'value':'value1', 'name':'name1','info':'info1'},
{'value':'value2', 'name':'name2','info':'info2'},
{'value':'value3', 'name':'name3','info':'info3'}]
for obj in my_dict:
value = obj['value']
name = obj['name']
info = obj['info']
to expand on #aws_apprentice's point, you can capture the data by creating some additional variables
my_dict = [{'value':'value1', 'name':'name1','info':'info1'},
{'value':'value2', 'name':'name2','info':'info2'},
{'value':'value3', 'name':'name3','info':'info3'}]
values = []
names = []
info = []
for obj in my_dict:
values.append(obj['value'])
names.append(obj['name'])
info.append(obj['info'])
I am making a generic tool which can take up any csv file.The file contains a city column which needs to be geocoded to latitudes and Longitudes. I have a csv file which looks something like this. The first row is the column name and the second row is the type of variable.
Time,M1,M2,M3,CityName
temp,num,num,num,loc
20-May-13,19,20,0,delhi
20-May-13,25,42,7,agra
20-May-13,23,35,4,mumbai
20-May-13,21,32,3,delhi
20-May-13,17,27,1,mumbai
20-May-13,16,40,5,delhi
First of all, I find the unique values in the City column and form a list of it.
filename = 'data_file.csv'
data_date = pd.read_csv(filename)
column_name = data_date.ix[:, data_date.loc[0] == "city"]
column_work = column_name.iloc[1:]
column_unique = column_work.iloc[:,3].unique().tolist()
Secondly, I have written code for geocoding my cities.
def geocode(address):
i = 0
try:
while i < len(geocoders):
# try to geocode using a service
location = geocoders[i].geocode(address)
# if it returns a location
if location != None:
# return those values
return [location.latitude, location.longitude]
else:
# otherwise try the next one
i += 1
except:
print (sys.exc_info()[0])
return ['null','null']
# if all services have failed to geocode, return null values
return ['null','null']
list = ['delhi', 'agra', 'mumbai']
j = 0
lat = []
for row in list:
print ('processing #',j)
j+=1
try:
state = row
address = state
result = geocode(address)
# add the lat/lon values to the row
lat.extend(result)
except:
# print 'Unsuccessful'
to_print = 'Unsuccessful'
# row.extend(to_print)
dout.append(row)
print(lat)
This gives me a list of latitudes and longitudes [28.7040592, 77.10249019999999, 27.1766701, 78.00807449999999, 19.0759837, 72.8776559]. I want to write this onto my CSV file as
Time,M1,M2,M3,CityName,Latitude,Longitude
temp,num,num,num,loc,lat,lng
20-May-13,19,20,0,delhi,28.7040592,77.10249019999999
20-May-13,25,42,7,agra,27.1766701,78.00807449999999
20-May-13,23,35,4,mumbai,19.0759837, 72.8776559
20-May-13,21,32,3,delhi,28.7040592,77.10249019999999
20-May-13,17,27,1,mumbai,19.0759837, 72.8776559
20-May-13,16,40,5,delhi,28.7040592,77.10249019999999
I tried making a separate list of latitudes and longitudes latitude = lat[0::2] longitude = lat[1::2] or convert it to into a dictionary {'delhi': [28.7040592, 77.10249019999999], 'agra': [27.1766701, 78.00807449999999], 'mumbai': [19.0759837, 72.8776559]} but somehow could not figure out how to write it on a csv file.
I think converting them into a dictionary is a good approach.
dic = {'delhi': [28.7040592, 77.10249019999999],
'agra': [27.1766701, 78.00807449999999],
'mumbai': [19.0759837, 72.8776559]}
# Create new columns
data_date["Latitude"] = data_date.apply(lambda row: dic.get(row["CityName"])[0], axis = 1)
data_date["Longitude"] = data_date.apply(lambda row: dic.get(row["CityName"])[1], axis = 1)
# Write the data back to csv file
data_date.to_csv(filename, index = False)
In this way it gets values of corresponding city names from the dictionary, and write them into the specified column. Finally it overwrites the old csv file with the new data frame.