I have the below string and need help on how write an if condition in a for loop that check if the row.startswith('name') then take the value and store is in a variable called name. Similarly for dob as well.
Once the for loop completes the output should be a dictionary as below which i can convert to a pandas dataframe.
'name john\n \n\nDOB\n12/08/1984\n\ncurrent company\ngoogle\n'
This is what i have tried so far but do not know how to get the values into a dictionary
for row in lines.split('\n'):
if row.startswith('name'):
name = row.split()[-1]
Final Ouput
data = {"name":"john", "dob": "12/08/1984"}
Try using a list comprehension and split:
s = '''name
john
dob
12/08/1984
current company
google'''
d = dict([i.splitlines() for i in s.split('\n\n')])
print(d)
Output:
{'name': 'john', 'dob': '12/08/1984', 'current company': 'google'}
Related
For example, for the txt file of
Math, Calculus, 5
Math, Vector, 3
Language, English, 4
Language, Spanish, 4
into the dictionary of:
data={'Math':{'name':[Calculus, Vector], 'score':[5,3]}, 'Language':{'name':[English, Spanish], 'score':[4,4]}}
I am having trouble with appending value to create list inside the smaller dict. I'm very new to this and I would not understand importing command. Thank you so much for all your help!
For each line, find the 3 values, then add them to a dict structure
from pathlib import Path
result = {}
for row in Path("test.txt").read_text().splitlines():
subject_type, subject, score = row.split(", ")
if subject_type not in result:
result[subject_type] = {'name': [], 'score': []}
result[subject_type]['name'].append(subject)
result[subject_type]['score'].append(int(score))
You can simplify it with the use of a defaultdict that creates the mapping if the key isn't already present
result = defaultdict(lambda: {'name': [], 'score': []}) # from collections import defaultdict
for row in Path("test.txt").read_text().splitlines():
subject_type, subject, score = row.split(", ")
result[subject_type]['name'].append(subject)
result[subject_type]['score'].append(int(score))
With pandas.DataFrame you can directly the formatted data and output the format you want
import pandas as pd
df = pd.read_csv("test.txt", sep=", ", engine="python", names=['key', 'name', 'score'])
df = df.groupby('key').agg(list)
result = df.to_dict(orient='index')
From your data:
data={'Math':{'name':['Calculus', 'Vector'], 'score':[5,3]},
'Language':{'name':['English', 'Spanish'], 'score':[4,4]}}
If you want to append to the list inside your dictionary, you can do:
data['Math']['name'].append('Algebra')
data['Math']['score'].append(4)
If you want to add a new dictionary, you can do:
data['Science'] = {'name':['Chemisty', 'Biology'], 'score':[2,3]}
I am not sure if that is what you wanted but I hope it helps!
I have two lists of dictionaries and json list and I need to grab a value of a specific key based on the value of a key from another dictionary. My data looks like this:
opps = [{'Product2Id': '100','Price': '1645'}, {'Product2Id': '101','Price': '5478'}]
products = [{'Id': '100', 'Name': 'Insertion'}, [{'Id': '101', 'Name': 'Print'}]
sales_json = {'Insertion': {'name': 'BAZ', 'id': '95'}, 'Print': {'name': 'BIC', 'id': '105'}
I need to loop through opps and assign a value to a new variable from sales_json. But for a specific Id that are stored in products and in opps
I tried the following:
for index, my_dict in enumerate(opps):
new_name = sales_json[products[my_dict["Product2Id"]]["Name"]]["name"]
Gives me an error.
The desired output is:
print(new_name)
BAZ,
BIC
You are trying to use the list products as a dictionary. Instead, you should first build a product number to name dictionary from it:
prod_num_to_name = {d['Id']: d['Name'] for d in products}
Then, you can run the loop you wanted, modified like this:
for index, my_dict in enumerate(opps):
new_name = sales_json[prod_num_to_name[my_dict["Product2Id"]]]["name"]
print new_name
To return a list of names that match the criteria, using a List Comprehension:
names = [ sales_json[product['Name']]['name'] for opp in opps for product in products if product['Id'] == opp['Product2Id']]
print (names)
Prints the list of names:
['BAZ', 'BIC']
I have a dataframe full of scientific paper information.
My Dataframe:
database authors title
0 sciencedirect [{'surname': 'Sharafaldin', 'first_name': 'Iman'}, An eval...
{'surname': 'Lashkari', 'first_name': 'Arash Habibi'}]
1 sciencedirect [{'surname': 'Srinivas', 'first_name': 'Jangirala'}, Governmen...
{'surname': 'Das', 'first_name': 'Ashok Kumar'}]
2 sciencedirect [{'surname': 'Bongiovanni', 'first_name': 'Ivano'}] The last...
3 ieeexplore [Igor Kotenko, Andrey Chechulin] Cyber Attac...
As you can see, the authors column contains a list of dictionarys, but only where the database is sciencedirect. In order to perform some analysis, I need to clean my data. Therefore, my goal is to put the names just into lists like in row 4.
What i want:
# From:
[{'surname': 'Sharafaldin', 'first_name': 'Iman'}, {'surname': 'Lashkari', 'first_name': 'Arash Habibi'}]
# To:
[Iman Sharafaldin, Arash Habibi Lashkari]
My appraoch is to createa two masks, one for the database column, extracting only sciencedirect papers and the other mask is the whole authors column. From these mask, a new dataframe is created, on which column "authors" i run the code shown below. It extracts the author names of each cell and stores them in a list, just as i want it:
scidir_mask = df["database"] == 'sciencedirect'
authors_col = df["authors"] is not None
only_scidir = df[authors_col & scidir_mask]
for cell in only_scidir["authors"]:
# get each list from cell
cell_list = []
for dictionary in cell:
# get the values from dict and reverse into list
name_as_list = [*dictionary.values()][::-1]
# make list from first and surname a string
author = ' '.join(name_as_list)
cell_list.append(author)
So at the end of the above code, the cell_list contains the authors names in the way I want.
But I can't get my head around, on how to store these cell_lists back into the original dataframe.
So, how do I get the authors cell, where the database is sciencedirect,perform my little function and store the output of my function back into the cell?
Idea is create custom function with f-strings and apply only to filtered rows:
scidir_mask = df["database"] == 'sciencedirect'
f = lambda x: [f"{y['first_name']} {y['surname']}" for y in x]
df.loc[scidir_mask, 'authors'] = df.loc[scidir_mask, 'authors'].apply(f)
print (df)
database authors title
0 sciencedirect [Iman Sharafaldin, Arash Habibi Lashkari] An eval
1 sciencedirect [Jangirala Srinivas, Ashok Kumar Das] Governmen
2 sciencedirect [Ivano Bongiovanni] The last
3 ieeexplore [Igor Kotenko, Andrey Chechulin] Cyber Attac
I want to create a dictionary that has predefined keys, like this:
dict = {'state':'', 'county': ''}
and read through and get values from a spreadsheet, like this:
for row in range(rowNum):
for col in range(colNum):
and update the values for the keys 'state' (sheet.cell_value(row, 1)) and 'county' (sheet.cell_value(row, 1)) like this:
dict[{}]
I am confused on how to get the state value with the state key and the county value with the county key. Any suggestions?
Desired outcome would look like this:
>>>print dict
[
{'state':'NC', 'county': 'Nash County'},
{'state':'VA', 'county': 'Albemarle County'},
{'state':'GA', 'county': 'Cook County'},....
]
I made a few assumptions regarding your question. You mentioned in the comments that State is at index 1 and County is at index 3; what is at index 2? I assumed that they occur sequentially. In addition to that, there needs to be a way in which you can map the headings to the data columns, hence I used a list to do that as it maintains order.
# A list containing the headings that you are interested in the order in which you expect them in your spreadsheet
list_of_headings = ['state', 'county']
# Simulating your spreadsheet
spreadsheet = [['NC', 'Nash County'], ['VA', 'Albemarle County'], ['GA', 'Cook County']]
list_of_dictionaries = []
for i in range(len(spreadsheet)):
dictionary = {}
for j in range(len(spreadsheet[i])):
dictionary[list_of_headings[j]] = spreadsheet[i][j]
list_of_dictionaries.append(dictionary)
print(list_of_dictionaries)
Raqib's answer is partially correct but had to be modified for use with an actual spreadsheet with row and columns and the xlrd mod. What I did was first use xlrd methods to grab the cell values, that I wanted and put them into a list (similar to the spreadsheet variable raqib has shown above). Not that the parameters sI and cI are the column index values I picked out in a previous step. sI=StateIndex and cI=CountyIndex
list =[]
for row in range(rowNum):
for col in range(colNum):
list.append([str(sheet.cell_value(row, sI)), str(sheet.cell_value(row, cI))])
Now that I have a list of the states and counties, I can apply raqib's solution:
list_of_headings = ['state', 'county']
fipsDic = []
print len(list)
for i in range(len(list)):
temp = {}
for j in range(len(list[i])):
tempDic[list_of_headings[j]] = list[i][j]
fipsDic.append(temp)
The result is a nice dictionary list that looks like this:
[{'county': 'Minnehaha County', 'state': 'SD'}, {'county': 'Minnehaha County', 'state': 'SD', ...}]
I am extracting a column from excel document with pandas. After that, I want to replace for each row of the selected column, all keys contained in multiple dictionaries grouped in a list.
import pandas as pd
file_loc = "excelFile.xlsx"
df = pd.read_excel(file_loc, usecols = "C")
In this case, my dataframe is called by df['Q10'], this data frame has more than 10k rows.
Traditionally, if I want to replace a value in df I use;
df['Q10'].str.replace('val1', 'val1')
Now, I have a dictionary of words like:
mydic = [
{
'key': 'wasn't',
'value': 'was not'
}
{
'key': 'I'm',
'value': 'I am'
}
... + tons of line of key value pairs
]
Currently, I have created a function that iterates over "mydic" and replacer one by one all occurrences.
def replaceContractions(df, mydic):
for cont in contractions:
df.str.replace(cont['key'], cont['value'])
Next I call this function passing mydic and my dataframe:
replaceContractions(df['Q10'], contractions)
First problem: this is very expensive because mydic has a lot of item and data set is iterate for each item on it.
Second: It seems that doesn't works :(
Any Ideas?
Convert your "dictionary" to a more friendly format:
m = {d['key'] : d['value'] for d in mydic}
m
{"I'm": 'I am', "wasn't": 'was not'}
Next, call replace with the regex switch and pass m to it.
df['Q10'] = df['Q10'].replace(m, regex=True)
replace accepts a dictionary of key-replacement pairs, and it should be much faster than iterating over each key-replacement at a time.