I am currently writing a lambda function (python 3.9) with the objective of finding students who are celebrating holiday in their respective country. To start off, I have a student.json file and a folder containing all the countries' holiday in the holiday_folder in my S3 bucket:
(main directory)
student.json
holiday_folder
(within holiday_folder)
holiday_us.json
holiday_cn.json
holiday_my.json
holiday_th.json
the list goes on...
The student.json file has the following content:
{'student': [{'name':'Adam', 'location':'US'}, {'name':'Ben', 'location':'SG'}...]}
While all the holiday json files are structured similarly as follow:
{ "holiday_us":[ { "date":"20220101", "desc":"New Year's Day" }, {"date"....}]}
To achieve my objective, I:
json.loads student.json content into variable student
Create a list called countries and run a for loop to append only the "location" values of all students in student variable
Create another list called holiday_calendar to store transformed location values a.k.a. transforming ['US', 'SG', ...] into ['holiday_us', 'holiday_sg' ...] so that I can convert these list values into variable names to load holiday dates
Run a conditional for loop on student. The loop will look into the holiday list that's only relevant to the specific student and return value accordingly. I wont be going deep on this as this is not the point of the question.
countries = []
holiday_calendar = []
for i in range(len(student)):
if student[i]['location'] not in countries:
countries.append(student[i]['location'])
holiday_calendar.append('holiday_' + student[i]['location'].lower())
print(holiday_calendar) # output: ['holiday_us', 'holiday_sg'...]
for i in range(0, len(holiday_calendar)):
print(holiday_calendar[i])
locals[holiday_calendar[i]] = json.loads(s3.Object('my_s3_bucket', 'holiday_folder' + holiday_calendar[i] + '.json')).get()['Body'].read())
My issue is that I cannot convert the holiday_calendar list values into variables. When I ran the above code, I got this error:
"errorMessage": "'builtin_function_or_method' object does not support item assignment",
"errorType": "TypeError"
I am pretty sure the error came from the globals() line as I have ran all the other parts of the code and they all worked flawlessly. I have also tried exec() and locals() as per suggested in a lot of SO topics but none worked. Any suggestions?
Related
I am new to python and am looking to analyze the S&P500 by sector. I have assigned symbols to all 11 sectors in the S&P with the first two looking like:
Financials = ['AFL', 'AIG', .... 'ZION']
Energy = ['APA', 'BKR', ... 'SLB']
I then create a new list (of lists) which might look like:
sectors_to_analyze = [Financials, Energy] or [Materials, ConsumerStaples]
My analysis is working perfectly, but I want to retrieve the names "Financials" and "Energy" to attach to the data produced and I cannot figure out how to do it other than make the name part of the list (Financials = ['Financials','AFL', 'AIG', .... 'ZION']
Can someone please point me in the right direction? Thank you.
Perhaps you could use a dictionary
sectors = {
'Financials':['AFL', ...],
# rest of your lists
}
Then you can iterate over the whole dict and access both names and data associated with those names
for key, value in sectors.items():
print(f'Sector name: {key}, List: {value}')
I think you want to use a dictionary instead of a "list of lists" (also called a two dimensional list). You could then loop over the dictionary almost the same way. Here's some example code:
Financials = ['AFL', 'AIG', 'ZION']
Energy = ['APA', 'BKR', 'SLB']
sectors = {"Finacials": Financials, "Energy": Energy}
# in this loop, sector is the sector's name, and symbols is the sector's
# list
for sector in sectors:
symbols = sectors[sector]
# ...
# do some analysis
# ...
I want to compare a (sub) dictionary created from RobotFramework(RF) with a (big) dictionary got from Python function
${house_1_key}= Set Variable House 1
${list_values}= Create List A n ${empty}
${house_1_dict}= create dictionary ${house_1_key}=${list_values}
and dictionary from python like:
def get_sum_dict():
....
total = {'House 1': ['A', 'n', ''], 'House 2':['','n',''],'House 3':['A','n','B']}
return total
I tried to compare in python:
def compare_dict(sub_dict, total_dict):
if sub_dict.items() <= total_dict.items():
return True
else:
return False
but this was FAIL
I tried to compare from RF too:
Compare dictionary
${sub_dict}= Convert To Dictionary ${house_1_dict}
${total_dict}= Run Keyword get_sum
${total_dict}= Convert To Dictionary ${total_dict}
Dictionary Should Contain Sub Dictionary ${total_dict} ${sub_dict}
But it fails too
and give back the error: Following keys missing from the first dictionary: House 1
Please help me to solve this problem.
#TodorMinakov I found that after get text from a website that actually return "House 1 " but not "House 1:" like I saw on difido report (after decoding from a report by html
I am working on an online course exercise (practice problem before the final test).
The test involves working with a big csv file (not downloadable) and answering questions about the dataset. You're expected to write code to get the answers.
The data set is a list of all documented baby names each year, along with
#how often each name was used for boys and for girls.
A sample list of the first 10 lines is also given:
Isabella,42567,Girl
Sophia,42261,Girl
Jacob,42164,Boy
and so on.
Questions you're asked include things like 'how many names in the data set', 'how many boys' names beginning with z' etc.
I can get all the data into a list of lists:
[['Isabella', '42567', 'Girl'], ['Sophia', '42261', 'Girl'], ['Jacob', '42164', 'Boy']]
My plan was to convert into a dictionary, as that would probably be easier for answering some of the other questions. The list of lists is saved to the variable 'data':
names = {}
for d in data:
names[d[0]] = d[1:]
print(names)
{'Isabella': ['42567', 'Girl'], 'Sophia': ['42261', 'Girl'], 'Jacob': ['42164', 'Boy']}
Works perfectly.
Here's where it gets weird. If instead of opening the sample file with 10 lines, I open the real csv file, with around 16,000 lines. everything works perfectly right up to the very last bit.
I get the complete list of lists, but when I go to create the dictionary, it breaks - here I'm just showing the first three items, but the full 16000 lines are all wrong in a similar way):
names = {}
for d in data:
names[d[0]] = d[1:]
print(names)
{'Isabella': ['56', 'Boy'], 'Sophia': ['48', 'Boy'], 'Jacob': ['49', 'Girl']
I know the data is there and correct, since I can read it directly:
for d in data:
print(d[0], d[1], d[2])
Isabella 42567 Girl
Sophia 42261 Girl
Jacob 42164 Boy
Why would this dictionary work fine with the cvs file with 10 lines, but completely break with the full file? I can't find any
Follow the comments to create two dicts, or a single dictionary with tuple keys. Using tuples as keys is fine if you keep your variables inside python, but you might get into trouble when exporting to json for example.
Try a dictionary comprehension with list unpacking
names = {(name, sex): freq for name, freq, sex in data}
Or a for loop as you started
names = dict()
for name, freq, sex in data:
names[(name, freq)] = freq
I'd go with something like
results = {}
for d in data:
name, amount, gender = d.split(',')
results[name] = data.get(name, {})
results[name].update({ gender: amount })
this way you'll get results in smth like
{
'Isabella': {'Girl': '42567', 'Boy': '67'},
'Sophia': {'Girl': '42261'},
'Jacob': {'Boy': '42164'}
}
However duplicated values will override previous, so you need to take that into account if there are some and it also assumes that the whole file matches format you've provided
I have a problem and I want to determine whether my approach is sound. Here is the idea:
I would be creating a primary dict called zip_codes, of which respective zipcodes (from a list) were the names of each of the nested dicts. Each would have keys for "members", "offices", "members per office"
It would look like this:
zips {
90219: {
"members": 120,
"offices": 18,
"membersperoffice": 28
},
90220: {
"members": 423,
"offices": 37,
"membersperoffice": 16
}
}
and so on and so forth.
I think I need to build the nested dicts, and then process several lists against conditionals, passing resulting values into the corresponding dicts on the fly (i.e. based on how many times a zip code exists in the list).
Is using nested dictionaries the most pythonic way of doing this? Is it cumbersome? Is there a better way?
Can someone drop me a hint about how to push key values into nested dicts from a loop? I've not been able to find a good resource describing what I'm trying to do (if this is, indeed, the best path).
Thanks.
:edit: a more specific example:
determine how many instances of a zipcode are in list called membersperzip
find corresponding nested dict with same name as zipcode, inside dict called zips
pass value to corresponding key, called "members" (or whatever key)
:edit 2:
MadPhysicist requested I give code examples (I don't even know where to start with this one and I can't find examples. All I've been able to do thus far is:
area_dict = {}
area_dict = dict.fromkeys(all_areas, 0) #make all of the zipscodes keys, add a zero in the first non-key index
dictkeys = list (area_dict.keys())
That gets me a dict with a bunch of zip codes as keys. I've discovered no way to iterate through a list and create nested dicts (yet). Hence the actual question.
Please don't dogpile me and do the usual stack overflow thing. This is not me asking anyone to do my homework. This is merely me asking someone to drop me a HINT.
:edit 3:
Ok. This is convoluted (my fault). Allow me to clarify further:
So, I have an example of what the nested dicts should look like. They'll start out empty, but I need to iterate through one of the zip code lists to create all the nested dicts... inside of zips.
This is a sample of the list that I want to use to create the nested dicts inside of the zips dict:
zips = [90272, 90049, 90401, 90402, 90403, 90404, 90291, 90292, 90290, 90094, 90066, 90025, 90064, 90073]
And this is what I want it to look like
zips {
90272: {
"members": ,
"offices": ,
"membersperoffice":
},
90049: {
"members": ,
"offices": ,
"membersperoffice":
}
}
....
etc, etc. ( creating a corresponding nested dict for each zipcode in the list)
After I achieve this, I have to iterate through several more zip code lists... and those would spit out the number of times a zip code appears in a given list, and then find the dict corresponding to the zip code in question, and append that value to the relevant key.
One I figure out the first part, I can figure this second part out on my own.
Thanks again. Sorry for any confusion.
You can do something like this:
all_areas = [90219, 90220]
zips = {zipcode: code_members(zipcode) for zipcode in all_areas}
def code_members(zipcode):
if zipcode == 90219:
return dict(members=120, offices=18, membersperoffice=28)
return dict(members=423, offices=37, membersperoffice=16)
I think I need to build the nested dicts, and then process several
lists against conditionals, passing resulting values into the
corresponding dicts on the fly (i.e. based on how many times a zip
code exists in the list).
Using the above approach, if a zipcode appears multiple times in the all_areas list, the resulting zip dictionary will only contain one instance of the zipcode.
Is using nested dictionaries the most pythonic way of doing this? Is
it cumbersome? Is there a better way?
May I suggest making a simple object that represents the value of each zipcode. Something simple like:
Using dataclass:
#dataclass.dataclass
class ZipProperties(object):
members: int
offices: int
membersperoffice: int
Using named tuple:
ZipProperties = collections.namedtuple('ZipProperties', ['members', 'offices', 'membersperoffice'])
You can then change the code_members function to this:
def code_members(zipcode):
if zipcode == 90219:
return ZipProperties(120, 18, 28)
return ZipProperties(423, 37, 16)
Addressing your concrete example:
determine how many instances of a zipcode are in list called membersperzip
find corresponding nested dict with same name as zipcode, inside dict called zips
pass value to corresponding key, called "members" (or whatever key)
membersperzip: typings.List[Tuple[int, int]] = [(90219, 54)]
for zip, members in membersperzip:
for zipcode, props in zips.items():
if zipcode == zip:
props.members = members
I would suggest you to append it when you have the actual value instead of initializing dictionary with empty values for each key. You have list of keys and I do not see why you want to put all of them to the dictionary without having value in the first place.
zips = [90272, 90049, 90401, 90402, 90403, 90404, 90291, 90292, 90290, 90094, 90066, 90025, 90064, 90073]
zips_dict = {}
for a_zip in zips:
if a_zip not in zips_dict:
# Initialize proper value here for members etc.
zips_dict[a_zip] = proper_value
If you insist to initialize dict with empty value for each keys, you could use this, which will also iterate through the list anyway but in python comprehension.
zips = [90272, 90049, 90401, 90402, 90403, 90404, 90291, 90292, 90290, 90094, 90066, 90025, 90064, 90073]
zips_dict = {
x:{
"members":None,
"offices":None,
"membersperoffice":None,
} for x in zips
}
Hope this helps
I'm having some trouble figuring out the best implementation
I have data in file in this format:
|serial #|machine_name|machine_owner|
If a machine_owner has multiple machines, I'd like the machines displayed in a comma separated list in the field. so that.
|1234|Fred Flinstone|mach1|
|5678|Barney Rubble|mach2|
|1313|Barney Rubble|mach3|
|3838|Barney Rubble|mach4|
|1212|Betty Rubble|mach5|
Looks like this:
|Fred Flinstone|mach1|
|Barney Rubble|mach2,mach3,mach4|
|Betty Rubble|mach5|
Any hints on how to approach this would be appreciated.
You can use dict as temporary container to group by name and then print it in desired format:
import re
s = """|1234|Fred Flinstone|mach1|
|5678|Barney Rubble|mach2|
|1313|Barney Rubble||mach3|
|3838|Barney Rubble||mach4|
|1212|Betty Rubble|mach5|"""
results = {}
for line in s.splitlines():
_, name, mach = re.split(r"\|+", line.strip("|"))
if name in results:
results[name].append(mach)
else:
results[name] = [mach]
for name, mach in results.items():
print(f"|{name}|{','.join(mach)}|")
You need to store all the machines names in a list. And every time you want to append a machine name, you run a function to make sure that the name is not already in the list, so that it will not put it again in the list.
After storing them in an array called data. Iterate over the names. And use this function:
data[i] .append( [ ] )
To add a list after each machine name stored in the i'th place.
Once your done, iterate over the names and find them in in the file, then append the owner.
All of this can be done in 2 steps.