i have a Problem with Django and dicts. I want to get only items that match a sting as below. but i can´t get it to work. Thanks for your help.
django_db_query = [{'time': '13:00 Uhr', 'titel': 'test1'}, {'time': '14:00 blah', 'titel': 'test2'}, {'time': '13:00 Uhr', 'titel': 'test3'},]
all_db_items = Django_db.objects.all()
only_13 = dict()
for item in all_db_items:
if item.time is "13":
only_13 += item
wanted: The datastructure and multiple values from my db in my dict but only with time 13:00 Uhr
for item in only_13:
print item.titel
console
test1
test2
Assuming your DjangoDbModel looks something like
def DjangoDbModel(models.Model):
time = models.DateTimeField()
title = models.CharField(max_length=256)
All you need to do in that case is
DjangoDbModels.objects.filter(time__hour=13) if you want to have only items that are from hour 13. You can apply similar filters to the day, year and month for example.
for item in all_db_items:
if item['time'] == "13:00 Uhr":
only_13.update(item)
is only works with identical operands
+ is not implemented for dicts, use update instead
more resources:
https://www.python-course.eu/dictionaries.php
Related
I'm trying to create an event manager in which a dictionary stores the events like this
my_dict = {'2020':
{'9': {'8': ['School ']},
'11': {'13': ['Doctors ']},
'8': {'31': ['Interview']}
},
'2021': {}}
In which the outer key is the year the middle key is a month and the most inner key is a date which leads to a list of events.
I'm trying to first sort it so that the months are in order then sort it again so that the days are in order. Thanks in advance
Use-case
DevOrangeCrush wishes to sort on keys in a nested dictionary where the nesting occurs on multiple levels
Solution
Normalize the data so that the dates match ISO8601 format, for easier sorting
In plain English, this means make sure you always use two digits for month and date, and always use four digits for year
Re-normalize the original dictionary data structure into a single list of dictionaries, where each dictionary represents a row, and the list represents an outer containing table
this is known as an Array of Hashes in perl-speak
this is known as a list of objects in JSON-speak
Once your data is restructured you are solving a much more well-known, well-documented, and more obvious problem, how to sort a simple list of dictionaries (which is already documented in the See also section of this answer).
Example
import pprint
## original data is formatted as a nested dictionary, which is clumsy
my_dict = {'2020':
{'9': {'8': ['School ']}, '11':
{'13': ['Doctors ']},'8':
{'31': ['Interview']}}, '2021': {}
}
## we want the data formatted as a standard table (aka list of dictionary)
## this is the most common format for this kind of data as you would see in
## databases and spreadsheets
mydata_table = []
ddtemp = dict()
for year in my_dict:
for month in my_dict[year].keys():
ddtemp['month'] = '{0:02d}'.format(*[int(month)])
ddtemp['year'] = year
for day in my_dict[year][month].keys():
ddtemp['day'] = '{0:02d}'.format(*[int(day)])
mydata_row = dict()
mydata_row['year'] = '{year}'.format(**ddtemp)
mydata_row['month'] = '{month}'.format(**ddtemp)
mydata_row['day'] = '{day}'.format(**ddtemp)
mydata_row['task_list'] = my_dict[year][month][day]
mydata_row['date'] = '{year}-{month}-{day}'.format(**ddtemp)
mydata_table.append(mydata_row)
pass
pass
pass
## output result is now easily sorted and there is no data loss
## you will have to modify this if you want to deal with years that
## do not have any associated task_list data
pprint.pprint(mydata_table)
'''
## now we have something that can be sorted using well-known python idioms
## and easily manipulated using data-table semantics
## (search, sort, filter-by, group-by, select, project ... etc)
[
{'date': '2020-09-08','day': '08',
'month': '09','task_list': ['School '],'year': '2020'},
{'date': '2020-11-13','day': '13',
'month': '11','task_list': ['Doctors '],'year': '2020'},
{'date': '2020-08-31','day': '31',
'month': '08','task_list': ['Interview'],'year': '2020'},
]
'''
See also
How to sort a python list-of-dictionary
How to sort objects by multiple keys
Why you should use ISO8601 date format
ISO8601 vs timestamp
To get sorted events data, you can do something like this:
def sort_events(my_dict):
new_events_data = dict()
for year, month_data in my_dict.items():
new_month_data = dict()
for month, day_data in month_data.items():
sorted_day_data = sorted(day_data.items(), key=lambda kv: int(kv[0]))
new_month_data[month] = OrderedDict(sorted_day_data)
sorted_months_data = sorted(new_month_data.items(), key=lambda kv: int(kv[0]))
new_events_data[year] = OrderedDict(sorted_months_data)
return new_events_data
Output:
{'2020': OrderedDict([('8', OrderedDict([('31', ['Interview'])])),
('9', OrderedDict([('8', ['School '])])),
('11', OrderedDict([('13', ['Doctors '])]))]),
'2021': OrderedDict()}
A simple dict can't be ordered, you could do it using a OrderedDict but if you simply need to get it sorted while iterating on it do like this
for year in sorted(map(int, my_dict)):
year_dict = my_dict[str(year)]
for month in sorted(map(int, year_dict)):
month_dict = year_dict[str(month)]
for day in sorted(map(int, month_dict)):
events = month_dict[str(day)]
for event in events:
print(year, month, day, event)
Online Demo
The conversion to int is to ensure right ordering between the numbers, without you'll get 1, 10, 11, .., 2, 20, 21
A dictionary in Python does not have an order, you might want to try the OrderedDict class from the collections Module which remembers the order of insertion.
Of course you would have to sort and reinsert the elements whenever you insert a new element which should be placed before any of the existing elements.
If you care about order maybe a different data structure works better. For example a list of lists.
I have the following models:
class Event(models.Model):
date = models.DateTimeField()
event_type = models.ForeignKey('EventType')
class EventType(models.Model):
name = models.CharField(unique=True)
I am trying to get a list of all dates, and what event types are available on that date.
Each item in the list would be a dictionary with two fields: date and event_types which would be a list of distinct event types available on that date.
Currently I have come up with a query to get me a list of all distinct dates, but this is only half of what I want to do:
query = Event.objects.all().select_related('event_type')
results = query.distinct('date').order_by('date').values_list('date', flat=True)
Now I can change this slightly to get me a list of all distinct date + event_type combinations:
query = Event.objects.all().select_related('event_type')
results = query.order_by('date').distinct('date', 'event_type').values_list('date', 'event_type__name')
But this will have an entry for each event type within a given date. I need to aggregate a list within each date.
Is there a way I can construct a queryset to do this? If not, how would I do this some other way to get to the same result?
You can perform such aggregate with the groupby function of itertools. It is a requirement that the elements appearch in "chunks" with respect to the "grouper criteria". But this is the case here, since you use order_by.
We can thus write it like:
from itertools import groupby
from operator import itemgetter
query = (Event.objects.all.select_related('event_type')
.order_by('date', 'event_type')
.distinct('date', 'event_type')
.values_list('date', 'event_type__name'))
result = [
{ 'date': k, 'datetypes': [v[1] for v in vs]}
for k, vs in groupby(query, itemgetter(0))
]
You also better use 'event_type' in the order by criterion.
This will result in something like:
[{'date': datetime.date(2018, 5, 19), 'datetypes': ['Famous person died',
'Royal wedding']},
{'date': datetime.date(2018, 5, 24), 'datetypes': ['Famous person died']},
{'date': datetime.date(2011, 5, 25), 'datetypes': ['Important law enforced',
'Referendum']}]
(based on quick Wikipedia scan of the last days in May).
The groupby works in linear time with the number of rows returned.
I have a list of lists called people in python
people = [['10000', '2018-02-04', 'Park', 'Chan'], ['10047', '2018-05-09', 'Tuckwell', 'Luke'], ['10207', '2018-05-06', 'Trentham', 'Sam'], ['10207', '2018-05-06', 'Smith', 'Tristin'], ['10511', '2018-02-07', 'Cotton', 'Marco'], ['10763', '2018-03-07', 'Wideman', 'Jocelyn'], ['10804', '2018-05-09', 'Hamm', 'Megan']]
Each individual list contains an ID, Expiry Date, Last Name, First Name.
What is the best way to go about defining a function that searches 'people' for a desired ID input and returns whether the expiry date of that person is passed today's date or not?
Thanks!
If you can it is better to convert your list of lists to a dictionary where the key is the ID, this way the searches (when performed by ID) will be more efficient.
One way is to use a dictionary to structure your data, mapping id to detail. Another improvement is to store your dates as datetime objects.
from datetime import datetime
people = [['10000', '2018-02-04', 'Park', 'Chan'], ['10047', '2018-05-09', 'Tuckwell', 'Luke'],
['10207', '2018-05-06', 'Trentham', 'Sam'], ['10207', '2018-05-06', 'Smith', 'Tristin'],
['10511', '2018-02-07', 'Cotton', 'Marco'], ['10763', '2018-03-07', 'Wideman', 'Jocelyn'],
['10804', '2018-05-09', 'Hamm', 'Megan']]
# create a dictionary mapping, convert dates to datetime objects
d = {k: [datetime.strptime(v[0], '%Y-%m-%d'), v[1], v[2]] for k, *v in people}
# function to calculate whether date has passed for given id
def return_date_passed(d, i):
return d[i][0] < datetime.now()
res = return_date_passed(d, '10000') # True
res = return_date_passed(d, '10207') # False
Tldr of Problem
Frontend is a form that requires a complex lookup with ranges and stuff across several models, given in a dict. Best way to do it?
Explanation
From the view, I receive a dict of the following form (After being processed by something else):
{'h_index': {"min": 10,"max":20},
'rank' : "supreme_overlord",
'total_citations': {"min": 10,"max":400},
'year_began': {"min": 2000},
'year_end': {"max": 3000},
}
The keys are column names from different models (Right now, 2 separate models, Researcher and ResearchMetrics), and the values are the range / exact value that I want to query.
Example (Above)
Belonging to model Researcher :
rank
year_began
year_end
Belonging to model ResearchMetrics
total_citations
h_index
Researcher has a One to Many relationship with ResearchMetrics
Researcher has a Many to Many relationship with Journals (not mentioned in question)
Ideally: I want to show the researchers who fulfill all the criteria above in a list of list format.
Researcher ID, name, rank, year_began, year_end, total_citations, h_index
[[123, "Thomas", "professor", 2000, 2012, 15, 20],
[ 343 ... ]]
What's the best way to go about solving this problem? (Including changes to form, etc?) I'm not very familiar with the whole form query model thing.
Thank you for your help!
To dynamically perform a query you pass a dict with items 'fieldname__lookuptype': value as **kwargs to Model.objects.filter.
So to filter for rank, year_began and year_end in your example above, you would do this:
How exactly you do the transformation depends on how variable this incoming dictionary is. An example could be something like this:
filter_in = {
'h_index': {"min": 10,"max":20},
'rank' : "supreme_overlord",
'total_citations': {"min": 10,"max":400},
'year_began': {"min": 2000},
'year_end': {"max": 3000},
}
LOOKUP_MAPPING = {
'min': 'gt',
'max': 'lt'
}
filter_kwargs = {}
for field in RESEARCHER_FIELDS:
if not field in filter_in:
continue
filter = filter_in[field]
if isinstance(filter, dict):
for filter_type, value in filter.items():
lookup_type = LOOKUP_MAPPING[filter_type]
lookup = '%s__%s' % (field, lookup_type)
filter_dict[lookup] = value
else:
filter_dict[field] = filter
This results in a dictionary like this:
{
'rank': 'supreme_overlord',
'year_began__gt': 2000,
'year_end__lt': 3000
}
Use it like this:
qs = Researcher.objects.filter(**filter_kwargs)
Regarding the fields total_citations and h_index from ResearchMetrics, I assume you want to aggregate the values. So in your example above you want either a sum or an average.
The principle is the same:
from django.db.models import Sum
METRICS_FIELDS = ['total_citations', 'h_index']
annotate_kwargs = {}
for field in METRICS_FIELDS:
if not field in filter_in:
continue
annotated_field = '%s_sum' % field
annotate_kwargs[annotated_field] = Sum('researchmetric__%s' % field)
filter = filter_in[field]
if isinstance(filter, dict):
for filter_type, value in filter.items():
lookup_type = LOOKUP_MAPPING[filter_type]
lookup = '%s__%s' % (annotated_field, lookup_type)
filter_dict[lookup] = value
else:
filter_kwargs[field] = filter
Now your filter_kwargs look like this:
{
'h_index_sum__gt': 10,
'h_index_sum__lt': 20,
'rank': 'supreme_overlord',
'total_citations_sum__gt': 10,
'total_citations_sum__lt': 400,
'year_began__gt': 2000,
'year_end__lt': 3000
}
And your annotate_kwargs look like this:
{
'h_index_sum': Sum('reasearchmetric__h_index')),
'total_citations_sum': Sum('reasearchmetric__total_citations'))
}
So your final call looks like this:
Researcher.objects.annotate(**annotate_kwargs).filter(**filter_kwargs)
There are some assumptions in my answer, but I hope you get the general idea.
There is one important point: make sure you properly validate the input to make sure that only the field can be filtered that you want the user to filter. In my approach, this is ensured by hard coding the field names in RESEARCHER_FIELDS and METRICS_FIELDS.
I am new to python so please excuse any dumb mistakes but after research, I can't figure this out. I am creating a dictionary from a list of days in the month taken from calendar. I had originally used dict.fromkeys() but found this submission that convinced me to change to the dictionary comprehension statement I have. Then I give each value in the dictionary another dictionary that has the day of the week as the key and another dictionary as the value. This dictionary is taskDic which has chores as the keys and will hold people's names as the values.
My problem is that my last statement in my loops is assigning the same person to do trash (etc) for every day even though the loop is just on the first day. I believe there is something wrong with how I start the dictionary because it is assigning values to all the dics as if they are the same.
Basically I have the same problem as the linked issue above but with nested dictionaries. Please let me know if I need to clarify anything. Thank you!
import calendar
week = ['Monday', 'Tuesday', 'Wednesday', 'Thursday', 'Friday', 'Saturday', 'Sunday']
taskDic = {'Trash':[], 'Groceries':[], 'Clean':[]}
teamList = ['Jane', 'Peter', 'Jake', 'Eliza', 'Sarah', 'Bill']
person = 0
cal = list(calendar.Calendar().itermonthdays(2015, 8))
cal = {k: {} for k in cal}
for i in cal:
cal[i] = {week[i % 7]: taskDic}
for i in cal:
if (cal[i].keys() != 'Saturday') and (cal[i].keys() != 'Sunday'):
for j in cal[i]:
for k in cal[i][j]:
cal[i][j][k] = teamList[person % len(teamList)]
person += 1
My result looks like this:
0 {'Monday': {'Trash': 'Eliza', 'Groceries': 'Sarah', 'Clean': 'Bill'}}
1 {'Tuesday': {'Trash': 'Eliza', 'Groceries': 'Sarah', 'Clean': 'Bill'}}
2 {'Wednesday': {'Trash': 'Eliza', 'Groceries': 'Sarah', 'Clean': 'Bill'}}
3 {'Thursday': {'Trash': 'Eliza', 'Groceries': 'Sarah', 'Clean': 'Bill'}}
etc...
the problem is here:
for i in cal:
cal[i] = {week[i % 7]: taskDic}
you're using the same copy of taskDic every day.
as commented on by #jojonas, a fix is this:
cal[i] = {week[i % 7]: taskDic.copy()}
the only problem with this is that you'll end up creating a bunch of unused copies, which is fine for this task.
what you want to do is something like:
week_of_tasks = [taskDic.copy() for _ in week]
cal[i] = {week[i % 7]: week_of_tasks[i % 7]}