I have my reference data which looks like this.
I'm trying to create a calendar using weekdays and 5 time slots as shown below.
Using the calendar, I want to identify if there are any potential clashes between departments.
What I mean by that is:
On any particular day, an employee must only come in once if they work in the department (regardless of the time slot)
If there are two two departments working in a particular day, then an error message such as "Tax clashes with Legal" is returned.
As you can see from the reference data, employees work in multiple fields and I'm trying to minimise the amount of overlaps/clashes between the departments.
Desired results would look something like this:
Any help/guidance is appreciated!
Related
I would like to create a price table by date, I tried to google this for python and django, but still have no idea for this. I don't want to create one to one relationship object like an options. but I would like to create the database associating date and price. Sorry that it may be simple question..
Would it be solution to create a database by using PostgreSQL, and read by django? or any resource / reference can help get me in right direction to access this problem?
Thanks so much
Well there is more to it then assigning a price to a date. You will need one or more tables that hold the establishment(hotels) data. These would include the room information as all rooms will not have the same price. Also the price will probably change over time, so you will need to track that. Then there is the reservation information to track. This is just some of the basics. It is not a simple task by any means. I would try a simpler project to start with to learn Django and how to get data in and out of it.
I have a Quote table that generates a price based on lots of parameters. Price is generated based on data from multiple other tables like: Charges, Coupon, Promotions etc. The best way to deal with it is using ForeignKeys. Here's the table (all _id fields are foreign keys):
Quote
#user input fields
charge_id
promotion_id
coupon_id
tariff_id
Everything looks good. Each quote record has very detailed data that tells you where the price comes from. The problem is the data in the tables that it depends on isn't guaranteed to stay the same. It might get deleted or get changed. Let's say a Quote has a foreign key to a tariff and then it gets changed. The records associated with it no longer tell the same story. How do you deal with something like this? Would really appreciate if you recommend some theory related to this.
If you don't want your quote values to be changed by change in foreign key related objects, the best bet is to add all fields from individual foreign keys models in the Quote class.
Then while calculating the values of Quote, you fetch data from all the (now not) related objects and save them in the Quotes table.
Now any changes to Foreign tables will not affect the Quotes table.
Another option would be to use a library like django-simple-history and that keeps track of all changes to the models with change of time.
I know we can use stripe customer id to retrieve list of cards attached to the customer id. Also, I know we can retrieve ONE specific card with the card's id....But what if I do not want to use the card id to retrieve the card. Instead I want to use exp month, year, last4. Is this possible?
I thought of
all_cards = stripe.Customer.retrieve(cus_id).sources.all(object=CARD)
the_card = all_cards.all(
exp_month=data.get(EXP_MONTH),
exp_year=data.get(EXP_YEAR),
last4=data.get(LAST_4)
)
but it says no such parameters such as exp_month
I thought of doing it in a way to loop through all the card and match the parameters which myself believes would be a bad idea if there is a better one.
Does anyone have a better idea or suggestions?
Thanks in advance for any advise
According to stripe documentation: no. https://stripe.com/docs/api/python#retrieve_card
There is a simple reason for that: the data might not be unique. Your customer, by some weird luck might have two or more cards ending the same month and year(well, actually that's not that weird) and with the same last 4 digits.
If you are working with some legacy system and need to retrieve some old data about the cards from Stripe, I'd create a management command to just fill the blanks: get the data from Stripe and attach card_id to the objects you already have.
Ps. you might be interested in this library: https://github.com/ArabellaTech/aa-stripe/ - disclaimer: I am one of the authors.
I am trying to optimise database insertion of a large data set through django.
Here is a simple sample model that should be enough to demonstrate the issue I face.
class School(models.Model):
name = models.CharField(max_length=32)
postcode = models.CharField(max_length=8)
class Student(models.Model):
name = models.CharField(max_length=32)
school = models.ForeignKey(School)
last_updated = models.DateTimeField(blank=True, null=True)
first_registered = models.DateTimeField(blank=True, null=True)
Given the example model above. I can currently create rows in the following manner
school = app.models.School.objects.get_or_create(name='School 1', postcode='AB12 3CD')
student = app.models.Student.objects.create(name='Student 1', school=school[0])
This works well for the simple "getting started" case, but there are potential issues when expanding to a larger data set and more complex models with more relationships.
The school entry requires a database round trip every time.
Race conditions are possible.
I'll define a more real world problem.
There is a master list of school and student data
The students and school list is updated daily
The data comes in no particular order
Previous data may be present, but could also be removed
Previous data may also be updated (outside scope of this question)
If it is removed then that indicates the student is no longer a student
In essence, each day a list is available with the current snapshot of all students and the school they attend.
Given the above, it's not hard to imagine perhaps 10,000 schools with 5,000,000 students, perhaps billions if looking on a global scale.
Some observations of the data include, that approx 95% of it is duplicated daily. So this data can be filtered quite quickly in a simple query.
Currently I have the following code that works well for filtering existing data, and also conditionally updating some of the fields too in a single SQL statement.
# See if the student already exists and is assigned to the school in the data set.
# If it does exist, get the database to update the last updated date and the first registration dates if needs be.
#
# Note: These updates are necessary as it is possible that data is imported in non-chronological order.
updated_fields = app.models.Student.objects.filter(
name=data['student_name'],
school__name=data['school_name'],
school__postcode=data['school_postcode'],
).update(
last_updated=Case(
When(last_updated__lt=data['last_updated'], then=data['last_updated']),
default=F('last_updated')
),
first_registered=Case(
When(first_registered_gt=data['first_registered'], then=data['first_registered']),
default=F('first_registered')
)
)
The problem I face is when I get to insertion of new data I am getting stuck doing multiple database queries. There are far fewer schools than students and they are added at a slower rate.
Apart from the initial import, in all subsequent daily updates, 99.9% of the time the school data already exists in the database and the student just needs assigning.
Currently my importing literally works in the primitive way as follows and there are more FK lookups than shown, hence I am keen to reduce database round trips.
for entry in data:
school = app.models.School.objects.get_or_create(name=entry['school_name'], postcode=entry['school_postcode'])
student = app.models.Student.objects.create(name=data['student_name'], school=school[0])
What I would like to do is move that get_or_create into the create statement, and have it create it in the database if it doesn't exist at the point of insertion. I feel of the right way to do this is to let the database do all the work.
The idea is to pass all the Student and School information upon each insertion. The insert query should then try and lookup the school information and assign that entry to the FK for Student.
If that returns no result, the school should be created and then assigned the resulting FK.
I have found a similar question for SQL. How to insert records to SQL with looked up values?
I think that is what I want, but with one key difference, my query should create the entry in the case of lookup failure
I was wondering if anyone has any idea how to do this in django, perhaps using functionality such as WHERE, CASE, etc..?
If the it is not possible through the ORM, some help via RAW SQL would be welcome.
FYI I am currently running django 1.10 and upgrading is not an issue. I am using sqlite backend (simply for ease rather than preference or features)
Many thanks,
Andy
In my App Engine datastore I've got an entity type which may hold a large number of entities, each of which will have the property 'customer_id'. For example, lets say a given customer_id has 10,000 entities, and there are 50,000 customer_ids.
I'm trying to filter this effectively, so that a user could get information for at least 2000 customer_ids at one time. That is, read them out to the UI within the 30 second time out limit (further filtering will be done at the front end, so the user isn't bombarded with all results at once).
Below I've listed a view of my current datastore models. 'Reports' refer to sets of customer_ids, so continuing the above example, I could get my 2000 customer_ids from ReportCids.
class Users(db.Model):
user = db.StringProperty()
report_keys_list = db.ListProperty(db.Key)
class Reports(db.Model):
#report_key
report_name = db.StringProperty()
class ReportCids(db.Model):
report_key_reference = db.ReferenceProperty(Reports, collection_name="report_cid_set")
customer_id = db.IntegerProperty()
start_timestamp = db.IntegerProperty()
end_timestamp = db.IntegerProperty()
class CustomerEvent(db.Model):
customer_id = db.IntegerProperty()
timestamp = db.IntegerProperty()
event_type = db.IntegerProperty()
The options I considered:
-Perform a separate query for each customer_in in my set of 2000
-Use lists of keys indicating customer events, but this is limited to 5000 entries in a list (so I've read)
-Get all entries, and filter in my code
I'd really appreciate if anyone had some advice on how to do this in the most efficient way, or if I'm approaching the problem in completely the wrong way. I'm a novice in terms of using the datastore effectively.
Of course happy to provide any clarification or information if it helps.
Thanks very much!
Thanks for getting back to me. It looks like I had an issue with the account used when I posted so I need to respond in the comments here.
Having thought about it and from what you've said, fetching that many results is not going to work.
Here's what I'm trying to do:
I'm trying to make a report that shows for multiple Customer IDs the events that happened for that group of customers. So lets say I have a report to view information for 2000 customers. I want to be able to get all events (CustomerEvent) and then filter this by event_type. I'm likely asking a lot here, but what I was hoping to do was get all of these events for 2000 customers, and then do the event_type filtering at the front end, so that a user could dynamically adjust the event_type they want to look for and get some information on successful actions for that event type.
So my main problem is getting the right entities out of CustomerEvent effectively.
Right now, I'm grabbing a list of Customer IDs like this:
cid_list = []
this_report = models.Reports.get(report_key)
if this_report.report_cid_set:
for entity in this_report.report_cid_set:
cid_list.append(entity.customer_id)
My estimation of 10,000 CustomerEvent entities was quite high, but theoretically it could happen. Perhaps when I go to get the report results, I could filter straight away by the event_type specified by the user. This means that I have to go back to the datastore each time they choose a new option which isn't ideal, but perhaps it's my only option given this set up.
Thanks very much for taking the time to look at this!