Django/Python: how to analyzing financial stock data (daily/monthly etc.) - python

I would like to do a lot of analysis/performance/statistics on my stock portfolio, which I plan to track with my app. E.g.:
Week performance
Month performance
Year performance
Best performer
and a lot of other things I can't imagine right now...
Where I'm struggling right now:
- What is a good/the best way to archive this? - Also to show this info on a dashboard
--> I think I should store this information somehow... But how to do this on a daily/weekly/whatever basis?
--> I don't see a way to do such things while runtime?
--> furthermore I need to know, when do do such thinks... It's end of the week so do weekly performance calculations...
--> Maybe there is also an MVP solution, that can evolve into a top-notch solution?
My models are locking like this at the moment - The position is the clamp around my orders and my EOD data. Right now I'm working with yfinance to get range financial data + finnhub API to get real time prices:
class Position(models.Model):
name = models.CharField(max_length=100)
shares = models.FloatField(default=0.0)
symbol = models.CharField(max_length=100, default="")
transaction_fee_sum = models.FloatField(default=0.0)
profit = models.FloatField(default=0.0)
average_price = models.FloatField(default=0.0)
cost_value = models.FloatField(default=0.0)
last_price = models.FloatField(default=0.0)
position_value = models.FloatField(default=0.0)
last_update = models.DateTimeField(default=datetime.now(), blank=True)
position_started = models.DateTimeField(default=datetime.now(), blank=True)
position_ended = models.DateTimeField(default=datetime.now(), blank=True)
isin = models.CharField(max_length=12)
depot = models.ForeignKey("Depot", blank=True, null=True, on_delete=models.SET_NULL)
def __str__(self):
return self.name
class PriceHistoryEOD(models.Model):
price = models.DecimalField(max_digits=8, decimal_places=2)
position = models.ForeignKey("Position", blank=True, null=True, on_delete=models.CASCADE)
price_date = models.DateField(default=date.today)
class Inventory_Position(models.Model):
shares = models.FloatField(default=0.0)
price = models.FloatField(default=0.0)
position = models.ForeignKey("Position", blank=True, null=True, on_delete=models.CASCADE)
def __str__(self):
return str(self.position.name) + "_Inventory_" + str(self.id)
class Order(models.Model):
ORDER_PLACES = (
('TRADEGATE', 'Tradegate'),
('FRANKFURT', 'Frankfurt'),
('STUTTGART', 'Stuttgart'),
)
ORDER_TYPE = (
('buy', 'buy'),
('sell', 'sell'),
)
name = models.CharField(max_length=100)
depot = models.ForeignKey("Depot", blank=True, null=True, on_delete=models.SET_NULL)
position = models.ForeignKey("Position", blank=True, null=True, on_delete=models.SET_NULL)
ko_identifier = models.BooleanField()
order_price = models.FloatField()
quantity = models.FloatField()
order_value = models.FloatField()
order_fees = models.DecimalField(decimal_places=2,max_digits=5,default=0.0, null=True)
order_date = models.DateTimeField('order date')
order_wkn = models.CharField(max_length=6)
order_isin = models.CharField(max_length=12)
order_TYPE = models.CharField(max_length=100, choices=ORDER_TYPE)
order_place = models.CharField(max_length=100, choices=ORDER_PLACES)
def __str__(self):
return self.name

I think the easiest way is to create a simple script for each type of analysis you need to do (weekly, monthly...) and then save the values to an sqlite3 DB. You can then import it into your django View as the data context. If you'd like to automate the whole process you can just set up a cronjob through crontab if you're using Linux, and if you're on Windows you can use the Windows task scheduler
Sqlite3 and python: How to work with sqlite3 and Python
If you are not familiar with crontab you can check out one of the many video on Youtube. As for the schedule expressions you can use this website, it makes it so much easier to find the expression you're looking for: https://crontab.guru/
finally one more tip, if you need to get stock prices and you feel like yfinance is kind of slow, you might want to try the Alpaca-trade-API (free API-KEY): https://github.com/alpacahq/alpaca-trade-api-python

Related

How do I filter django models based on product fields?

I am trying to run a filter command, using related fields; and am unsure how to go about it:
class Listing(models.Model):
name = models.CharField(max_length=150)
slug = models.SlugField()
description = models.TextField()
catchphrase = models.TextField(null=True, blank=True)
picture_0 = models.ImageField(upload_to = "mainimages")
picture_1 = models.ImageField(null=True, blank=True, upload_to = "./product/static/product/imgs")
picture_2 = models.ImageField(null=True, blank=True, upload_to = "./product/static/product/imgs")
short_term_price = models.IntegerField(default=0)
long_term_price = models.IntegerField(default=0)
tax = models.IntegerField(default=0)
tag = models.ForeignKey('Tag', on_delete=models.PROTECT)
def __str__(self):
return self.name
class Order(models.Model):
listing = models.ForeignKey(Listing, on_delete=models.PROTECT)
customer = models.ForeignKey(Customer, on_delete=models.PROTECT)
lease_date = models.DateField()
clear_date = models.DateField()
price = models.IntegerField(default=0) #cents
def __str__(self):
return self.listing.name
def get_display_price(self):
return "{0:.2f}".format(self.price / 100)
The general idea is that the user provides a start date and an end date and Django only returns the listings that aren't already in an order in that timeframe. I am unsure how to go about the view function:
def search_products(request, start_date, end_date):
listing = Listing.objects.select_related('order').all()
I will provide an answer as if you are using the lease_date to do the filtering. There is a couple of ways to achieve this. One is:
listing_qs = Listing.objects.filter(
pk__in=Order.objects.exclude(lease_date__range(start_date,end_date)).select_related('listing').values_list('listing__pk')
)
Code breakdown:
retrieve the orders by excluding those whose lease date is in between the provided timeframe
selecting the listing's pk (via values('listing__pk')) you can select any other attribute you want
using the result of the 2 previous instructions to get the Listing objects since we have the list of pk.
Another way:
Just exclude all the Listing objects whose lease date is in between the provided timeframe
Listing.objects.exclude(order_set__lease_date__range=(start_date,end_date))
I hope this helps.

How to insert JSON data to database in Django?

I've built a scraper that gets product data from different shopping websites.
When I run python scraper.py the program will print a JSON object containing all the data like this:
{ 'ebay': [ { 'advertiser': 'ebay',
'advertiser_url': 'https://rover.ebay.com/rover/1/711-53200-19255-0/1?ff3=2&toolid=10041&campid=5338482617&customid=&lgeo=1&vectorid=229466&item=302847614914',
'description': '30-Day Warranty - Free Charger & Cable - '
'Easy Returns!',
'main_image': 'https://thumbs1.ebaystatic.com/pict/04040_0.jpg',
'price': '290.0',
'title': 'Apple iPhone 8 Plus Smartphone AT&T Sprint '
'T-Mobile Verizon or Unlocked 4G LTE'}
]}
I want this data to be added to the database automatically every time I run the scraper.
Here's my database structure:
models.py
class Product(models.Model):
similarity_id = models.CharField(max_length=255, blank=True, null=True)
name = models.CharField(max_length=255, blank=True, null=True)
url = models.SlugField(blank=True, unique=True, allow_unicode=True)
advertiser_url = models.TextField(blank=True, null=True)
main_image = models.TextField(blank=True, null=True)
second_image = models.TextField(blank=True, null=True)
third_image = models.TextField(blank=True, null=True)
old_price = models.FloatField(default=0.00)
price = models.FloatField(default=0.00)
discount = models.FloatField(default=0.00)
currency = models.CharField(max_length=255, default="$")
description = models.TextField(blank=True, null=True)
keywords = models.CharField(max_length=255, blank=True, null=True)
asin = models.CharField(max_length=80, blank=True, null=True)
iban = models.CharField(max_length=255, blank=True, null=True)
sku = models.CharField(max_length=255, blank=True, null=True)
seller = models.CharField(max_length=255, blank=True, null=True)
free_shipping = models.BooleanField(default=False)
in_stock = models.BooleanField(default=True)
sold_items = models.IntegerField(default=0)
likes_count = models.IntegerField(default=0)
category = models.CharField(max_length=255, blank=True, null=True)
sub_category = models.CharField(max_length=255, blank=True, null=True)
reviews_count = models.IntegerField(default=0)
rating = models.FloatField(default=0)
active = models.BooleanField(default=True)
is_prime = models.BooleanField(default=False)
created_on = models.DateTimeField(auto_now_add=True)
advertiser = models.CharField(max_length=255, blank=True, null=True)
objects = ProductManager()
class Meta:
verbose_name_plural = "products"
def __str__(self):
return self.name
Add this to scrapper.py:
import path.to.model
product = Product()
product.<key> = <value> #Where key is the field and value is the value you need to fill
and after you assign every field, add
product.save()
Trick
If all the keys in the json response match the fields in the model, you can do:
for k, v in response.items():
setattr(product, k, v)
product.save()
That will save you a lot of lines and time :)
I work with json a lot; I have API caching where I receive a lot of json-based API data and I want to store it in a database for querying and caching. If you use postgres (for instance), you will see that if has extensions for json. This means that you can save json data in a special json field. But better, there are sql extensions that let you run queries on the json data. That is, postgres has "no sql" capabilities. This lets you work with json natively. I find it very compelling and I recommend it highly. It is a learning curve because it uses non-traditional sql, but heck, we have stackoverflow.
see: https://django-postgres-extensions.readthedocs.io/en/latest/json.html
here is a little example:
product_onhand_rows = DearCache.objects.filter(
object_type=DearObjectType.PRODUCT_AVAILABILITY.value).filter(
dear_account_id=self.dear_api.account_id).filter(jdata__Location=warehouse).filter(jdata__SKU=sku).all()
in this example, I have the json stored in a field jdata.
jdata__Location accesses the key Location in the json.
It nests and so on. For advanced queries, I resort to sql
select object_type,last_modified, jdata
from cached_dear_dearcache
where object_type = 'orders'
and jdata->>'Status' in ('ESTIMATING','ESTIMATED')
order by last_modified;
and there's more, you can 'unroll' lists (this is what I would call a complicated example, my json has lists of invoices, each of which has a list of lines...)
/* 1. listing invoice lines. We have to iterate over the array of invoices to get each invoice, and then inside the invoice object find the array of lines */
select object_type,last_modified, jsonb_array_elements(jsonb_array_elements(cached_dear_dearcache.jdata#>'{Invoices}')->'Lines') as lines,
jsonb_array_elements(cached_dear_dearcache.jdata#>'{Invoices}')->'InvoiceDate' as invoice_date,
jsonb_array_elements(cached_dear_dearcache.jdata#>'{Invoices}')->'InvoiceNumber' as invoice_number
from cached_dear_dearcache
where object_type = 'orders' order by last_modified;
Your approach is to convert the json data to a traditional sql model. That will work too. It's not very flexible ... if the json "schema" changes, your database schema may need to change. Philosophically, I think it is better to go with the flow, and use the json extensions, this is the best of both worlds. Performance is good, by the way.

I want to make a changing "array" of color options and size options per item in the admin panel

Title and code. I'm working on an E-Commerce site where I will have multiple options per item (color/size). In the code you can see price, price(1-3), the idea is to create a button or option for the admin to add additional sizes and therefor prices without hard coding sizename(1-4) and the same goes for colorName(1-4). I hope that you can understand what I'm going for, I just started learning python and all of this bootstrap and django stuff yesterday. BTW If you know of an easier way to handle all of this, please do let me know, I'm lost as to how to make a shopping cart.
I just started this
class Product(models.Model):
name = models.CharField(max_length=255)
price = models.DecimalField(decimal_places=2, max_digits=20)
stock = models.IntegerField()
image_url = models.CharField(max_length=2083)
category = models.CharField(choices=MY_CHOICES, max_length=25, default='Default')
multSizes = models.BooleanField(default=False)
numberSizes = models.IntegerField(default=1)
price1 = models.DecimalField(decimal_places=2, max_digits=20, default=0)
price2 = models.DecimalField(decimal_places=2, max_digits=20, default=0)
price3 = models.DecimalField(decimal_places=2, max_digits=20, default=0)
sizename1 = models.CharField(max_length=255, default='N/A')
sizename2 = models.CharField(max_length=255, default='N/A')
sizename3 = models.CharField(max_length=255, default='N/A')
sizename4 = models.CharField(max_length=255, default='N/A')
numberColors = models.IntegerField(default=1)
colorName = models.CharField(max_length=255, default='N/A')
colorName1 = models.CharField(max_length=255, default='N/A')
colorName2 = models.CharField(max_length=255, default='N/A')
colorName3 = models.CharField(max_length=255, default='N/A')
colorName4 = models.CharField(max_length=255, default='N/A')
I want to be able to change the amount of different sizes/prices and the amount of colors through the django admin panel. I do not want to hard code the fields as I have there right now.
If you simply have a single "color" & "size" field for the Product model, you could create multiple instances of the model with different properties.
class Product(models.Model):
name = models.CharField(max_length=255)
price = models.DecimalField(decimal_places=2, max_digits=20)
stock = models.IntegerField()
image_url = models.CharField(max_length=2083)
category = models.CharField(choices=MY_CHOICES, max_length=25, default='Default')
sizename = models.CharField(max_length=255, default='N/A')
colorName = models.CharField(max_length=255, default='N/A')
>>> p1 = Product(name="shirt", price=10, sizename="M", colorname="red")
>>> p2 = Product(name="pants", price=20, sizename="M", colorname="yellow")
However, I think the issue that you are getting at is that for a product with a single set of some properties-- name, price, stock, image_url, and category-- there are multiple sizes and colors. Is that correct?
If so, you could still create multiple instances that share those properties:
>>> p1 = Product(name="shirt", price=10, sizename="M", colorname="red")
>>> p2 = Product(name="shirt", price=10, sizename="L", colorname="yellow")
However, in database design, this is not considered "normal form." See: Second Normal Form (Wikipedia).
A better design might be to pull those attributes into a separate table, ie, a second Django model, that references the first model with a foreign key. Something like this:
class Product(models.Model):
name = models.CharField(max_length=255)
price = models.DecimalField(decimal_places=2, max_digits=20)
stock = models.IntegerField()
image_url = models.CharField(max_length=2083)
category = models.CharField(choices=MY_CHOICES, max_length=25, default='Default')
class ProductVariant(models.Model):
product = models.ForeignKey(Product, on_delete=models.CASCADE)
sizename = models.CharField(max_length=255, default='N/A')
colorName = models.CharField(max_length=255, default='N/A')
For more info, see the Django docs on many-to-one relationships.

Django Query a query?

Trying to speed up the performance of my app, and post some analysis with debug toolbar I can see I am doing 68 queries.
I query the circuits for every showroom (68 of), I thought if I just query the circuits once, could I then requery the existing query instead of calling the DB again for each Circuit?
something like adding:
crData = Circuits.objects.only('circuit_preference','site_data__id')
then how to query crData again? to match each statement?
Current code below
# get shworoom data
srData = SiteData.objects.only('location','subnet').filter(is_live=True).exclude(site_type='Major Site')
for sr in srData:
site = SiteType()
site.type = checkRoute(sr.subnet)
site.location = sr.location
if 'MPLS' in site.type:
mpls = site.type.split('-')
try:
d = Circuits.objects.only('circuit_preference','site_data').filter(site_data__id=sr.id,provider=mpls[0],circuit_type__icontains=mpls[1])
site.preference = d[0].circuit_preference
except:
pass
elif site.type == '4G':
try:
d = Circuits.objects.only('circuit_preference','site_data').filter(site_data__id=sr.id,provider='EE',circuit_type__icontains=site.type)
site.preference = d[0].circuit_preference
except:
pass
elif site.type == 'DSL' or site.type == 'FIBRE':
try:
d = Circuits.objects.only('circuit_preference','site_data').filter(site_data__id=sr.id,circuit_type__icontains=site.type)
site.preference = d[0].circuit_preference
except:
pass
**EDIT: models below
class SiteData(models.Model):
location = models.CharField(max_length=50)
site_type = models.CharField(max_length=20, verbose_name="Site Type", \
choices=settings.SITE_TYPE)
subnet = models.GenericIPAddressField(protocol='IPv4')
bgp_as = models.CharField(max_length=6, verbose_name="BGP AS Number")
opening_date = models.DateField(verbose_name="Showroom opening date")
last_hw_refresh_date = models.DateField(verbose_name="Date of latest hardware refresh", \
blank=True, null=True)
is_live = models.BooleanField(default=False, verbose_name="Is this a live site?")
tel = models.CharField(max_length=20, blank=True, null=True)
notes = models.TextField(blank=True)
class Meta:
verbose_name = "Site Data"
verbose_name_plural = "Site Data"
ordering = ('location',)
def __unicode__(self):
return self.location
class Circuits(models.Model):
site_data = models.ForeignKey(SiteData, verbose_name="Site", on_delete=models.PROTECT)
order_no = models.CharField(max_length=200, verbose_name="Order No")
install_date = models.DateField(blank=True, null=True)
circuit_type = models.CharField(max_length=100, choices=settings.CIRCUIT_CHOICES)
circuit_preference = models.CharField(max_length=20, verbose_name="Circuit Preference", \
choices=settings.CIRCUIT_PREFERENCE, blank=True, null=True)
circuit_speed_down = models.DecimalField(max_digits=10, decimal_places=1, blank=True, null=True)
circuit_speed_up = models.DecimalField(max_digits=10, decimal_places=1, blank=True, null=True)
circuit_bearer = models.IntegerField(blank=True, null=True)
provider = models.CharField(max_length=200, choices=settings.PROVIDER_CHOICES)
ref_no = models.CharField(max_length=200, verbose_name="Reference No")
cost_per_month = models.DecimalField(decimal_places=2, max_digits=8)
contract_length = models.IntegerField(verbose_name="Contact length in years")
service_contacts = models.ForeignKey(ServiceContacts, on_delete=models.PROTECT)
subnet = models.GenericIPAddressField(protocol='IPv4', verbose_name="Subnet", \
blank=True, null=True)
default_gateway = models.GenericIPAddressField(protocol='IPv4', \
verbose_name="Default Gateway", blank=True, null=True)
subnet_mask = models.CharField(max_length=4, verbose_name="Subnet Mask", \
choices=settings.SUBNET_MASK_CHOICES, blank=True, null=True)
internet_device = models.ForeignKey(ConfigTemplates, \
verbose_name="is this circuit the active internet line for a device?", \
default=6, on_delete=models.PROTECT)
decommissioned = models.BooleanField(default=False, verbose_name="Decomission this circuit?")
I was going to point you toward Pickling, but I suppose that doesn't make sense unless you need to cache the querysets to re-use in another location.
Actually I'm pretty sure querysets are pretty good for only hitting the database when they need to, which is when they're first evaluated. However, I think redeclaring the queryset would cause it to be re-evaluated, but if you create a list/dictionary of querysets I imagine you should be able to just re-use them without hitting the database again (unless you need to run new filters on them.) So I don't think you have much choice than to hit the database for each time you fetch a crData queryset, but you should at least be able to store the querysets and reuse them without it hitting the database for each one again.
Something like this should work I think. Would love to know if I'm wrong.
crData = []
for sr in srData:
# ...
crData.append(d)
for cr in crData:
# Do stuff
EDIT: Here's another relevant question: Django ORM and hitting DB

What is the recommended schema design for dynamic dates in Django for PostgreSQL?

we have a Django app focusing on timeline evolution visualization. There we have conceptually the relationship of:
1 Item with 1 or more Lifecycles (more for versioning purposes)
1 Lifecycle has 0..n Milestones
1 Milestone is a date stored as a string in form YYYY-MM-DD or a special tag "today", which means daily changing date (dynamic - the date was not stated, but until today is some state valid - if today is smaller then the next milestone).
The character of the data is that there are very diverse interpretations of milestones and phases in between them. Also the amount of milestones is diverse. However there seam to be used a maximum of 7 milestones. The characteristics of lifecycle records can be grouped (same amount of milestones with the same meanings).
We are using Django on top of PostgreSQL with model schema like this:
class Item(models.Model):
... other attributes
lifecycle_actual = models.IntegerField(null=True, default=-1, help_text="Selectable actual roadmap. Can be used to override the imported data. Use the ID of particular roadmap or -1 for the latest import.")
class Lifecycle(models.Model):
... other attributes
lifecycle_group = models.ForeignKey(LifecycleGroup, help_text="Vizualization group.")
date0 = models.CharField(max_length=10, blank=True)
date1 = models.CharField(max_length=10, blank=True)
date2 = models.CharField(max_length=10, blank=True)
date3 = models.CharField(max_length=10, blank=True)
date4 = models.CharField(max_length=10, blank=True)
date5 = models.CharField(max_length=10, blank=True)
date6 = models.CharField(max_length=10, blank=True)
item = models.ForeignKey(Item, null=True, blank=True)
def __unicode__(self):
return self.item.fullname
class LifecycleGroup(models.Model):
name = models.CharField(max_length=220, help_text="Name of the group")
era0_name = models.CharField(max_length=100, blank=True)
era1_name = models.CharField(max_length=100, blank=True)
era2_name = models.CharField(max_length=100, blank=True)
era3_name = models.CharField(max_length=100, blank=True)
era4_name = models.CharField(max_length=100, blank=True)
era5_name = models.CharField(max_length=100, blank=True)
era6_name = models.CharField(max_length=100, blank=True)
era0_start_name = models.CharField(max_length=100, blank=True)
era1_start_name = models.CharField(max_length=100, blank=True)
era2_start_name = models.CharField(max_length=100, blank=True)
era3_start_name = models.CharField(max_length=100, blank=True)
era4_start_name = models.CharField(max_length=100, blank=True)
era5_start_name = models.CharField(max_length=100, blank=True)
era6_start_name = models.CharField(max_length=100, blank=True)
era0_css_classes = models.CharField(max_length=150, blank=True)
era1_css_classes = models.CharField(max_length=151, blank=True)
era2_css_classes = models.CharField(max_length=152, blank=True)
era3_css_classes = models.CharField(max_length=153, blank=True)
era4_css_classes = models.CharField(max_length=154, blank=True)
era5_css_classes = models.CharField(max_length=155, blank=True)
era6_css_classes = models.CharField(max_length=156, blank=True)
def __unicode__(self):
return self.name
Overall it works fine, however we have issues with reporting questions such as:
Which items will hit milestones of certain characteristics in December 2015?
Even if we changed the model code to this:
class Item(models.Model):
... other attributes
lifecycle_actual = models.IntegerField(null=True, default=-1, help_text="Selectable actual roadmap. Can be used to override the imported data. Use the ID of particular roadmap or -1 for the latest import.")
class Lifecycle(models.Model):
... other attributes
# lifecycle group - not used anymore - have to duplicate info somehow in milestones
# lifecycle_group = models.ForeignKey(LifecycleGroup, help_text="Vizualization group.")
item = models.ForeignKey(Item, null=True, blank=True)
def __unicode__(self):
return self.item.fullname
class Milestone(models.Model):
lifecycle = models.ForeignKey(Lifecycle, null=True, blank=True)
date = models.CharField(max_length=10, blank=True)
name = models.CharField(max_length=100, blank=True)
next_era = models.ForeignKey(Era, null=True, blank=True)
impact = ... cca 4 choices
order = models.PositiveIntegerField()
class Era(models.Model):
name = models.CharField(max_length=100, blank=True)
css_classes = models.CharField(max_length=150, blank=True)
We got still several problems:
we would have to always join milestones under lifecycle for every vizualization query we have (seams to be contradictory to this normalization)
What is the recommended schema design for such needs?
dynamic date "today" in Milestone date field
How to store dynamic(changing) date in the DB, so it would become valid for SELECTS and comparable with stored static dates?
So we can do:
SELECT * FROM item, lifecycle, milestone
WHERE item.id = lifecycle.item AND milestone.lifecycle = lifecycle.id
AND milestone.impact = 'huge'
AND milestone.date between '2015-12-01' AND '2015-12-31'
We would like to enhance the "today" control string
So we can store milestone definition like this:
"today +365d" or "today -20d", resp. “YYYY-MM-DD<today<YYYY-MM-DD”.
Thanks in advance for any comments, suggestions!
EDIT
Imagine data like this:
(item lifecycle => milestone name: date, ...)
item1 => born: 2011-12-02,
decline: 2015-06-01,
end of life:2017-06-01
item2 => lifecycle check: 2015-08-01,
some significant milestone: 2017-09-01,
depreciation ends: 2019-04-15,
to be decommissioned: 2022-04-01
item3 => initiated: 2012-05-08,
life until at least: *today*,
end of life: not declared
item4 => initiated: 2012-05-08,
productive life until at least: *today +2 years*,
end of life: 2032-08-01
item5 => born: unknown but latest *today*,
end of life:2017-06-01
Where today is the ongoing date, i.e. the every current date in the future when user uses the data.
Let's assume we should select all items, which have any milestone between 2015-10-01 and 2015-12-01. If we run the SELECT today (2015-10-29) the item3 and item5 should be in the output. If we run that SELECT on 2015-12-15 the item3 and item5 must not be in the output.
You should use models.DateTimeField(default=timezone.now) in your dates and use a models.BooleanField to define TODAY behavior milestone.
I guess that is better:
class Milestone(models.Model):
lifecycle = models.ForeignKey(Lifecycle, null=True, blank=True)
date = models.DateTimeField(max_length=10, blank=True)
today = models.BooleanField(default=False)
name = models.CharField(max_length=100, blank=True)
next_era = models.ForeignKey(Era, null=True, blank=True)
Seconding arkadyzalko's recommendation on DateTimeField but would note a few additional things.
First I would recommend reading this documentation and focus on range types. If each era hits a range (you known in advance when the era will end) then it becomes easy to add indexes to determine what is in an era -- i.e. thequestion is whether the date falls within a range and you can join on that as well.
So from a database design perspective, I would look at
using a range type for era boundaries
using an exclusion constraint to ensure they do not overlap
Joining on the overlap between the date of the event and the era.
Django should support all of these (though the exclusion constraint you might have to do yourself).
As an example of some daterange queries:
test=# select '[2011-01-01,2011-02-01)'::daterange #> '2011-01-15'::date;
?column?
----------
t
(1 row)
test=# select '[2011-01-01,2011-02-01)'::daterange #> '2011-01-1'::date;
?column?
----------
t
(1 row)
test=# select '[2011-01-01,2011-02-01)'::daterange #> '2011-02-1'::date;
?column?
----------
f
(1 row)
But this means you can join on a value being in a range, s
FROM dates JOIN epoch ON epoch.range #> dates.date
GiST indexes also allow you to do this with index lookups.

Categories

Resources