I have a page that is meant to show company financials based on customers input(they input which company they're looking for). Once they submit, in the view function I want to create 5 API urls which then get the json data, create a list with the dates from 1 API result (they will all contain the same dates, so I will use the same list for all), then create new dictionaries with specific data from each api call, as well as the list of dates. I then want to create dataframes for each dictionary, then render each as html to the template.
My first attempt at this I attempted to do all requests.get and jsons.load calls one after another in a try block within " if request.method == 'POST' " block. This worked well when only grabbing data from one API call, but did not work with 5. I would get the local variable referenced before assigned error, which makes me think either the multiple requests.get or json.loads was creating the error.
My current attempt(which was created out of curiosity to see if it worked this way) does work as expected, but is obv not correct as it is calling the API multiple times in the for loop, as shown. (I have taken out some code for simplicity)
def get_financials(request, *args, **kwargs):
pd.options.display.float_format = '{:,.0f}'.format
IS_financials = {} #Income statement dictionary
BS_financials = {} #Balance sheet dictionary
dates = []
if request.method == 'POST':
ticker = request.POST['ticker']
IS_Url = APIURL1
BS_URL = APIURL2
try:
IS_r = requests.get(IS_Url)
IS = json.loads(IS_r.content)
for year in IS:
y = year['date']
dates.append(y)
for item in range(len(dates)):
IS_financials[dates[item]] = {}
IS_financials[dates[item]]['Revenue'] = IS[item]['revenue'] / thousands
IS_financials[dates[item]]["Cost of Revenue"] = IS[item]['costOfRevenue'] / thousands
IS_fundementals = pd.DataFrame.from_dict(IS_financials, orient="columns")
for item in range(len(dates)):
BS_r = requests.get(BS_URL)
BS = json.loads(BS_r.content)
BS_financials[dates[item]] = {}
BS_financials[dates[item]]['Cash and Equivalents'] = BS[item]['cashAndCashEquivalents'] / thousands
BS_financials[dates[item]]['Short Term Investments'] = BS[item]['shortTermInvestments'] / thousands
BS_fundementals = pd.DataFrame.from_dict(BS_financials, orient="columns")
except Exception as e:
apiList = "Error..."
return render(request, 'financials.html', {'IS': IS_fundementals.to_html(), 'BS': BS_fundementals.to_html()})
else:
return render(request, 'financials.html', {})
I'm trying to think of the proper way to do this. I'm new to django/python and not quite sure the best practice for a problem like this would be. I thought about making separate functions for each API, but then I would be unable to render them all on the same page. Can I use nested functions? Where only the main function renders to template, and all inner functions simply return the dataframe to outer function? Would class based views be better for something like this? I have never worked with class based views yet so would be a bit of a learning curve.
Another question I have is how to change the html in the table that is rendered from dataframe? The table/font that is currently rendered is quite large.
Thanks for any tips/advice!
It's not common to use pandas only for it's .to_html() method, but I have invoked pandas in a django method for less.
A more common approach is to loop over the IS and BS objects using django template's loop methods to generate the html tables.
To make this method more efficient move the BS api call out of the date loop, As long as the API call is not changed by the date.
Reasonable timeouts on the api calls would help also.
def get_financials(request, *args, **kwargs):
pd.options.display.float_format = '{:,.0f}'.format
IS_financials = {} #Income statement dictionary
BS_financials = {} #Balance sheet dictionary
dates = []
if request.method == 'POST':
ticker = request.POST['ticker']
IS_Url = APIURL1
BS_URL = APIURL2
try:
IS_r = requests.get(IS_Url, timeout=10)
IS = json.loads(IS_r.content)
BS_r = requests.get(BS_URL, timeout=10)
BS = json.loads(BS_r.content)
for year in IS:
y = year['date']
dates.append(y)
for item in range(len(dates)):
IS_financials[dates[item]] = {}
IS_financials[dates[item]]['Revenue'] = IS[item]['revenue'] / thousands
IS_financials[dates[item]]["Cost of Revenue"] = IS[item]['costOfRevenue'] / thousands
IS_fundementals = pd.DataFrame.from_dict(IS_financials, orient="columns")
for item in range(len(dates)):
BS_financials[dates[item]] = {}
BS_financials[dates[item]]['Cash and Equivalents'] = BS[item]['cashAndCashEquivalents'] / thousands
BS_financials[dates[item]]['Short Term Investments'] = BS[item]['shortTermInvestments'] / thousands
BS_fundementals = pd.DataFrame.from_dict(BS_financials, orient="columns")
except Exception as e:
apiList = "Error..."
return render(request, 'financials.html', {'IS': IS_fundementals.to_html(), 'BS': BS_fundementals.to_html()})
else:
return render(request, 'financials.html', {})
Related
I have a pretty reasonable use case: Multiple possible filter_by matches for a single column. Basically, a multiselect JS dropdown on front end posts multiple company industries to the backend. I need to know how to write the SQLAlchemy query and am surprised at how I couldn't find it.
{ filters: { type: "Industry", minmax: false, value: ["Financial Services", "Biotechnology"] } }
#app.route("/dev/api/saved/symbols", methods=["POST"])
#cross_origin(origin="*")
def get_saved_symbols():
req = request.get_json()
# res = None
# if "minmax" in req["filters"]:
# idx = req["filters"].index("minmax")
# if req["filters"][idx] == "min":
# res = db.session.query.filter(Company[req["filter"]["type"]] >= req["filters"]["value"])
# else:
# res = db.session.query.filter(Company[req["filter"]["type"]] <= req["filters"]["value"])
# else:
res = db.session.query.filter_by(Company[req["filters"]["type"]] == req["filters"]["value"])
return jsonify(res)
As you can see I am also working on a minmax which is like an above or below filter for other columns like price or market cap. However, the multiselect OR dynamic statement is really what I am stuck on...
I ended up creating a separate filter function for this that I can than loop over results with.
I will just show the first case for brevity. I am sending a list of strings in which I create a list of filters and then use the or_ operator imported from sqlalchemy package.
def company_filter(db, filter_type, filter_value, minmax):
match filter_type:
case "industry":
filter_list = []
for filter in filter_value:
filter_list.append(Company.industry == filter)
return db.query(Company).with_entities(Company.id, Company.symbol, Company.name, Company.monthly_exp).filter(or_(*filter_list))
...
I have to make 2 search bars. I don't know how to add multiple fiels...
match= Staff.objects.filter(id=srch1...) here how can I add name=srch1
over here after trying many ways I found it but the problem is all input here is string how to change it to int
def search(request):
# Catch the data and search in Staff model.
if request.method=='POST':
srch1 = request.POST['srch']
print(type(srch1))
if type(srch1)== int:
match= Staff.objects.filter(id=srch1)
if match :
return render(request,'search.html',{'sr': match})
else:
messages.error(request,'no results,found')
elif type(srch1)== str:
catch= Staff.objects.filter(name=srch1)
if catch:
return render(request,'search.html',{'sr': catch})
else:
messages.error(request,'no results,found')
else:
return HttpResponseRedirect("/search")
return render(request,"search.html")
You should be using a GET request with querysting parameters or (url parameters, which is a bit more complicated) for this, not a POST. Here's how I would do this:
def search(request, *args, **kwargs):
# Initial empty query dictionary for use with query later.
query = {}
# Getting 'name' and 'id' from querystring parameters (i.e. ?id=1&name=foo)
name = request.GET.get('name', None)
id = request.GET.get('id', None)
# Add 'id' to the query dictionary if it exists
if id is not None:
query['id'] = id
# Add name to the query dictionary if it exists
if name is not None:
query['name'] = name
# If the query dictionary has name or id, get the Staff entry from the database
if query.get('name', None) or query.get('id', None):
# Note that .filter() returns a QuerySet. You should probably use .get()
# since my guess is that you only want one Staff object (judging by your
# search parameters). Also note that since we are using **query we will be
# using BOTH 'name' AND 'id' to search for the Staff, as long as both exist in
# the query dictionary.
match = Staff.objects.get(**query)
# If there is a match, send it in the rendered response context dict
if match:
return render(request, 'search.html', {'sr': match})
# no match, send message notifying that a Staff entry was not found matching
# the desired criteria
return render(request, 'search.html', {message: 'Not Found'}
# There were no query parameters, so we are not searching for anything.
else:
return render(request, 'search.html')
You can see that the above code is much simpler, and more concise. This will help you or anyone else checking out your code in the future to better understand what you're trying to acheive.
P.S. I typically only use POST requests when I am creating an entry in the database. Maybe this is preference, but to my knowledge it is best practice to use a GET request for search.
How can i split by big dataframe into smaller dataframe and able to print all the dataframe separately on web? any idea on edit code can place a loop in context?
here is my code:
def read_raw_data(request):
Wb = pd.read_excel(r"LookAhead.xlsm", sheetname="Step")
Step1 = Wb.replace(np.nan, '', regex=True)
drop_column =
Step1_Result.drop(['facility','volume','indicator_product'], 1)
uniquevaluesproduct = np.unique(drop_column[['Product']].values)
total_count=drop_column['Product'].nunique()
row_array=[]
for name, group in drop_column.groupby('Product')
group=group.values.tolist()
row_array.append(group)
i=1
temp=row_array[0]
while i<total_count:
newb = temp + row_array[i]
temp=newb
i = i + 1
b = ['indicator', 'Product']
test=pd.DataFrame.from_records(temp, columns=b)
table = test.style.set_table_attributes('border="" class = "dataframe table table-hover table-bordered"').set_precision(10).render()
context = { "result": table}
return render(request, 'result.html', context)
If you want to show a big dataframe in different pages, I recommend you using a Paginator. The documentation has a good example on how to implement it.
https://docs.djangoproject.com/en/1.10/topics/pagination/#using-paginator-in-a-view
I've got the following view:
def search_events(request):
term = request.GET.get('term', '')
adminDivision = request.GET.get('adminDivision', '')
events = Event.objects.filter(event_name__icontains=term, city__admin1=adminDivision)
data= serializers.serialize('json', events);
return HttpResponse(data, content_type='application/json')
It can receive 2 parameters from the request, term and adminDivision, and then it makes a search on the DB.
The way it works now is that when one of them is empty, term or adminDivision, then I get no results, as all the objects has some value for those fields.
What I want is, if one or even both filters are empty, then don't apply that filter.
Example,
if I've got these objects:
[event_name='foo', adminDivision='1']
[event_name='bar', adminDivision='2']
[event_name='foo bar', adminDivision='3']
With term=foo (no value for adminDivision) at the moment, I'am getting no results, but I would want it to return the first and the third.
With both values empty, I am also getting no results, and I'd like to have all them.
Is there an elegant way to achieve this?
Thank you!
You are not getting results because you are performing an AND query in your filter. Try like this:
def search_events(request):
term = request.GET.get('term', '')
adminDivision = request.GET.get('adminDivision', '')
events = Event.objects.all()
if term:
events = events.filter(event_name__icontains=term)
if adminDivision:
events = events.filter(city__admin1=adminDivision)
data= serializers.serialize('json', events);
return HttpResponse(data, content_type='application/json')
Or:
from django.db.models import Q
...
events = Event.objects.filter(Q(event_name__icontains=term)|Q(city__admin1=adminDivision))
...
I have a database of artists and paintings, and I want to query based on artist name and painting title. The titles are in a json file (the artist name comes from ajax) so I tried a loop.
def rest(request):
data = json.loads(request.body)
artistname = data['artiste']
with open('/static/top_paintings.json', 'r') as fb:
top_paintings_dict = json.load(fb)
response_data = []
for painting in top_paintings_dict[artist_name]:
filterargs = {'artist__contains': artistname, 'title__contains': painting}
response_data.append(serializers.serialize('json', Art.objects.filter(**filterargs)))
return HttpResponse(json.dumps(response_data), content_type="application/json")
It does not return a list of objects like I need, just some ugly double-serialized json data that does no good for anyone.
["[{\"fields\": {\"artist\": \"Leonardo da Vinci\", \"link\": \"https://trove2.storage.googleapis.com/leonardo-da-vinci/the-madonna-of-the-carnation.jpg\", \"title\": \"The Madonna of the Carnation\"}, \"model\": \"serve.art\", \"pk\": 63091}]",
This handler works and returns every painting I have for an artist.
def rest(request):
data = json.loads(request.body)
artistname = data['artiste']
response_data = serializers.serialize("json", Art.objects.filter(artist__contains=artistname))
return HttpResponse(json.dumps(response_data), content_type="application/json")
I just need to filter my query by title as well as by artist.
inYour problem is that you are serializing the data to json twice - once with serializers.serialize and then once more with json.dumps.
I don't know the specifics of your application, but can chain filters in django. So I would go with your second approach and just replace the line
response_data = serializers.serialize("json", Art.objects.filter(artist__contains=artistname))
with
response_data = serializers.serialize("json", Art.objects.filter(artist__contains=artistname).filter(title__in=paintings))
Check the queryset documentation.
The most efficient way to do this for a __contains search on painting title would be to use Q objects to or together all your possible painting names:
from operator import or_
def rest(request):
data = json.loads(request.body)
artistname = data['artiste']
with open('/static/top_paintings.json', 'r') as fb:
top_paintings_dict = json.load(fb)
title_filters = reduce(or_, (Q(title__contains=painting) for painting in top_paintings_dict[artist_name]))
paintings = Art.objects.filter(title_filters, artist__contains=artist_name)
That'll get you a queryset of paintings. I suspect your double serialization is not correct, but it seems you're happy with it in the single artist name case so I'll leave that up to you.
The reduce call here is a way to build up the result of |ing together multiple Q objects - operator.or_ is a functional handle for |, and then I'm using a generator expression to create a Q object for each painting name.