I know this could be a repeated question for many of you but I have not been able to find a proper answer for this yet. I am a beginner to Django and Python. I have a python code which runs and produce output on cli at present but I want the same program to run its output on web.
I read that for web django is best suitable framework and for this purpose I started to study django. I see in every tutorial people have discussed apps, views urls etc but not seen an example which integrate a python code with django.
All I am looking for to understand how can I integrate my python script with Django and where do I place my code in Django project or app. Should I import it within views? if yes, then how to present my output to web.
Here is the sample code I am running, it basically opens two files and run some regex to extract the desired information.
import re
def vipPoolFileOpen(): # function opens vip and pool config file and store them to vip_config and pool_config variables
with open("pool_config.txt",'rb') as pool_config:
pool_config = pool_config.read()
pool_config = pool_config.split('ltm')
with open("vip_config.txt",'rb') as vip_config:
vip_config = vip_config.read()
vip_config = vip_config.split('ltm')
return vip_config,pool_config
def findWidth(vip_config): # function to find the maximum length of vip in entire file, this will be used to adjust column space
colWidth=0
for item in vip_config:
i=0
if colWidth<len(item):
while i<len(vip_config)-1:
if len(item)>=len(vip_config[i+1]):
colWidth=len(item)
i=i+1
else:
i+=1
continue
return colWidth
def regexFunction():
vip_config, pool_config = vipPoolFileOpen()
findWidth(vip_config)
for vip in vip_config:
regVip = re.compile(r'pool (.+)\r')
poolByVip = regVip.findall(vip) # poolByVip holds pool name from the vip_config file
for poolblock in pool_config:
regPool = re.compile(r'pool (.+) {')
poolByConfig = regPool.findall(poolblock)
if poolByVip == poolByConfig:
print vip + poolblock
break
elif poolByVip == ['none']:
print vip
break
else:
continue
Yes, you should present your output to the web via view. You need to write a view function (or a class view) in views.py and provide an url where you want to have it in urls.py
If you rewrite your function to return desired result instead of printing it, tou can do the following:
write this in views.py
from django.http import HttpResponse
from wherever_you_have_it import regexFunction
def bar(request):
result = regexFunction() # result should be a string
return HttpResponse(result)
and in urls.py:
from .views import bar
urlpatterns = [
url(r'^foo$', bar),
]
Providing of course you have created your Django app at the first place.
Your result should be displayed as plain text on address localhost:8000/foo - but you need to:
python menage.py runserver
In your terminal first
And of course feel free to look at:
https://github.com/Ergaro/CheckMyChords
to see how a simple django app looks like
Related
I currently have my flask running static where I run the ETL job independently and then use the result dataset in my Flask to display chartjs line charts.
However, I'd like to integrate the ETL piece in my web framework where my users can login and submit the input parameter(locations of the input files and an added version id) using HTML form, which will then be used by my ETL job to run and use the resultant data directly to display the charts on same page.
Current Setup:
My custom ETL module has submodules that act together to form a simple pipeline process:
globals.py - has my globals such as location of the s3 and the etc. ideally i'd like my user's form inputs to be stored here so that they can be used directly in all my submodules wherever necessary.
s3_bkt = 'abc' #change bucket here
s3_loc = 's3://'+s3_bkt+'/'
ip_loc = 'alv-input/'
#Ideally ,I'd like my users form inputs to be sitting here
# ip1 = 'alv_ip.csv'
# ip2 = 'Input_product_ICL_60K_Seg15.xlsx'
#version = 'v1'
op_loc = 'alv-output/'
---main-module.py - main function
import module3 as m3
import globals as g
def main(ip1,ip2,version):
data3,ip1,ip2,version = m3.module3(ip1,ip2,version)
----perform some actions on the data and return---
return res_data
---module3.py
import module2 as m2
def mod3(ip1,ip2,version):
data2,ip1,ip2,version = m2.mod2(ip1,ip2,version)
----perform some actions on the data and return---
return data3
---module2.py
import module1 as m1
import globals as g
def mod2(ip1,ip2,version):
data1,ip1,ip2,version = m1.mod1(ip1,ip2,version)
data_cnsts = pd.read_csv(ip2) #this is where i'll be using the user's input for ip2
----perform some actions on the datasets and write them to location with version_id to return---
data1.to_csv(g.op_loc+'data2-'+ version + '.csv', index=False)
return data2
---module1.py
import globals as g
def mod1(ip1,ip2,version):
#this is the location where the form input for the data location should be actually used
data = pd.read_csv(g.s3_loc+g.ip_loc+ip1)
----perform some actions on the data and return---
return data1
Flask setup:
import main-module as mm
app = Flask(__name__)
#this is where the user first hits and submits the form
#app.route('/form')
def form():
return render_template('form.html')
#app.route('/result/',methods=['GET', 'POST'])
def upload():
msg=''
if request.method == 'GET':
return f"The URL /data is accessed directly. Try going to '/upload' to submit form"
if request.method == 'POST':
ip1 = request.form['ip_file']
ip2 = request.form['ip_sas_file']
version = request.form['version']
data = mm.main(ip1,ip2,version)
grpby_vars = ['a','b','c']
grouped_data = data.groupby(['mob'])[grpby_vars].mean().reset_index()
#labels for the chart
a = [val for val in grouped_data['a']]
#values for the chart
b = [round(val*100,3) for val in grouped_data['b']]
c = [round(val*100,3) for val in grouped_data['c']]
d = [val for val in grouped_data['d']]
return render_template('results.html', title='Predictions',a=a,b=b,c=c,d=d)
The Flask setup works perfectly fine without using any form inputs from the user(if the ETL job and Flask is de-coupled i.e. when I run the ETL and supply the result data location to Flask directly.
Problem:
The problem after integration is that I'm not very sure how to pass these inputs from users to all my sub-module. Below is the error I get when I use the current setup.
data3,ip1,ip2,version = m3.module3(ip1,ip2,version)
TypeError: module3() missing 3 required positional arguments: 'ip1', 'ip2', and 'version'
So, definitely this is due to the issue with my param passing across my sub-modules.
So my question is how do I use the data from the form as a global variable across my sub-modules. Ideally I'd like them to be stored in my globals so that I'd not have to be passing them as a param through all modules.
Is there a standard way to achieve that? Might sound very trivial but I'm struggling hard to get to my end-state.
Thanks for reading through :)
I realized my dumb mistake, I should've have just included
data,ip1,ip2,version = mm.main(ip1,ip2,version)
I could also instead use my globals file by initiating the inputs with empty string and then import my globals in the flask file and update the values. This way I can avoid the route of passing the params through my sub-modules.
I'm trying to use a FilteredElementCollector inside my pyRevit script to collect all views (sections, elevations, plan callouts etc.) in the active view.
from pyrevit.framework import clr
from pyrevit import revit, DB
clr.AddReference('RevitAPI')
clr.AddReference('RevitAPIUI')
from Autodesk.Revit.DB import *
from pyrevit import forms
doc = __revit__.ActiveUIDocument.Document
view = doc.ActiveView
AllStuff = FilteredElementCollector(doc,doc.ActiveView.Id).WhereElementIsNotElementType().ToElements()
AllViews = []
try:
for x in AllStuff:
if "View" in x.Category.Name:
AllViews.append(x)
This will return some, but not all of the views. For example, some sections are included but others are not and I can't tell why.
If I add ".OfCategory(BuiltInCategory.OST_Views)" I get nothing at all. Do I need to break it down into several more specific categories? Thanks for any help.
There is no view in FilteredElementCollector(doc, doc.ActiveView.Id), you can see it by doing :
for el in FilteredElementCollector(doc, doc.ActiveView.Id):
print(el)
There is an Element which is not of category OST_Views and is not a view even if it has the same name as your view. To see this you can use RevitLookUp.
I found a way to retrieve the actual view (I don't know any other way at the moment) by looking at VIEW_FIXED_SKETCH_PLANE BuiltInParameter which refer to the SketchPlane whiche reference the actual view as Element.OwnerViewId. Then you can make sure that the element is of class View :
for el in FilteredElementCollector(doc,doc.ActiveView.Id):
sketch_parameter = el.get_Parameter(BuiltInParameter.VIEW_FIXED_SKETCH_PLANE)
# If parameter do not exist skip the element
if not sketch_parameter:
continue
view_id = doc.GetElement(sketch_parameter.AsElementId()).OwnerViewId
view = doc.GetElement(view_id)
if isinstance(view, View):
print(view)
I'm currently using the shared_contacts_profiles.py script to load contacts from an external system into our Google Shared Domain contacts. I'd like to make the process more automated so I've tried to create a shared contact (with just a full name and email address) using a basic python script. The contact is created but it gets added to the administrator's contacts and not the Directory.
My code is
#!/usr/bin/python
import atom
import gdata.data
import gdata.contacts.client
import gdata.contacts.data
def main():
admin_email = 'admin#mydomain.com'
admin_password = 'P4ssw0rd'
domain_index = admin_email.find('#')
domain = admin_email[domain_index+1:]
contacts_client = gdata.contacts.client.ContactsClient(domain=domain)
contacts_client.client_login(email=admin_email,
password=admin_password,
source='shared_contacts_profiles',
account_type='HOSTED')
new_contact = gdata.contacts.data.ContactEntry()
new_contact.name = gdata.data.Name(
full_name=gdata.data.FullName(text='John Doe'))
new_contact.email.append(gdata.data.Email(address='john.doe#example.com',
primary='true',rel=gdata.data.WORK_REL))
contact_entry = contacts_client.CreateContact(new_contact)
print "Contact's ID: %s" % contact_entry.id.text
if __name__ == '__main__':
main()
I must be missing something fairly simple, but just can't see what it is.
EDIT * I think that shared_contacts_profiles.py sets the domain contact list when it sends batches to Google. I wasn't going to use batches as there are only ever a couple of contacts to add. I also suspect I should be using gdata.contacts.service.ContactsService and not gdata.contacts.client.ContactsClient
Thanks
Dave
In the end I used the original code as shown above with some additions. I needed to get the feed uri for the shared domain contact list and then supply that uri in the CreateContact.
feed_url = contacts_client.GetFeedUri(contact_list=domain, projection='full')
contact_entry = contacts_client.CreateContact(new_contact,insert_uri=feed_url)
Thanks
Dave
I want to see all the routes which my application has. Return them as a response like key=>value pair:
'route1' => '{foo:\w+}'
'route2' => '{baz:\w+\d+}'
... and so on
But I don't know how to get them within my view. For example, this is my view. I want it to return a map of routes. I do this:
#view_config(route_name='route1')
def someView(request):
routes = request.registry.settings.getRoutes() ## what should I print here to get a map of routes?
r = ''
for k,v in sorted(routes.items()):
r += str(k) + "=>" + str(v) + "<br/>";
return Response(r)
There is a RoutesConfiguratorMixin class with get_routes_mapper method. I tried to import the class and called its method but got an error that no registry was in the instance of it:
from pyramid.config.routes import RoutesConfiguratorMixin as Router
r = Router();
routes = r.get_routes_mapper();
## ... and the same code as above
Doesn't work.
There are 2 ways, one is supported (public) and one is unsupported (private).
Option #1 is to use the introspector and is explained here.
Option #2 is to use the route mapper (which is not a public api), in the way that the pyramid debugtoolbar does in its routes panel.
Pyramid installs a bin script called proutes for that purpose.
Install pshell then
pshell to login to pshell with your app config.
then run
print("\n".join([r.path for r in app.routes_mapper.routelist]))
I'm brand new at Python and I'm trying to write an extension to an app that imports GA information and parses it into MySQL. There is a shamfully sparse amount of infomation on the topic. The Google Docs only seem to have examples in JS and Java...
...I have gotten to the point where my user can authenticate into GA using SubAuth. That code is here:
import gdata.service
import gdata.analytics
from django import http
from django import shortcuts
from django.shortcuts import render_to_response
def authorize(request):
next = 'http://localhost:8000/authconfirm'
scope = 'https://www.google.com/analytics/feeds'
secure = False # set secure=True to request secure AuthSub tokens
session = False
auth_sub_url = gdata.service.GenerateAuthSubRequestUrl(next, scope, secure=secure, session=session)
return http.HttpResponseRedirect(auth_sub_url)
So, step next is getting at the data. I have found this library: (beware, UI is offensive) http://gdata-python-client.googlecode.com/svn/trunk/pydocs/gdata.analytics.html
However, I have found it difficult to navigate. It seems like I should be gdata.analytics.AnalyticsDataEntry.getDataEntry(), but I'm not sure what it is asking me to pass it.
I would love a push in the right direction. I feel I've exhausted google looking for a working example.
Thank you!!
EDIT: I have gotten farther, but my problem still isn't solved. The below method returns data (I believe).... the error I get is: "'str' object has no attribute '_BecomeChildElement'" I believe I am returning a feed? However, I don't know how to drill into it. Is there a way for me to inspect this object?
def auth_confirm(request):
gdata_service = gdata.service.GDataService('iSample_acctSample_v1.0')
feedUri='https://www.google.com/analytics/feeds/accounts/default?max-results=50'
# request feed
feed = gdata.analytics.AnalyticsDataFeed(feedUri)
print str(feed)
Maybe this post can help out. Seems like there are not Analytics specific bindings yet, so you are working with the generic gdata.
I've been using GA for a little over a year now and since about April 2009, i have used python bindings supplied in a package called python-googleanalytics by Clint Ecker et al. So far, it works quite well.
Here's where to get it: http://github.com/clintecker/python-googleanalytics.
Install it the usual way.
To use it: First, so that you don't have to manually pass in your login credentials each time you access the API, put them in a config file like so:
[Credentials]
google_account_email = youraccount#gmail.com
google_account_password = yourpassword
Name this file '.pythongoogleanalytics' and put it in your home directory.
And from an interactive prompt type:
from googleanalytics import Connection
import datetime
connection = Connection() # pass in id & pw as strings **if** not in config file
account = connection.get_account(<*your GA profile ID goes here*>)
start_date = datetime.date(2009, 12, 01)
end_data = datetime.date(2009, 12, 13)
# account object does the work, specify what data you want w/
# 'metrics' & 'dimensions'; see 'USAGE.md' file for examples
account.get_data(start_date=start_date, end_date=end_date, metrics=['visits'])
The 'get_account' method will return a python list (in above instance, bound to the variable 'account'), which contains your data.
You need 3 files within the app. client_secrets.json, analytics.dat and google_auth.py.
Create a module Query.py within the app:
class Query(object):
def __init__(self, startdate, enddate, filter, metrics):
self.startdate = startdate.strftime('%Y-%m-%d')
self.enddate = enddate.strftime('%Y-%m-%d')
self.filter = "ga:medium=" + filter
self.metrics = metrics
Example models.py: #has the following function
import google_auth
service = googleauth.initialize_service()
def total_visit(self):
object = AnalyticsData.objects.get(utm_source=self.utm_source)
trial = Query(object.date.startdate, object.date.enddate, object.utm_source, ga:sessions")
result = service.data().ga().get(ids = 'ga:<your-profile-id>', start_date = trial.startdate, end_date = trial.enddate, filters= trial.filter, metrics = trial.metrics).execute()
total_visit = result.get('rows')
<yr save command, ColumnName.object.create(data=total_visit) goes here>