I have a JSON file with data as such :
['dbname' : 'A', 'collection' : 'ACollection', 'fields' : ['name', 'phone_no', 'address']}
['dbname' : 'B', 'collection' : 'BCollection', 'fields' : ['name', 'phone_no', 'address', 'class']}
These are 2 examples amongst many other dictionaries of the same format.
I have a python code that does the following : Accepts 2 inputs from the user - phone_no and dbname. For example, the user enters phone_no as xxxxxxxxxx and dbname as A. The python code then reads the JSON file and matches the user input with the dictionary element having the name of the database as 'A'. It then opens the database 'A', opens the respective collection 'ACollection' and prints the respective fields of posts within the collection that have the phone_no value as xxxxxxxxxx. The databases are implemented with mongoDB.
I need to build a django rest api for this code. The end goal is to access the code from a browser. The user provides the 2 inputs in the browser and the code is executed, returning the data, which is displayed on the browser. I have gone through the django-rest framework documentation but I'm new to this whole concept and would like some guidance.
How do I implement these functions and create an API? What code should the models, serializers, views and urls files have related to my program?
models.py
from django.db import models
class App(object):
def __init__(self, phone_no, name, address, categories):
self.phone_no = phone_no
self.name = name
self.address = address
self.categories = categories
This is what I'm working with so far, to get started. The problem, however, is that the models class should essentially be dynamic. Ex: If 'A' is the database, the program returns 3 fields but if 'B' is the database, the program returns 4 values so I'm not sure what the models class would be like.
views.py
from django.views.decorators.csrf import csrf_exempt
from rest_framework.decorators import api_view
from rest_framework.response import Response
from pymongo import Connection
from models import App
from serializers import AppSerializer
import json
import pymongo
from os import listdir
import re
from django import forms
#csrf_exempt
#api_view(['GET'])
def pgs(request):
#connect to our local mongodb
db = Connection('localhost',27017)
#get a connection to our database
dbconn = db.general
dbCollection = dbconn['data']
if request.method == 'GET':
#get our collection
items = []
for r in dbCollection.find():
post = App(r["phone_no"],r["name"],r["address"],r["categories"])
items.append(post)
serializedList = AppSerializer(items, many=True)
return Response(serializedList.data)
Let's say you have identical tables in two different databases. We'd start by creating two db connections in settings.py. Let's say those are called db_a and db_b. We might model this as so:
class PhoneGenome(models.Model):
phone_no = models.CharField(length=255)
name = models.CharField(length=255)
# and so on...
class Meta:
# if database pre-exists, may want to use managed=False
managed = False
This gives us a model. Now we choose which database to pull from based on user input. In a view, you might have something like:
db_used = request.GET.get('database')
db_conns = {'a': 'db_a', 'b': 'db_b'}
if db_conns.has_key(db_used):
records = PhoneGenome.objects.using(db_conns[db_used]).filter( user_criteria_here)
The using() method in your queryset is what allows you to select which database to run the query against.
There's a lot to manage here potentially, so this would be a good time to look at the docs: https://docs.djangoproject.com/en/1.7/topics/db/multi-db/
And if you haven't already, you really should work through the Django tutorial at the very least before going much further.
Related
I am totally new to Django and I'm having a problem displaying data from a MariaDB MySQL database on a HTML page.
I have a legacy database called Vision which has a table within it called SensorResult. I have added this database to the settings.py file and made a routings folder with a router_db.py file to tell the app when to use the Vision database.
From this, I ran the following command:
python manage.py inspectdb --database=Vision
This returned a printout of each table within the database and all columns. I then used this to create a standard models.py file using this information as shown below:
from django.db import models
class Sensorresult(models.Model):
sensorresult = models.AutoField(db_column='SensorResult', primary_key=True) # Field name made lowercase.
sensorid = models.IntegerField(db_column='SensorID') # Field name made lowercase.
visionid = models.IntegerField(db_column='VisionID') # Field name made lowercase.
userinputid = models.IntegerField(db_column='UserInputID', blank=True, null=True) # Field name made lowercase.
value = models.FloatField(db_column='Value') # Field name made lowercase.
timestamp = models.DateTimeField(db_column='TimeStamp') # Field name made lowercase.
class Meta:
db_table = 'SensorResult'
From this, I used makemigrations and migrate commands to submit this model. Now I can open the Django Shell and query the database and get a response, which makes me think that the models.py, settings.py and db_routings.py files are all working correctly (please tell me if this is not the case)? An example of the shell commands and printout response I get is as follows:
python manage.py shell
>>> from authenticate.models import Sensorresult
>>> qs = Sensorresult.objects.using('Vision').get(sensorresult="1")
>>> qs.value
556746.0
From this I have created the following views.py file, which has an identical SQL command as that used in the shell:
from django.http import HttpResponse
from django.shortcuts import redirect, render
from .models import Sensorresult
def db(request):
qs = Sensorresult.objects.using('Vision').get(sensorresult='1')
return render(request, 'authenticate/instruments.html',{'qs':qs})
and then an instruments.html which is as follows:
<div class="overviewcard">
<div class="overviewcard__icon">Engine</div>
<div class="overviewcard__info">{{qs.value}}</div>
However for some reason I dont get any data displaying, just blank. I also have a URLs.py page which is as follows:
from django.urls import path, include
from . import views
from . import models
#used for routing within the app
#use the below format to add further pages
urlpatterns = [
path('instruments/', views.instruments, name="instruments"),
path('instruments/',views.db, name='db'),
]
I've been through loads of online tutorials and YouTube videos; however, none have worked so far. If anyone has any suggestions as to what is going wrong that would be greatly appreciated.
You should use unique paths for your url patterns or you will get conflicts, the first path is always matching so your db view is never called.
urlpatterns = [
path('instruments/', views.db, name='db'),
path('instruments/foo/', views.instruments, name="instruments"),
]
The url dispatcher iterates over your urlpatterns in order and returns the first one that matches the incoming path, if you have two patterns that could match the path then the first one will always be used
I want to store network devices data through a Django model Device to my database.
Workflow
Host configuration needs to be setup by User inside a View (Host Model)
When the Host configuration is finished the Network should be scanned for devices (Device Model)
The data should be stored inside the DB
Problem:
The fuction create_devices() only is allowed to get called when the Host is configured, but if Host.objects.values(): isnt working.
How is it possible to call the function create_devices() only if one Host Model exists?
Is it correct to use a view to store Dynamic and Static Data in to the DB without User interaction?
Models:
class Host(models.Model):
hostname = models.CharField(default="noads", max_length=6)
ipv4_address = models.GenericIPAddressField('IPv4')
ipv4_subnet = models.GenericIPAddressField('IPv4')
gateway = models.GenericIPAddressField('IPv4')
class Device(models.Model):
hostname = models.CharField(max_length=64)
mac_address = models.CharField(max_length=64)
ipv4_address = models.GenericIPAddressField('IPv4')
My View:
from webapp.models import Host, Device
from django.views import View
from django.views.generic.detail import DetailView
import multiprocessing.dummy
import multiprocessing
def create_devices():
"""
Creates DB entry of devices if they dont already exist
:return: List of mulitple devices stored in objects
:rtype: list ["Device", "Device", ...]
"""
available_devices = get_available_devices_in_list()
arp_table_of_all_hosts = get_arp_table_linux()
dev_list = []
for deviceip in available_devices:
#If device already exists in DB continue
if arp_table_of_all_hosts.get(deviceip):
if arp_table_of_all_hosts[deviceip] in Device.objects.filter(mac_address = arp_table_of_all_hosts[deviceip]):
continue
else:
devmac = arp_table_of_all_hosts[deviceip]
devname = "unknown" #socket.gethostbyaddr(deviceip)
dev = Device(hostname=devname, mac_address=devmac, ipv4_address=deviceip)
dev.save()
dev_list.append(dev)
return dev_list
class DeviceGetAll(DetailView):
if Host.objects.values():
create_devices()
model = Device
pass
Your view DeviceGetAll is not written correctly. The if condition needs to be placed in some of the methods of the view, not in the class definition.
I don't understand what exactly you are trying to do, so I cannot know in which method you need to add the code to, but you can look at the base code of DetailView and see if some of those methods are useful to you.
I don't even know, if the detail view is the best place to put code that creates instances of the same model; maybe it could also go after the creation of the host.
But if you want to use the DetailView, the you could for example override the method get_object() from the SingleObjectMixin (which is one of the parents of DetailView), the your code could look like this:
class DeviceGetAll(DetailView):
model = Device
def get_object(self, *args, **kwargs):
if Host.objects.exists():
create_devices()
obj = super().get_object(*args, **kwargs)
return obj
Also, you probably mean to use the .exists() method in the if-condition instead of .values().
Say I have two servers prod and API.
On prod I have a Django application running and say I have the following model
class MyModel(model.Models):
name = models.Charfield()
age = models.IntField()
birthdate = models.DateTimeField()
has_birthday = models.BooleanField()
thus the model in my database would look like
name | age | birthdate | has_birthday
-----+-------+-------------+-------------
john 30 1990-01-30 0
doe 20 1987-05-01 0
....
Each day on API I run a daily script which checks, if someone has birthday - if they do, set has_birthday=1 (note, everything above is just an example for illustration purposes).
Since API just is a server for daily jobs, I have not deployed a Django-application on that server, thus I wonder, what is the usual/best way to update the MyModel table with the following logic?
My first intuition is just to make a plain and simple update- SQL statement e.g
from utils import get_con_to_db
con = get_con_to_db()
query = <SQL update query>
con.execute(query)
but that is rather error prone in case I decide to change my model in some way. Is there a better/more secure way of doing so, without having to create a Django application on API thus maintaining two Django applications?
You can define an url with view function do_orm_operation on prod server, then you can execute ORM operation on API server by send a POST request to the url,
its specific implementation is as follows(core idea is to use eval,):
from django.shortcuts import HttpResponse
import django
# Create your views here.
models = django.apps.apps.get_models()
models_names = {mod.__name__: mod for mod in models}
def do_orm_operation(request):
"""
This method supports CURD, you just need to send a request by `POST` method,
the post data should be the format `{'model': 'your_model_name', 'cmd': 'your_orm_operation_begins_with_objects'}`,
Eg: requests.post(data={'model': 'Book', 'cmd': 'objects.filter(name='book1').update(name="mybook")'}, url='http://127.0.0.1:8000/do_orm_operation/')
"""
try:
data = request.POST
model_name = data['model']
cmd = data['cmd']
model = models_names[model_name] # this variable will be used in eval
query = 'model.' + cmd
res = eval(query)
print('res:', res)
# then do whatever you want
return HttpResponse('success')
except:
return HttpResponse('error')
Note:If you use this method, you must pay attention to security issues to prevent malicious execution of database operations.
I am new to Flask and bit confused about the database modeling for it. Please have my apologies if by any mean this isn't a question eligible for post.
I need to create a multi choice model field in Flask and I need to be able to access it from backend admin panel to set values for it. It does shows options in docs with WTF forms to create multiple choice field. I am confused that how to create forms attached to Database. Can someone clear it up for me because I am a Django user and in Django Forms and ModelForms have different approach so trying to understand what would it be in Flask. How to render Database based forms in Flask? How would I create a multi choice field with database created for it. Please help.
What you are looking for is SQLAlchemy built-in ORM to build forms from models or integrated to database. There are other options to overcome limitations of Flask ORM when needed. Following is the example that would give you some clarity.
from flask import Flask, render_template, redirect, flash
from flask.wtf import Form
from flask.ext.sqlalchemy import SQLAlchemy
from wtf.ext.sqlalchemy.orm import model_form
app=Flask(__app__)
app.config['SECRET_KEY'] = 'secret'
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:////tmp/employees.sqlite'
app.config['SQLALCHEMY_ECHO'] = True
# Here you initiate the ext
db=SQLAlchemy(app)
#Let's define a model
class Employee(db.Model)
__tablename__ = 'employee'
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String(100), nullable=False
birthday = db.Column(db.Date, nullable=False
def __repr__(self):
return 'employee %s' %self.name
# Now go to your shell inside your env not in gloabal shell outside your env and run this command.
# From your .py file where you've created all above configration first make an import to db from shell
from file.py import db
#Then create a database with following command in shell
db.create.all()
#Your auto generated database based form below
EmployeeForm() = model_form(Employee, base_class=Form, field_args{'name':{'class':'employee'}})
#Let's create a view with model_form or database based form in your case.
#app.route('/', methods=['GET', 'POST'])
def index()
#request.POST does same in Django or any other Python based web framework like Bottle, Tornado etc
form = EmployeeForm()
try:
if form_validate_on_submit():
employee=Employee() #load the model values
form.populate_obj(Employee) #populates the form with respective values
db.session.add(employee) #gathers the session based data to be added in DB
db.session.commit() #Adds data to DB
flash('New Employee added to database successfully.') #Display a message to end user at front end.
retrun redirect('/') # redirects upon success to your homepage.
except Exception e:
# logs the errors
db.session.rollback()
flash('There was a problem registering new employee. Please contact the site administrator at site#site.com')
employee_list = Employe.query.all() #equailent to django style "item.objects.all() to show list of all existing items.
return render_template('index.html', form=form, employee_list=employee_list)
In last line above you did three things. You got your form variable or context variable like you do in Django as "form" so your end user can enter data.
Then you have your model data that is saved in db as "employee_list=employee_list" that will show all the list to end users. "flash" is just like Django
messaging framework.
Now for multiple choices its model has same as djagno choice arguement for key value like below:
With my experience I would suggest you to install "peewee" a simple ORM for Python connected Databases.
choices = (('key', 'value')('key', 'value'))
employee_type = db.Model(db.String(90), choices=('key1', 'key2)
Hope this helps.
I am using Factory Boy to create test factories for my django app. The model I am having an issue with is a very basic Account model which has a OneToOne relation to the django User auth model (using django < 1.5):
# models.py
from django.contrib.auth.models import User
from django.db import models
class Account(models.Model):
user = models.OneToOneField(User)
currency = models.CharField(max_length=3, default='USD')
balance = models.CharField(max_length="5", default='0.00')
Here are my factories:
# factories.py
from django.db.models.signals import post_save
from django.contrib.auth.models import User
import factory
from models import Account
class AccountFactory(factory.django.DjangoModelFactory):
FACTORY_FOR = Account
user = factory.SubFactory('app.factories.UserFactory')
currency = 'USD'
balance = '50.00'
class UserFactory(factory.django.DjangoModelFactory):
FACTORY_FOR = User
username = 'bob'
account = factory.RelatedFactory(AccountFactory)
So I am expecting the factory boy to create a related UserFactory whenever AccountFactory is invoked:
# tests.py
from django.test import TestCase
from factories import AccountFactory
class AccountTest(TestCase):
def setUp(self):
self.factory = AccountFactory()
def test_factory_boy(self):
print self.factory.id
When running the test however, it looks like multiple User models are being create, and I am seeing an integriy error:
IntegrityError: column username is not unique
The documentation does mention watching out for loops when dealing with circular imports, but I am not sure whether that is whats going on, nor how I would remedy it. docs
If anyone familiar with Factory Boy could chime in or provide some insight as to what may be causing this integrity error it would be much appreciated!
I believe this is because you have a circular reference in your factory definitions. Try removing the line account = factory.RelatedFactory(AccountFactory) from the UserFactory definition. If you are always going to invoke the account creation through AccountFactory, then you shouldn't need this line.
Also, you may consider attaching a sequence to the name field, so that if you ever do need more than one account, it'll generate them automatically.
Change: username = "bob" to username = factory.Sequence(lambda n : "bob {}".format(n)) and your users will be named "bob 1", "bob 2", etc.
To pass result of calling UserFactory to AccountFactory you should use factory_related_name (docs)
Code above works next way:
AccountFactory for instantiating needs SubFactory(UserFactory).
UserFactory instantiates User.
UserFactory after instantiating calls RelatedFactory(AccountFactory)
Recursion,.. that is broken due to unique username constraint (you probably want to generate usernames via FuzzyText or Sequence)
So you need write UserFactory like this:
class UserFactory(factory.django.DjangoModelFactory):
account = factory.RelatedFactory(AccountFactory, factory_related_name='user')
username = factory.Sequence(lambda a: 'email%04d#somedomain.com' % a)
# rest of code
But you can still experience issues with already written tests. Imagine you have in tests places like next:
user = UserFactory()
account = Account(user=user)
Then adding RelatedFactory will break tests. If you haven't lots of tests and contributors in your project, you could rewrite them. But if not, it is not an option. Here is how it could be handled:
class UserFactory(factory.django.DjangoModelFactory):
class Params:
generate_account = factory.Trait(
account=factory.RelatedFactory(AccountFactory, factory_related_name='user')
)
Then code above won't be broken, because default call of UserFactory won't instantiate AccountFactory. To instantiate user with account:
user_with_account = UserFactory(generate_account=True)