I want to store network devices data through a Django model Device to my database.
Workflow
Host configuration needs to be setup by User inside a View (Host Model)
When the Host configuration is finished the Network should be scanned for devices (Device Model)
The data should be stored inside the DB
Problem:
The fuction create_devices() only is allowed to get called when the Host is configured, but if Host.objects.values(): isnt working.
How is it possible to call the function create_devices() only if one Host Model exists?
Is it correct to use a view to store Dynamic and Static Data in to the DB without User interaction?
Models:
class Host(models.Model):
hostname = models.CharField(default="noads", max_length=6)
ipv4_address = models.GenericIPAddressField('IPv4')
ipv4_subnet = models.GenericIPAddressField('IPv4')
gateway = models.GenericIPAddressField('IPv4')
class Device(models.Model):
hostname = models.CharField(max_length=64)
mac_address = models.CharField(max_length=64)
ipv4_address = models.GenericIPAddressField('IPv4')
My View:
from webapp.models import Host, Device
from django.views import View
from django.views.generic.detail import DetailView
import multiprocessing.dummy
import multiprocessing
def create_devices():
"""
Creates DB entry of devices if they dont already exist
:return: List of mulitple devices stored in objects
:rtype: list ["Device", "Device", ...]
"""
available_devices = get_available_devices_in_list()
arp_table_of_all_hosts = get_arp_table_linux()
dev_list = []
for deviceip in available_devices:
#If device already exists in DB continue
if arp_table_of_all_hosts.get(deviceip):
if arp_table_of_all_hosts[deviceip] in Device.objects.filter(mac_address = arp_table_of_all_hosts[deviceip]):
continue
else:
devmac = arp_table_of_all_hosts[deviceip]
devname = "unknown" #socket.gethostbyaddr(deviceip)
dev = Device(hostname=devname, mac_address=devmac, ipv4_address=deviceip)
dev.save()
dev_list.append(dev)
return dev_list
class DeviceGetAll(DetailView):
if Host.objects.values():
create_devices()
model = Device
pass
Your view DeviceGetAll is not written correctly. The if condition needs to be placed in some of the methods of the view, not in the class definition.
I don't understand what exactly you are trying to do, so I cannot know in which method you need to add the code to, but you can look at the base code of DetailView and see if some of those methods are useful to you.
I don't even know, if the detail view is the best place to put code that creates instances of the same model; maybe it could also go after the creation of the host.
But if you want to use the DetailView, the you could for example override the method get_object() from the SingleObjectMixin (which is one of the parents of DetailView), the your code could look like this:
class DeviceGetAll(DetailView):
model = Device
def get_object(self, *args, **kwargs):
if Host.objects.exists():
create_devices()
obj = super().get_object(*args, **kwargs)
return obj
Also, you probably mean to use the .exists() method in the if-condition instead of .values().
Related
I have a program that lets users upload data files and lookup tables (both which are ID'd to a specific company) and map them together. One page lets users choose which company they want to map data for by looking at which companies have both data files and lookup tables, which I use a queryset/model manager for. The problem is if I load a new data file and hierarchy the queryset doesn't pick them up until the server restarts. The queryset returns all the companies that have a data file and hierarchies at the time the server starts, but not anything that's added afterwards. I think this must be because the queryset is defined at startup, but I'm not sure. Is there a way to work around this?
forms.py
class CompanySelectionForm(forms.Form):
companies = RawData.objects.get_companyNames(source="inRDandH")
companiesTuple = makeTuple(companies)
print(companiesTuple)
company = forms.ChoiceField(widget=forms.Select(attrs={'class': 'form-select'}), choices=companiesTuple)
managers.py
class RawDataManager(models.Manager):
def get_queryset(self):
return RawDataQuerySet(self.model, using=self._db)
def get_companyNames(self, source):
return self.get_queryset().get_companyNames(source)
class RawDataQuerySet(models.QuerySet):
def get_companyNames(self, source):
if (source == 'inRDandH'):
distinct_companiesRD = self.filter(fileType=0).values_list('companyName', flat=True).distinct()
distinct_companiesH = self.filter(fileType=1).values_list('companyName', flat=True).distinct()
distinct_companies = set(distinct_companiesRD).intersection(set(distinct_companiesH))
else:
distinct_companies = self.values_list('companyName', flat=True).distinct()
return distinct_companies
The problem is that this code runs only once, when the code is initialised on server start, because it is part of your form class definition:
companies = RawData.objects.get_companyNames(source="inRDandH")
The solution is to make choices a callable, which is run every time the form is instantiated. define that field dynamically, in the __init__ method of the form:
def get_companies_tuple():
companies = RawData.objects.get_companyNames(source="inRDandH")
return makeTuple(companies)
class CompanySelectionForm(forms.Form):
company = forms.ChoiceField(
widget=forms.Select(attrs={'class': 'form-select'}),
choices=get_companies_tuple
)
This will now fetch the data from the database every time the form is initialised, rather than only once during startup.
First time using Django Forms. I'm stuck trying to get the dropdown choices to reload. My forms.py is below. When the database state changes, the choices do not. I suppose that this is because they are defined at a class level, which means the query happens at initialisation of the module? I've found that the only way to get my dropdowns to update is to restart the webserver.
How can I have the database queries evaluate on every request?
forms.py
from django import forms
from app.models import Collection, ErrorMessage, Service
class FailureForm(forms.Form):
collections = [(collection.value,)*2 for collection in Collection.objects.all()]
error_messages = [(message.value,)*2 for message in ErrorMessage.objects.all()]
services = [(service.value,)*2 for service in Service.objects.all()]
collection = forms.CharField(label='collection', max_length=100, widget=forms.Select(choices=collections))
error_message = forms.CharField(label='error_message', max_length=400, widget=forms.Select(choices=error_messages))
service = forms.CharField(label='service', max_length=100, widget=forms.Select(choices=services))
class FailureForm(forms.Form):
collection = forms.ChoiceField(widget=forms.Select, choices=[])
... # etc
def __init__(self, *args, **kwargs):
super(FailureForm, self).__init__(*args, **kwargs)
self.fields['collection'].choices = [(collection.value,)*2 for collection in Collection.objects.all()]
... # etc
Note: label='collection' is obsolete. It will be "collection" by default. Same with error_message and service
Did some more digging in the documentation and noticed that callables on a ChoiceField are called every initialisation. Therefore, the solution below was I think preferable for me.
class FailureForm(forms.Form):
collection = forms.ChoiceField(choices=lambda: [(collection.value,)*2 for collection in Collection.objects.all()])
error_message = forms.ChoiceField(choices=lambda: [(message.value,)*2 for message in ErrorMessage.objects.all()])
service = forms.ChoiceField(choices=lambda: [(service.value,)*2 for service in Service.objects.all()])
I'm writing a web scraper to get information about customers and appointment times to visit them. I have a class called Job that stores all the details about a specific job. (Some of its attributes are custom classes too e.g Client).
class Job:
def __init__(self, id_=None, client=Client(None), appointment=Appointment(address=Address(None)), folder=None,
notes=None, specific_reqs=None, system_notes=None):
self.id = id_
self.client = client
self.appointment = appointment
self.notes = notes
self.folder = folder
self.specific_reqs = specific_reqs
self.system_notes = system_notes
def set_appointment_date(self, time, time_format):
pass
def set_appointment_address(self, address, postcode):
pass
def __str__(self):
pass
My scraper works great as a stand alone app producing one instance of Job for each page of data scraped.
I now want to save these instances to a Django database.
I know I need to create a model to map the Job class onto but that's where I get lost.
From the Django docs (https://docs.djangoproject.com/en2.1/howto/custom-model-fields/) it says in order to use my Job class in the Django model I don't have to change it at all. That's great - just what I want. but I can't follow how to create a model that maps to my Job class.
Should it be something like
from django.db import models
import Job ,Client
class JobField(models.Field):
description = "Job details"
def __init__(self, *args, **kwargs):
kwargs['id_'] = Job.id_
kwargs['client'] = Client(name=name)
...
super().__init__(*args, **kwargs)
class Job(models.Model):
job = JobField()
And then I'd create a job using something like
Job.objects.create(id_=10101, name="Joe bloggs")
What I really want to know is am I on the right lines? Or (more likely) how wrong is this approach?
I know there must be a big chunk of something missing here but I can't work out what.
By mapping I'm assuming you want to automatically generate a Django model that can be migrated in the database, and theoretically that is possible if you know what field types you have, and from that code you don't really have that information.
What you need to do is to define a Django model like exemplified in https://docs.djangoproject.com/en/2.1/topics/db/models/.
Basically you have to create in a project app's models.py the following class:
from django import models
class Job(models.Model):
client = models.ForeignKey(to=SomeClientModel)
appointment = models.DateTimeField()
notes = models.CharField(max_length=250)
folder = models.CharField(max_length=250)
specific_reqs = models.CharField(max_length=250)
system_notes = models.CharField(max_length=250)
I don't know what data types you actually have there, you'll have to figure that out yourself and cross-reference it to https://docs.djangoproject.com/en/2.1/ref/models/fields/#model-field-types. This was just an example for you to understand how to define it.
After you have these figured out you can do the Job.objects.create(...yourdata).
You don't need to add an id field, because Django creates one by default for all models.
For the life of me I cannot figure this out and I'm having trouble finding information on it.
I have a Django view which accepts an argument which is a primary key (e.g: URL/problem/12) and loads a page with information from the argument's model.
I want to mock models used by my view for testing but I cannot figure it out, this is what I've tried:
#patch('apps.problem.models.Problem',)
def test_search_response(self, problem, chgbk, dispute):
problem(problem_id=854, vendor_num=100, chgbk=122)
request = self.factory.get(reverse('dispute_landing:search'))
request.user = self.user
request.usertype = self.usertype
response = search(request, problem_num=12)
self.assertTemplateUsed('individual_chargeback_view.html')
However - I can never get the test to actually find the problem number, it's as if the model does not exist.
I think that's because if you mock the entire model itself, the model won't exist because any of the functions to create/save it will have been mocked. If Problem is just a mock model class that hasn't been modified in any way, it knows nothing about interacting with the database, the ORM, or anything that could be discoverable from within your search() method.
One approach you could take rather than mocking models themselves would be to create FactoryBoy model factories. Since the test database is destroyed with each test run, these factories are a great way to create test data:
http://factoryboy.readthedocs.io/en/latest/
You could spin up a ProblemFactory like so:
class ProblemFactory(factory.Factory):
class Meta:
model = Problem
problem_id = factory.Faker("pyint")
vendor_num = factory.Faker("pyint")
chgbk = factory.Faker("pyint")
Then use it to create a model that actually exists in your database:
def test_search_response(self, problem, chgbk, dispute):
problem = ProblemFactory(problem_id=854, vendor_num=100, chgbk=122)
request = self.factory.get(reverse('dispute_landing:search', kwargs={'problem_id':problem.id}))
request.user = self.user
request.usertype = self.usertype
response = search(request, problem_num=854)
self.assertTemplateUsed('individual_chargeback_view.html')
I have a JSON file with data as such :
['dbname' : 'A', 'collection' : 'ACollection', 'fields' : ['name', 'phone_no', 'address']}
['dbname' : 'B', 'collection' : 'BCollection', 'fields' : ['name', 'phone_no', 'address', 'class']}
These are 2 examples amongst many other dictionaries of the same format.
I have a python code that does the following : Accepts 2 inputs from the user - phone_no and dbname. For example, the user enters phone_no as xxxxxxxxxx and dbname as A. The python code then reads the JSON file and matches the user input with the dictionary element having the name of the database as 'A'. It then opens the database 'A', opens the respective collection 'ACollection' and prints the respective fields of posts within the collection that have the phone_no value as xxxxxxxxxx. The databases are implemented with mongoDB.
I need to build a django rest api for this code. The end goal is to access the code from a browser. The user provides the 2 inputs in the browser and the code is executed, returning the data, which is displayed on the browser. I have gone through the django-rest framework documentation but I'm new to this whole concept and would like some guidance.
How do I implement these functions and create an API? What code should the models, serializers, views and urls files have related to my program?
models.py
from django.db import models
class App(object):
def __init__(self, phone_no, name, address, categories):
self.phone_no = phone_no
self.name = name
self.address = address
self.categories = categories
This is what I'm working with so far, to get started. The problem, however, is that the models class should essentially be dynamic. Ex: If 'A' is the database, the program returns 3 fields but if 'B' is the database, the program returns 4 values so I'm not sure what the models class would be like.
views.py
from django.views.decorators.csrf import csrf_exempt
from rest_framework.decorators import api_view
from rest_framework.response import Response
from pymongo import Connection
from models import App
from serializers import AppSerializer
import json
import pymongo
from os import listdir
import re
from django import forms
#csrf_exempt
#api_view(['GET'])
def pgs(request):
#connect to our local mongodb
db = Connection('localhost',27017)
#get a connection to our database
dbconn = db.general
dbCollection = dbconn['data']
if request.method == 'GET':
#get our collection
items = []
for r in dbCollection.find():
post = App(r["phone_no"],r["name"],r["address"],r["categories"])
items.append(post)
serializedList = AppSerializer(items, many=True)
return Response(serializedList.data)
Let's say you have identical tables in two different databases. We'd start by creating two db connections in settings.py. Let's say those are called db_a and db_b. We might model this as so:
class PhoneGenome(models.Model):
phone_no = models.CharField(length=255)
name = models.CharField(length=255)
# and so on...
class Meta:
# if database pre-exists, may want to use managed=False
managed = False
This gives us a model. Now we choose which database to pull from based on user input. In a view, you might have something like:
db_used = request.GET.get('database')
db_conns = {'a': 'db_a', 'b': 'db_b'}
if db_conns.has_key(db_used):
records = PhoneGenome.objects.using(db_conns[db_used]).filter( user_criteria_here)
The using() method in your queryset is what allows you to select which database to run the query against.
There's a lot to manage here potentially, so this would be a good time to look at the docs: https://docs.djangoproject.com/en/1.7/topics/db/multi-db/
And if you haven't already, you really should work through the Django tutorial at the very least before going much further.