Django & D3 - Best way to link csv files to front-end - python

I am using Python Django to create an interactive Dashboard. The main reason why I decided to use Django is because I can create users and groups to give different access to different people. Thought this was the better way.
I have massive reports built on excel sheets that I can convert to .CSV and machine readable. I read the best way to load data to D3 is through d3.csv(). I've tried that without success and now have tried with d3.json() but still I have used another code sample to try and plot my own but could have any luck.
The code is as follows. Important note: In dashboard.html, there is a selector, thus the HTML Template has that extends the "dashboard" template.
# app/models.py
from django.db import models
class wrtes(models.Model):
name = models.CharField(max_length=150)
date = models.DateTimeField()
# app/urls.py
from django.conf.urls import url
from . import views
from .views import graph, oneModel
urlpatterns = [
# Dashboard parts
url(r'^oneModel/', graph),
url(r'^api/oneModel', oneModel, name='oneModel'),
]
# app/views.py
def graph(request):
return render(request, 'app/oneModel.html')
def oneModel(request):
data = oneModel.objects.all() \
.extra(select={'month': connections[oneModel.objects.db].ops.date.trunc_sql('month', 'date')}) \
.values('month') \
.annotate(count_items=Count('id'))
return JsonResponse(list(data), safe=False)
<!-- app/oneModel.html -->
{% extends 'app/dashboard.html' %}
{% load staticfiles %}
{% block content %}
<script src="https://d3js.org/d3.v3.js"></script> <!-- D3 obligatory tags -->
<script>
var margin = {top: 20, right: 20, bottom: 30, left: 50},
width = 960 - margin.left - margin.right,
height = 500 - margin.top - margin.bottom;
var parseDate = d3.time.format("%Y-%m-%d").parse; // for dates like "2014-01-01"
//var parseDate = d3.time.format("%Y-%m-%dT00:00:00Z").parse; // for dates like "2014-01-01T00:00:00Z"
var x = d3.time.scale()
.range([0, width]);
var y = d3.scale.linear()
.range([height, 0]);
var xAxis = d3.svg.axis()
.scale(x)
.orient("bottom");
var yAxis = d3.svg.axis()
.scale(y)
.orient("left");
var line = d3.svg.line()
.x(function(d) { return x(d.month); })
.y(function(d) { return y(d.count_items); });
var svg = d3.select("body").append("svg")
.attr("width", width + margin.left + margin.right)
.attr("height", height + margin.top + margin.bottom)
.append("g")
.attr("transform", "translate(" + margin.left + "," + margin.top + ")");
d3.json("{% url "oneModel" %}", function(error, data) {
data.forEach(function(d) {
d.month = parseDate(d.month);
d.count_items = +d.count_items;
});
x.domain(d3.extent(data, function(d) { return d.month; }));
y.domain(d3.extent(data, function(d) { return d.count_items; }));
svg.append("g")
.attr("class", "x axis")
.attr("transform", "translate(0," + height + ")")
.call(xAxis);
svg.append("g")
.attr("class", "y axis")
.call(yAxis)
.append("text")
.attr("transform", "rotate(-90)")
.attr("y", 6)
.attr("dy", ".71em")
.style("text-anchor", "end")
.text("Play count");
svg.append("path")
.datum(data)
.attr("class", "line")
.attr("d", line);
});
</script>
{% endblock %}
When running the local server, it does not render anything in the graphics part. It renders the remaining parts - navbar, selector & so on. It makes me consider the dataset is not being loaded in any way. This one in particular, does not even make sense :-D But when I tried with the d3.csv() method, it was not working either way.
What is wrong with the code?
What is the best way to pass .csv files to D3 to plot cute stuff? If you'll be referencing guides, please consider I am not a programmer nor a very experienced one.
Thank you for your time.

Related

How to print leaflet map in pdf using django-wkhtmltopdf?

I have a django app I am trying to print a leaflet map in pdf using django-wkhtmltopdf, but the map is not rendering correctly. I even try to give more time to the 'javascript-delay' parameter but is giving me the same result.
Here is the image of the map:
from wkhtmltopdf.views import PDFTemplateResponse
class MyPDFView(View):
template='agrimensuras/project_pdf.html' # the template
def get(self, request, pk):
project = get_object_or_404(Project, id = int(pk))
data = {"project": project}
response = PDFTemplateResponse(request=request,
template=self.template,
filename="hello.pdf",
context= data,
show_content_in_browser=False,
cmd_options={'margin-top': 10,
"zoom":1,
"viewport-size" :"1366 x 513",
'javascript-delay':1000,
'footer-center' :'[page]/[topage]',
"no-stop-slow-scripts":True},
)
return response
In the template (I am just showing the relevant pieces of code):
<!--Importing leaflet-->
<link rel="stylesheet" href="https://unpkg.com/leaflet#1.9.3/dist/leaflet.css"
integrity="sha256-kLaT2GOSpHechhsozzB+flnD+zUyjE2LlfWPgU04xyI="
crossorigin=""/>
<script src="https://unpkg.com/leaflet#1.9.3/dist/leaflet.js"
integrity="sha256-WBkoXOwTeyKclOHuWtc+i2uENFpDZ9YPdf5Hf+D7ewM="
crossorigin=""></script>
<!--Defining the div-->
<div id="map" ></div>
<!--Defining the map-->
<script type="text/javascript">
var map = L.map('map').setView([51.505, -0.09], 13);
L.tileLayer('https://tile.openstreetmap.org/{z}/{x}/{y}.png', {
minZoom: 5,
maxZoom: 19,
attribution: '© OpenStreetMap'
}).addTo(map);
var marker = L.marker([51.5, -0.09]).addTo(map);
var circle = L.circle([51.508, -0.11], {
color: 'red',
fillColor: '#f03',
fillOpacity: 0.5,
radius: 500
}).addTo(map);
var polygon = L.polygon([
[51.509, -0.08],
[51.503, -0.06],
[51.51, -0.047]
]).addTo(map);
</script>

Saving image captured from webcam

I'm building a Django project in which I need to take a picture from webcam, and then store it in my database and as a file. I'm saving the source in the database, but I'm having some trouble saving it as a file.
Here is my code:
html:
<form method="POST" action="{% url 'takePhoto' %}" enctype="multipart/form-data">
{% csrf_token %}
<video id="video" autoplay ></video>
<canvas id="canvas"></canvas>
<input type="hidden" name="photo" id="photo" value=""/>
<button id="startbutton1" class="btn btn-outline-secondary btn-sm">Take Photo</button>
<button id="submit" type="submit">submit</button>
<script src="{% static 'scripts/script.js' %}"></script>
javascript:
(function() {
var width = 320;
var height = 0;
var streaming = false;
var video = null;
var canvas = null;
var photo = null;
var startbutton1 = null;
function startup() {
video = document.getElementById('video');
canvas = document.getElementById('canvas');
photo = document.getElementById('photo');
startbutton1 = document.getElementById('startbutton1');
navigator.mediaDevices.getUserMedia({video: true, audio: false})
.then(function(stream) {
video.srcObject = stream;
video.play();
})
.catch(function(err) {
console.log("An error occurred: " + err);
});
video.addEventListener('canplay', function(ev){
if (!streaming) {
height = video.videoHeight / (video.videoWidth/width);
if (isNaN(height)) {
height = width / (4/3);
}
video.setAttribute('width', width);
video.setAttribute('height', height);
canvas.setAttribute('width', width);
canvas.setAttribute('height', height);
streaming = true;
}
}, false);
startbutton1.addEventListener('click', function(ev){
takepicture();
ev.preventDefault();
}, false);
clearphoto();
}
function clearphoto() {
var context = canvas.getContext('2d');
context.fillStyle = "#AAA";
context.fillRect(0, 0, canvas.width, canvas.height);
var data = canvas.toDataURL('image/png');
photo.setAttribute('src', data);
}
function takepicture() {
var context = canvas.getContext('2d');
if (width && height) {
canvas.width = width;
canvas.height = height;
context.drawImage(video, 0, 0, width, height);
var data = canvas.toDataURL('image/png');
photo.value=data;
} else {
clearphoto();
}
}
window.addEventListener('load', startup, false);
})();
views:
def takePhoto(request):
print("works")
if not request.session.get('logged_in'):
return redirect('/appChallenge/login')
if request.method== 'POST':
user = User.objects.get(username=request.session["username"])
img = request.POST.get("photo")
image = img
imagePath="/media"
a=str(img)
image = image.save(f"{imagePath}/{a}.png")
imgInfo= Image(user=user, img=img)
imgInfo.save()
print("works")
return render(request, 'page.html')
When I click submit, it says "'str' object has no attribute 'save'"
Please help. Thanks.
If your js/html code works, you will receive a raw image data encoded to base64 in your view 'takePhoto'. Try
print(img) to see something like "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAUAAAADwCAYAA...."
Maybe you will see just "iVBORw0KGgoAAAANSUhEUgAAAUAAAADwCAYAA...." without meta data.
So your 'img' is just a string with base64 encoded value, that hasn't method 'save'.
First, you need to get rid of "data:image/png;base64," in your raw image data, if your image string contains it. It is meta data and base64 could not decode it. You can do it at front end side:
var data = canvas.toDataURL('image/png').replace(/^data:image\/png;base64,/, "")
Or back end:
img = request.POST.get("photo").replace('data:image/png;base64,', '')
Next you need to use django ContentFile to create a file-like object. You need to simulate file with its method .read(). Also you need to decode base64 encoded image data:
import base64
from django.core.files.base import ContentFile
def takePhoto(request):
print("works")
if not request.session.get('logged_in'):
return redirect('/appChallenge/login')
if request.method== 'POST':
user = request.user
# remove meta data from base64 encoded data, you can also
# use 'split(',')[1]' to remove all before ','
img = request.POST.get("photo").replace('data:image/png;base64,', '')
# create a file-like object with your image data
image_file_like = ContentFile(base64.b64decode(img))
# first you need to create object
image = Image(user=user)
# and then save image into your model object
# note: only if your 'img' field is 'ImageField' or similar
image.img.save("filename.png", image_file_like)
image.save()
print("works")
return render(request, 'page.html')
P.S:
You don't need to add 'media' prefix in your image file name while saving it.
You should add MEDIA_ROOT variable in your settings.py like this:
BASE_DIR = Path(__file__).resolve().parent.parent # or manually set the path of your project
MEDIA_ROOT = os.path.join(BASE_DIR, 'media')
And all files automatically will be saved in your project media directory.

Generate dynamic div containers on submit in Flask app

I am new in Flask. My goal is to generate dynamic div containers every time I upload a dxf file and next submit happens. For example: one file uploaded- one div shown; two files uploaded- two divs shown and so on.
I can convert uploaded dxf files to .png images and I would like to show these images in div elements displayed after every upload.
I use input tag to upload files (type='file') and Java Script to generate dynamic elements (divs and their child tags).
The problem is that every time I upload file, the template is loading again and no new content is shown except the image of the last uploaded dxf. Please, give me a piece of advice to solve it.
HTML
...
<form enctype="multipart/form-data" id="uploadForm" action="/upload_files" name="uploadForm" method="post">
DXF file: <input type="file" id="dxfUpload" onchange="form.submit(); createConfigure();" name="dxfUpload" />
<div id="calcHolder" name="calcHolder">
<script type="text/javascript">
function createConfigure() {
var div = document.createElement("div");
div.id = "dxf-"+Math.random() * 100000000000000000 + "-"
+ window.performance.now() * 100000000000000000;
id_div=div.id;
div.className = 'border pad';
div.style.width = "640px";
div.style.height = "200px";
document.getElementById("calcHolder").appendChild(div);
var img = document.createElement("img");
img.setAttribute("src", "{{url_for('static', filename=dxfName+'.png')}}");
img.setAttribute("alt", "no image");
img.setAttribute("height", "120px");
img.setAttribute("width", "120px");
document.getElementById(id_div).appendChild(img);
var array = ["Carbon Steel","Stainless Steel","Aluminium"];
var selectMaterial = document.createElement("select");
document.getElementById(id_div).appendChild(selectMaterial);
for (var i = 0; i < array.length; i++) {
var option = document.createElement("option");
option.value = array[i];
option.text = array[i];
selectMaterial.appendChild(option);
}
var selectThickness = document.createElement("select");
document.getElementById(id_div).appendChild(selectThickness);
for (i = 1; i <= 16; i++) {
var opt = document.createElement('option');
//opt.value = i;
opt.innerHTML = i + ' mm';
selectThickness.appendChild(opt);
}
var quantity = document.createElement("input")
quantity.type="number";
quantity.value="1";
quantity.name="quantity";
quantity.min="1";
quantity.max="50";
quantity.onkeyup= function maxReach(){if(quantity.value > 50) quantity.value=50;};
document.getElementById(id_div).appendChild(quantity);
var btn = document.createElement("button");
btn.innerHTML = "Delete";
btn.type = "button";
document.getElementById(id_div).appendChild(btn);
btn.onclick = function() {div.remove();};
}
</script>
{{ html | safe }}
</div>
</form>
...
Python
#app.route('/upload_files', methods=['POST'])
def upload_files():
try:
if request.method == 'POST':
dxf_file = request.files['dxfUpload']
full_filename = os.path.join(app.config['UPLOAD_FOLDER'],dxf_file.filename)
dxf_file.save(full_filename)
first = DXF2IMG()
first.convert_dxf2img([full_filename],img_format='.png')
html="<img src="+url_for('static', filename=dxf_file.filename+'.png' )+" width='120' height='120' />"
return render_template('upload_files.html',dxfName=dxf_file.filename, html=html)
except:
...
#something happens
The result now
Desired result
Once the form.submit() function is executed, the form will be sent as a regular post request. For this reason, the following function is no longer executed and the entire page is reloaded.
In order to submit the form and change the content of the existing page, it is necessary to use AJAX.
This example shows you how to submit the form to the server and receive a JSON response containing the URLs of the received file and the generated image.
As soon as the submit button is pressed, the form data is packed into a FormData object and sent via AJAX using the fetch function. The browser's default behavior for a submit event is suppressed and the form is reset. The received file is processed by the server and the associated URLs are sent back to the client in JSON format. Now the document can be changed with the received data.
Remember this is just a minimal example to help you achieve your goals and implement your concept.
Flask (app.py)
import os
import ezdxf
from ezdxf.addons.drawing import matplotlib
from flask import Flask
from flask import (
jsonify,
make_response,
render_template,
url_for
)
from werkzeug.utils import secure_filename
app = Flask(__name__)
#app.route('/')
def index():
return render_template('index.html')
def dxf2png(source, target):
doc = ezdxf.readfile(source)
msp = doc.modelspace()
auditor = doc.audit()
if auditor.has_errors:
raise Exception('Conversion failed.')
matplotlib.qsave(doc.modelspace(), target)
#app.route('/upload', methods=['POST'])
def upload():
if 'dxf-file' in request.files:
file = request.files['dxf-file']
if file.filename != '':
filename = secure_filename(file.filename)
filepath = os.path.join(app.static_folder, filename)
destname, _ = os.path.splitext(filename)
destname = f'{destname}.png'
destpath = os.path.join(app.static_folder, destname)
file.save(filepath)
try:
dxf2png(filepath, destpath)
except:
os.remove(filepath)
return make_response('', 400)
return make_response(
jsonify(
target=url_for('static', filename=filename),
preview=url_for('static', filename=destname)
),
200
)
return make_response('', 400)
HTML (templates/index.html)
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8">
<title>Index</title>
<style media="screen">
.preview {
width: 120px;
height: auto;
}
</style>
</head>
<body>
<form name="dxf-upload" method="post" enctype="multipart/form-data">
<input type="file" name="dxf-file" />
<input type="submit" />
</form>
<div id="dxf-files"></div>
<script type="text/javascript">
((uri) => {
function createPreview(target, preview) {
const divElem = document.createElement('div');
divElem.innerHTML = `<img src="${preview}" class="preview" />`;
const outElem = document.getElementById('dxf-files');
outElem.append(divElem);
}
const form = document.querySelector('form[name="dxf-upload"]');
form.addEventListener('submit', evt => {
evt.preventDefault();
const formData = new FormData(evt.target);
fetch(uri, {
method: 'POST',
body: formData
}).then(resp => resp.json())
.then(data => {
const { target, preview } = data;
createPreview(target, preview);
});
evt.target.reset();
});
})({{ url_for('.upload') | tojson }});
</script>
</body>
</html>

Jinja .css template logic

I am wiring my framework to wagtail. It is a custom module css framework called reely.io that is made for Hubspot. It has jinja variables for breakpoints, colors, etc. one of the variables is a multiplier for padding and margin settings.
I have the css file set up and coming through to the template. the issue is with any logic in the css file:
{{ spacing_multiplier * 1 + 'px' }}
{{ break_sm + 30 + 'px' }}
Yes, this works in Hubspot's HubL, which I always considered a derivative of Jinja, but the syntax is a bit different. am I doing it wrong?
views.py
from django.views.generic import TemplateView
class Themes(TemplateView):
template_name = "css/reely.css"
content_type = "text/css"
def get(self, request, *args, **kwargs):
context = self.get_context_data(**kwargs)
''' Responsive Breaks '''
context["break_xl"] = 1773
context["break_lg"] = 1292
context["break_md"] = 768
context["break_sm"] = 576
context["break_xs"] = 396
''' Spacing Multiplier '''
context["spacing_multiplier"] = 10
''' Font '''
context['google_font_link'] = "https://fonts.googleapis.com/css2?family=IBM+Plex+Sans:wght#300;400;600;700&display=swap"
context['font_family_1'] = "'IBM Plex Sans', sans-serif;"
''' Color Set 1 '''
context["primary_color"] = '#333333' '''rgb(51,51,51)'''
context["secondary_color"] = '#555555' '''rgb(85,85,85)'''
context["tertiary_color"] = '#FFFFFF' '''rgb(255,255,255)'''
''' Color Set 2 '''
context["primary_font_color"] = '#333333' '''rgb(51,51,51)'''
context["secondary_font_color"] = '#555555' '''rgb(85,85,85)'''
context["tertiary_font_color"] = '#FFFFFF' '''rgb(255,255,255)'''
''' Color Set 3 '''
context["accent_color_1"] = '#DD0000' '''rgb(221,0,0)'''
context["accent_color_2"] = '#84C318' '''rgb(132,195,24)'''
context["accent_color_3"] = '#F96900' '''rgb(249,105,0)'''
context["success_color"] = '#28A745' '''rgb(40,167,69)'''
context["info_color"] = '#17A2B8' '''rgb(23,162,184)'''
context["warning_color"] = '#FFC107' '''rgb(255,193,7)'''
context["danger_color"] = '#DC3545' '''rgb(255,193,7)'''
return self.render_to_response(context)
urls.py
from django.urls import path
from . import views
urlpatterns = [
path("config/reely/css", views.Themes.as_view(),
name="reelycss"),
]
expected output:
context["spacing_multiplier"] = 10
{{spacing multiplier * 30 +'px'}} = 10 * 3 = 30 + 'px' = '30px'
and I get that there are strings and ints here HubL doesn't have an issue with it. This doesn't work either.
context["spacing_multiplier"] = 10
{{spacing multiplier * 30 }} = 10 * 3 = 30 = 30
The answer is to use a custom template filter.
css:
{% load reely_tags %}
.ry [class*="pr3" i] {
padding-right: {{spacing_multiplier|space:3}};
box-sizing: border-box;
}
app/templatetags/reely_tags.py:
from django import template
register = template.Library()
#register.filter(name="space")
def space(value, size):
output = str(value * size)
return f"{output}px"
output css:
.ry [class*="pr3" i] {
padding-right: 30px;
box-sizing: border-box;
}
I'd be interested to know how Hubspot made HubL compared to Jinja2 now, or even what makes the difference in this type of functionality, but custom template tags are a good solution for this. I'm now able to wire the context variables for the css into my wagtail admin, which was the goal.

Call ajax on dropdown button

I've created a form and populating its value from Category model in drop down box. Its working.
I displayed this form in template like this {{ language_form }}.It worked. Now I want to implement onchange event on this drop down with ajax. Function will call on change of drop down value.
Edit:it can be done like this without django forms . <select onchange='ajaxfunction()'></select> But I'm using django forms.
Form
from django import forms
from myapp.movie.models import Category
class Language(forms.Form):
language = forms.ModelChoiceField(queryset=Category.objects.all())
Here is my Ajax function
function showMovie(str) {
if (str == "") {
document.getElementById("txtHint").innerHTML = "";
return;
}
if (window.XMLHttpRequest) {// code for IE7+, Firefox, Chrome, Opera, Safari
xmlhttp = new XMLHttpRequest();
}
else {// code for IE6, IE5
xmlhttp = new ActiveXObject("Microsoft.XMLHTTP");
}
xmlhttp.onreadystatechange = function() {
if (xmlhttp.readyState == 4 && xmlhttp.status == 200) {
document.getElementById("txtHint").innerHTML = xmlhttp.responseText;
}
}
xmlhttp.open("GET", "movie_list?q=" + str, true);
xmlhttp.send();
}
If you want to just add onchange attr, then:
class Language(forms.Form):
language = forms.ModelChoiceField(queryset=Category.objects.all(), widget=forms.Select(attrs={'onchange':'ajaxfunction()'}))
Or you can render form manually.
Add this code in the end of your form.
...
</form>
<script type="text/javascript">
var ddl = document.getElementById('ddlYourDropDownListID');
ddl.onchange = function() {
var str = 'someValue';
showMovie(str);
};
</script>
</body>
</html>

Categories

Resources