I am working on a Species Recommendation tool that will find all of the trees in a GPS range, average out the trees of allowed species, and then return the results in a JSON dict that can be parsed and read out.
I was working with folks from Code for SF who were using the K Nearest Neighbors algorithm, which was really nice, but kept crashing the application and was only pulling from a csv I uploaded, rather than the database of trees and planting sites I created.
Is there a way to implement K Nearest Neighbors to search my database that's better than using latitude__range and longitude__range? This is my current view:
def sg_species_guide_latlong_json(request, latitude=1, longitude=1):
min_latitude = decimal.Decimal(latitude)
min_latitude -= decimal.Decimal(0.01)
max_latitude = decimal.Decimal(latitude)
max_latitude += decimal.Decimal(0.01)
min_longitude = decimal.Decimal(longitude)
min_longitude -= decimal.Decimal(0.01)
max_longitude = decimal.Decimal(longitude)
max_longitude += decimal.Decimal(0.01)
response = JsonResponse(dict(species_list=list(RecommendedSpecies.objects.values(
"species__id_species", "species__trees_of_species__condition__score", "species__name_scientific",
).filter(
Q(species__trees_of_species__planting_site__coord_lat__range=[min_latitude, max_latitude]),
Q(species__trees_of_species__planting_site__coord_long__range=[min_longitude, max_longitude]),
).annotate(avg_condition=(Avg('species__trees_of_species__condition__score')),
num_trees=(Count('species__trees_of_species'))).order_by('-avg_condition'))))
return response
It's VERY slow, especially for what should be an AJAX/JSON call.
In addition, I want to print out the results in a format that contains the species name, number of trees, and average condition. Sorting that is easy in my view, but printing it out in JSON has been really tough. The dicts that JSON returns look like this:
{"species_list": [{"num_trees": 102, "species__trees_of_species__condition__score": 5, "species__id_species": 88, "avg_condition": 5.0, "species__name_scientific": "Arbutus x marina"}, {"num_trees": 828, "species__trees_of_species__condition__score": 4, "species__id_species": 88, "avg_condition": 4.0, "species__name_scientific": "Arbutus x marina"}]}
I would like to parse each object and display the scientific name, average condition, and number of trees.
Currently this is my HTML:
<script type="text/javascript">
function if_gmap_updateInfoWindow()
{
var lng = gmapmarker.getPosition().lng().toFixed(6)
var lat = gmapmarker.getPosition().lat().toFixed(6)
infoWindow.setContent("Longitude: " + lng + "<br>" + "Latitude: " + lat);
document.getElementById("location-info").innerHTML = 'Lat / Long: ' + lat + ' x ' + lng;
// Call the model api
// TODO consts
$.ajax({
url: '/species-guide/json/' + lat + '/' + lng,
dataType: 'json',
type: 'get',
success: function(response){
document.getElementById("species-recommendations").innerHTML = response['species_list'].join('<br>')
console.log(response)
}
})
} // end of if_gmap_bindInfoWindow
</script>
which returns a list of [object Object] instead of keys & values
I would love to have it spit out keys & values using something like this:
var species_recs = response['species_list']\
document.getElementById("species-recommendations").innerHTML =
species_recs.forEach(species => {
Object.keys(species).forEach(key => {
'<pre>' + species[key] + '</pre><br>';
})
})
but that returns undefined. Is there a better way to get it to yield the data?
Related
*New to Programming
Question: I need to use the below "Data" (two rows as arrays) queried from sql and use it to create the message structure below.
data from sql using fetchall()
Data = [[100,1,4,5],[101,1,4,6]]
##expected message structure
message = {
"name":"Tom",
"Job":"IT",
"info": [
{
"id_1":"100",
"id_2":"1",
"id_3":"4",
"id_4":"5"
},
{
"id_1":"101",
"id_2":"1",
"id_3":"4",
"id_4":"6"
},
]
}
I tried to create below method to iterate over the rows and then input the values, this is was just a starting, but this was also not working
def create_message(data)
for row in data:
{
"id_1":str(data[0][0],
"id_2":str(data[0][1],
"id_3":str(data[0][2],
"id_4":str(data[0][3],
}
Latest Code
def create_info(data):
info = []
for row in data:
temp_dict = {"id_1_tom":"","id_2_hell":"","id_3_trip":"","id_4_clap":""}
for i in range(0,1):
temp_dict["id_1_tom"] = str(row[i])
temp_dict["id_2_hell"] = str(row[i+1])
temp_dict["id_3_trip"] = str(row[i+2])
temp_dict["id_4_clap"] = str(row[i+3])
info.append(temp_dict)
return info
Edit: Updated answer based on updates to the question and comment by original poster.
This function might work for the example you've given to get the desired output, based on the attempt you've provided:
def create_info(data):
info = []
for row in data:
temp_dict = {}
temp_dict['id_1_tom'] = str(row[0])
temp_dict['id_2_hell'] = str(row[1])
temp_dict['id_3_trip'] = str(row[2])
temp_dict['id_4_clap'] = str(row[3])
info.append(temp_dict)
return info
For the input:
[[100, 1, 4, 5],[101,1,4,6]]
This function will return a list of dictionaries:
[{"id_1_tom":"100","id_2_hell":"1","id_3_trip":"4","id_4_clap":"5"},
{"id_1_tom":"101","id_2_hell":"1","id_3_trip":"4","id_4_clap":"6"}]
This can serve as the value for the key info in your dictionary message. Note that you would still have to construct the message dictionary.
I'm trying to convert some python code into C#. The python code uses a dictionary structure and I've created a similar object in C#. I'm hoping to get help on a specific section:
'''
letterHTMLTemplates = {}
cursor = conn.cursor()
cursor.execute("select lower(cast([id] as nvarchar(36))) as [id], [template] from dbo.
[lookup.letter]")
for row in cursor:
letterHTMLTemplates.update( {row.id : str(row.template)} )
# Replace the <head> tag in all letters, as it contains the confetti nonsense
for letterKey in letterHTMLTemplates:
startHead = letterHTMLTemplates[letterKey].find("<head>")
endHead = letterHTMLTemplates[letterKey].find("</head>") + len("</head>")
beforeHead = letterHTMLTemplates[letterKey][:startHead]
endHead = letterHTMLTemplates[letterKey][endHead:]
newHead = '<head><meta charset="utf-8"></head>' # Replace the head with this - necessary to render weird characters
letterHTMLTemplates[letterKey] = beforeHead + newHead + endHead
'''
I've written this so far in C# but I'm having trouble with the the find part in C#:
'''
public static void parseSlateDocs()
{
SqlConnection conn = new SqlConnection(GetSlateConnectionString());
string query = "select lower(cast([id] as nvarchar(36))) as [id], [template] from dbo.[lookup.letter]";
Dictionary<int, string> letterTemplate = new Dictionary<int, string>();
using (SqlCommand cmd = new SqlCommand(query,conn))
{
using (var reader = cmd.ExecuteReader())
{
while (reader.Read())
{
foreach(KeyValuePair<int,string> kvp in letterTemplate)
{
letterTemplate.Add(kvp.Key, kvp.Value);
//find the "<head>" tag
}
}
}
'''
Here you need IndexOf(string str) method:
string str = "Hello Friends....How are you...";
int i = str.IndexOf("How");
// As this string is present in the
// main string then it will obviously
// output the value as 17
Console.WriteLine("First value Index of 'How' is " + i);
// now the following string is not present
// So as per the rules, it will return -1
int i1 = str.IndexOf("Chair");
// As this string is present in
// the main string then it will
// obviously output the value as -1
Console.WriteLine("First value Index of 'Chair' is " + i1);
Use this method to find head.
Example from GeeksForGeeks.
I'm invoking a python script from node js. The python script retrieves data from a REST API and stores it in a dataframe and then there's a search function based on user input. I'm confused as to what variable type does python send the data to node js in? I've tried to convert into a string but in node js it says it is an unresolved variable type. Here's the code:
r = requests.get(url)
data = r.json()
nested = json.loads(r.text)
nested_full = json_normalize(nested)
req_data= json_normalize(nested,record_path ='items')
search = req_data.get(["name", "id"," ])
#search.head(10)
filter = sys.argv[1:]
print(filter)
input = filter[0]
print(input)
result = search[search["requestor_name"].str.contains(input)]
result = result.to_String(index=false)
response = '```' + str(result) + '```'
print(response)
sys.stdout.flush()
Here's the node js program that invokes the above python script. How do i store the output in a format which i can pass to another function in node?
var input = 'robert';
var childProcess = require("child_process").spawn('python', ['./search.py', input], {stdio: 'inherit'})
const stream = require('stream');
const format = require('string-format')
childProcess.on('data', function(data){
process.stdout.write("python script output",data)
result += String(data);
console.log("Here it is", data);
});
childProcess.on('close', function(code) {
if ( code === 1 ){
process.stderr.write("error occured",code);
process.exit(1);
}
else{
process.stdout.write('done');
}
});
According to the docs:
childProcess.stdout.on('data', (data) => {
console.log(`stdout: ${data}`);
});
1.
I had to send request.POST a dict of dict,
big_dict = {k:{kk:vv, kk1: vv1}, k1:{kk:vv, kk1: vv1}}
and ajax always somehow made it into one dict like
one_dict = {k[kk]:vv, k[kk1]:vv1, k1[kk]:vv, k1[kk1]:vv1}
someone told me to
JSON.stringify(big_dict)
before ajax request and then json.loads it in my python django views.
json.loads(request.POST.get('data'))
it worked all damn fine till someone uploaded a chinese character named file.
At the third step of json.loads in my view,
(wrong)
the key and value of my big_dict is still good, looking fine, displaying proper chinese character.
(edited)
the key and value of my big_dict, chinese characters in one value looks good, and not in the other.
{'0': {'designer': 'EV', 'number': '0229', 'version': '', 'bracket': '(120门幅)', 'imgUrl': 'http://127.0.0.1:8000/media/cache/temp/EV-0229(120%E9%97%A8%E5%B9%85%EF%BC%89.jpg'}}
but when I try to
for k, v in big_dict.items():
for kk, vv in v.items():
# do something with kk and vv
chinese characters within vv becomes all wrong.
and os is telling me no such file.
It looked like a simple problem, I searched a web for over an hour and tried things, nothing worked..
Thanks anyone in advance for helping me out.
EDIT
script from template
$('.designInfoSave').click(function() {
let num = $('.designBox').length;
let bcontext = {};
for (let i = 0; i<num; i++) {
let infoBox = $('#'+i+"-infoBox");
let image = $('#image'+i);
image.toArray();
let children = infoBox.children();
children.toArray();
bcontext[i] = {
'designer': children[2].value,
'number': children[4].value,
'version': children[6].value,
'bracket': children[8].value,
'imgUrl': image[0].currentSrc,
};
}
$.ajax({
type: 'POST',
url: "{% url 'update_des_info' %}",
data: {'csrfmiddlewaretoken': '{{ csrf_token }}', 'context':JSON.stringify(bcontext)},
dataType: "json",
success: function(response){
$("#album").hide();
$(".ajaxResult").show();
}
}
)
});
views.py
import json
import os.path
def update_des_info(request):
if request.method != 'POST':
return
queryset = json.loads(request.POST.get('context'))
print(queryset)
exist = 0
created = 0
for k, v in queryset.items():
designer = v['designer']
number = v['number']
version = v['version']
bracket = v['bracket']
fp = v['imgUrl'][v['imgUrl'].rfind('/media/')+6:]
n, c = Design.objects.get_or_create(designer=designer, number=number, version=version)
if c is True:
created += 1
else:
exist += 1
if n.bracket != bracket:
n.bracket = bracket
fpp = settings.MEDIA_ROOT + fp
p, fn = os.path.split(fpp)
nfp = os.path.join(settings.MEDIA_ROOT, 'designs', 'image', fn)
shutil.move(fpp, nfp)
if not n.image:
n.image = nfp
n.save()
I have been messing with Json a bit today and doing some trial and error but I couldn't seem to make this work:
def checkForNewBooty(chan):
j = urllib2.urlopen('http://tmi.twitch.tv/group/user/' + chan + '/chatters')
j_obj = json.load(j)
viewers = j_obj['chatters']['viewers']
moderators = j_obj['chatters']['moderators']
for x in viewers and moderators:
print(json.dumps('users' = {'johhny' = {'Points' = 0, 'Time Joined' = 9938}}))
Json example of what I'm trying to do:
{
users = {
"johhnyknoxville"
}
}
What is the proper way of doing this?
Python dictionaries (which are serialized using JSON) use : and not =.
Try:
json.dumps({'users': {'johhny': {'Points': 0, 'Time Joined': 9938}}})