I am creating a rest-api using the Python+Flask_Restful combo, find it amazing.
Currently, i am allowing user to run a sql-query using the browser and it works fine, but the problem is, headers information is not displayed in the response.
Here is the code that i am using:
class RunCustomeSQL(Resource):
def get(self, enter_db_name, query):
if not os.path.isfile(enter_db_name+'.db'):
raise BadRequest("Database Doesn't Exist. Please select a valid database")
conn = sqlite3.connect(enter_db_name+'.db')
search_out = []
cursor = conn.execute(query)
row = None
for row in cursor:
search_out.append(row)
if not row: #This means empty response
raise BadRequest("No Results Found")
conn.commit()
conn.close()
return search_out
While, this code works great, it doesn't print the header values in the json-response. The current response is :
[
[
"dusiri_bibi",
"11",
"----------",
" srt/None ",
"14.30 MB",
"2017-12-13 23:43:54",
"C:/Test_Software/vc_redist.x64.exe"
],
]
Expected Output :
[
[
"Machine Name" : "dusiri_bibi",
"LABEL" : "11",
"PERMISSIONS" : "----------",
"USER" : " srt/None ",
"SIZE" : "14.30 MB",
"CREATED" : "2017-12-13 23:43:54",
"FILENAME" : "C:/Test_Software/vc_redist.x64.exe"
],
]
All the above text such as "machine name, label etc." are my table headers, I am not sure how to print them along with my output.
What if the user runs select user, size from table_name only
or
What if the user runs select * from table_name
In both scenario's, the output should display the table headers
Thanks
UPDATE #1 (25 April) : I managed to answer my first question and able to display a proper json response if the user selects the SELECT * statement in SQL but still facing issue with the second piece of it
Here is the answer to first part if anyone is looking for it : Using Regex
row = None
if re.search(r'(?<=SELECT)(.*)(?=FROM)',query, re.IGNORECASE).group(1) == ' * ':
for row in cursor:
search_out.append({'NAME' : row[0], 'LABEL_NUMBER' : row[1], 'PERM' : row[2], 'USER' : row[3] , 'SIZE' : row[4], 'DATE' : row[5], 'FILENAME' : row[6]})
if not row: #This means empty response
raise BadRequest("No Results Found")
Part II : Unanswered Query:
For the second piece, i now have two list :
list_1 : [[u'LABEL_NUMBER', u'PERM', u'FILENAME']]
list_2 : [(u'11', u'----------', u'C:/Test_Software/26.avi'), (u'11', u'----------', u'C:/Test_Software/6.avi'), (u'11', u'-rwx------', u'C:/Test_Software/Debug/Current_Frame1.avi'), (u'10', u'-rwxrwx---', u'C:/Windows/WinSxS/boxed-split.avi')]
As you can see, i have two list and i want to combine them into a dict to show the response like this:
[
{
LABEL_NUMBER : '11' ,
PERM : '-----------',
FILENAME : 'C:/Test_Software/26.avi'
},
...
....
......
{
LABEL_NUMBER : '10' ,
PERM : '-rwxrwx---',
FILENAME : 'C:/Windows/WinSxS/boxed-split.avi'
},
]
i am using the following code to do the same :
chunks = [list_2[idx:idx+3] for idx in range(0, len(list_2), 3)]
output = []
for each in chunks:
output.append(dict(zip(list_1, each)))
print(output)
But, this is failing with "TypeError: unhashable type: 'list'", i understand that lists are mutable and which is why i am getting this error but then how can i get the desired dict response? what am i doing wrong here?
You can use a list comprehension combined with zip for this:
list_1 = [[u'LABEL_NUMBER', u'PERM', u'FILENAME']]
list_2 = [(u'11', u'----------', u'C:/Test_Software/26.avi'), (u'11', u'----------', u'C:/Test_Software/6.avi'), (u'11', u'-rwx------', u'C:/Test_Software/Debug/Current_Frame1.avi'), (u'10', u'-rwxrwx---', u'C:/Windows/WinSxS/boxed-split.avi')]
d = [dict(zip(list_1[0], i)) for i in list_2]
Result:
[{'FILENAME': 'C:/Test_Software/26.avi',
'LABEL_NUMBER': '11',
'PERM': '----------'},
{'FILENAME': 'C:/Test_Software/6.avi',
'LABEL_NUMBER': '11',
'PERM': '----------'},
{'FILENAME': 'C:/Test_Software/Debug/Current_Frame1.avi',
'LABEL_NUMBER': '11',
'PERM': '-rwx------'},
{'FILENAME': 'C:/Windows/WinSxS/boxed-split.avi',
'LABEL_NUMBER': '10',
'PERM': '-rwxrwx---'}]
Related
*New to Programming
Question: I need to use the below "Data" (two rows as arrays) queried from sql and use it to create the message structure below.
data from sql using fetchall()
Data = [[100,1,4,5],[101,1,4,6]]
##expected message structure
message = {
"name":"Tom",
"Job":"IT",
"info": [
{
"id_1":"100",
"id_2":"1",
"id_3":"4",
"id_4":"5"
},
{
"id_1":"101",
"id_2":"1",
"id_3":"4",
"id_4":"6"
},
]
}
I tried to create below method to iterate over the rows and then input the values, this is was just a starting, but this was also not working
def create_message(data)
for row in data:
{
"id_1":str(data[0][0],
"id_2":str(data[0][1],
"id_3":str(data[0][2],
"id_4":str(data[0][3],
}
Latest Code
def create_info(data):
info = []
for row in data:
temp_dict = {"id_1_tom":"","id_2_hell":"","id_3_trip":"","id_4_clap":""}
for i in range(0,1):
temp_dict["id_1_tom"] = str(row[i])
temp_dict["id_2_hell"] = str(row[i+1])
temp_dict["id_3_trip"] = str(row[i+2])
temp_dict["id_4_clap"] = str(row[i+3])
info.append(temp_dict)
return info
Edit: Updated answer based on updates to the question and comment by original poster.
This function might work for the example you've given to get the desired output, based on the attempt you've provided:
def create_info(data):
info = []
for row in data:
temp_dict = {}
temp_dict['id_1_tom'] = str(row[0])
temp_dict['id_2_hell'] = str(row[1])
temp_dict['id_3_trip'] = str(row[2])
temp_dict['id_4_clap'] = str(row[3])
info.append(temp_dict)
return info
For the input:
[[100, 1, 4, 5],[101,1,4,6]]
This function will return a list of dictionaries:
[{"id_1_tom":"100","id_2_hell":"1","id_3_trip":"4","id_4_clap":"5"},
{"id_1_tom":"101","id_2_hell":"1","id_3_trip":"4","id_4_clap":"6"}]
This can serve as the value for the key info in your dictionary message. Note that you would still have to construct the message dictionary.
I have following document in MongoDB 3.6
forum_collection = { '_id' : id,
'tags': tags,
'sub_topic_name' : sub_topic_name,
'topic_creator' : creator_name,
'main_forum_name' : forum_name,
'threads' : [ {'thread_id' = uuid4,
'thread_title = title,
'thread_author = author,
'thread_comment_cout = 0,
'thread_body' = content,
'thread_comments' = [ {'thread_comment_id' : uuid4,
'thread_comment_body': content,
'thread_commenter' : author,
'thread_comment_time' : time
},
]
'thread_time' = time,
},
],
SO I want to have forum_collection with multiple sub_topics, each sub_topics have threads and each thread have comments.
How can I add a new sub comment and update the thread_comment_count to 1 ? My following attempt fails miserably...my function passes three variables, sub_topic (to query the sub_topic_name), thread (to query thread_id) and thread_vals (to append new comment to thread_id)
forum_collection.find_one_and_update({'sub_topic_name': sub_topic, 'threads.thread_id' : thread},
{'$inc': {'threads.'+thread+'.thread_comments_count': 1},
'$addToSet': {thread + '.threads_comment': thread_vals}},
return_document=pymongo.ReturnDocument.AFTER
)
How does it fail?
I don't know if it was a copy/paste error but your "thread_comment_cout is misspelled .
Also the $addToSet should refer to "threads" not "thread"
I needed to add $ at appropriate places....so not 'threads.'+thread+'.thread_comments_count' but 'threads.$.thread_comments_count'...
forum_collection.find_one_and_update({'sub_topic_name': sub_topic, 'threads.thread_id' : thread},
{'$inc': {'threads.$.thread_comments_count': 1},
'$addToSet': {'threads.$.thread_comments': thread_vals}},
return_document=pymongo.ReturnDocument.AFTER
)
I use openpyxl to read data from excel files to provide a json file at the end. The problem is that I cannot figure out an algorithm to do a hierarchical organisation of the json (or python dictionary).
The data form is like the following:
The output should be like this:
{
'id' : '1',
'name' : 'first',
'value' : 10,
'children': [ {
'id' : '1.1',
'name' : 'ab',
'value': 25,
'children' : [
{
'id' : '1.1.1',
'name' : 'abc' ,
'value': 16,
'children' : []
}
]
},
{
'id' : '1.2',
...
]
}
Here is what I have come up with, but i can't go beyond '1.1' because '1.1.1' and '1.1.1.1' and so on will be at the same level as 1.1.
from openpyxl import load_workbook
import re
from json import dumps
wb = load_workbook('resources.xlsx')
sheet = wb.get_sheet_by_name(wb.get_sheet_names()[0])
resources = {}
prev_dict = {}
list_rows = [ row for row in sheet.rows ]
for nrow in range(list_rows.__len__()):
id = str(list_rows[nrow][0].value)
val = {
'id' : id,
'name' : list_rows[nrow][1].value ,
'value' : list_rows[nrow][2].value ,
'children' : []
}
if id[:-2] == str(list_rows[nrow-1][0].value):
prev_dict['children'].append(val)
else:
resources[nrow] = val
prev_dict = resources[nrow]
print dumps(resources)
You need to access your data by ID, so first step is to create a dictionary where the IDs are the keys. For easier data manipulation, string "1.2.3" is converted to ("1","2","3") tuple. (Lists are not allowed as dict keys). This makes the computation of a parent key very easy (key[:-1]).
With this preparation, we could simply populate the children list of each item's parent. But before doing that a special ROOT element needs to be added. It is the parent of top-level items.
That's all. The code is below.
Note #1: It expects that every item has a parent. That's why 1.2.2 was added to the test data. If it is not the case, handle the KeyError where noted.
Note #2: The result is a list.
import json
testdata="""
1 first 20
1.1 ab 25
1.1.1 abc 16
1.2 cb 18
1.2.1 cbd 16
1.2.1.1 xyz 19
1.2.2 NEW -1
1.2.2.1 poz 40
1.2.2.2 pos 98
2 second 90
2.1 ezr 99
"""
datalist = [line.split() for line in testdata.split('\n') if line]
datadict = {tuple(item[0].split('.')): {
'id': item[0],
'name': item[1],
'value': item[2],
'children': []}
for item in datalist}
ROOT = ()
datadict[ROOT] = {'children': []}
for key, value in datadict.items():
if key != ROOT:
datadict[key[:-1]]['children'].append(value)
# KeyError = parent does not exist
result = datadict[ROOT]['children']
print(json.dumps(result, indent=4))
I have a pymongo collection in the form of:
{
"_id" : "R_123456789",
"supplier_ids" : [
{
"id" : "S_987654321",
"file_version" : ISODate("2016-03-15T00:00:00Z"),
"latest" : false
},
{
"id" : "S_101010101",
"file_version" : ISODate("2016-03-29T00:00:00Z"),
"latest" : true
}
]
}
when I get new supplier data, if the supplier ID has changed, I want to capture that by setting latest on the previous 'latest' to False and the $push the new record.
$set is not working as I am trying to employ it (commented code after 'else'):
import pymongo
from dateutil.parser import parse
new_id = 'S_323232323'
new_date = parse('20160331')
with pymongo.MongoClient() as client:
db = client.transactions
collection_ids = db.ids
try:
collection_ids.insert_one({"_id": "R_123456789",
"supplier_ids": ({"id": "S_987654321",
"file_version": parse('20160315'),
"latest": False},
{"id": "S_101010101",
"file_version": parse('20160329'),
"latest": True})})
except pymongo.errors.DuplicateKeyError:
print('record already exists')
record = collection_ids.find_one({'_id':'R_123456789'})
for supplier_id in record['supplier_ids']:
print(supplier_id)
if supplier_id['latest']:
print(supplier_id['id'], 'is the latest')
if supplier_id['id'] == new_id:
print(new_id, ' is already the latest version')
else:
# print('setting', supplier_id['id'], 'latest flag to False')
# <<< THIS FAILS >>>
# collection_ids.update_one({'_id':record['_id']},
# {'$set':{'supplier_ids.latest':False}})
print('appending', new_id)
data_to_append = {"id" : new_id,
"file_version": new_date,
"latest": True}
collection_ids.update_one({'_id':record['_id']},
{'$push':{'supplier_ids':data_to_append}})
any and all help is much appreciated.
This whole process seems unnaturally verbose - should I be using a more streamlined approach?
Thanks!
You can try with positional operators.
collection_ids.update_one(
{'_id':record['_id'], "supplier_ids.latest": true},
{'$set':{'supplier_ids.$.latest': false}}
)
This query will update supplier_ids.latest = false, if it's true in document and matches other conditions.
The catch is you have to include field array as part of condition too.
For more information see Update
I want to build dynamic dictionary in python like this,,
users = {log_id : { "message_id" : "1", "sent_to" : "taqi.official#outlook.com" , "unique_arguments" : "03455097679"},
log_id : { "message_id" : "1", "sent_to" : "taqi.hass#cogilent.com" , "unique_arguments" : "03455097679" },
log_id : { "message_id" : "2 Turab", "sent_to" : "taqi.official#gmailllllll.com" , "unique_arguments" : "4534534535" }}
I have write this code but it is not building as i desired;
cur = conn.cursor()
cur.execute("select log_id, message_id, sent_to, unique_arguments from sendmessage_log_messages where log_status = 'Queue'")
rows = cur.fetchall()
count = 0
for row in rows:
temp['message_id'] = row[1]
temp['sent_to'] = str(row[2])
temp['unique_arguments'] = row[3]
log_dictionary[row[0]] = temp
print log_dictionary
It produces the this output,
{1: {'unique_arguments': 'log_8_taqi.official#gmailllllll.com', 'message_id': 8, 'sent_to': 'taqi.official#gmailllllll.com'},
2: {'unique_arguments': 'log_8_taqi.official#gmailllllll.com', 'message_id': 8, 'sent_to': 'taqi.official#gmailllllll.com'},
3: {'unique_arguments': 'log_8_taqi.official#gmailllllll.com', 'message_id': 8, 'sent_to': 'taqi.official#gmailllllll.com'},
4: {'unique_arguments': 'log_8_taqi.official#gmailllllll.com', 'message_id': 8, 'sent_to': 'taqi.official#gmailllllll.com'}}
Since no answer is given here, I am just giving the explanation here.
Here, you are replacing the temp dictionary again and again. When you do log_dictionary[row[0]] = temp, this just points to the existing temp dictionary. Thus whenever you change any value in the temp dictionary and you read the log_dictionary, this will always give you the updated temp dictionary only.
Try this:
for row in rows:
temp['message_id'] = row[1]
temp['sent_to'] = str(row[2])
temp['unique_arguments'] = row[3]
log_dictionary[row[0]] = temp
print log_dictionary
temp['message_id'] = 0
temp['send_to'] = 'stackoverflow'
temp['unique_arguments'] = None
print log_dictionary
This will give you the updated temp dictionary values as it is just referenced with the log_dictionary. As Ashwini Chaudhary mentioned in your comment, if you initialize a temp dictionary within the for loop before referencing it to log_dictionary, you will get the desired values. (eg)
for row in rows:
temp = {}
temp['message_id'] = row[1]
temp['sent_to'] = str(row[2])
temp['unique_arguments'] = row[3]
log_dictionary[row[0]] = temp