I am trying to create a search function in tkinter from a sql table.
for konto in sql_konto_search(db_file, "kontoplan", keywords):
tv.insert('', tk.END, values=((konto[konto_q][0], konto[konto_q][1]), konto[konto_q][2]), tags=('fg', 'fs', 'bg', 'anch'))
konto_q += 1
if not keywords:
tv.delete(*tv.get_children())
tv_index()
Here is my SQL code (function). I am trying to create a query that is searching for multiple keywords - but it is not working that well.
def sql_konto_search(db_file, table, keywords):
keywords_split = keywords.split()
keywords_list = []
for keyword in keywords_split:
keywords_list.append(keyword)
ant_que = len(keywords_list)
keywords_list = tuple(keywords_list)
try:
rows = []
conn = sqlite3.connect(db_file)
cur = conn.cursor()
xq = 0
for ant_q in range(ant_que):
cur.execute(f"SELECT konto, konto_t, beskrivelse FROM {table} WHERE konto_t LIKE ('%{keywords_list[xq]}%') OR beskrivelse LIKE ('%{keywords[xq]}%')")
xq += 1
rows.append(cur.fetchall())
return rows
except Error as e:
print(e)
As you can see "keywords" are user input keywords - and I want to search for every single word.
Any suggestions?
You'll want something like this to properly compose a single SQL query from your keyword search string.
This also properly uses parameter substitution, so your program is no longer vulnerable to SQL injection issues. (I recommend reading the sqlite3 module documentation; search for "Never do this".)
import sqlite3
def sql_konto_search(db_file, table, keywords):
# Get an unique set of keywords from the string
keywords_set = set(keyword.strip() for keyword in keywords.split())
# Initialize a list for the where clauses we'll OR together
where_clauses = []
# Initialize a list for the `?` parameter placeholders.
parameters = []
for keyword in keywords_set:
# If the keyword is empty, skip it.
if not keyword:
continue
# Add a parenthesized fragment for the search with two parameter placeholders...
where_clauses.append("(konto_t LIKE ? OR beskrivelse LIKE ?)")
# ... so add two parameters.
keyword_wildcard = f"%{keyword}%"
parameters.append(keyword_wildcard)
parameters.append(keyword_wildcard)
# Compose the final query. Start with the select...
query_fragments = [f"SELECT konto, konto_t, beskrivelse FROM {table}"]
if where_clauses:
# and if there are where clauses, add the WHERE
# and join the clauses with ORs (they're already parenthesized above)
query_fragments.append(" WHERE ")
query_fragments.append(" OR ".join(where_clauses))
# Join the fragments into a single SQL statement...
sql = "".join(query_fragments)
print(sql, parameters) # Just so you can see what happens.
# ... and execute it.
with sqlite3.connect(db_file) as db:
cur = db.cursor()
cur.execute(sql, parameters)
return cur.fetchall()
Related
I am trying to query on a local MySQL database using Python's (3.4) MySQL module with the following code:
class databases():
def externaldatabase(self):
try:
c = mysql.connector.connect(host="127.0.0.1", user="user",
password="password", database="database")
if c.is_connected():
c.autocommit = True
return(c)
except:
return(None)
d = databases().externaldatabase()
c = d.cursor()
r = c.execute('''select * from tbl_wiki''')
print(r)
> Returns: None
As far as I can tell, the connection is successful, the database is composed of several rows but the query always returns the none type.
What instances does the MySQL execute function return None?
Query executions have no return values.
The pattern you need to follow is:
cursor creation;
cursor, execute query;
cursor, *fetch rows*;
Or in python:
c = d.cursor()
c.execute(query) # selected rows stored in cursor memory
rows = c.fetchall() # get all selected rows, as Barmar mentioned
for r in rows:
print(r)
Also some db modules allow you to iterate over the cursor using the for...in pattern, but triple-check that regarding mysql.
For my case, I return the cursor as I need the value to return a string specifically, for instance, I return the password (string) for inspect whether user used the same password twice. Here's how I did it (In my case):
def getUserPassword(metadata):
cursorObject.execute("SELECT password FROM users WHERE email=%s AND password=%s LIMIT 1", (metadata['email'], metadata['password']))
return cursorObject.fetchall()[0]['password']
Which I can easily call from another class by calling the method:
assert getUserPassword({"email" : "email", "password" : "oldpass"}) is not None
And which the getUserPassword itself is returning a string
I am currently trying to make a command line todo manager that will allow the user to input a task(s), remove it and list the task(s) out. From what I tried visualizing it didn't do as I thought it would, it's my first time using sqlite3.
What I am trying to achieve:
Storing the task(s) in the database which will automatically add an incrementing ID to it.
Example:
python todo.py -add do the laundry on Sunday
[in the database]
Id Task
1 do the laundry on Sunday
My code.
import sqlite3
import argparse
def parse_args():
desc = 'Todo manager for storing and removing tasks'
parser = argparse.ArgumentParser(description=desc)
parser.add_argument("-a", "--add", "-add", help="To add a new item to the list",
type=str, nargs="+")
parser.add_argument("-r", "-remove", "--remove", help="To remove an item from the list",
type=int)
parser.add_argument("-l", "-list", "--list", help="displays the tasks or task in the list",
nargs="*")
args = parser.parse_args()
return args
#staticmethod
def dict_factory(cursor, row):
d = {}
for idx, col in enumerate(cursor.description):
d[col[0]] = row[idx]
return d
def get_todo_list():
database_connection.row_factory = dict_factory
cursor = database_connection.cursor()
cursor.execute("select rowid, * FROM todo_list")
return cursor.fetchall()
def add_to_todo_list(num,task):
cursor = database_connection.cursor()
cursor.execute("INSERT INTO todo_list VALUES (?)", (str(task),))
database_connection.commit()
def remove_from_todo_list(rowid):
cursor = database_connection.cursor()
cursor.execute("DELETE FROM todo_list WHERE rowid = ?", (rowid,))
database_connection.commit()
if __name__ == '__main__':
commands = parse_args()
# Creating table for database using sqlite
database_connection = sqlite3.connect('todo_list.db')
cursor = database_connection.cursor()
cursor.execute('''CREATE TABLE if not exists todo_list(
description TEXT);''')
database_connection.commit()
if commands.add:
# Stops accepting tasks when there is a blank task as input.
if not commands.add == ' ':
add_to_todo_list(commands.add)
elif commands.remove:
remove_from_todo_list(commands.remove)
elif commands.list:
get_todo_list()
However, my database is not accepting any values when I am trying to store data. By putting Id as Id INTEGER PRIMARY KEY when creating the table i.e.
cursor.execute('''CREATE TABLE if not exists todo_list(
Id INTEGER PRIMARY KEY
description TEXT);''')
Will the Id increment as I add data to the database?
sqlite3.InterfaceError: Error binding parameter 1 - probably unsupported type.
Your inputs from argparse are coming in as str, yet you defined your column ID as an INTEGER in your db. Fix is:
cursor.execute("INSERT INTO todo_list VALUES (?,?)", (int(num), task,))
Storing the task(s) in the database which will automatically add an incrementing ID to it.
According to the sqlite docs here, defining a INTEGER PRIMARYKEY will auto-increment. Simply pass a null value to it, and sqlite takes care of the rest for you.
You have a few issues on your code when it comes to displaying and adding the tasks. First, initializing the DB:
cursor.execute(
"""CREATE TABLE if not exists todo_list(
id INTEGER PRIMARY KEY,
description TEXT);"""
)
How you had it before your post edit was fine. Then, the add_to_todo_list:
def add_to_todo_list(task):
cursor = database_connection.cursor()
cursor.execute("INSERT INTO todo_list VALUES (?,?)", (None, str(task)))
database_connection.commit()
Notice the removal of num from the functions input, and the passing of None for the column ID. Within the get_todo_list() you can fetch it more easily as so:
def get_todo_list():
cursor = database_connection.cursor()
cursor.execute("select * FROM todo_list")
return cursor.fetchall()
A fix is also needed in the way you parse your args; for commands.list you need to do the following:
elif commands.list is not None:
print(get_todo_list())
This is since commands.list will be a [] when you do app.py -list, which Python evaluates to False (empty lists are falsey). You also ought to print the contents of the function to terminal -- so don't forget that. With the edits above I can do on my terminal:
python test.py -add Random Task!
python test.py -add Hello World!
python test.py -list
[(1, "['Random', 'Task!']"), (2, "['Hello', 'World!']")]
I'm learning programming with python and trying to implement the safest possible MySQL queries starting with the simple SELECT ones. The problem is whenever I use coma in a query I got the following error:
cursor.execute(query)
File "C:\Users\username\AppData\Local\Programs\Python\Python37-32\lib\site-packages\mysql\connector\cursor.py", line 536, in execute
stmt = operation.encode(self._connection.python_charset)
AttributeError: 'tuple' object has no attribute 'encode'
I am aware of the fact that coma itself isn't a source of a problem but I tried many different MySQL syntax and everytime I use a come I got this "AttributeError: 'tuple' object has no attribute 'encode'" error.
I also tried to change MySQL database encoding - nothing changes. The code is below.
import mysql.connector
conn = mysql.connector.connect(
charset='utf8',
# init_command='SET NAMES UTF8',
host="10.0.0.234",
user="x",
passwd="x>",
database="x",
)
print(conn.is_connected())
param = "test"
cursor = conn.cursor()
# =========== query below does work ========
# query = ("SELECT * from list WHERE username LIKE '%test%'")
# ============ query below does work =======
# query = ("SELECT * from list HAVING username = '%s'" % param)
# ============ query below doesn't work =====
# query = ("SELECT * from list HAVING username = %s", (param,))
# ============= query below doesn't work =====
query = "SELECT * from list WHERE username = :name", {'name': param}
cursor.execute(query)
result = cursor.fetchall()
for x in result:
print(x)
conn.close()
Any ideas what am I doing wrong?
The answer is a little bit tricky, but it is in essence because of what the actual value of the 'query' variable is...
For example:
# 1.
query = ("SELECT * from list WHERE username LIKE '%test%'")
# when you do this, query is a string variable,
# NB: the parentheses are not necessary here
# so when you call
cursor.execute(query)
# the value passed into the execute call is the string "SELECT * from list WHERE username LIKE '%test%'"
# 2.
query = ("SELECT * from list HAVING username = '%s'" % param)
# when you do this, query is the result of a string formatting operation
# This is a Python 2 form of string formatting
# The discussion here probably makes it more clear:
# https://stackoverflow.com/questions/13945749/string-formatting-in-python-3
# it is almost the same as doing this:
query = "SELECT * from list HAVING username = 'test'"
# so when you call
cursor.execute(query)
# the value passed into the execute call is the string "SELECT * from list HAVING username = 'test'"
# 3.
query = ("SELECT * from list HAVING username = %s", (param,))
# This operation is assigning a 2-value tuple into the query variable
# The first value in the tuple is the string "SELECT * from list HAVING username = %s"
# The second value in the tuple is a 1-value, with 'test' as its first value
# 4.
query = "SELECT * from list WHERE username = :name", {'name': param}
# This is similar to #3, but the values in the tuple are instead
# query[0] == "SELECT * from list WHERE username = :name"
# query[1] is a dictionary: {'name': param}
Both 3 and 4 above are not calling the MySQL execute with the parameters you are expecting (see API here). You probably need to do one of:
unpack the query tuple into separate variables, and call the function with them
operation, params = query # unpack the first elem into operation, and second into params
cursor.execute(operation, params)
just index into the query tuple
cursor.execute(query[0], query[1])
# NB: you could also use the 'named parameters' feature in Python
cursor.execute(query[0], params=query[1])
Use the 'unpacking arguments list' (SPLAT operator)
cursor.execute(*query)
I know that this question has been asked in the past, but thorough searching hasn't seemed to fix my issue. I'm probably just missing something simple, as I'm new to the Python-mysql connector supplied by mysql.
I have a Python script which accesses a mysql database, but I'm having issues with removing quotes from my query. Here is my code:
import mysql.connector
try:
db = mysql.connector.connect(user='root', password='somePassword', host='127.0.0.1', database='dbName')
cursor = db.cursor()
query = "select * from tags where %s = %s"
a = 'tag_id'
b = '0'
cursor.execute(query, (a, b))
print cursor
data = cursor.fetchall()
print data
except mysql.connector.Error as err:
print "Exception tripped..."
print "--------------------------------------"
print err
cursor.close()
db.close()
My database is set up properly (as I'll prove shortly).
My output for this program is:
MySQLCursor: select * from tags where 'tag_id' = '0'
[]
Yet when I change my query to not use variables, for example:
cursor.execute("select * from tags where tag_id = 0")
Then my output becomes:
MySQLCursor: select * from tags where tag_id = 0
[(0, u'192.168.1.110')]
To me, this means that the only difference between my Cursor queries are the quotes.
How do I remove them from the query?
Thanks in advance.
I personally believe this code is correct and safe, but you should be extremely skeptical of using code like this without carefully reviewing it yourself or (better yet) with the help of a security expert. I am not qualified to be such an expert.
Two important things I changed:
I changed b = '0' to b = 0 so it ends up as a number rather than a quoted string. (This part was an easy fix.)
I skipped the built-in parameterization for the column name and replaced it with my own slight modification to the escaping/quoting built in to mysql-connector. This is the scary part that should give you pause.
Full code below, but again, be careful with this if the column name is user input!
import mysql.connector
def escape_column_name(name):
# This is meant to mostly do the same thing as the _process_params method
# of mysql.connector.MySQLCursor, but instead of the final quoting step,
# we escape any previously existing backticks and quote with backticks.
converter = mysql.connector.conversion.MySQLConverter()
return "`" + converter.escape(converter.to_mysql(name)).replace('`', '``') + "`"
try:
db = mysql.connector.connect(user='root', password='somePassword', host='127.0.0.1', database='dbName')
cursor = db.cursor()
a = 'tag_id'
b = 0
cursor.execute(
'select * from tags where {} = %s'.format(escape_column_name(a)),
(b,)
)
print cursor
data = cursor.fetchall()
print data
except mysql.connector.Error as err:
print "Exception tripped..."
print "--------------------------------------"
print err
cursor.close()
db.close()
I encountered a similar problem using pymysql and have shown my working code here, hope this will help.
What I did is overwrite the escape method in class 'pymysql.connections.Connection', which obviously adds "'" arround your string.
better have shown my code:
from pymysql.connections import Connection, converters
class MyConnect(Connection):
def escape(self, obj, mapping=None):
"""Escape whatever value you pass to it.
Non-standard, for internal use; do not use this in your applications.
"""
if isinstance(obj, str):
return self.escape_string(obj) # by default, it is :return "'" + self.escape_string(obj) + "'"
if isinstance(obj, (bytes, bytearray)):
ret = self._quote_bytes(obj)
if self._binary_prefix:
ret = "_binary" + ret
return ret
return converters.escape_item(obj, self.charset, mapping=mapping)
config = {'host':'', 'user':'', ...}
conn = MyConnect(**config)
cur = conn.cursor()
I'm busy writting a python script that is querying two db tables to build a single row of data per row it finds. Here is my script at the moment
#========================================================================
# DB CONNECT FUNCTION
#========================================================================
def f_connect(status):
global gv_conn
global gv_curs
if status == 1:
gv_conn = sqlite3.connect("./data.db")
gv_curs = gv_conn.cursor()
else
gv_conn.close()
#========================================================================
# PREPARE SQL STATEMENTS
#========================================================================
def f_statements():
global users_stmt
users_stmt = ("select * from users")
global users_curs
users_curs = gv_conn.cursor()
global uinfo_stmt
uinfo_stmt = ("select * from uinfo" +
"where ui_u_id = ?")
global uinfo_curs
uinfo_curs = gv_conn.cursor()
#========================================================================
#
# MAIN SCRIPT START
#
#========================================================================
f_connect(1)
f_statements()
la_users = []
for u_row in users_curs.execute(users_stmt):
# THIS LINE GETS USERS FROM THE ABOVE STATEMENT
# AND ADDS THEM TO THE DICTIONARY
la_users.append({"u_id": u_row[0], "u_name": u_row[1]})
# THIS LINE EXECUTES ANOTHER QUERY TO RETRIEVE
# A SINGLE ROW OF DATA FROM ANOTHER TABLE
la_uinfo = uinfo_curs.execute(uinfo_stmt, "1")
f_connect(0)
My problem is that when I execute the first sql statement I can get get the data by looping using a for loop which is storing the data so i can access it using u_row[int].
When I execute the second query it is storing it inside la_uinfo although when I try to get the data from la_uinfo[int] it doesn't work? How can I retrieve the data from my second query without using another for loop? (I shouldn't have to considering it only returns one row)
Cursors are not indexable, so cursor[0] will not work. To retrieve the first row of a cursor, you should use cursor.fetchone().