I know that this question has been asked in the past, but thorough searching hasn't seemed to fix my issue. I'm probably just missing something simple, as I'm new to the Python-mysql connector supplied by mysql.
I have a Python script which accesses a mysql database, but I'm having issues with removing quotes from my query. Here is my code:
import mysql.connector
try:
db = mysql.connector.connect(user='root', password='somePassword', host='127.0.0.1', database='dbName')
cursor = db.cursor()
query = "select * from tags where %s = %s"
a = 'tag_id'
b = '0'
cursor.execute(query, (a, b))
print cursor
data = cursor.fetchall()
print data
except mysql.connector.Error as err:
print "Exception tripped..."
print "--------------------------------------"
print err
cursor.close()
db.close()
My database is set up properly (as I'll prove shortly).
My output for this program is:
MySQLCursor: select * from tags where 'tag_id' = '0'
[]
Yet when I change my query to not use variables, for example:
cursor.execute("select * from tags where tag_id = 0")
Then my output becomes:
MySQLCursor: select * from tags where tag_id = 0
[(0, u'192.168.1.110')]
To me, this means that the only difference between my Cursor queries are the quotes.
How do I remove them from the query?
Thanks in advance.
I personally believe this code is correct and safe, but you should be extremely skeptical of using code like this without carefully reviewing it yourself or (better yet) with the help of a security expert. I am not qualified to be such an expert.
Two important things I changed:
I changed b = '0' to b = 0 so it ends up as a number rather than a quoted string. (This part was an easy fix.)
I skipped the built-in parameterization for the column name and replaced it with my own slight modification to the escaping/quoting built in to mysql-connector. This is the scary part that should give you pause.
Full code below, but again, be careful with this if the column name is user input!
import mysql.connector
def escape_column_name(name):
# This is meant to mostly do the same thing as the _process_params method
# of mysql.connector.MySQLCursor, but instead of the final quoting step,
# we escape any previously existing backticks and quote with backticks.
converter = mysql.connector.conversion.MySQLConverter()
return "`" + converter.escape(converter.to_mysql(name)).replace('`', '``') + "`"
try:
db = mysql.connector.connect(user='root', password='somePassword', host='127.0.0.1', database='dbName')
cursor = db.cursor()
a = 'tag_id'
b = 0
cursor.execute(
'select * from tags where {} = %s'.format(escape_column_name(a)),
(b,)
)
print cursor
data = cursor.fetchall()
print data
except mysql.connector.Error as err:
print "Exception tripped..."
print "--------------------------------------"
print err
cursor.close()
db.close()
I encountered a similar problem using pymysql and have shown my working code here, hope this will help.
What I did is overwrite the escape method in class 'pymysql.connections.Connection', which obviously adds "'" arround your string.
better have shown my code:
from pymysql.connections import Connection, converters
class MyConnect(Connection):
def escape(self, obj, mapping=None):
"""Escape whatever value you pass to it.
Non-standard, for internal use; do not use this in your applications.
"""
if isinstance(obj, str):
return self.escape_string(obj) # by default, it is :return "'" + self.escape_string(obj) + "'"
if isinstance(obj, (bytes, bytearray)):
ret = self._quote_bytes(obj)
if self._binary_prefix:
ret = "_binary" + ret
return ret
return converters.escape_item(obj, self.charset, mapping=mapping)
config = {'host':'', 'user':'', ...}
conn = MyConnect(**config)
cur = conn.cursor()
Related
I have a problem with my python code which I want to use for a REST API server.
The current problem is that my database query is returning null when I know that the value is there
The code for the specific path:
#app.route('/data/active_predicted/<int:ticketId>', methods=['GET'])
def search_db_tickId_act(ticketId):
cursor = db.cursor()
db_query = cursor.execute("select * from active_predicted where ticketId=" + str(ticketId))
json_output = json.dumps(dict(cursor.fetchall()))
cursor.close()
if not cursor.fetchall():
return "Nothing found \n SQL Query: " + "select * from active_predicted where ticketId=" + str(ticketId)
else:
return str(cursor.fetchall())
When I access this URL I get returned the following:
Nothing found SQL Query: select * from active_predicted where ticketId=1324
When I plug this SQL query I get the result I want, 1 row with 2 columns but it seems as though the program cannot locate the row?
The problems:
As #pvg mentioned, you need to escape your input values when querying database;
If you want to fetch a dictionary-like result, passing dictionary=True when you initialize the cursor;
In your original code, you didn't return the variable json_output;
To fetch only one result, use fetchone instead fetchall;
After cursor.close() got called, you can obtain nothing from that cursor no matter you fetched before or not;
Use try-finally to ensure that cursor always get closed (at last).
Here's the fixed code:
#app.route('/data/active_predicted/<int:ticketId>', methods=['GET'])
def search_db_tickId_act(ticketId):
try:
cursor = db.cursor(dictionary=True)
db_query = cursor.execute("select * from active_predicted where ticketId=%s LIMIT 1", ticketId)
row = cursor.fetchone()
if row:
return json.dumps(row)
else:
return "Nothing found \n SQL Query: " + "select * from active_predicted where ticketId=" + str(ticketId)
finally:
cursor.close()
I've written a simple python program, which get Data form database successfully. but unable to update table in DB.
When executing update statement it's get stuck and nothing happen, no any exception.
My code is as follows. Any idea whyis this ?
from java.sql import DriverManager
def updateDB():
url = "jdbc:oracle:thin:#192.1.1.1:1521:auid"
uname = "dbtstj1"
pword = "dbtstj321"
conn = None
stmt = None
try:
conn = DriverManager.getConnection(url,uname,pword)
stmt = conn.createStatement()
rs = stmt.executeQuery("select PKG_NAME from PkgData")
while rs.next():
print rs.getString(1)
pkgName = "'Test Pkg Name'"
pkgID = "'T1234'"
updateQuary = "UPDATE PkgData SET PKG_NAME =%s WHERE PKG_ID =%s" %(pkgName, pkgID)
stmt.execute(updateQuary)
except Exception , e:
print 'Error:', e[0]
finally:
if stmt is not None:
stmt.close()
if conn is not None:
conn.close()
updateDB()
you need to commit your changes to the database:
stmt.execute(updateQuary)
conn.commit()
These type of issues can be happen when query request datatype and required datatype is difference.
It seems to be there was a mismatch with database's datatype and your query. Can you recheck with database's datatype with your query.
For Ex: PKG_ID =%s can be another data type in database as digit or etc...
Somewhere in here lies a problem. http://paste.pocoo.org/show/528559/
Somewhere between lines 32 and 37. As you can see the DELETE FROM is inside a for loop.
Running the script just makes the program go through the loop and exit, without actually removing any records.
Any help would be greatly appreciated! Thanks!
#!/usr/bin/env python
# encoding: utf-8
import os, os.path, MySQLdb, pprint, string
class MySQLclass(object):
"""Learning Classes"""
def __init__(self, db):
self.db=db
self.cursor = self.db.cursor()
def sversion(self):
self.cursor.execute ("SELECT VERSION()")
row = self.cursor.fetchone ()
server_version = "server version:", row[0]
return server_version
def getRows(self, tbl):
""" Returns the content of the table tbl """
statmt="select * from %s" % tbl
self.cursor.execute(statmt)
rows=list(self.cursor.fetchall())
return rows
def getEmailRows(self, tbl):
""" Returns the content of the table tbl """
statmt="select email from %s" % tbl
self.cursor.execute(statmt)
rows=list(self.cursor.fetchall())
return rows
def removeRow(self,tbl,record):
""" Remove specific record """
print "Removing %s from table %s" %(record,tbl)
print tbl
self.cursor.execute ("""DELETE FROM maillist_frogs where email LIKE %s""", (record,))
def main():
#####connections removed
sql_frogs = MySQLclass(conn_frogs)
sql_mailgust = MySQLclass(conn_mailgust)
frogs_emails = sql_frogs.getEmailRows ("emails")
frogs_systemcatch = sql_frogs.getEmailRows ("systemcatch")
mailgust_emails = sql_mailgust.getEmailRows ("maillist_frogs")
aa = set(mailgust_emails)
remove = aa.intersection(frogs_emails)
remove = remove.union(aa.intersection(frogs_systemcatch))
for x in remove:
x= x[0]
remove_mailgust = sql_mailgust.removeRow ("maillist_frogs",x)
conn_frogs.close ()
conn_mailgust.close ()
if __name__ == '__main__':
main()
the problem is python-msyqldb specific.:
Starting with 1.2.0, MySQLdb disables autocommit by default, as required by the DB-API standard (PEP-249). If you are using InnoDB tables or some other type of transactional table type, you'll need to do connection.commit() before closing the connection, or else none of your changes will be written to the database.
therefore, after the DELETE, you must self.db.commit
The removeRow() method does not return a value, yet remove_mailgust is expecting to receive this non-existent value.
Also, your removeRow() class method is statically fixed to only search the table maillist_frogs in its query. You should probably set the table name to accept the second parameter of the method, tbl.
Finally, your removeRow() method is comparing the value of a record (presumably, an id) using LIKE, which is typically used for more promiscuous string comparison. Is email your primary key in this table? If so, I would suggest changing it to:
self.cursor.execute ("""DELETE FROM %s where email_id = %s""", (tbl, record,))
How would you go about executing several SQL statements (script mode) with python?
Trying to do something like this:
import MySQLdb
mysql = MySQLdb.connect(host='host...rds.amazonaws.com', db='dbName', user='userName', passwd='password')
sql = """
insert into rollout.version (`key`, `value`) VALUES ('maxim0', 'was here0');
insert into rollout.version (`key`, `value`) VALUES ('maxim1', 'was here1');
insert into rollout.version (`key`, `value`) VALUES ('maxim2', 'was here1');
"""
mysql.query(sql)
Fails with:
ProgrammingError: (2014, "Commands out
of sync; you can't run this command
now")
I'm writing an deployment engine that would accept SQL delta changes from several people and apply them to the DB on version deployment.
I've looked into this code http://sujitpal.blogspot.com/2009/02/python-sql-runner.html and implemented __sanitize_sql:
def __sanitize_sql(sql):
# Initial implementation from http://sujitpal.blogspot.com/2009/02/python-sql-runner.html
sql_statements = []
incomment = False
in_sqlcollect = False
sql_statement = None
for sline in sql.splitlines():
# Remove white space from both sides.
sline = sline.strip()
if sline.startswith("--") or len(sline) == 0:
# SQL Comment line, skip
continue
if sline.startswith("/*"):
# start of SQL comment block
incomment = True
if incomment and sline.endswith("*/"):
# end of SQL comment block
incomment = False
continue
# Collect line which is part of
if not incomment:
if sql_statement is None:
sql_statement = sline
else:
sql_statement += sline
if not sline.endswith(";"):
in_sqlcollect = True
if not in_sqlcollect:
sql_statements.append(sql_statement)
sql_statement = None
in_sqlcollect = False
if not incomment and not sql_statement is None and len(sql_statement) != 0:
sql_statements.append(sql_statement)
return sql_statements
if __name__ == "__main__":
sql = sql = """update tbl1;
/* This
is my
beautiful
comment*/
/*this is comment #2*/
some code...;
-- comment
sql code
"""
print __sanitize_sql(sql)
Don't know if it's the best solution but seems to work for not too complex to parse SQL statements.
The question now how to run this code, I can do something like this dude but it seems ugly, I'm not a python expert (we've been doing python here for just the past 2 weeks) but it seems that abusing cursor this way is hackish and not a good practice.
Ideas / blog posts would be helpful.
Thanks you,
Maxim.
Here is how you could use executemany():
import MySQLdb
connection = MySQLdb.connect(host='host...rds.amazonaws.com', db='dbName', user='userName', passwd='password')
cursor = connection.cursor()
my_data_to_insert = [['maxim0', 'was here0'], ['maxim1', 'was here1'], ['maxim2', 'was here1']]
sql = "insert into rollout.version (`key`, `value`) VALUES (%s, %s);"
cursor.executemany(sql, my_data_to_insert)
connection.commit()
connection.close()
Call the executemany method on the cursor object. More info here:
http://mysql-python.sourceforge.net/MySQLdb.html
I need to process mysql data one row at a time and i have selected all rows put them in a tuple but i get the error above.
what does this mean and how do I go about it?
Provide some code.
You probably call some function that should update database, but the function does not return any data (like cursor.execute()). And code:
data = cursor.execute()
Makes data a None object (of NoneType). But without code it's hard to point you to the exact cause of your error.
It means that the object you are trying to iterate is actually None; maybe the query produced no results?
Could you please post a code sample?
The function you used to select all rows returned None. This "probably" (because you did not provide code, I am only assuming) means that the SQL query did not return any values.
Try using the cursor.rowcount variable after you call cursor.execute(). (this code will not work because I don't know what module you are using).
db = mysqlmodule.connect("a connection string")
curs = dbo.cursor()
curs.execute("select top 10 * from tablename where fieldA > 100")
for i in range(curs.rowcount):
row = curs.fetchone()
print row
Alternatively, you can do this (if you know you want ever result returned):
db = mysqlmodule.connect("a connection string")
curs = dbo.cursor()
curs.execute("select top 10 * from tablename where fieldA > 100")
results = curs.fetchall()
if results:
for r in results:
print r
This error means that you are attempting to loop over a None object. This is like trying to loop over a Null array in C/C++. As Abgan, orsogufo, Dan mentioned, this is probably because the query did not return anything. I suggest that you check your query/databse connection.
A simple code fragment to reproduce this error is:
x = None
for each i in x:
#Do Something
pass
This may occur when I try to let 'usrsor.fetchone' execute twice. Like this:
import sqlite3
db_filename = 'test.db'
with sqlite3.connect(db_filename) as conn:
cursor = conn.cursor()
cursor.execute("""
insert into test_table (id, username, password)
values ('user_id', 'myname', 'passwd')
""")
cursor.execute("""
select username, password from test_table where id = 'user_id'
""")
if cursor.fetchone() is not None:
username, password = cursor.fetchone()
print username, password
I don't know much about the reason. But I modified it with try and except, like this:
import sqlite3
db_filename = 'test.db'
with sqlite3.connect(db_filename) as conn:
cursor = conn.cursor()
cursor.execute("""
insert into test_table (id, username, password)
values ('user_id', 'myname', 'passwd')
""")
cursor.execute("""
select username, password from test_table where id = 'user_id'
""")
try:
username, password = cursor.fetchone()
print username, password
except:
pass
I guess the cursor.fetchone() can't execute twice, because the cursor will be None when execute it first time.
I know it's an old question but I thought I'd add one more possibility. I was getting this error when calling a stored procedure, and adding SET NOCOUNT ON at the top of the stored procedure solved it. The issue is that earlier selects that are not the final select for the procedure make it look like you've got empty row sets.
Try to append you query result to a list, and than you can access it. Something like this:
try:
cursor = con.cursor()
getDataQuery = 'SELECT * FROM everything'
cursor.execute(getDataQuery)
result = cursor.fetchall()
except Exception as e:
print "There was an error while getting the values: %s" % e
raise
resultList = []
for r in result:
resultList.append(r)
Now you have a list that is iterable.