I am attempting to update a single value in a single cell in a SQL table using the mysql connector for python. Using the following code, I get no error messages, but nor does the table actually update. The value of the cell that I am attempting to update is sometimes empty, sometimes NULL, and sometimes contains a string. Here is my code:
query = ("UPDATE data_set SET %s = '%s' WHERE id = %s") % (column_to_change, change_to_value, row_id)
What am I doing wrong?
Edit: Thanks for the replies so far. I do not think there is any functional issue with the surrounding code (outside of the vulnerability to SQL injection, which I have fixed, here and elsewhere), as I have been effective executing similar code with different queries. Here is my code now:
column_to_change = "column2"
change_to_value = "james"
id = "1234"
cnx = mysql.connector.connect(user='user', password='password',
host='db.website.com',
database='database')
cursor = cnx.cursor()
query = ("UPDATE data_set SET %s = %s WHERE policy_key = %s")
cursor.execute(query, (column_to_change, change_to_value, id))
cursor.close()
cnx.close()
If it's relevant, it turns out the cells I'm trying to insert into are formatted as VARCHAR(45). When I run a SELECT query on a cell, it returns a name formatted like: (u'James',)
If I set change_to_value = "(u'James',)", I receive the following error message:
mysql.connector.errors.ProgrammingError: 1064 (42000): You have an error in your SQL syntac; check the manual that corresponds to your MariaDB server version for the right syntax to use near ''column1' = '(u\'James\',)' WHERE id = '1234'' at line 1
Make sure you are following all the steps:
conn = pyodbc.connect("SERVER=my.server;DATABASE=my_database;UID=my_user;PWD=my_password;",ansi=True)
cursor = conn.cursor()
query = ("UPDATE data_set SET %s = '%s' WHERE id = %s") % (column_to_change, change_to_value, row_id)
cursor.execute(query)
And also verify the SQL query that your passing to .execute() is same as that you want to run on your database
Depending on your version of Python, perhaps it is an issue with your string interpolation. Otherwise, you may not be connecting your cursor and executing the query successfully. I am assuming you are executing your query elsewhere in your code, but in the instance you are not, this should work:
cursor.execute ("""
UPDATE data_set
SET %s=%s
WHERE id=%s
""", (column_to_change, change_to_value, row_id))
Alternatively, you could store this query in a variable as you have done, assign the respective variables and execute afterward like so:
query = (“UPDATE data_set SET %s=%s WHERE id=%s”)
column_to_change = [YOUR ASSIGNMENT HERE]
change_to_value = [YOUR ASSIGNMENT HERE--if this is a string, it should be formatted as such here]
row_id = [YOUR ASSIGNMENT HERE]
cursor.execute(query, (column_to_change, change_to_value, row_id))
Basic string interpolation is prone to SQL injection and should be avoided.
For more reference, see here
Related
I have created a test database called test inside it has a table called testTable with an autoincrement id value and a name field that takes a varchar(30).
The PREPARE statement queries (4 of them) execute fine when copied into phpmyadmin but I get the error 👍 2021-01-08 18:26:53,022 (MainThread) [ERROR] (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'SET\n #name = 'fred';\nEXECUTE\n statement USING #name;\nDEALLOCATE\nPREPARE\n ' at line 5")
The test code:
import pymysql
import logging
class TestClass():
def __init__(self):
# mysqlconnections
self.mySQLHostName = "localhost"
self.mySQLHostPort = 3306
self.mySQLuserName = "userName"
self.mySQLpassword = "pass"
self.MySQLauthchandb = "mysql"
def QueryMYSQL (self, query):
try:
#logging.info("QueryMYSQL : " + str( query)) # Uncomment to print all mysql queries sent
conn = pymysql.connect(host=self.mySQLHostName, port=self.mySQLHostPort, user=self.mySQLuserName, passwd=self.mySQLpassword, db=self.MySQLauthchandb, charset='utf8')
conn.autocommit(True)
cursor = conn.cursor()
if cursor:
returnSuccess = cursor.execute(query)
if cursor:
returnValue = cursor.fetchall()
#logging.info ("return value : " + str(returnValue)) # Uncomment to print all returned mysql queries
if cursor:
cursor.close()
if conn:
conn.close()
return returnValue
except Exception as e:
logging.error("Problem in ConnectTomySQL")
logging.error(query)
logging.error(e)
return False
# Default error logging log file location:
logging.basicConfig(format='%(asctime)s (%(threadName)-10s) [%(levelname)s] %(message)s', filename= 'ERROR.log',filemode = "w", level=logging.DEBUG)
logging.info("Logging Started")
test = TestClass()
result = test.QueryMYSQL("Describe test.testTable")
print(result)
query = """
PREPARE
statement
FROM
'INSERT INTO test.testTable (id, name) VALUES (NULL , ?)';
SET
#name = 'fred';
EXECUTE
statement USING #name;
DEALLOCATE
PREPARE
statement;
"""
result = test.QueryMYSQL(query)
print(result)
I'm assuming this is a library issue rather than a mysql issue? I am trying to use prepared statements to prevent code injection from user input as I understand this prepared statements are the best way to do this rather than trying to pre filter user input and missing something.
I asked this question on the github but one of the authors (methane Inada Naoki) replied with this:
========
Multistatement can be used by attacker when there is a query injection vulnerability. So it is disabled by default.
as I understand this prepared statements are the best way
You are totally wrong. Your use of prepared statement doesn't protect you from SQL injection at all. If you enable multistatement, your "prepared statement" can be attacked by SQL injection.
But I am not free tech support nor free teacher for you. OSS maintainers are not. Please don't ask here.
and he closed the issue.
Is he correct?
The author book I am reading Robin Nixon,"Learning PHP, MySQL and JavaScript" O'Reilly 5th edition. He appears to be under the misconception and I quote "Let me introduce the best and recommended way to interact with MySQL, which is pretty much bulletproof in terms of Security" Its in the Using Placeholders section pg 260. Is he wrong?
Because I bought this book to improve my security practices and now I'm not sure what is correct.
I found out from the developer of pymysql that the library does not support the PREPARE mysql statement. Also the pymysql library by default does not execute multi-statements.
I understand that my first attempt at substituting values into the INSERT statement is inherently unsafe if multi-statements are enabled. This can be done by using the client_flag=pymysql.constants.CLIENT.MULTI_STATEMENTS in the connect constructor.
The pymysql library does however allow for placeholders to be used in MySQL queries using the cursor.execute(query, (tuple)) method.
To demonstrate this I wrote the following test code example.
import pymysql
import logging
class TestClass():
def __init__(self):
# mysqlconnections
self.mySQLHostName = "localhost"
self.mySQLHostPort = 3306
self.mySQLuserName = "name"
self.mySQLpassword = "pw"
self.MySQLauthchandb = "mysql"
def QueryMYSQL (self, query, data = ()):
try:
logging.info("QueryMYSQL : " + str( query)) # Uncomment to print all mysql queries sent
conn = pymysql.connect(host=self.mySQLHostName, port=self.mySQLHostPort, user=self.mySQLuserName, passwd=self.mySQLpassword, db=self.MySQLauthchandb, charset='utf8', client_flag=pymysql.constants.CLIENT.MULTI_STATEMENTS) #code injection requires multistatements to be allowed this is off in pymysql by default and has to be set on manually.
conn.autocommit(True)
cursor = conn.cursor()
if cursor:
if data:
returnSuccess = cursor.execute(query, data)
else:
returnSuccess = cursor.execute(query)
if cursor:
returnValue = cursor.fetchall()
logging.info ("return value : " + str(returnValue)) # Uncomment to print all returned mysql queries
if cursor:
cursor.close()
if conn:
conn.close()
return returnValue
except Exception as e:
logging.error("Problem in ConnectTomySQL")
logging.error(e)
logging.error(query)
if data:
logging.error("Data {}".format(str(data)))
return False
# Default error logging log file location:
logging.basicConfig(format='%(asctime)s (%(threadName)-10s) [%(levelname)s] %(message)s', filename= 'ERROR.log',filemode = "w", level=logging.DEBUG)
logging.info("Logging Started")
def usePlaceholder(userInput):
query = "INSERT INTO test.testTable (id, name) VALUES (NULL , %s)"
data = (userInput,)
result = test.QueryMYSQL(query,data)
print(result)
def useSubstitution(userInput):
query = "INSERT INTO test.testTable (id, name) VALUES (NULL , '{}')".format(userInput) # this is unsafe.
result = test.QueryMYSQL(query)
print(result)
test = TestClass()
#Create the test database and testTable.
query = "CREATE DATABASE test"
test.QueryMYSQL(query)
query = "CREATE TABLE `test`.`testTable` ( `id` INT NOT NULL AUTO_INCREMENT , `name` VARCHAR(256) NULL DEFAULT NULL , PRIMARY KEY (`id`)) ENGINE = InnoDB;"
test.QueryMYSQL(query)
#Simulated user input.
legitUserEntry = "Ringo"
injectionAttempt = "333' ); INSERT INTO test.testTable (id, name) VALUES (NULL , 666);#" #A simulated user sql injection attempt.
useSubstitution(legitUserEntry) # this will also insert Ringo - but could be unsafe.
usePlaceholder(legitUserEntry) # this will insert Ringo - but is safer.
useSubstitution(injectionAttempt) # this will inject the input code and execute it.
usePlaceholder(injectionAttempt) # this will insert the input into the database without executing the injected code.
So from this exercise, I shall henceforth improve my security by keeping multi-statements set to off (the default) AND using the placeholders and data tuple rather than substitution.
I have a database created from
CREATE TABLE `ip` (
`idip` int(11) NOT NULL AUTO_INCREMENT,
`ip` decimal(45,0) DEFAULT NULL,
`mask` int(11) DEFAULT NULL,
PRIMARY KEY (`idip`),
UNIQUE KEY `ip_UNIQUE` (`ip`)
)
And I've made some insertions into this table
But when I try to execute on python:
sql = "select idip from ip where ip=%s and mask=%s" % (long(next_hop), 'DEFAULT')
cursor.execute(sql)
idnext_hop = cursor.fetchone()[0]
I get the following error:
Inserting routes into table routes (1/377)...('insere_tabela_routes: Error on insertion at table routes, - SQL: ', 'select idip from ip where ip=0 and mask=DEFAULT')
1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '' at line 1
Does anyone have a clue on what is the problem?
mysql.connector uses %s instead of ? as the parameter marker, but you are circumventing it by using Python string formatting. Try this:
sql = "select idip from ip where ip=%s and mask=%s"
cursor.execute(sql, (long(next_hop), 'DEFAULT'))
idnext_hop = cursor.fetchone()[0]
You are munging the query string with parameters, rather than passing them in as, well, parameters. The code should look like this:
sql = "select idip from ip where ip = ? and mask = ?"
cursor.execute(sql, (long(next_hop), 'DEFAULT'))
idnext_hop = cursor.fetchone()[0]
In other words, you want the query engine to do the substitution into the query. You don't want Python to do the substitution into the query string.
I'm having trouble passing data into %s token. I've looked around and noticed that this Mysql module handles the %s token differently, and that it should be escaped for security reasons, my code is throwing this error.
mysql.connector.errors.ProgrammingError: 1064 (42000): You have an error in your
SQL syntax; check the manual that corresponds to your MySQL server version for
the right syntax to use near '%s)' at line 1
If I do it like this:
sql_insert = ("INSERT INTO `Products` (title) VALUES(%s)"),(data)
I get a tuple error..
import mysql.connector
from mysql.connector import errorcode
cnx = mysql.connector.connect (user='userDB1', password='UserPwd1',
host='somedatabase.com', database='mydatabase1')
cursor = cnx.cursor()
sql_insert = ("INSERT INTO `Products` (title) VALUES(%s)")
data=('HelloSQLWORLD')
cursor.execute(sql_insert,data)
cnx.commit()
cnx.close()
No, don't do it the way #Jeon suggested - by using string formatting you are exposing your code to SQL injection attacks. Instead, properly parameterize the query:
query = """
INSERT INTO
Products
(title)
VALUES
(%s)"""
cursor.execute(query, ('HelloSQLWORLD', ))
Note how the query parameters are put into a tuple.
Pythonic string formatting is:
str1 = 'hello'
str2 = 'world'
'%s, %s' % (str1, str2)
Use % with tuple, not ,
For your particular case, try:
cursor.execute(sql_insert % (data))
I have what would appear to be a straight forward insert statement for Oracle SQL. It works properly in Oracle SQL Developer but the same command will not work in Python, complaining about
cx_Oracle.DatabaseError: ORA-00933: SQL command not properly ended.
This happens on the line for the cursor.execute() call. The query itself is:
insert into TestNamesTable (TestName, TheUser, TheProject) values ('mytest.s', 'bjurasz', 'Beta');
If run in SQL Developer I get a new row. Inside Python I get the termination error. From what I can tell its properly formed and terminated.
Here is how I construct the query in python:
sql = "insert into TestNamesTable (TestName, TheUser, TheProject) values ('%s', '%s', '%s');" % (diagname, username, project)
print sql
cursor.execute(sql)
connection.commit()
Try this:
# These are just random definitions, must be of type table requires
diagname = "DEFINE HERE"
username = "SOMEBODY"
project = "PROJECT NAME"
# Assuming you've defined your connection before
cursor.execute("""insert into TestNamesTable (TestName, TheUser, TheProject) values (:diagname, :username, :project)""",
{"diagname": diagname, #cx_Oracle likes this bind-variable looking syntax
"username": username.
"project": project})
connection.commit()
I'm having some troubles with this python method.
I don't have any problem getting the select results but when I've tried to execute the update I don't get any results.
I have tried to generate another cursor object, redefine the cursor, generate another connection, use a different sql query (without the use of the %s) and I didn't have any results.
If you could give me any help i would be really appreciate.
def getTarea():
conn = db.connect('url','user','pass','dbInstance')
with conn:
try:
cursor = conn.cursor(db.cursors.DictCursor)
sql = "SELECT CMD, ID_TAREA FROM TAREAS WHERE OBTENIDA = '0' AND DEVICE_ID = '1001' ORDER BY FECHA_TAREA DESC LIMIT 1"
cursor.execute(sql)
f.write(sql+"\n")
# fetch all of the rows from the query
data = cursor.fetchone()
# print the rows
f.write("CMD: "+data["CMD"]+"\n")
f.write("ID_TAREA: "+ str(data["ID_TAREA"])+"\n")
idTarea = str(data["ID_TAREA"])
obtenido = 1
cursor.execute("""UPDATE TAREAS SET OBTENIDA=%s WHERE ID_TAREA =%s""", (obtenido, idTarea))
cursor.close()
conn.close()
except Exception as e:
f.write("error \n"+e)
return cmd
conn.commit() will commit the changes, as documented in this similar post: Database does not update automatically with MySQL and Python