I have created a test database called test inside it has a table called testTable with an autoincrement id value and a name field that takes a varchar(30).
The PREPARE statement queries (4 of them) execute fine when copied into phpmyadmin but I get the error 👍 2021-01-08 18:26:53,022 (MainThread) [ERROR] (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'SET\n #name = 'fred';\nEXECUTE\n statement USING #name;\nDEALLOCATE\nPREPARE\n ' at line 5")
The test code:
import pymysql
import logging
class TestClass():
def __init__(self):
# mysqlconnections
self.mySQLHostName = "localhost"
self.mySQLHostPort = 3306
self.mySQLuserName = "userName"
self.mySQLpassword = "pass"
self.MySQLauthchandb = "mysql"
def QueryMYSQL (self, query):
try:
#logging.info("QueryMYSQL : " + str( query)) # Uncomment to print all mysql queries sent
conn = pymysql.connect(host=self.mySQLHostName, port=self.mySQLHostPort, user=self.mySQLuserName, passwd=self.mySQLpassword, db=self.MySQLauthchandb, charset='utf8')
conn.autocommit(True)
cursor = conn.cursor()
if cursor:
returnSuccess = cursor.execute(query)
if cursor:
returnValue = cursor.fetchall()
#logging.info ("return value : " + str(returnValue)) # Uncomment to print all returned mysql queries
if cursor:
cursor.close()
if conn:
conn.close()
return returnValue
except Exception as e:
logging.error("Problem in ConnectTomySQL")
logging.error(query)
logging.error(e)
return False
# Default error logging log file location:
logging.basicConfig(format='%(asctime)s (%(threadName)-10s) [%(levelname)s] %(message)s', filename= 'ERROR.log',filemode = "w", level=logging.DEBUG)
logging.info("Logging Started")
test = TestClass()
result = test.QueryMYSQL("Describe test.testTable")
print(result)
query = """
PREPARE
statement
FROM
'INSERT INTO test.testTable (id, name) VALUES (NULL , ?)';
SET
#name = 'fred';
EXECUTE
statement USING #name;
DEALLOCATE
PREPARE
statement;
"""
result = test.QueryMYSQL(query)
print(result)
I'm assuming this is a library issue rather than a mysql issue? I am trying to use prepared statements to prevent code injection from user input as I understand this prepared statements are the best way to do this rather than trying to pre filter user input and missing something.
I asked this question on the github but one of the authors (methane Inada Naoki) replied with this:
========
Multistatement can be used by attacker when there is a query injection vulnerability. So it is disabled by default.
as I understand this prepared statements are the best way
You are totally wrong. Your use of prepared statement doesn't protect you from SQL injection at all. If you enable multistatement, your "prepared statement" can be attacked by SQL injection.
But I am not free tech support nor free teacher for you. OSS maintainers are not. Please don't ask here.
and he closed the issue.
Is he correct?
The author book I am reading Robin Nixon,"Learning PHP, MySQL and JavaScript" O'Reilly 5th edition. He appears to be under the misconception and I quote "Let me introduce the best and recommended way to interact with MySQL, which is pretty much bulletproof in terms of Security" Its in the Using Placeholders section pg 260. Is he wrong?
Because I bought this book to improve my security practices and now I'm not sure what is correct.
I found out from the developer of pymysql that the library does not support the PREPARE mysql statement. Also the pymysql library by default does not execute multi-statements.
I understand that my first attempt at substituting values into the INSERT statement is inherently unsafe if multi-statements are enabled. This can be done by using the client_flag=pymysql.constants.CLIENT.MULTI_STATEMENTS in the connect constructor.
The pymysql library does however allow for placeholders to be used in MySQL queries using the cursor.execute(query, (tuple)) method.
To demonstrate this I wrote the following test code example.
import pymysql
import logging
class TestClass():
def __init__(self):
# mysqlconnections
self.mySQLHostName = "localhost"
self.mySQLHostPort = 3306
self.mySQLuserName = "name"
self.mySQLpassword = "pw"
self.MySQLauthchandb = "mysql"
def QueryMYSQL (self, query, data = ()):
try:
logging.info("QueryMYSQL : " + str( query)) # Uncomment to print all mysql queries sent
conn = pymysql.connect(host=self.mySQLHostName, port=self.mySQLHostPort, user=self.mySQLuserName, passwd=self.mySQLpassword, db=self.MySQLauthchandb, charset='utf8', client_flag=pymysql.constants.CLIENT.MULTI_STATEMENTS) #code injection requires multistatements to be allowed this is off in pymysql by default and has to be set on manually.
conn.autocommit(True)
cursor = conn.cursor()
if cursor:
if data:
returnSuccess = cursor.execute(query, data)
else:
returnSuccess = cursor.execute(query)
if cursor:
returnValue = cursor.fetchall()
logging.info ("return value : " + str(returnValue)) # Uncomment to print all returned mysql queries
if cursor:
cursor.close()
if conn:
conn.close()
return returnValue
except Exception as e:
logging.error("Problem in ConnectTomySQL")
logging.error(e)
logging.error(query)
if data:
logging.error("Data {}".format(str(data)))
return False
# Default error logging log file location:
logging.basicConfig(format='%(asctime)s (%(threadName)-10s) [%(levelname)s] %(message)s', filename= 'ERROR.log',filemode = "w", level=logging.DEBUG)
logging.info("Logging Started")
def usePlaceholder(userInput):
query = "INSERT INTO test.testTable (id, name) VALUES (NULL , %s)"
data = (userInput,)
result = test.QueryMYSQL(query,data)
print(result)
def useSubstitution(userInput):
query = "INSERT INTO test.testTable (id, name) VALUES (NULL , '{}')".format(userInput) # this is unsafe.
result = test.QueryMYSQL(query)
print(result)
test = TestClass()
#Create the test database and testTable.
query = "CREATE DATABASE test"
test.QueryMYSQL(query)
query = "CREATE TABLE `test`.`testTable` ( `id` INT NOT NULL AUTO_INCREMENT , `name` VARCHAR(256) NULL DEFAULT NULL , PRIMARY KEY (`id`)) ENGINE = InnoDB;"
test.QueryMYSQL(query)
#Simulated user input.
legitUserEntry = "Ringo"
injectionAttempt = "333' ); INSERT INTO test.testTable (id, name) VALUES (NULL , 666);#" #A simulated user sql injection attempt.
useSubstitution(legitUserEntry) # this will also insert Ringo - but could be unsafe.
usePlaceholder(legitUserEntry) # this will insert Ringo - but is safer.
useSubstitution(injectionAttempt) # this will inject the input code and execute it.
usePlaceholder(injectionAttempt) # this will insert the input into the database without executing the injected code.
So from this exercise, I shall henceforth improve my security by keeping multi-statements set to off (the default) AND using the placeholders and data tuple rather than substitution.
Related
I tried to write a function to drop database :
def deleteDb(self, dbName: str):
conn = psycopg2.connect(dbname="postgres", user="postgres")
conn.autocommit = True
curs = conn.cursor()
curs.execute("DROP DATABASE {};".format(dbName))
curs.close()
conn.close()
When I try to test it with an existing db :
def test_deleteDb(self):
self.deleteDb("dbTest")
I get this error :
"psycopg2.errors.InvalidCatalogName: database "dbtest" does not exist"
I tried to play with the isolation level, to drop all the connections to the database and to connect directly to the database but it did not work
Remember to put quotes around identifiers that contain upper-case letters:
curs.execute('DROP DATABASE "{}";'.format(dbName))
Note that string substitution into SQL-statements is generally a bad idea beause it is vulnerable to SQL-injection.
I'm still using Flask-mysql.
I'm getting the database context (the mysql variable) just fine, and can query on the database / get results. It's only the insert that is not working: it's not complaining (throwing Exceptions). It returns True from the insert method.
This should be done inserting the record when it commits, but for some reason, as I watch the MySQL database with MySQL Workbench, nothing is getting inserted into the table (and it's not throwing exceptions from the insert method):
I'm passing in this to insertCmd:
"INSERT into user(username, password) VALUES ('test1','somepassword');"
I've checked the length of the column in the database, and copied the command into MySQL Workbench (where it successfully inserts the row into the table).
I'm at a loss. The examples I've seen all seem to follow this format, and I have a good database context. You can see other things I've tried in the comments.
def insert(mysql, insertCmd):
try:
#connection = mysql.get_db()
cursor = mysql.connect().cursor()
cursor.execute(insertCmd)
mysql.connect().commit()
#mysql.connect().commit
#connection.commit()
return True
except Exception as e:
print("Problem inserting into db: " + str(e))
return False
You need to keep a handle to the connection; you keep overriding it in your loop.
Here is a simplified example:
con = mysql.connect()
cursor = con.cursor()
def insert(mysql, insertCmd):
try:
cursor.execute(insertCmd)
con.commit()
return True
except Exception as e:
print("Problem inserting into db: " + str(e))
return False
If mysql is your connection, then you can just commit on that, directly:
def insert(mysql, insertCmd):
try:
cursor = mysql.cursor()
cursor.execute(insertCmd)
mysql.commit()
return True
except Exception as e:
print("Problem inserting into db: " + str(e))
return False
return False
Apparently, you MUST separate the connect and cursor, or it won't work.
To get the cursor, this will work: cursor = mysql.connect().cursor()
However, as Burchan Khalid so adeptly pointed out, any attempt after that to make a connection object in order to commit will wipe out the work you did using the cursor.
So, you have to do the following (no shortcuts):
connection = mysql.connect()
cursor = connection.cursor()
cursor.execute(insertCmd)
connection.commit()
I am attempting to update a single value in a single cell in a SQL table using the mysql connector for python. Using the following code, I get no error messages, but nor does the table actually update. The value of the cell that I am attempting to update is sometimes empty, sometimes NULL, and sometimes contains a string. Here is my code:
query = ("UPDATE data_set SET %s = '%s' WHERE id = %s") % (column_to_change, change_to_value, row_id)
What am I doing wrong?
Edit: Thanks for the replies so far. I do not think there is any functional issue with the surrounding code (outside of the vulnerability to SQL injection, which I have fixed, here and elsewhere), as I have been effective executing similar code with different queries. Here is my code now:
column_to_change = "column2"
change_to_value = "james"
id = "1234"
cnx = mysql.connector.connect(user='user', password='password',
host='db.website.com',
database='database')
cursor = cnx.cursor()
query = ("UPDATE data_set SET %s = %s WHERE policy_key = %s")
cursor.execute(query, (column_to_change, change_to_value, id))
cursor.close()
cnx.close()
If it's relevant, it turns out the cells I'm trying to insert into are formatted as VARCHAR(45). When I run a SELECT query on a cell, it returns a name formatted like: (u'James',)
If I set change_to_value = "(u'James',)", I receive the following error message:
mysql.connector.errors.ProgrammingError: 1064 (42000): You have an error in your SQL syntac; check the manual that corresponds to your MariaDB server version for the right syntax to use near ''column1' = '(u\'James\',)' WHERE id = '1234'' at line 1
Make sure you are following all the steps:
conn = pyodbc.connect("SERVER=my.server;DATABASE=my_database;UID=my_user;PWD=my_password;",ansi=True)
cursor = conn.cursor()
query = ("UPDATE data_set SET %s = '%s' WHERE id = %s") % (column_to_change, change_to_value, row_id)
cursor.execute(query)
And also verify the SQL query that your passing to .execute() is same as that you want to run on your database
Depending on your version of Python, perhaps it is an issue with your string interpolation. Otherwise, you may not be connecting your cursor and executing the query successfully. I am assuming you are executing your query elsewhere in your code, but in the instance you are not, this should work:
cursor.execute ("""
UPDATE data_set
SET %s=%s
WHERE id=%s
""", (column_to_change, change_to_value, row_id))
Alternatively, you could store this query in a variable as you have done, assign the respective variables and execute afterward like so:
query = (“UPDATE data_set SET %s=%s WHERE id=%s”)
column_to_change = [YOUR ASSIGNMENT HERE]
change_to_value = [YOUR ASSIGNMENT HERE--if this is a string, it should be formatted as such here]
row_id = [YOUR ASSIGNMENT HERE]
cursor.execute(query, (column_to_change, change_to_value, row_id))
Basic string interpolation is prone to SQL injection and should be avoided.
For more reference, see here
I've created a new lambda function in Python with a handler called handler. In the Configuration section, AWS requires me to put the name of the function in the form file-name.function-name (as described here).
However I have no idea what the filename is supposed to be. I've created a blank function and didn't specify a filename at any point. My function name is "MySQLTest" so I've tried various things like "MySQLTest.handler", "my-sql-test.handler", "mysqltest.handler" but none of it seems to work.
Any idea what I should put as a filename?
For info, this is the test code I'm using:
import sys
import logging
import rds_config
import pymysql
#rds settings
rds_host = "*******"
# "rds-instance-endpoint"
name = rds_config.db_username
password = rds_config.db_password
db_name = rds_config.db_name
port = 3306
logger = logging.getLogger()
logger.setLevel(logging.INFO)
server_address = (rds_host, port)
try:
conn = pymysql.connect(rds_host, user=name, passwd=password, db=db_name, connect_timeout=5)
except:
logger.error("ERROR: Unexpected error: Could not connect to MySql instance.")
sys.exit()
logger.info("SUCCESS: Connection to RDS mysql instance succeeded")
def handler(event, context):
"""
This function fetches content from mysql RDS instance
"""
item_count = 0
try:
with conn.cursor() as cur:
cur.execute("create table Employee3 ( EmpID int NOT NULL, Name varchar(255) NOT NULL, PRIMARY KEY (EmpID))")
cur.execute('insert into Employee3 (EmpID, Name) values(1, "Joe")')
cur.execute('insert into Employee3 (EmpID, Name) values(2, "Bob")')
cur.execute('insert into Employee3 (EmpID, Name) values(3, "Mary")')
cur.execute("select * from Employee3")
for row in cur:
item_count += 1
logger.info(row)
#print(row)
finally:
conn.close()
return "Added %d items from RDS MySQL table" %(item_count)
If you are adding the function via the AWS Console the name you supply should be <name of lambda function>.handler (assuming you called your Python function handler.
So, if the name of my Lambda function is fooBar and my Python function is called handler I would use fooBar.handler.
I'm having some troubles with this python method.
I don't have any problem getting the select results but when I've tried to execute the update I don't get any results.
I have tried to generate another cursor object, redefine the cursor, generate another connection, use a different sql query (without the use of the %s) and I didn't have any results.
If you could give me any help i would be really appreciate.
def getTarea():
conn = db.connect('url','user','pass','dbInstance')
with conn:
try:
cursor = conn.cursor(db.cursors.DictCursor)
sql = "SELECT CMD, ID_TAREA FROM TAREAS WHERE OBTENIDA = '0' AND DEVICE_ID = '1001' ORDER BY FECHA_TAREA DESC LIMIT 1"
cursor.execute(sql)
f.write(sql+"\n")
# fetch all of the rows from the query
data = cursor.fetchone()
# print the rows
f.write("CMD: "+data["CMD"]+"\n")
f.write("ID_TAREA: "+ str(data["ID_TAREA"])+"\n")
idTarea = str(data["ID_TAREA"])
obtenido = 1
cursor.execute("""UPDATE TAREAS SET OBTENIDA=%s WHERE ID_TAREA =%s""", (obtenido, idTarea))
cursor.close()
conn.close()
except Exception as e:
f.write("error \n"+e)
return cmd
conn.commit() will commit the changes, as documented in this similar post: Database does not update automatically with MySQL and Python