Develop Reference

Syntax error near (%s %s %s) using mysql.connector for Python

I have several queries. Most of my insert queries work and they follow the same format as this one (I even copy pasted and modified as needed). For some reason, this query is throwing me a syntax error and I'm not sure why.
I've looked at the various solutions on SO for resolving this error. Seems like people get it for a variety of different reasons, and I'm not sure exactly what the reason is for this error being thrown at me. I can't see the problem with the statement.
def set_prices(game_name, vendor_id, vendor_name, game_id, game_platform, vendor_store_link, current_price_at_vendor, previous_price_at_vendor, historical_low_at_vendor):
# Create the query
insert_price_data = """ INSERT INTO game_vendors
(game_name, vendor_id, vendor_name, game_id, game_platform, vendor_store_link, current_price_at_vendor, previous_price_at_vendor, historical_low_at_vendor, historical_low_date)
VALUES (%s,%s,%s,%s,%s,%s,%s,%s,%s,%s); """
vals = (game_name, vendor_id, vendor_name, game_id, game_platform, vendor_store_link, current_price_at_vendor, previous_price_at_vendor, historical_low_at_vendor, datetime.datetime.now().strftime("%Y/%m/%d").replace('/', '-'))
send_to_db(insert_price_data, vals)
set_prices("borderlands 3", "1", "xbox", "26", "xbox one", "xbox.com", "45", "45", "25")
I wasn't expected to receive this error, as it had not occurred with any of my previous SQL statements.
"1064 (42000): You have an error in your SQL syntax; check the manual
that corresponds to your MySQL server version for the right syntax to
use near '%s,%s,%s,%s,%s,%s,%s,%s,%s,%s)' at line 4"
I bet it is something minor I'm overlooking.
Update send_to_db() function:
def send_to_db(query, multi_result=False, vals=None):
# this function opens a database query for a connection
# build engine for database
db_engine = mysql.connector.connect(
host="localhost",
user="root",
passwd="",
database="mydb",
)
# Initiate the cursor
cursor = db_engine.cursor()
# row = None
# rows = None
try:
if vals:
# insert queries require vals
cursor.execute(query, vals)
db_engine.commit()
else:
# select queries which don't require vals
cursor.execute(query)
if multi_result:
rows = cursor.fetchall()
return rows
else:
row = cursor.fetchone()
return row
except mysql.connector.Error as error:
print(error)
finally:
if db_engine.is_connected():
cursor.close()
db_engine.close()
print("MySQL connection is closed")
db_engine.disconnect()

Removing quotes from mysql query in Python

I know that this question has been asked in the past, but thorough searching hasn't seemed to fix my issue. I'm probably just missing something simple, as I'm new to the Python-mysql connector supplied by mysql.
I have a Python script which accesses a mysql database, but I'm having issues with removing quotes from my query. Here is my code:
import mysql.connector
try:
db = mysql.connector.connect(user='root', password='somePassword', host='127.0.0.1', database='dbName')
cursor = db.cursor()
query = "select * from tags where %s = %s"
a = 'tag_id'
b = '0'
cursor.execute(query, (a, b))
print cursor
data = cursor.fetchall()
print data
except mysql.connector.Error as err:
print "Exception tripped..."
print "--------------------------------------"
print err
cursor.close()
db.close()
My database is set up properly (as I'll prove shortly).
My output for this program is:
MySQLCursor: select * from tags where 'tag_id' = '0'
[]
Yet when I change my query to not use variables, for example:
cursor.execute("select * from tags where tag_id = 0")
Then my output becomes:
MySQLCursor: select * from tags where tag_id = 0
[(0, u'192.168.1.110')]
To me, this means that the only difference between my Cursor queries are the quotes.
How do I remove them from the query?
Thanks in advance.

I personally believe this code is correct and safe, but you should be extremely skeptical of using code like this without carefully reviewing it yourself or (better yet) with the help of a security expert. I am not qualified to be such an expert.
Two important things I changed:
I changed b = '0' to b = 0 so it ends up as a number rather than a quoted string. (This part was an easy fix.)
I skipped the built-in parameterization for the column name and replaced it with my own slight modification to the escaping/quoting built in to mysql-connector. This is the scary part that should give you pause.
Full code below, but again, be careful with this if the column name is user input!
import mysql.connector
def escape_column_name(name):
# This is meant to mostly do the same thing as the _process_params method
# of mysql.connector.MySQLCursor, but instead of the final quoting step,
# we escape any previously existing backticks and quote with backticks.
converter = mysql.connector.conversion.MySQLConverter()
return "`" + converter.escape(converter.to_mysql(name)).replace('`', '``') + "`"
try:
db = mysql.connector.connect(user='root', password='somePassword', host='127.0.0.1', database='dbName')
cursor = db.cursor()
a = 'tag_id'
b = 0
cursor.execute(
'select * from tags where {} = %s'.format(escape_column_name(a)),
(b,)
)
print cursor
data = cursor.fetchall()
print data
except mysql.connector.Error as err:
print "Exception tripped..."
print "--------------------------------------"
print err
cursor.close()
db.close()

I encountered a similar problem using pymysql and have shown my working code here, hope this will help.
What I did is overwrite the escape method in class 'pymysql.connections.Connection', which obviously adds "'" arround your string.
better have shown my code:
from pymysql.connections import Connection, converters
class MyConnect(Connection):
def escape(self, obj, mapping=None):
"""Escape whatever value you pass to it.
Non-standard, for internal use; do not use this in your applications.
"""
if isinstance(obj, str):
return self.escape_string(obj) # by default, it is :return "'" + self.escape_string(obj) + "'"
if isinstance(obj, (bytes, bytearray)):
ret = self._quote_bytes(obj)
if self._binary_prefix:
ret = "_binary" + ret
return ret
return converters.escape_item(obj, self.charset, mapping=mapping)
config = {'host':'', 'user':'', ...}
conn = MyConnect(**config)
cur = conn.cursor()

Update statement not working in python?

I've written a simple python program, which get Data form database successfully. but unable to update table in DB.
When executing update statement it's get stuck and nothing happen, no any exception.
My code is as follows. Any idea whyis this ?
from java.sql import DriverManager
def updateDB():
url = "jdbc:oracle:thin:#192.1.1.1:1521:auid"
uname = "dbtstj1"
pword = "dbtstj321"
conn = None
stmt = None
try:
conn = DriverManager.getConnection(url,uname,pword)
stmt = conn.createStatement()
rs = stmt.executeQuery("select PKG_NAME from PkgData")
while rs.next():
print rs.getString(1)
pkgName = "'Test Pkg Name'"
pkgID = "'T1234'"
updateQuary = "UPDATE PkgData SET PKG_NAME =%s WHERE PKG_ID =%s" %(pkgName, pkgID)
stmt.execute(updateQuary)
except Exception , e:
print 'Error:', e[0]
finally:
if stmt is not None:
stmt.close()
if conn is not None:
conn.close()
updateDB()

you need to commit your changes to the database:
stmt.execute(updateQuary)
conn.commit()

These type of issues can be happen when query request datatype and required datatype is difference.
It seems to be there was a mismatch with database's datatype and your query. Can you recheck with database's datatype with your query.
For Ex: PKG_ID =%s can be another data type in database as digit or etc...

Python insert into SQL unicode error

I have the following error :
db = MySQLdb.connect(host="localhost", # host
user="root", # username
passwd="", # password
db="test",charset='utf8') #
cur = db.cursor()
x = "испытание" # random unicode characters
sql = "INSERT INTO links(test) VALUES(N'%s');"
lst = ( x ) #x is unicode data
cur.execute(sql,lst)
The error I get is : MySQL Error [1064]: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax ...

x = "испытание" # random unicode characters
(What Python version are you using? If 2.x, those are not Unicode characters, they're bytes.)
sql = "INSERT INTO links(test) VALUES(N'%s');"
When you use parameterised queries, you don't include the string literal delimiters ''. For MySQLdb where the parameter marker is %s, it should just be:
sql = "INSERT INTO links(test) VALUES(%s);"
(Note also NVARCHAR is unnecessary in MySQL.)
lst = ( x ) #x is unicode data
Here lst is the same value as x, you haven't got a tuple. If you really want a tuple-of-one then say (x,), but probably using an actual list [x] is clearer.

what value are you trying to insert? nothing. what does N'%s' represents. I guess you are trying to insert from a form so I have work on this code, tested it and is running.
First create database with 2 column employee_id and lastname. Then test this code. it will work for you. If you get it right please tick this as answer. Sectona...
#!c:/python25/python
import MySQLdb
# Import modules for CGI handling
import cgi, cgitb
# Create instance of FieldStorage
form = cgi.FieldStorage()
# get form data from form field
employee_id = form.getvalue('employee_id')
lastname = form.getvalue('lastname')
# Open database connection
db = MySQLdb.connect("localhost","sectona","sectona90","sectona_db" )
# prepare a cursor object using cursor() method
cursor = db.cursor()
sql = "INSERT INTO EMPLOYEE(employee_id, \
lastname) \
VALUES ('%s', '%s')" % \
(employee_id, lastname)
try:
# Execute the SQL command
cursor.execute(sql)
# Commit your changes in the database
db.commit()
except:
# Rollback in case there is any error
db.rollback()
# disconnect from server
db.close()
print "Content-type:text/html\r\n\r\n"
print '<html>'
print '<body>'
print '<h2>Data Submitted Successfully</h2>'
print "<b>Employee Identity:</b> %s<br>" % (employee_id)
print "<b>Last Name:</b> %s<br>" % (lastname)
print '</body>'
print '</html>'

Checking if a postgresql table exists under python (and probably Psycopg2)

How can I determine if a table exists using the Psycopg2 Python library? I want a true or false boolean.

How about:
>>> import psycopg2
>>> conn = psycopg2.connect("dbname='mydb' user='username' host='localhost' password='foobar'")
>>> cur = conn.cursor()
>>> cur.execute("select * from information_schema.tables where table_name=%s", ('mytable',))
>>> bool(cur.rowcount)
True
An alternative using EXISTS is better in that it doesn't require that all rows be retrieved, but merely that at least one such row exists:
>>> cur.execute("select exists(select * from information_schema.tables where table_name=%s)", ('mytable',))
>>> cur.fetchone()[0]
True

I don't know the psycopg2 lib specifically, but the following query can be used to check for existence of a table:
SELECT EXISTS(SELECT 1 FROM information_schema.tables
WHERE table_catalog='DB_NAME' AND
table_schema='public' AND
table_name='TABLE_NAME');
The advantage of using information_schema over selecting directly from the pg_* tables is some degree of portability of the query.

select exists(select relname from pg_class
where relname = 'mytablename' and relkind='r');

The first answer did not work for me. I found success checking for the relation in pg_class:
def table_exists(con, table_str):
exists = False
try:
cur = con.cursor()
cur.execute("select exists(select relname from pg_class where relname='" + table_str + "')")
exists = cur.fetchone()[0]
print exists
cur.close()
except psycopg2.Error as e:
print e
return exists

#!/usr/bin/python
# -*- coding: utf-8 -*-
import psycopg2
import sys
con = None
try:
con = psycopg2.connect(database='testdb', user='janbodnar')
cur = con.cursor()
cur.execute('SELECT 1 from mytable')
ver = cur.fetchone()
print ver //здесь наш код при успехе
except psycopg2.DatabaseError, e:
print 'Error %s' % e
sys.exit(1)
finally:
if con:
con.close()

I know you asked for psycopg2 answers, but I thought I'd add a utility function based on pandas (which uses psycopg2 under the hood), just because pd.read_sql_query() makes things so convenient, e.g. avoiding creating/closing cursors.
import pandas as pd
def db_table_exists(conn, tablename):
# thanks to Peter Hansen's answer for this sql
sql = f"select * from information_schema.tables where table_name='{tablename}'"
# return results of sql query from conn as a pandas dataframe
results_df = pd.read_sql_query(sql, conn)
# True if we got any results back, False if we didn't
return bool(len(results_df))
I still use psycopg2 to create the db-connection object conn similarly to the other answers here.

The following solution is handling the schema too:
import psycopg2
with psycopg2.connect("dbname='dbname' user='user' host='host' port='port' password='password'") as conn:
cur = conn.cursor()
query = "select to_regclass(%s)"
cur.execute(query, ['{}.{}'.format('schema', 'table')])
exists = bool(cur.fetchone()[0])

Expanding on the above use of EXISTS, I needed something to test table existence generally. I found that testing for results using fetch on a select statement yielded the result "None" on an empty existing table -- not ideal.
Here's what I came up with:
import psycopg2
def exist_test(tabletotest):
schema=tabletotest.split('.')[0]
table=tabletotest.split('.')[1]
existtest="SELECT EXISTS (SELECT 1 FROM information_schema.tables WHERE table_schema = '"+schema+"' AND table_name = '"+table+"' );"
print('existtest',existtest)
cur.execute(existtest) # assumes youve already got your connection and cursor established
# print('exists',cur.fetchall()[0])
return ur.fetchall()[0] # returns true/false depending on whether table exists
exist_test('someschema.sometable')

You can look into pg_class catalog:
The catalog pg_class catalogs tables and most everything else that has
columns or is otherwise similar to a table. This includes indexes (but
see also pg_index), sequences (but see also pg_sequence), views,
materialized views, composite types, and TOAST tables; see relkind.
Below, when we mean all of these kinds of objects we speak of
“relations”. Not all columns are meaningful for all relation types.
Assuming an open connection with cur as cursor,
# python 3.6+
table = 'mytable'
cur.execute(f"SELECT EXISTS(SELECT relname FROM pg_class WHERE relname = {table});")
if cur.fetchone()[0]:
# if table exists, do something here
return True
cur.fetchone() will resolve to either True or False because of the EXISTS() function.