Develop Reference

Passing a folder location as an SQL parameter in python causes an error

I am fairly new to python and the only SQL I know is from this project so forgive the lack of technical knowledge:
def importFolder(self):
user = getuser()
filename = askopenfilename(title = "Choose an image from the folder to import", initialdir='C:/Users/%s' % user)
for i in range (0,len(filename) - 1):
if filename[-i] == "/":
folderLocation = filename[:len(filename) - i]
break
cnxn = pyodbc.connect('DRIVER={Microsoft Access Driver (*.mdb, *.accdb)};DBQ=C:\Users\Public\dbsDetectorBookingSystem.accdb')
cursor = cnxn.cursor()
cursor.execute("SELECT * FROM tblRuns")
cursor.execute("insert into tblRuns(RunID,RunFilePath,TotalAlphaCount,TotalBetaCount,TotalGammaCount) values (%s,%s,0,0,0)" %(str(self.runsCount + 1), folderLocation))
cnxn.commit()
self.runsCount = cursor.rowcount
rowString = str(self.runsCount) + " " + folderLocation + " " + str(0) + " " + str(0) + " " + str(0) + " " + str(0)
self.runsTreeView.insert("","end", text = "", values = (rowString))
That is one routine from my current program meant to create a new record which is mostly empty apart from an index and a file location. This location needs to be saved as a string however when it is passed as a paramenter to the SQL string the following error occurs:
cursor.execute("insert into tblRuns(RunID,RunFilePath,TotalAlphaCount,TotalBetaCount,TotalGammaCount) values (%s,%s,0,0,0)" %(str(self.runsCount + 1), folderLocation))
ProgrammingError: ('42000', "[42000] [Microsoft][ODBC Microsoft Access Driver] Syntax error (missing operator) in query expression 'C:/Users/Jacob/Documents/USB backup'. (-3100) (SQLExecDirectW)") I assume this is because the SQL recognises a file path and wantsto user it. Does anybody know how to fix this?

You're not using the db-api correctly. Instead of using string formatting to pass your query params - which is error-prone (as you just noticed) AND a security issue, you want to pass them as arguments to cursor.execute(), ie:
sql = "insert into tblRuns(RunID, RunFilePath, TotalAlphaCount, TotalBetaCount, TotalGammaCount) values (%s, %s, 0, 0, 0)"
cursor.execute(sql, (self.runsCount + 1, folderLocation))
Note that we DONT use string formatting here (no "%" between sql and the params)
NB : note that the placeholder for parameterized queries depends on your db connector. python-MySQLdb uses % but your one may use a ? or anything else.
wrt/ your exact problem: since you didn't put quotes around your placeholders, the sql query you send looks something like:
"insert into tblRuns(
RunID, RunFilePath,
TotalAlphaCount, TotalBetaCount, TotalGammaCount
)
values (1,/path/to/folder,0,0,0)"
Which cannot work, obviously (it needs quotes around /path/to/folder to be valid SQL).
By passing query parameters the right way, your db connector will take care of all the quoting and escaping.

Use a python dictionary to insert into mysql

I am trying to take the data from a dictionary (the example is simplified for readability) and insert it into a mysql database.
I have the following piece of code.
import pymysql
conn = pymysql.connect(server, user , password, "db")
cur = conn.cursor()
ORFs={'E7': '562', 'E6': '83', 'E1': '865', 'E2': '2756 '}
table="genome"
cols = ORFs.keys()
vals = ORFs.values()
sql = "INSERT INTO %s (%s) VALUES(%s)" % (
table, ",".join(cols), ",".join(vals))
print sql
print ORFs.values()
cur.execute(sql, ORFs.values())
cur.close()
conn.close()
the print sql statement returns
INSERT INTO genome (E7,E6,E1,E2) VALUES(562,83,865,2756 )
when I type this directly into the mysql command line, the mysql command works. But when I run the python script I get an error:
<type 'exceptions.TypeError'>: not all arguments converted during string formatting
args = ('not all arguments converted during string formatting',)
message = 'not all arguments converted during string formatting'
As always, any suggestions would be highly appreciated.

The previous answer doesn't work for non string dictionary value. This one is a revised version.
format_string = ','.join(['%s'] * len(dict))
self.db.set("""INSERT IGNORE INTO listings ({0}) VALUES ({1})""".format(", ".join(dict.keys()),format_string),
(dict.values()))

sql = "INSERT INTO %s (%s) VALUES(%s)" % (
table, ",".join(cols), ",".join(vals))
This SQL includes values and cur.execute(sql, ORFs.values()) has values, too.
So, it should be cur.execute(sql).

In my case, I will skip null columns.
data = {'k': 'v'}
fs = ','.join(list(map(lambda x: '`' + x + '`', [*data.keys()])))
vs = ','.join(list(map(lambda x: '%(' + x + ')s', [*data.keys()])))
sql = "INSERT INTO `%s` (%s) VALUES (%s)" % (table, fs, vs)
count = cursor.execute(sql, data)

Python insert into SQL unicode error

I have the following error :
db = MySQLdb.connect(host="localhost", # host
user="root", # username
passwd="", # password
db="test",charset='utf8') #
cur = db.cursor()
x = "испытание" # random unicode characters
sql = "INSERT INTO links(test) VALUES(N'%s');"
lst = ( x ) #x is unicode data
cur.execute(sql,lst)
The error I get is : MySQL Error [1064]: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax ...

x = "испытание" # random unicode characters
(What Python version are you using? If 2.x, those are not Unicode characters, they're bytes.)
sql = "INSERT INTO links(test) VALUES(N'%s');"
When you use parameterised queries, you don't include the string literal delimiters ''. For MySQLdb where the parameter marker is %s, it should just be:
sql = "INSERT INTO links(test) VALUES(%s);"
(Note also NVARCHAR is unnecessary in MySQL.)
lst = ( x ) #x is unicode data
Here lst is the same value as x, you haven't got a tuple. If you really want a tuple-of-one then say (x,), but probably using an actual list [x] is clearer.

what value are you trying to insert? nothing. what does N'%s' represents. I guess you are trying to insert from a form so I have work on this code, tested it and is running.
First create database with 2 column employee_id and lastname. Then test this code. it will work for you. If you get it right please tick this as answer. Sectona...
#!c:/python25/python
import MySQLdb
# Import modules for CGI handling
import cgi, cgitb
# Create instance of FieldStorage
form = cgi.FieldStorage()
# get form data from form field
employee_id = form.getvalue('employee_id')
lastname = form.getvalue('lastname')
# Open database connection
db = MySQLdb.connect("localhost","sectona","sectona90","sectona_db" )
# prepare a cursor object using cursor() method
cursor = db.cursor()
sql = "INSERT INTO EMPLOYEE(employee_id, \
lastname) \
VALUES ('%s', '%s')" % \
(employee_id, lastname)
try:
# Execute the SQL command
cursor.execute(sql)
# Commit your changes in the database
db.commit()
except:
# Rollback in case there is any error
db.rollback()
# disconnect from server
db.close()
print "Content-type:text/html\r\n\r\n"
print '<html>'
print '<body>'
print '<h2>Data Submitted Successfully</h2>'
print "<b>Employee Identity:</b> %s<br>" % (employee_id)
print "<b>Last Name:</b> %s<br>" % (lastname)
print '</body>'
print '</html>'

sqlite3.Warning: You can only execute one statement at a time

I get the error when running this code:
import sqlite3
user_name = raw_input("Please enter the name: ")
user_email = raw_input("Please enter the email: ")
db = sqlite3.connect("customer")
cursor=db.cursor()
sql = """INSERT INTO customer
(name, email) VALUES (?,?);,
(user_name, user_email)"""
cursor.execute(sql)
Why is this happening?

While the other posters are correct about your statement formatting you are receiving this particular error because you are attempting to perform multiple statements in one query (notice the ; in your query which separates statements).
From Python sqlite3 docs:
"execute() will only execute a single SQL statement. If you try to execute more than one
statement with it, it will raise a Warning. Use executescript() if you want to execute
multiple SQL statements with one call."
https://docs.python.org/2/library/sqlite3.html
Now your statement will not execute properly even if you use executescript() because there are other issues with the way it is formatted (see other posted answers). But the error you are receiving is specifically because of your multiple statements. I am posting this answer for others that may have wandered here after searching for that error.

Use executescript instead of execute
execute() will only execute a single SQL statement. If you try to execute more than one statement with it, it will raise a Warning. Use executescript() if you want to execute multiple SQL statements with one call.
https://docs.python.org/2/library/sqlite3.html#sqlite3.Cursor.execute

You have a ;, in the middle of the query string - that is an invalid syntax. Pass a dictionary as a second argument to execute if you want to use a named parameter binding.
sql = "INSERT INTO customer (name, email) VALUES (:name, :email)"
cursor.execute(sql, {'name':user_name, 'email':user_email})

Try this:
sql = """INSERT INTO customer
(name, email) VALUES (?,?)"""
cursor.execute(sql, (user_name, user_email))

import sqlite3
def DB():
List = {"Name":"Omar", "Age":"33"}
columns = ', '.join("" + str(x).replace('/', '_') + "" for x in List.keys())
values = ', '.join("'" + str(x).replace('/', '_') + "'" for x in List.values())
sql_qry = "INSERT INTO %s ( %s ) values (?,?) ; ( %s )" % ('Table Name', columns, values)
conn = sqlite3.connect("DBname.db")
curr = conn.cursor()
# curr.execute("""create table if not exists TestTable(
# Name text,
# Age text
# )""")
# print columns
# print values
# print sql
# sql = 'INSERT INTO yell (Name , Age) values (%s, %s)'
curr.execute(sql_qry)
DB()

Python/postgres/psycopg2: getting ID of row just inserted

I'm using Python and psycopg2 to interface to postgres.
When I insert a row...
sql_string = "INSERT INTO hundred (name,name_slug,status) VALUES ("
sql_string += hundred_name + ", '" + hundred_slug + "', " + status + ");"
cursor.execute(sql_string)
... how do I get the ID of the row I've just inserted? Trying:
hundred = cursor.fetchall()
returns an error, while using RETURNING id:
sql_string = "INSERT INTO domes_hundred (name,name_slug,status) VALUES ("
sql_string += hundred_name + ", '" + hundred_slug + "', " + status + ") RETURNING id;"
hundred = cursor.execute(sql_string)
simply returns None.
UPDATE: So does currval (even though using this command directly into postgres works):
sql_string = "SELECT currval(pg_get_serial_sequence('hundred', 'id'));"
hundred_id = cursor.execute(sql_string)
Can anyone advise?
thanks!

cursor.execute("INSERT INTO .... RETURNING id")
id_of_new_row = cursor.fetchone()[0]
And please do not build SQL strings containing values manually. You can (and should!) pass values separately, making it unnecessary to escape and SQL injection impossible:
sql_string = "INSERT INTO domes_hundred (name,name_slug,status) VALUES (%s,%s,%s) RETURNING id;"
cursor.execute(sql_string, (hundred_name, hundred_slug, status))
hundred = cursor.fetchone()[0]
See the psycopg docs for more details: http://initd.org/psycopg/docs/usage.html#passing-parameters-to-sql-queries

I ended up here because I had a similar problem, but we're using Postgres-XC, which doesn't yet support the RETURNING ID clause. In that case you can use:
cursor.execute('INSERT INTO ........')
cursor.execute('SELECT LASTVAL()')
lastid = cursor.fetchone()['lastval']
Just in case it was useful for anyone!

Consider a RETURNING clause http://www.postgresql.org/docs/8.3/static/sql-insert.html

For me, neither ThiefMaster's answer worked nor Jamie Brown's. What worked for me was a mix of both, and I'd like to answer here so it can help someone else.
What I needed to do was:
cursor.execute('SELECT LASTVAL()')
id_of_new_row = cursor.fetchone()[0]
The statement lastid = cursor.fetchone()['lastval'] didn't work for me, even after cursor.execute('SELECT LASTVAL()'). The statement id_of_new_row = cursor.fetchone()[0] alone didn't work either.
Maybe I'm missing something.

ThiefMaster's approach worked for me, for both INSERT and UPDATE commands.
If cursor.fetchone() is called on a cursor after having executed an INSERT/UPDATE command but lacked a return value (RETURNING clause) an exception will be raised: ProgrammingError('no results to fetch'))
insert_query = """
INSERT INTO hundred (id, name, name_slug, status)
VALUES (DEFAULT, %(name)s, %(name_slug)s, %(status)s)
RETURNING id;
"""
insert_query_values = {
"name": "",
"name_slug": "",
"status": ""
}
connection = psycopg2.connect(host="", port="", dbname="", user="", password="")
try:
with connection:
with connection.cursor() as cursor:
cursor.execute(insert_query, insert_query_values)
num_of_rows_affected = cursor.rowcount
new_row_id = cursor.fetchone()
except psycopg2.ProgrammingError as ex:
print("...", ex)
raise ex
finally:
connection.commit()
connection.close()