I'm using Python and psycopg2 to interface to postgres.
When I insert a row...
sql_string = "INSERT INTO hundred (name,name_slug,status) VALUES ("
sql_string += hundred_name + ", '" + hundred_slug + "', " + status + ");"
cursor.execute(sql_string)
... how do I get the ID of the row I've just inserted? Trying:
hundred = cursor.fetchall()
returns an error, while using RETURNING id:
sql_string = "INSERT INTO domes_hundred (name,name_slug,status) VALUES ("
sql_string += hundred_name + ", '" + hundred_slug + "', " + status + ") RETURNING id;"
hundred = cursor.execute(sql_string)
simply returns None.
UPDATE: So does currval (even though using this command directly into postgres works):
sql_string = "SELECT currval(pg_get_serial_sequence('hundred', 'id'));"
hundred_id = cursor.execute(sql_string)
Can anyone advise?
thanks!
cursor.execute("INSERT INTO .... RETURNING id")
id_of_new_row = cursor.fetchone()[0]
And please do not build SQL strings containing values manually. You can (and should!) pass values separately, making it unnecessary to escape and SQL injection impossible:
sql_string = "INSERT INTO domes_hundred (name,name_slug,status) VALUES (%s,%s,%s) RETURNING id;"
cursor.execute(sql_string, (hundred_name, hundred_slug, status))
hundred = cursor.fetchone()[0]
See the psycopg docs for more details: http://initd.org/psycopg/docs/usage.html#passing-parameters-to-sql-queries
I ended up here because I had a similar problem, but we're using Postgres-XC, which doesn't yet support the RETURNING ID clause. In that case you can use:
cursor.execute('INSERT INTO ........')
cursor.execute('SELECT LASTVAL()')
lastid = cursor.fetchone()['lastval']
Just in case it was useful for anyone!
Consider a RETURNING clause http://www.postgresql.org/docs/8.3/static/sql-insert.html
For me, neither ThiefMaster's answer worked nor Jamie Brown's. What worked for me was a mix of both, and I'd like to answer here so it can help someone else.
What I needed to do was:
cursor.execute('SELECT LASTVAL()')
id_of_new_row = cursor.fetchone()[0]
The statement lastid = cursor.fetchone()['lastval'] didn't work for me, even after cursor.execute('SELECT LASTVAL()'). The statement id_of_new_row = cursor.fetchone()[0] alone didn't work either.
Maybe I'm missing something.
ThiefMaster's approach worked for me, for both INSERT and UPDATE commands.
If cursor.fetchone() is called on a cursor after having executed an INSERT/UPDATE command but lacked a return value (RETURNING clause) an exception will be raised: ProgrammingError('no results to fetch'))
insert_query = """
INSERT INTO hundred (id, name, name_slug, status)
VALUES (DEFAULT, %(name)s, %(name_slug)s, %(status)s)
RETURNING id;
"""
insert_query_values = {
"name": "",
"name_slug": "",
"status": ""
}
connection = psycopg2.connect(host="", port="", dbname="", user="", password="")
try:
with connection:
with connection.cursor() as cursor:
cursor.execute(insert_query, insert_query_values)
num_of_rows_affected = cursor.rowcount
new_row_id = cursor.fetchone()
except psycopg2.ProgrammingError as ex:
print("...", ex)
raise ex
finally:
connection.commit()
connection.close()
Related
I have a problem with my python code which I want to use for a REST API server.
The current problem is that my database query is returning null when I know that the value is there
The code for the specific path:
#app.route('/data/active_predicted/<int:ticketId>', methods=['GET'])
def search_db_tickId_act(ticketId):
cursor = db.cursor()
db_query = cursor.execute("select * from active_predicted where ticketId=" + str(ticketId))
json_output = json.dumps(dict(cursor.fetchall()))
cursor.close()
if not cursor.fetchall():
return "Nothing found \n SQL Query: " + "select * from active_predicted where ticketId=" + str(ticketId)
else:
return str(cursor.fetchall())
When I access this URL I get returned the following:
Nothing found SQL Query: select * from active_predicted where ticketId=1324
When I plug this SQL query I get the result I want, 1 row with 2 columns but it seems as though the program cannot locate the row?
The problems:
As #pvg mentioned, you need to escape your input values when querying database;
If you want to fetch a dictionary-like result, passing dictionary=True when you initialize the cursor;
In your original code, you didn't return the variable json_output;
To fetch only one result, use fetchone instead fetchall;
After cursor.close() got called, you can obtain nothing from that cursor no matter you fetched before or not;
Use try-finally to ensure that cursor always get closed (at last).
Here's the fixed code:
#app.route('/data/active_predicted/<int:ticketId>', methods=['GET'])
def search_db_tickId_act(ticketId):
try:
cursor = db.cursor(dictionary=True)
db_query = cursor.execute("select * from active_predicted where ticketId=%s LIMIT 1", ticketId)
row = cursor.fetchone()
if row:
return json.dumps(row)
else:
return "Nothing found \n SQL Query: " + "select * from active_predicted where ticketId=" + str(ticketId)
finally:
cursor.close()
I know that this question has been asked in the past, but thorough searching hasn't seemed to fix my issue. I'm probably just missing something simple, as I'm new to the Python-mysql connector supplied by mysql.
I have a Python script which accesses a mysql database, but I'm having issues with removing quotes from my query. Here is my code:
import mysql.connector
try:
db = mysql.connector.connect(user='root', password='somePassword', host='127.0.0.1', database='dbName')
cursor = db.cursor()
query = "select * from tags where %s = %s"
a = 'tag_id'
b = '0'
cursor.execute(query, (a, b))
print cursor
data = cursor.fetchall()
print data
except mysql.connector.Error as err:
print "Exception tripped..."
print "--------------------------------------"
print err
cursor.close()
db.close()
My database is set up properly (as I'll prove shortly).
My output for this program is:
MySQLCursor: select * from tags where 'tag_id' = '0'
[]
Yet when I change my query to not use variables, for example:
cursor.execute("select * from tags where tag_id = 0")
Then my output becomes:
MySQLCursor: select * from tags where tag_id = 0
[(0, u'192.168.1.110')]
To me, this means that the only difference between my Cursor queries are the quotes.
How do I remove them from the query?
Thanks in advance.
I personally believe this code is correct and safe, but you should be extremely skeptical of using code like this without carefully reviewing it yourself or (better yet) with the help of a security expert. I am not qualified to be such an expert.
Two important things I changed:
I changed b = '0' to b = 0 so it ends up as a number rather than a quoted string. (This part was an easy fix.)
I skipped the built-in parameterization for the column name and replaced it with my own slight modification to the escaping/quoting built in to mysql-connector. This is the scary part that should give you pause.
Full code below, but again, be careful with this if the column name is user input!
import mysql.connector
def escape_column_name(name):
# This is meant to mostly do the same thing as the _process_params method
# of mysql.connector.MySQLCursor, but instead of the final quoting step,
# we escape any previously existing backticks and quote with backticks.
converter = mysql.connector.conversion.MySQLConverter()
return "`" + converter.escape(converter.to_mysql(name)).replace('`', '``') + "`"
try:
db = mysql.connector.connect(user='root', password='somePassword', host='127.0.0.1', database='dbName')
cursor = db.cursor()
a = 'tag_id'
b = 0
cursor.execute(
'select * from tags where {} = %s'.format(escape_column_name(a)),
(b,)
)
print cursor
data = cursor.fetchall()
print data
except mysql.connector.Error as err:
print "Exception tripped..."
print "--------------------------------------"
print err
cursor.close()
db.close()
I encountered a similar problem using pymysql and have shown my working code here, hope this will help.
What I did is overwrite the escape method in class 'pymysql.connections.Connection', which obviously adds "'" arround your string.
better have shown my code:
from pymysql.connections import Connection, converters
class MyConnect(Connection):
def escape(self, obj, mapping=None):
"""Escape whatever value you pass to it.
Non-standard, for internal use; do not use this in your applications.
"""
if isinstance(obj, str):
return self.escape_string(obj) # by default, it is :return "'" + self.escape_string(obj) + "'"
if isinstance(obj, (bytes, bytearray)):
ret = self._quote_bytes(obj)
if self._binary_prefix:
ret = "_binary" + ret
return ret
return converters.escape_item(obj, self.charset, mapping=mapping)
config = {'host':'', 'user':'', ...}
conn = MyConnect(**config)
cur = conn.cursor()
I get the error when running this code:
import sqlite3
user_name = raw_input("Please enter the name: ")
user_email = raw_input("Please enter the email: ")
db = sqlite3.connect("customer")
cursor=db.cursor()
sql = """INSERT INTO customer
(name, email) VALUES (?,?);,
(user_name, user_email)"""
cursor.execute(sql)
Why is this happening?
While the other posters are correct about your statement formatting you are receiving this particular error because you are attempting to perform multiple statements in one query (notice the ; in your query which separates statements).
From Python sqlite3 docs:
"execute() will only execute a single SQL statement. If you try to execute more than one
statement with it, it will raise a Warning. Use executescript() if you want to execute
multiple SQL statements with one call."
https://docs.python.org/2/library/sqlite3.html
Now your statement will not execute properly even if you use executescript() because there are other issues with the way it is formatted (see other posted answers). But the error you are receiving is specifically because of your multiple statements. I am posting this answer for others that may have wandered here after searching for that error.
Use executescript instead of execute
execute() will only execute a single SQL statement. If you try to execute more than one statement with it, it will raise a Warning. Use executescript() if you want to execute multiple SQL statements with one call.
https://docs.python.org/2/library/sqlite3.html#sqlite3.Cursor.execute
You have a ;, in the middle of the query string - that is an invalid syntax. Pass a dictionary as a second argument to execute if you want to use a named parameter binding.
sql = "INSERT INTO customer (name, email) VALUES (:name, :email)"
cursor.execute(sql, {'name':user_name, 'email':user_email})
Try this:
sql = """INSERT INTO customer
(name, email) VALUES (?,?)"""
cursor.execute(sql, (user_name, user_email))
import sqlite3
def DB():
List = {"Name":"Omar", "Age":"33"}
columns = ', '.join("" + str(x).replace('/', '_') + "" for x in List.keys())
values = ', '.join("'" + str(x).replace('/', '_') + "'" for x in List.values())
sql_qry = "INSERT INTO %s ( %s ) values (?,?) ; ( %s )" % ('Table Name', columns, values)
conn = sqlite3.connect("DBname.db")
curr = conn.cursor()
# curr.execute("""create table if not exists TestTable(
# Name text,
# Age text
# )""")
# print columns
# print values
# print sql
# sql = 'INSERT INTO yell (Name , Age) values (%s, %s)'
curr.execute(sql_qry)
DB()
I can't show the data from database sqlite in python.
connection = sqlite3.connect('db')
connection.cursor().execute('CREATE TABLE IF NOT EXISTS users ( \
id TEXT, \
name TEXT, \
avatar TEXT \
)')
# In cycle:
query = 'INSERT INTO users VALUES ("' + str(friend.id) + '", "' + friend.name + '", "' + friend.avatar +'" )'
print query
connection.cursor().execute(query)
connection.commit()
# After cycle
print connection.cursor().fetchall()
Sample output of query variable:
INSERT INTO users VALUES ("111", "Some Name", "http://avatar/path" )
In result, fetchall returns empty tuple. Why?
UPD
Forgotten code:
connection.cursor().execute('SELECT * FROM users')
connection.cursor().fetchall()
→
[]
INSERT does not return data. To get the data back out, you'll have to issue a SELECT statement.
import sqlite3
con = sqlite3.connect("db")
con.execute("create table users(id, name, avatar)")
con.execute("insert into users(id, name, avatar) values (?, ?, ?)", (friend.id, friend.name, friend.avatar))
con.commit()
for row in con.execute("select * from users")
print row
con.close()
Because the create table string as displayed is syntactically invalid Python, as is the insert into string.
Actually, the answer to your first question is: because you use different cursors.
connection.cursor() creates a new cursor in the connection you created before. fetchall() gives you the results of the query you executed before in that same cursor. I.e. what you did was this:
# After cycle
cursor1 = connection.cursor()
cursor1.execute('SELECT * FROM users')
cursor2 = connection.cursor()
cursor2.execute("")
cursor2.fetchall()
What you should have done was this:
# After cycle
cursor = connection.cursor()
cursor.execute('SELECT * FROM users')
print cursor.fetchall()
I need to process mysql data one row at a time and i have selected all rows put them in a tuple but i get the error above.
what does this mean and how do I go about it?
Provide some code.
You probably call some function that should update database, but the function does not return any data (like cursor.execute()). And code:
data = cursor.execute()
Makes data a None object (of NoneType). But without code it's hard to point you to the exact cause of your error.
It means that the object you are trying to iterate is actually None; maybe the query produced no results?
Could you please post a code sample?
The function you used to select all rows returned None. This "probably" (because you did not provide code, I am only assuming) means that the SQL query did not return any values.
Try using the cursor.rowcount variable after you call cursor.execute(). (this code will not work because I don't know what module you are using).
db = mysqlmodule.connect("a connection string")
curs = dbo.cursor()
curs.execute("select top 10 * from tablename where fieldA > 100")
for i in range(curs.rowcount):
row = curs.fetchone()
print row
Alternatively, you can do this (if you know you want ever result returned):
db = mysqlmodule.connect("a connection string")
curs = dbo.cursor()
curs.execute("select top 10 * from tablename where fieldA > 100")
results = curs.fetchall()
if results:
for r in results:
print r
This error means that you are attempting to loop over a None object. This is like trying to loop over a Null array in C/C++. As Abgan, orsogufo, Dan mentioned, this is probably because the query did not return anything. I suggest that you check your query/databse connection.
A simple code fragment to reproduce this error is:
x = None
for each i in x:
#Do Something
pass
This may occur when I try to let 'usrsor.fetchone' execute twice. Like this:
import sqlite3
db_filename = 'test.db'
with sqlite3.connect(db_filename) as conn:
cursor = conn.cursor()
cursor.execute("""
insert into test_table (id, username, password)
values ('user_id', 'myname', 'passwd')
""")
cursor.execute("""
select username, password from test_table where id = 'user_id'
""")
if cursor.fetchone() is not None:
username, password = cursor.fetchone()
print username, password
I don't know much about the reason. But I modified it with try and except, like this:
import sqlite3
db_filename = 'test.db'
with sqlite3.connect(db_filename) as conn:
cursor = conn.cursor()
cursor.execute("""
insert into test_table (id, username, password)
values ('user_id', 'myname', 'passwd')
""")
cursor.execute("""
select username, password from test_table where id = 'user_id'
""")
try:
username, password = cursor.fetchone()
print username, password
except:
pass
I guess the cursor.fetchone() can't execute twice, because the cursor will be None when execute it first time.
I know it's an old question but I thought I'd add one more possibility. I was getting this error when calling a stored procedure, and adding SET NOCOUNT ON at the top of the stored procedure solved it. The issue is that earlier selects that are not the final select for the procedure make it look like you've got empty row sets.
Try to append you query result to a list, and than you can access it. Something like this:
try:
cursor = con.cursor()
getDataQuery = 'SELECT * FROM everything'
cursor.execute(getDataQuery)
result = cursor.fetchall()
except Exception as e:
print "There was an error while getting the values: %s" % e
raise
resultList = []
for r in result:
resultList.append(r)
Now you have a list that is iterable.