I'm aware that the best way to prevent sql injection is to write Python queries of this form (or similar):
query = 'SELECT %s %s from TABLE'
fields = ['ID', 'NAME']
cur.execute(query, fields)
The above will work for a single query, but what if we want to do a UNION of 2 SQL commands? I've set this up via sqlite3 for sake of repeatability, though technically I'm using pymysql. Looks as follows:
import sqlite3
conn = sqlite3.connect('dummy.db')
cur = conn.cursor()
query = 'CREATE TABLE DUMMY(ID int AUTO INCREMENT, VALUE varchar(255))'
query2 = 'CREATE TABLE DUMMy2(ID int AUTO INCREMENT, VALUE varchar(255)'
try:
cur.execute(query)
cur.execute(query2)
except:
print('Already made table!')
tnames = ['DUMMY1', 'DUMMY2']
sqlcmds = []
for i in range(0,2):
query = 'SELECT %s FROM {}'.format(tnames[i])
sqlcmds.append(query)
fields = ['VALUE', 'VALUE']
sqlcmd = ' UNION '.join(sqlcmds)
cur.execute(sqlcmd, valid_fields)
When I run this, I get a sqlite Operational Error:
sqlite3.OperationalError: near "%": syntax error
I've validated the query prints as expected with this output:
INSERT INTO DUMMY VALUES(%s) UNION INSERT INTO DUMMY VALUES(%s)
All looks good there. What is the issue with the string substitutions here? I can confirm that running a query with direct string substitution works fine. I've tried it with both selects and inserts.
EDIT: I'm aware there are multiple ways to do this with executemany and a few other. I need to do this with UNION for the purposes I'm using this for because this is a very, very simplified example fo the operational code I'm using
The code below executes few INSERTS at once
import sqlite3
conn = sqlite3.connect('dummy.db')
cur = conn.cursor()
query = 'CREATE TABLE DUMMY(ID int AUTO INCREMENT NOT NULL, VALUE varchar(255))'
try:
cur.execute(query)
except:
print('Already made table!')
valid_fields = [('ya dummy',), ('stupid test example',)]
cur.executemany('INSERT INTO DUMMY (VALUE) VALUES (?)',valid_fields)
I have an issue to run my SQL queries on a Postgres ElephantSql hosted:
This is my code to connect (except dynamo, user, password which are replaced by XXX
DATABASE_URL = 'postgres://YYYY:ZZZZ#drona.db.elephantsql.com:5432/YYYY'
# ---------------------------- CONNECT ELEPHANT DB
def ElephantConnect():
up.uses_netloc.append("postgres")
url = up.urlparse(DATABASE_URL)
conn = psycopg2.connect(dbname='YYYY',
user='YYYY',
password='ZZZZ',
host='drona.db.elephantsql.com',
port='5432'
)
cursor = conn.cursor()
# cursor.execute("CREATE TABLE notes(id integer primary key, body text, title text);")
#conn.commit()
# conn.close()
return conn
this code seems to connect well to db
My issue is when I want to delete a table:
def update(df, table_name, deleteYes= 'Yes'):
conn = ElephantConnect()
db = create_engine(DATABASE_URL)
cursor =conn.cursor()
if deleteYes == 'Yes': # delete
queryCount = "SELECT count(*) FROM {};".format(table_name)
queryDelete = "DELETE FROM {};".format(table_name)
count = db.execute(queryCount)
rows_before = count.fetchone()[0]
try:
db.execute(queryDelete)
logging.info('Deleted {} rows into table {}'.format(rows_before, table_name))
except:
logging.info('Deleted error into table {}'.format(table_name))
else:
pass
It seems when I run db.execute(queryDelete), it goes to the exception.
I have no message of error. But the query with count data is working...
thanks
I think that the reason for the error is because there are foreign keys against the table. In order to be sure, assign the exception into a variable and print it:
except Exception as ex:
print(ex)
By the way, if you want to quickly delete all of the rows from a table then
It will be much more efficient to truncate the table instead of deleting all the rows:
truncate table table_name
Delete is more useful when you want to delete rows under some conditions:
delete from table_name where ...
I tried writing this code hoping that it will auto increment, but somehow it is not working and the output entries in id column are set to 'None'.I have also tried other answers but none of them are working.Please help if possible.
Here is the code:
import sqlite3
def connect():
conn=sqlite3.connect("books.db")
cur=conn.cursor()
cur.execute("CREATE TABLE IF NOT EXISTS book (id INTEGER PRIMARY KEY,title text,author text,year int,isbn int)")
conn.commit()
conn.close()
def insert(title,author,year,isbn):
conn=sqlite3.connect("books.db")
cur=conn.cursor()
cur.execute("INSERT INTO book VALUES (?,?,?,?)",(title,author,year,isbn))
conn.commit()
conn.close()
def view():
conn=sqlite3.connect("books.db")
cur=conn.cursor()
cur.execute("SELECT * FROM book ")
rows=cur.fetchall()
conn.close()
return rows
connect()
insert("sample","abc",2003,123456)
insert("sample2","def",2003,123457)
print(view())
This is the output:
[(None, 'sample', 'abc', 2003, 123456), (None, 'sample2', 'def', 2003, 123457)]
Answer Edited: Thanks to Shawn's comment I went back to play with the code and hunt down the problem It is true that AUTOINCREMENT is not needed and as such is not the problem (I learned something new about sqlite).
The following code does work. Notice, since you're not supplying data to all columns in the table that you must specify which columns you are inserting data into in your insert statement. I have removed the unnecessary AUTOINCREMENT, and modified the insert statement to work correctly.
Also note: As others have stated, you should not use * wild card for selecting all columns in production code, but instead list all columns individual.
import sqlite3
def connect():
conn=sqlite3.connect("books.db")
cur=conn.cursor()
cur.execute("CREATE TABLE IF NOT EXISTS book (id INTEGER PRIMARY KEY,title text,author text,year int,isbn int)")
conn.commit()
conn.close()
def insert(title,author,year,isbn):
conn=sqlite3.connect("books.db")
cur=conn.cursor()
cur.execute("INSERT INTO book (title, author, year, isbn) VALUES (?,?,?,?)",(title,author,year,isbn))
conn.commit()
conn.close()
def view():
conn=sqlite3.connect("books.db")
cur=conn.cursor()
cur.execute("SELECT * FROM book ")
rows=cur.fetchall()
conn.close()
return rows
connect()
insert("sample","abc",2003,123456)
insert("sample2","def",2003,123457)
print(view())
The produced output is:
[(1, 'sample', 'abc', 2003, 123456), (2, 'sample2', 'def', 2003, 123457)]
First SQLite recommends that you not use auto increment as your primary, you should select fields that will define a unique record whenever possible.
Second the data type you are passing in is “int” and requires the autoincrement keyword following primary key.
Third you should avoid using * in your select statement. If you simply need a row number back you can query the fields you need and add in the standard field “rowid”.
I'm using SQLite3 in Python to build a database and tables. I want to create a table according to the value of a variable (name: A). I found this (answer) and it says "Unfortunately, tables can't be the target of parameter substitution". And the full table name should be like "table_"+str(A). I tried several methods:
new_field = 'my_1st_column'
field_type = 'INTEGER'
tablename = "table_"+str(A)
databasecur.execute('CREATE TABLE IF NOT EXISTS {tn} ({nf} {ft})' .format(tn=tablename, nf=new_field, ft=field_type))
or
databasecur.prepare("CREATE TABLE IF NOT EXISTS" + tablename + "(col1, col2)")
All of them are not working. So right now the way I can figure out is like:
A=1140
tablelist = [1140, 1150, 1160, 1170,....]
tablenum = tablelist.index(A)
con = sqlite3.connect('test.db')
cur = con.cursor()
if tablenum == 0:
cur.execute("create table if not exists table_1140 (col1, col2)")
elif tablenum == 1:
cur.execute("create table if not exists table_1150 (col1, col2)")
......
con.close()
Does anyone have more effective way to do this? Thank you very much.
I have the following function:
def credential_check(username, password):
conn = sqlite3.connect('pythontkinter.db')
c = conn.cursor()
idvalue = c.execute('''SELECT ID FROM userdetails WHERE username = "{0}"'''.format(username)).fetchall()
print(idvalue)
I wish to assign the value of ID in my userdetails table to the variable idvalue in the row where the inputted username = userdetails username, however when I use this fetchall() I get [('0',)] printed out rather than just 0.
How do I go about doing this?
Thanks
You can use fetchone() if you only want one value. However, the result will still be returned as a tuple, just without the list.
import sqlite3
conn = sqlite3.connect('test.db')
c = conn.cursor()
c.execute('''CREATE TABLE IF NOT EXISTS testing(id TEXT)''')
conn.commit()
c.execute("""INSERT INTO testing (id) VALUES ('0')""")
conn.commit()
c.execute("""SELECT id FROM testing""")
data = c.fetchone()
print data
# --> (u'0',)
You can also use LIMIT if you want to restrict the number of returned values with fetchall().
More importantly, don't format your queries like that. Get used to using the ? placeholder as a habit so that you are not vulnerable to SQL injection.
idvalue = c.execute("""SELECT ID FROM userdetails WHERE username = ?""", (username,)).fetchone()