db.execute('CREATE TABLE IF NOT EXISTS test (name TEXT)')
print("Table created successfully")
db.execute('INSERT INTO %s VALUES (%s)' % ('test', 'sample'))
db.close()
I am using Python. The table is created successfully with a column of "name", but I am unable to insert anything.
I get the error:
sqlite3.OperationalError: no such column: sample
Why?
I think you need something like:
INSERT INTO test (name)
VALUES
('sample');
Translated into python:
db.execute('INSERT INTO %s (name) VALUES (\'%s\')' % ('test', 'sample'))
You forgot the column name?
db.execute('CREATE TABLE IF NOT EXISTS test (name TEXT)')
print("Table created successfully")
db.execute('INSERT INTO %s (name) VALUES (%s)' % ('test', 'sample'))
db.close()
You need to commit your changes. If you don't call db.commit(), your changes (including creating the table) will be rolled back when you close the database.
This should work:
db.execute('INSERT INTO test VALUES (%s)' % ('sample'))
You definitely don't want to use string substitutions. That opens you to https://www.owasp.org/index.php/SQL_Injection. Instead, use database binding which incidentally fixes your actual error (which is that you are treating sample as a column name, rather than data).
Ok, might as well write it up:
import sqlite3
db = sqlite3.connect(':memory')
#you control this stuff, as the db schema isn't typically coming from user data
#so less likely to be a mess...
#i.e. build your query templates with string substitutions, but exec with binds.
tablename = 'test'
db.execute('CREATE TABLE IF NOT EXISTS %s (name TEXT)' % (tablename))
print("Table created successfully")
qry = 'INSERT INTO %s VALUES (?)' % (tablename)
#the data is where you want to be careful
db.execute(qry, ('sample',))
print ("insert done")
db.close()
which gives:
Table created successfully
insert done
Check out docs # https://docs.python.org/2/library/sqlite3.html Starting at # Never do this -- insecure!.
Related
I am building a pipeline to upload data from local folders into snowflake cloud.
The last step after uploading each table is to alter the table itself named after the CSV file name, to add a foreign key to the master table. I am doing the following:
query = """ALTER TABLE """+filename+""" ADD FOREIGN KEY (SURVEY_ID) REFERENCES SURVEYS(SURVEY_ID)"""
if(cursor.execute(query)):
print(filename+ " FOREIGN KEY added successfully")
else:
print(filename+" FOREIGN KEY was not added")
I have got the following error:
Table 'MY_DB.PUBLIC.SURVEY_AST' does not exist or not
authorized.
I know the table name should be like above in order to be found and the query will be executed:
"MY_DB"."PUBLIC"."SURVEY_AST"
If I print the filename I will get:
SURVEY_AST
I tried to use %s but it is not possible to bind the table name itself using snowflake cursor:
query = """ALTER TABLE %s ADD FOREIGN KEY (SURVEY_ID) REFERENCES SURVEYS(SURVEY_ID)"""
if(cursor.execute(query, (filename)):
print(filename+ " FOREIGN KEY added successfully")
else:
print(filename+" FOREIGN KEY was not added")
How can I make the engine stop binding extra text to the table itself in order to find it and execute the query?
Take note that the previous queries are working normally (INSERT, SELECT...).
EDIT
conn = snowCtx.connect(
user=user,
password=password,
account=account,
database='MY_DB',
schema='PUBLIC',
warehouse='COMPUTE_WH',
role='SYSADMIN'
)
if(success):
print(filename+' columns uploaded')
# UPDATE now variable
now = date.today().strftime("%Y-%m-%d")
# To insert the data files into the new table
data = data.applymap(str)
data['_xform_id'] = data['_xform_id'].apply(lambda x: int(float(x)))
data['SURVEY_ID'] = data['SURVEY_ID'].apply(lambda x: int(str(x)))
try:
data.to_sql(name=filename,
con=engine.connect(),
if_exists="replace",
index=False)
print(filename+' to_sql() done.')
print("--- %s seconds ---" % (time.time() - start_time))
log_frame.append({'Survey Name': filename, 'Survey ID': nextval}, ignore_index=True)
except sqlalchemy.exc.Error as e:
print("Error adding the table: "+ e)
# Add a foreign key
finally:
# query = "select count (*) from information_schema.tables WHERE table_type = 'BASE TABLE' AND table_name="+filename
print(filename)
query = """ALTER TABLE {0} ADD FOREIGN KEY (SURVEY_ID) REFERENCES SURVEYS(SURVEY_ID)""".format(filename.replace('\'', '\'\''))
print(query)
if(cursor.execute(query)):
print(filename+ " FOREIGN KEY added successfully")
else:
print(filename+" FOREIGN KEY was not added")
I don't think that it's adding any extra text. As I see, The error message includes the database and schema name (which is expected). For example:
create table DENEME ( i integer );
alter table DENEME2 add column z varchar;
SQL compilation error: Table 'GOKHAN_DB.PUBLIC.DENEME2' does not exist or not authorized.
Please check the query history to see the SQL text:
https://docs.snowflake.com/en/sql-reference/functions/query_history.html
Do you think that the table name should be like below?
"MY_DB"."PUBLIC"."SURVEY_AST"
It's not different than MY_DB.PUBLIC.SURVEY_AST (because all characters are uppercase). So it should work. Please check the previous commands to see how you create table (maybe you create it in a different schema).
The solution was to do the following on the filename itself:
filename = "\""+database+"\".\""+schema+"\".\""+filename+"\""
Which resulted exactly:
"MY_DB"."SCHEMA"."TABLENAME"
I am attempting to write an SQL query to insert rows into a table when the script is run. I have been able to get it working whilst duplicating data, but am unable to set up an insert that can determine whether or not the record is unique.
I have tried following these threads:
Most efficient way to do a SQL 'INSERT IF NOT EXISTS'
Python MySQLdb / MySQL INSERT IGNORE & Checking if Ignored
Currently, I can get the system to add all records, but due to the frequency the script is ran at the database is being filled with the same records. This is my current code:
for post in r.get_subreddit("listentothis").get_new(limit=5):
if 'youtube.com' in post.url or 'youtu.be' in post.url:
print(str(post.title))
print(str(post.url))
sql = """INSERT INTO listentothis2(title,url) VALUES ('%s', '%s') WHERE NOT EXISTS (SELECT 1 FROM listentothis2 WHERE url = '%s') """ % (post.title, post.url, post.url)
try:
cursor.execute(sql)
db.commit()
except:
db.rollback()
I have also tried:
sql = """INSERT IGNORE INTO listentothis2(title,url) VALUES ('%s', '%s') """ % (post.title, post.url)
However, this adds the 3 most recent records whether or not they are commited to the database already. Any help is greatly appreciated!
i'm trying to write a program where i loop through a list of strings and use those strings to make names for my tables, however i don't quite understand how to properly escape everything without getting an error.
Heres my code right now:
cursor = db.cursor()
cursor.execute('DROP TABLE IF EXISTS %s' %data[1])
0L
sql ="""CREATE TABLE %s (IP TEXT, AVAILIBILITY INT)""" %data[1]
cursor.execute(sql)
0L
for key in data[0]:
cursor.execute("""INSERT INTO %s VALUES (%s,%s)""" %(data[1],key,data[0][key]))
db.commit()
1L
cursor.execute("""SELECT * FROM %s""" %data[1])
1L
print cursor.fetchall()
((188L, 90),)
db.close()
My code works when i enter a string rather than a variable that represents a string, but fails when using a variable
you should use string formatting in this line
sql ="""CREATE TABLE data[1] (IP TEXT, AVAILIBILITY INT, AP TEXT )"""
when you call cur.execute(sql) method on it, this will raise errorclass or/and errorvalue error.
try something like this:
sql ="""CREATE TABLE %s (IP TEXT, AVAILIBILITY INT, AP TEXT )""" %data[1]
I have created table using this create command as:
CREATE TABLE test_table(id INT PRIMARY KEY,name
VARCHAR(50),price INT)
i want to insert into this table wherein values are stored already in variable
bookdb=# name = 'algorithms'
bookdb-# price = 500
bookdb-# INSERT INTO test_table VALUES(1,'name',price);
I get the following error:
ERROR: syntax error at or near "name"
LINE 1: name = 'algorithms'
Can anyone point out the mistake and propose solution for the above?
Thanks in advance
Edit:
import psycopg2
import file_content
try:
conn = psycopg2.connect(database='bookdb',user='v22')
cur = conn.cursor()
cur.execute("DROP TABLE IF EXISTS book_details")
cur.execute("CREATE TABLE book_details(id INT PRIMARY KEY,name VARCHAR(50),price INT)")
cur.execute("INSERT INTO book_details VALUES(1,'name',price)")
conn.commit()
except:
print "unable to connect to db"
I have used the above code to insert values into table,variables name and price containing the values to be inserted into table are available in file_content python file and i have imported that file.The normal INSERT statement takes values manually but i want my code to take values which are stored in variables.
SQL does not support the concept of variables.
To use variables, you must use a programming language, such as Java, C, Xojo. One such language is PL/pgSQL, which you can think of as a superset of SQL. PL/PgSQL is often bundled as a part of Postgres installers, but not always.
I suggest you read some basic tutorials on SQL.
See this similar question: How do you use script variables in PostgreSQL?
don't have postgres installed here, but you can try this
import psycopg2
import file_content
try:
conn = psycopg2.connect(database='bookdb',user='v22')
cur = conn.cursor()
cur.execute("DROP TABLE IF EXISTS book_details")
cur.execute("CREATE TABLE book_details(id INT PRIMARY KEY,name VARCHAR(50),price INT)")
cur.execute("INSERT INTO book_details VALUES(1, '%s', %s)" % (name, price))
conn.commit()
except:
print "unable to connect to db"
If you are using PSQL console:
\set name 'algo'
\set price 10
insert into test_table values (1,':name',:price)
\g
I have created a database with MySQLdb.
In database I have a table with name student with columns:
id(is int),
id_user(is int),
f_name(is str),
l_name(is str)
I want to update a row.
My code is below:
db=mdb.connect(host="localhost", use_unicode="True", charset="utf8",
user="", passwd="", db="test")
# prepare a cursor object using cursor() method
cursor = db.cursor()
sql="""SELECT id_user FROM student"""
try:
# Execute the SQL command
cursor.execute(sql)
# Commit your changes in the database
db.commit()
except:
# Rollback in case there is any error
db.rollback()
rows = cursor.fetchall()
the=int(7)
se=str('ok')
for row in rows:
r=int(row[0])
if r==the:
sql2 = """UPDATE student
SET f_name=%s
WHERE id_user = %s"""% (se,the)
# Execute the SQL command
cursor.execute(sql2)
# Commit your changes in the database
db.commit()
db.rollback()
# disconnect from server
db.close()
When I run it I take the error there is column with name ok why?
Can anyone help me find what I am doing wrong please?
str doesn't wrap its argument in quotation marks, so your statement is this:
UPDATE student SET f_name=ok WHERE id_user = 7
when it needs to be this:
UPDATE student SET f_name='ok' WHERE id_user = 7
So, either change this line:
SET f_name=%s
to this:
SET f_name='%s'
or else change this line:
se=str('ok')
to this:
se="'" + str('ok') + "'"
Though I recommend reading about SQL injection, which will become a concern as soon as you start using user-supplied data instead of hard-coded values.
You should run the query like this:
sql2 = """UPDATE student
SET f_name = %s
WHERE id_user = %s"""
cursor.execute(sql2, (se, the))
Don't use string interpolation, let the database driver handle passing the parameters for you. Otherwise you have to deal with syntax errors like this, or worse, SQL injection.
More details here.
You should always enclose your data with quotes.
Instead of %s solely use '%s' the only types you dont need it are numeric ones, but even there i would enclose %d with '%d' cos it is more save.
And you should use at least db.escape_string(your_data) before inserting or updating same values into your database.
Or have a look at the pdo-using style of mysqldb:
http://mysql-python.sourceforge.net/MySQLdb.html#some-examples
c=db.cursor()
max_price=5
c.execute("""SELECT spam, eggs, sausage FROM breakfast
WHERE price < %s""", (max_price,))