Develop Reference

Error executing cursor.execute when quotes are required

fairly new to SQL in general. I'm currently trying to bolster my general understanding of how to pass commands via cursor.execute(). I'm currently trying to grab a column from a table and rename it to something different.
import mysql.connector
user = 'root'
pw = 'test!*'
host = 'localhost'
db = 'test1'
conn = mysql.connector.connect(user=user, password=pw, host=host, database=db)
cursor = conn.cursor(prepared=True)
new_name = 'Company Name'
query = f'SELECT company_name AS {new_name} from company_directory'
cursor.execute(query)
fetch = cursor.fetchall()
I've also tried it like this:
query = 'SELECT company_name AS %s from company_directory'
cursor.execute(query, ('Company Name'),)
fetch = cursor.fetchall()
but that returns the following error:
stmt = self._cmysql.stmt_prepare(statement)
_mysql_connector.MySQLInterfaceError: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '? from company_directory' at line 1
I'm using python and mySQL. I keep reading about database injection and not using string concatenation but every time I try to use %s I get an error similar to the one below where. I've tried switching to ? syntax but i get the same error.
If someone could ELI5 what the difference is and what exactly database injection is and if what I'm doing in the first attempt qualifies as string concatenation that I should be trying to avoid.
Thank you so much!

If a column name or alias contains spaces, you need to put it in backticks.
query = f'SELECT company_name AS `{new_name}` from company_directory'
You can't use a placeholder for identifiers like table and column names or aliases, only where expressions are allowed.

You can't make a query parameter in place of a column alias. The rules for column aliases are the same as column identifiers, and they must be fixed in the query before you pass the query string.
So you could do this:
query = f"SELECT company_name AS `{'Company Name'}` from company_directory'
cursor.execute(query)

copy LONGTEXT from MySQL to CITEXT in PostgreSQL using Python

I have data in MySQL table which I want to copy to a PostgreSQL table.
Everything works except when the MySQL contains a string with " and/or '
For example:
The data in MySQL:
When I run my code I get:
ProgrammingError: ERROR: syntax error at or near "t"
(the t of the can't)
This is my code:
postgre = pg.connect(dbname=DB,user=USR,passwd=PASSWD,host=HOST, port=PORT)
crs = db_remote.cursor(MySQLdb.cursors.DictCursor)
crs.execute ("""select post_id, post_excerpt from tabl""")
data = crs.fetchall ()
for row in data :
postgre.query("""INSERT INTO importfrommysql(id,message)
VALUES ('%s','%s')"""%(row["post_id"],row["post_excerpt"]))
the connection pg.connect is from PygreSQL package.
What can I do? Is it possible to get the text as it is? or the only solution is to drop all " / ' before the insert?

Use the Psycopg cursor.execute parameter passing:
import psycopg2
conn = psycopg2.connect(database='DB')
cursor = conn.cursor()
for row in data :
cursor.execute ("""
INSERT INTO importfrommysql (id,message)
VALUES (%s,%s)
""",
(row["post_id"],row["post_excerpt"])
)
It will escape and quote as necessary.

Syntax error when creating table in Vertica with PYODBC

I am trying to load a big list of sql queries into a table in Vertica using PYODBC. Here's my code:
tablename = DVXTEMP.my_table_name
sql = my_sql_query.strip().strip(';')
samplesize = 1000
createstring = 'CREATE TABLE %s AS %s \n limit %s;' %(tablename, sql, samplesize)
cursor.execute(createstring)
when I print createstring and run it in Toad, it works fine. when I try to execute it in pyodbc, it gives me the following error:
'Syntax error at or near "DVXTEMP" at character 1\n (4856) (SQLExecDirectW)'
We are using Vertica Analytic Database v7.1.2-6
Any ideas what might be causing this?
Thanks

1) did you import pyodbc?
2) did you define "cursor" from "pyodbc.connect"?
import pyodbc
DB = '[string for dbfile]'
DRV = '[string of which driver you are going to use]'
con = pyodbc.connect('DRIVER={};DBQ={}'.format(DRV,DB))
cursor = con.cursor()
##build SQL code and execute as you have done
Try SQL commands after you can connect without an error.
3) I use pyodbc for mdb files (MS Access) and some of my queries will not run unless I put single quotes outside double quotes on table/field names.
mytbl_1 = "mytbl"
SQL = 'SELECT * FROM ' + mytbl_1
print SQL
print result -> SELECT * FROM mytbl
(this fails)
mytbl_2 = '"mytbl"' #single quotes outside of double quote
SQL = 'SELECT * FROM ' + mytbl_2
print SQL
print result -> SELECT * FROM "mytbl"
(this string gets passed w/o error works for me with MDB files)

Error while testing postgresql database with python

I wanted to start into using databases in python. I chose postgresql for the database "language". I already created several databases, but now I want simply to check if the database exists with python. For this I already read this answer: Checking if a postgresql table exists under python (and probably Psycopg2) and tried to use their solution:
import sys
import psycopg2
con = None
try:
con = psycopg2.connect(database="testdb", user="test", password="abcd")
cur = con.cursor()
cur.execute("SELECT exists(SELECT * from information_schema.testdb)")
ver = cur.fetchone()[0]
print ver
except psycopg2.DatabaseError, e:
print "Error %s" %e
sys.exit(1)
finally:
if con:
con.close()
But unfortunately, I only get the output
Error relation "information_schema.testdb" does not exist
LINE 1: SELECT exists(SELECT * from information_schema.testdb)
Am I doing something wrong, or did I miss something?

Your question confuses me a little, because you say you want to look to see if a database exists, but you look in the information_schema.tables view. That view would tell you if a table existed in the currently open database. If you want to check if a database exists, assuming you have access to the 'postgres' database, you could:
import sys
import psycopg2, psycopg2.extras
cur = conn.cursor(cursor_factory=psycopg2.extras.DictCursor)
dbname = 'db_to_check_for_existance'
con = None
try:
con = psycopg2.connect(database="postgres", user="postgres")
cur = con.cursor(cursor_factory=psycopg2.extras.DictCursor)
cur.execute("select * from pg_database where datname = %(dname)s", {'dname': dbname })
answer = cur.fetchall()
if len(answer) > 0:
print "Database {} exists".format(dbname)
else:
print "Database {} does NOT exist".format(dbname)
except Exception, e:
print "Error %s" %e
sys.exit(1)
finally:
if con:
con.close()
What is happening here is you are looking in the database tables called pg_database. The column 'datname' contains each of the database names. Your code would supply db_to_check_for_existance as the name of the database you want to check for existence. For example, you could replace that value with 'postgres' and you would get the 'exists' answer. If you replace the value with aardvark you would probably get the does NOT exist report.

If you're trying to see if a database exists:
curs.execute("SELECT exists(SELECT 1 from pg_catalog.pg_database where datname = %s)", ('mydb',))
It sounds like you may be confused by the difference between a database and a table.

How to use a variable path in MySQL query in a Python script

I am having trouble getting a variable path into a MySQL query in a python script. The path variable is either resolved with double backslashes or none at all.
This works:
cursor.execute ("""load data local infile 'M:/Users/Jonathan/Dropbox/BCHS_3015/Spatial Data/Cartographic Data/USA/acs_data/Sequence_Number_and_Table_Number_Lookup.csv'
into table Sequence_Table_Lookup
fields terminated by ','enclosed by '"'
lines terminated by '\r\n'
ignore 1 lines
(File_ID,Table_ID,Sequence_Number,Line_Number, Subject_Area)""");
This following returns the error:
_mysql_exceptions.InternalError: (22, "File 'M:UsersJonathanDropbox\x08chs_3015spatial datacartographic datausaacs_dataSequence_Number_and_Table_Number_Lookup.txt' not found (Errcode: 22)")
cursor.execute ("""load data local infile '%s'
into table Sequence_Table_Lookup
fields terminated by ','enclosed by '"'
lines terminated by '\r\n'
ignore 1 lines
(File_ID,Table_ID,Sequence_Number,Line_Number, Subject_Area)""" % filepath);
Removing the single quotes around %s yields
_mysql_exceptions.ProgrammingError: (1064, "You have an error in your SQL syntax;
check the manual that corresponds to your MySQL server version for the right
syntax to use near 'M:\\Users\\Jonathan\\Dropbox\\bchs_3015\\spatial data\\cartographic data\\usa\\acs_data\\' at line 1")
I would appreciate any help in understanding how to insert a variable path into a MySQL query.
I am using PyDev in Eclipse on a windows machine. Python 2.7 and MySQLdb connector.
The full block of relevant code
conn = MySQLdb.connect (host = "localhost",
user = "user",
passwd = "pwd",
db = "gis_census_acs")
#finds census directory
dropbox = navigation.get_dropbox_home()
acs_data = os.path.join(dropbox,'bchs_3015','spatial data','cartographic data','usa','acs_data');
for filepath in navigation.get_filepaths(acs_data):
filename = os.path.split(filepath)[1]
if filename == 'Sequence_Number_and_Table_Number_Lookup.txt':
print filepath;
tablename = filename.split('.')[0].replace(' ','_')[0:64]
cursor = conn.cursor()
cursor.execute ('create table if not exists ' + tablename + """(
File_ID varchar(255),
Table_ID varchar(255),
Sequence_Number varchar(255),
Line_Number varchar(255),
Start_Position varchar(255),
Total_cells_in_Table varchar(255),
Total_Cells_in_Sequence varchar(255),
Table_title text,
Subject_Area text
)""");
cursor.execute ("""load data local infile '%s'
into table Sequence_Table_Lookup
fields terminated by ','enclosed by '"'
lines terminated by '\r\n'
ignore 1 lines
(File_ID,Table_ID,Sequence_Number,Line_Number, Start_Position,
Total_cells_in_Table, Total_Cells_in_Sequence, Table_title, Subject_Area)""" % filepath);
print "Number of rows inserted: %d" % cursor.rowcount
cursor.close()
else:
print "not the file"
conn.close ()

This file exists:
M:/Users/Jonathan/Dropbox/BCHS_3015/Spatial Data/Cartographic Data/USA/acs_data/Sequence_Number_and_Table_Number_Lookup.csv
As you expect, this strange one doesnt:
M:UsersJonathanDropbox\x08chs_3015spatial datacartographic datausaacs_dataSequence_Number_and_Table_Number_Lookup.txt
Seems like there's something wrong with your filepath. Try checking it out.