I'm trying to use OPENJSON in a Python script to import some basic JSON into a SQL database. I initially tried with a more complex JSON file, but simplified it for the sake of this post. Here's what I have:
sql_statement = "declare #json nvarchar(max) = '{\"name\":\"James\"}'; SELECT * FROM OPENJSON(#json) WITH (name nvarchar(20))"
cursor.execute(sql_statement)
cursor.commit()
connection.close()
The error I receive:
pypyodbc.ProgrammingError: (u'42000', u"[42000] [Microsoft][ODBC SQL
Server Driver][SQL Server]Incorrect syntax near the keyword 'with'. If
this statement is a common table expression, an xmlnamespaces clause
or a change tracking context clause, the previous statement must be
terminated with a semicolon.")
Any thoughts on why I'm seeing this error? I was successfully able to execute other SQL queries with the same pypyodbc / database configuration.
The problem could be that your database is running in an older compatibility level, where OPEN JSON is not available.
To find the compatibility level of your database, run following SQL statement:
SELECT compatibility_level FROM sys.databases WHERE name = 'your_db_name';
If the result is 120 or lower, you'll need to update your compatibility level to 130, by running:
ALTER DATABASE your_db_name SET COMPATIBILITY_LEVEL = 130;
Note: In case your database is actually Azure SQL DB, you should check the version as well, as OPEN JSON is not available for versions prior to 12.x
Related
I'm using Python + MySQL and want to use parameterized query. I'm stuck. I've encountered an error and can't figure out how to solve it. I've spent a day, checked dozens of articles, used various options (sinle quotes, double quotes, prepared statements) and still no luck.
Requirements: use Parameterized Query
Here is basic demo of the issue:
#!/usr/bin/python3
import mysql.connector as mysql
conn = mysql.connect(host=server, user=username, passwd=password, autocommit=True)
try:
create_database_query = "CREATE DATABASE %s;"
db_name = "BOOKS"
cursor = conn.cursor()
print(f"Creating {db_name} database... ", end='')
cursor.execute(create_database_query, (db_name,))
print("Success")
except mysql.Error as error:
print("Parameterized query failed {}".format(error))
Output:
Creating BOOKS database... Parameterized query failed 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ''BOOKS'' at line 1
So it looks like it uses too many quotes (2 single quotes on each side). The code above works fine if I change the following line:
create_database_query = "CREATE DATABASE %s;"
and put backtick around %s
The problem that now it creates a database but with invalid chars - 'BOOKS' (quotes are now part of db name). Duh...
If I use prepared statements then the same issue occurs but slightly different error message:
1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '?' at line 1
Environment:
MacOS Catalina
Python 3.8
PyCharm 2019.3 IDE
MySQL 8.0.19
mysql-connector-python module 8.0.19
What is going on? Any ideas?
Thanks
You can't use query parameters for identifiers (like a database name or table name or column name).
Query parameters can be used only in place of a constant value — a quoted string, quoted date/time, or a numeric value. Not identifiers, expressions, SQL keywords, etc.
To combine a database name with your CREATE DATABASE statement, you have to format it into the string in a way that forms the full statement before it is sent to MySQL.
db_name = "BOOKS"
create_database_query = "CREATE DATABASE %s;" % db_name
cursor.execute(create_database_query)
Because this creates a risk of SQL injection when you format variables into your string, it's up to you to make sure the db_name is safe.
Update: Thanks to #Parfait for the reminder about current best practices of string-formatting.
Prefer:
db_name = "BOOKS"
create_database_query = "CREATE DATABASE {};".format(db_name)
Or F-strings:
db_name = "BOOKS"
create_database_query = f"CREATE DATABASE {db_name};"
(In other words, Python has become Ruby ;-)
I'm using Python + MySQL and want to use parameterized query. I'm stuck. I've encountered an error and can't figure out how to solve it. I've spent a day, checked dozens of articles, used various options (sinle quotes, double quotes, prepared statements) and still no luck.
Requirements: use Parameterized Query
Here is basic demo of the issue:
#!/usr/bin/python3
import mysql.connector as mysql
conn = mysql.connect(host=server, user=username, passwd=password, autocommit=True)
try:
create_database_query = "CREATE DATABASE %s;"
db_name = "BOOKS"
cursor = conn.cursor()
print(f"Creating {db_name} database... ", end='')
cursor.execute(create_database_query, (db_name,))
print("Success")
except mysql.Error as error:
print("Parameterized query failed {}".format(error))
Output:
Creating BOOKS database... Parameterized query failed 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ''BOOKS'' at line 1
So it looks like it uses too many quotes (2 single quotes on each side). The code above works fine if I change the following line:
create_database_query = "CREATE DATABASE %s;"
and put backtick around %s
The problem that now it creates a database but with invalid chars - 'BOOKS' (quotes are now part of db name). Duh...
If I use prepared statements then the same issue occurs but slightly different error message:
1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '?' at line 1
Environment:
MacOS Catalina
Python 3.8
PyCharm 2019.3 IDE
MySQL 8.0.19
mysql-connector-python module 8.0.19
What is going on? Any ideas?
Thanks
You can't use query parameters for identifiers (like a database name or table name or column name).
Query parameters can be used only in place of a constant value — a quoted string, quoted date/time, or a numeric value. Not identifiers, expressions, SQL keywords, etc.
To combine a database name with your CREATE DATABASE statement, you have to format it into the string in a way that forms the full statement before it is sent to MySQL.
db_name = "BOOKS"
create_database_query = "CREATE DATABASE %s;" % db_name
cursor.execute(create_database_query)
Because this creates a risk of SQL injection when you format variables into your string, it's up to you to make sure the db_name is safe.
Update: Thanks to #Parfait for the reminder about current best practices of string-formatting.
Prefer:
db_name = "BOOKS"
create_database_query = "CREATE DATABASE {};".format(db_name)
Or F-strings:
db_name = "BOOKS"
create_database_query = f"CREATE DATABASE {db_name};"
(In other words, Python has become Ruby ;-)
I am trying to create an 'upsert' function for pypyodbc SQL Server. I have validated that the query built up will run in SSMS with the desired outcome, but when trying to execute and commit with pypyodbc I receive the following error: 'HY007', '[HY007] [Microsoft][ODBC SQL Server Driver]Associated statement is not prepared'.
Here is the upsert function:
def sql_upsert(sql_connection, table, key_field, key_value, **kwargs):
keys = ["{key}".format(key=k) for k in kwargs]
values = ["'{value}'".format(value=v) for v in kwargs.values()]
update_columns = ["{key} = '{value}'".format(key=k, value=v) for k, v in kwargs.items()]
sql = list()
#update
sql.append("UPDATE {table} SET ".format(table=table))
sql.append(", ".join(update_columns))
sql.append(" WHERE {} = '{}'".format(key_field, key_value))
sql.append(" IF ##ROWCOUNT=0 BEGIN ")
# insert
sql.append("INSERT INTO {table} (".format(table=table))
sql.append(", ".join(keys))
sql.append(") VALUES (")
sql.append(", ".join(values))
sql.append(")")
sql.append(" END")
query = "".join(sql)
print(query)
The function builds up a query string in a format based on this other thread How to insert or update using single query?
Here is an example of the output:
UPDATE test SET name='john' WHERE id=3012
IF ##ROWCOUNT=0 BEGIN
INSERT INTO test(name) VALUES('john')
END
The error message you cited is produced by the ancient "SQL Server" ODBC driver that ships as part of Windows. A more up-to-date driver version like "ODBC Driver 17 for SQL Server" should produce a meaningful error message.
If you look here or here you'll see people complaining about this over a decade ago.
Apparently SQL Server's ODBC driver returns that error when you're executing two statements that fail due to a field value being too long, or perhaps due to foreign key violations.
Using SSMS to see which statement causes this problem, or better - stop using ODBC and use pymssql
This error may also come when you don't give correct permissions to stored procedure
Go the SQL server --> Right click on your sp-->properties-->permissions
Add required users and roles which are going to execute this sp
This may help resolving the issue
I am currently trying to use pyodbc to select data from a table within Database A and insert it into a table within Database B. I was able to establish connections with both databases, so I know there is no error there. Additionally, my first cursor.execute command (line #9) works as I was able to print all the data.
The issue I am running into is when I try and insert the data from the first cursor.execute command into Database B. There are a few questions on SO regarding this same error, however I have checked to ensure I am not committing on of those errors. All the data types are accepted within SQL Server, I have the correct number of parameters and parameter markers, and I have ensured that the columns within my Python code match both the input and output tables. I am completely stuck and would greatly appreciate any help.
The specific error I am getting is:
('HYC00', '[HYC00] [Microsoft][ODBC SQL Server Driver]Optional feature
not implemented (0) (SQLBindParameter)')
Please see my code below:
import pyodbc
import time
cnxn1 = pyodbc.connect(r"DRIVER={SQL Server Native Client 11.0};SERVER='Server';" + \
"DATABASE='DatabaseA';Trusted_Connection=Yes")
cursor1 = cnxn1.cursor()
cnxn2 = pyodbc.connect(r"DRIVER={SQL Server};SERVER='Server'," + \
"user='Username', password='Password', database='DatabaseB'")
cursor2 = cnxn2.cursor()
SQL = cursor1.execute("select * from table.DatabaseA")
SQL2 = """insert into table.DatabaseB([col1], [col2], [col3], [col4],[col5], [col6], [col7],
[col8], [col9], [col10], [col11], [col12], [col13], [col14],
[col15], [col16],[col17], [col18], [col19], [col20], [col21],
[col22], [col23], [col24], [col25], [col26], [col27], [col28],
[col29], [col30], [col31])
values (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)"""
for row in cursor1.fetchall():
cursor2.execute(SQL2,row)
In regard to the last two lines of code, I have also tried the following with no success:
for row in SQL:
cursor2.execute(SQL2,row)
In SQL Server Management Studio I can run this query:
select a.foo, colX, colY, colZ from tblA a
join tblB b
on (a.foo = b.foo)
Running the same query through pyodbc in python script, I get:
pyodbc.ProgrammingError: ('42000', '[42000] [Microsoft][ODBC SQL Server Driver][
SQL Server]"b" is not a recognized table hints option. If it is intended as a p
arameter to a table-valued function or to the CHANGETABLE function, ensure that
your database compatibility mode is set to 90. (321) (SQLExecDirectW)')
Can anyone explain why this query works directly against SQL Server but there seems to be a problem with the table alias when using pyodbc?
This is what I've been able to find out:
SQL Server running on Microsoft Windows NT 6.1 (7601)
SQL Server version is 11.0.5532.0
pyodbc library: pyodbc-3.0.7.win32-py2.7
1) I do not know how to produce a reproducible example and I'm not a DBA; I can just about get by in SQL. 2) I cannot create tables on this database. However: I can do the full query with 3 joins / 3 aliases works fine using the RODBC r package and the sqlQuery() call against the same db. Therefore the culprit is pyodbc so I will try to find a python dev to try reproduce and post back the findings.
UPDATE + FIX
The issue is caused by concatenation errors:
Got python developer help to find and print raw sql before it was executed in python:
select a.foo, colX, colY, colZ from tblA a join tblB bon (a.foo = b.foo)
...and similar issue at each join, caused by missing whitespace within the sql string:
sql = "select a.foo, colX, colY, colZ from tblA a "
sql += "join tblB" #no whitespace at end of string
sql += "on (a.foo = b.foo)" #runs into next segment.
Worked in r with RODBC because string in r built like:
sql = "
...full query...
"
...with no opportunity for concatenation errors.