I try to store pickled numpy array in SQL Server as a VARBINARY(MAX) object using pyodbc. INSERT statement from SQL Server looks like this:
INSERT INTO [dbo].[Images]
([UserId]
,[FileName]
,[FeaturesVector])
VALUES
(<UserId, int,>
,<FileName, nchar(100),>
,<FeaturesVector, varbinary(max),>)
In my python code I build query as a fstring:
query = f"INSERT INTO Images(UserID, FileName, FeaturesVector) \
VALUES ('{user_id}', '{file_name}', '{features_vector}')"
When I try to insert object to database:
features_vector = np.arange(1)
features_vector.astype('float32')
features_vector = pickle.dumps(features_vector)
query = f"INSERT INTO Images(UserID, FileName, FeaturesVector) \
VALUES ('{user_id}', '{file_name}', '{features_vector}')"
cnxn.execute(query)
cnxn.commit()
I get an error:
('42000', "[42000] [Microsoft][ODBC Driver 17 for SQL Server][SQL Server]Incorrect syntax near '\'. (102) (SQLExecDirectW)")
Reliable Insert Method With pyodbc - Why?
#quarkpol, I feel your pain! I ran into this a while back, and I don't appreciate why I had to do it the way that I had to do it, but I had to do it this way to get it to work, and I noticed others doing this too.
First, when you do something like the following on reading data, the following ...
query_string = f'SELECT {col_name_1}, {col_name_2} FROM {some_table} WHERE {other_col} = {some_val}
cursor.execute(query_string)
# do some stuff with what the cursor got
the python f'stuff' type statements work great!
When INSERT'ing however, I have found that I must do the following for it to work ...
command_string = f'''INSERT INTO My_Table
(Col_Name_1, Col_Name_2, Col_Name_3) VALUES (?, ?, ?);'''
cursor.execute(command_string, val_1, val_2, val_3)
conn.commit()
Again, I don't know why, but at least this works.
Related
I am trying to insert data into ms access
I successfully insert some rows and suddenly I get an error. After the error, I can't INSERT more rows.
An example:
import pyodbc
cnxn = pyodbc.connect(your_driver_and_path_to_ms_access)
crsr = cnxn.cursor()
query1 = '''CREATE TABLE tbl01 (Id INT, name VARCHAR(25)) '''
crsr.execute(query1)
cnxn.commit()
query2 = ''' INSERT INTO tbl01 (Id,name) VALUES(?,?) '''
for i in range(5000):
param = [i] + ['test']
crsr.execute(query2, param)
cnxn.commit()
crsr.close()
Error: ('HY000', "[HY000] [Microsoft][ODBC Microsoft Access Driver] Cannot open database '|'. It may not be a database that your application recognizes, or the file may be corrupt. (-1206) (SQLExecDirectW)")
Inserting up 2000 most of the time works fine.
And also reading the table after the Error works.
query3 = ''' select * from tbl01 '''
crsr.execute(query3)
crsr.fetchone()
Thanks in advance
I'm currently trying to query a deltadna database. Their Direct SQL Access guide states that any PostgreSQL ODBC compliant tools should be able to connect without issue. Using the guide, I set up an ODBC data source in windows
I have tried adding Set nocount on, changed various formats for the connection string, changed the table name to be (account).(system).(tablename), all to no avail. The simple query works in Excel and I have cross referenced with how Excel formats everything as well, so it is all the more strange that I get the no query problem.
import pyodbc
conn_str = 'DSN=name'
query1 = 'select eventName from table_name limit 5'
conn = pyodbc.connect(conn_str)
conn.setdecoding(pyodbc.SQL_CHAR,encoding='utf-8')
query1_cursor = conn.cursor().execute(query1)
row = query1_cursor.fetchone()
print(row)
Result is ProgrammingError: No results. Previous SQL was not a query.
Try it like this:
import pyodbc
conn_str = 'DSN=name'
query1 = 'select eventName from table_name limit 5'
conn = pyodbc.connect(conn_str)
conn.setdecoding(pyodbc.SQL_CHAR,encoding='utf-8')
query1_cursor = conn.cursor()
query1_cursor.execute(query1)
row = query1_cursor.fetchone()
print(row)
You can't do the cursor declaration and execution in the same row. Since then your query1_cursor variable will point to a cursor object which hasn't executed any query.
I am saving MS Access tables as CSV files using Python. There is a table in the MS Access database that is named 'Perm_Site Info'. There is a space in the naming in MS Access. When I run the below snippet, the code blows up. I have tried having single and as well as double quotes in the cursor.execute but no fruition. I request your kind assistance in order to understand how to fix this.
import pyodbc
import csv
conn_string = ("DRIVER={Microsoft Access Driver (*.mdb, *.accdb)};DBQ=C:\\Access\\permissions.accdb")
conn = pyodbc.connect(conn_string)
cursor = conn.cursor()
cursor.execute("select * from Perm_Site Info;")
with open('C:\\Desktop\\Python Files\\Perms_Site_Info.csv','wb') as csvfile:
writer = csv.writer(csvfile)
writer.writerow([i[0] for i in cursor.description])
writer.writerows(cursor)
cursor.close()
conn.close()
print 'All done for now'
The error:
cursor.execute("select * from Perm_Site Info;")
ProgrammingError: ('42S02', "[42S02] [Microsoft][ODBC Microsoft Access Driver] The Microsoft Access database engine cannot find the input table or query 'Perm_Site'. Make sure it exists and that its name is spelled correctly. (-1305) (SQLExecDirectW)")
Try using brackets around the entire table name. It's barking because it doesn't know what to do with the space.
cursor.execute("select * from [Perm_Site Info];")
I have data in MySQL table which I want to copy to a PostgreSQL table.
Everything works except when the MySQL contains a string with " and/or '
For example:
The data in MySQL:
When I run my code I get:
ProgrammingError: ERROR: syntax error at or near "t"
(the t of the can't)
This is my code:
postgre = pg.connect(dbname=DB,user=USR,passwd=PASSWD,host=HOST, port=PORT)
crs = db_remote.cursor(MySQLdb.cursors.DictCursor)
crs.execute ("""select post_id, post_excerpt from tabl""")
data = crs.fetchall ()
for row in data :
postgre.query("""INSERT INTO importfrommysql(id,message)
VALUES ('%s','%s')"""%(row["post_id"],row["post_excerpt"]))
the connection pg.connect is from PygreSQL package.
What can I do? Is it possible to get the text as it is? or the only solution is to drop all " / ' before the insert?
Use the Psycopg cursor.execute parameter passing:
import psycopg2
conn = psycopg2.connect(database='DB')
cursor = conn.cursor()
for row in data :
cursor.execute ("""
INSERT INTO importfrommysql (id,message)
VALUES (%s,%s)
""",
(row["post_id"],row["post_excerpt"])
)
It will escape and quote as necessary.
So i am trying to call the mysql function TIMEDATEDIFF with the SECOND constant as the first parameter like so
query = session.query(func.TIME(Log.IncidentDetection).label('detection'), func.TIMESTAMPDIFF(SECOND,Log.IncidentDetection, Log.IncidentClear).label("duration")).all()
print(query)
I have tried it as a string and I get a mysql/mariadb error:
query = session.query(func.TIME(Log.IncidentDetection).label('detection'), func.TIMESTAMPDIFF("SECOND",Log.IncidentDetection, Log.IncidentClear).label("duration")).all()
print(query)
Gives me this
sqlalchemy.exc.ProgrammingError: (mysql.connector.errors.ProgrammingError) 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near ''SECOND', log.`IncidentDetection`, log.`IncidentClear`) AS duration
FROM log' at line 1 [SQL: 'SELECT TIME(log.`IncidentDetection`) AS detection, TIMESTAMPDIFF(%(TIMESTAMPDIFF_1)s, log.`IncidentDetection`, log.`IncidentClear`) AS duration \nFROM log'] [parameters: {'TIMESTAMPDIFF_1': 'SECOND'}]
I am sure it is something simple, some sort of escape sequence or import that I am missing. I have looked through the sqlalchemy documentation to no avail.
To get sqlalchemy to parse the string exactly into the query I used the _literal_as_text() function
Working solution
from sqlalchemy.sql.expression import func, _literal_as_text
# ...
query = session.query(func.TIME(Log.IncidentDetection).label('detection'), func.TIMESTAMPDIFF(_literal_as_text("SECOND"),Log.IncidentDetection, Log.IncidentClear).label("duration")).all()
print(query)