I've written a bit of python code that essentially will take data from one database (SQL Server 2008) and insert it into another (MySQL). I am fairly new to python so am struggling to find the errors in my code.
My code is:
import mysql.connector
import pyodbc
def insert_VPS(SageResult):
query = """
INSERT INTO SOPOrderReturn(SOPOrderReturnID,DocumentTypeID,DocumentNo,DocumentDate,CustomerID,CustomerTypeID,CurrencyID,SubtotalGoodsValue,TotalNetValue,TotalTaxValue,TotalGrossValue,SourceTypeID,SourceDocumentNo)
VALUES(%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s,%s)"""
try:
mydbVPS = mysql.connector.connect(
host="serveraddress",
user="username",
passwd="password;",
database="databse"
)
VPScursor = mydbVPS.cursor()
print(SageResult)
VPScursor.executemany(query, SageResult)
mydbVPS.commit()
except Exception as e:
print('InsertError:', e)
finally:
VPScursor.close()
mydbVPS.close()
def main():
selectQuery = """
SELECT TOP 1 [SOPOrderReturnID]
,[DocumentTypeID]
,[DocumentNo]
,[DocumentDate]
,[CustomerID]
,[CustomerTypeID]
,[CurrencyID]
,[SubtotalGoodsValue]
,[TotalNetValue]
,[TotalTaxValue]
,[TotalGrossValue]
,[SourceTypeID]
,[SourceDocumentNo]
FROM [Live].[dbo].[SOPOrderReturn]
"""
try:
mydbSage = pyodbc.connect('Driver={SQL Server};'
'Server=CRMTEST;'
'Database=Live;'
'UID=sa;'
'PWD=password;')
Sagecursor = mydbSage.cursor()
Sagecursor.execute(selectQuery)
SageResult = tuple(Sagecursor.fetchall())
mydbSage.commit()
except Exception as e:
print('MainError:', e)
finally:
Sagecursor.close()
mydbSage.close()
insert_VPS(SageResult)
if __name__ == '__main__':
main()
The error I get:
D:\xampp\htdocs\stripe\group\beta>sql-sync.py
((10447177, 0, '0000091897', datetime.datetime(2010, 8, 18, 0, 0), 186150, 1, 1, Decimal('18896.95'), Decimal('18896.95'), Decimal('3779.39'), Decimal('22676.34
'), 0, ''),)
InsertError: Failed executing the operation; Could not process parameters
I have tested the select query (but not the INSERT one) and both connections in a more basic script and those all work fine. Can anyone see the issues?
That should have been except Exception as e instead of except Error as e:
Consider workarounds as the special datetime and decimal types may not translate effectively from pyodbc to mysql connector DB-APIs.
CSV
Use the popular form of data transfer using MySQL's fast LOAD DATA method.
import csv
...
# SQL SERVER CSV EXPORT
mydbSage = pyodbc.connect('...')
Sagecursor = mydbSage.cursor()
Sagecursor.execute(selectQuery)
SageResult = Sagecursor.fetchall()
with open("/path/to/SageResult.csv", "w", newline='') as csv_file:
cw = csv.writer(csv_file)
cw.writerow([i[0] for i in Sagecursor.description]) # WRITE HEADERS
cw.writerows(SageResult) # WRITE DATA ROWS
# MYSQL CSV IMPORT
mydbVPS = mysql.connector.connect(...)
query = """LOAD DATA LOCAL INFILE '/path/to/SageResult.csv'
INTO TABLE SOPOrderReturn
FIELDS TERMINATED BY
ENCLOSED BY '"'
LINES TERMINATED BY '\r\n'
"""
VPScursor = mydbVPS.cursor()
VPScursor.execute(query)
PyODBC
Run both database connections with same API which requires downloading the MySQL ODBC driver for your OS and replace the mysql.connector). This may resolve handling of those specific types.
# SQL SERVER SELECT QUERY
mydbSage = pyodbc.connect(driver="SQL Server", host="CRMTEST", database="LIVE",
uid="sa", pwd="password")
Sagecursor = mydbSage.cursor()
Sagecursor.execute(selectQuery)
SageResult = tuple(Sagecursor.fetchall())
# MYSQL APPEND QUERY
mydbVPS = pyodbc.connect(driver="ODBC Driver Name", host="hostname",
uid="username", pwd="password", database="database")
query = """INSERT INTO SOPOrderReturn (SOPOrderReturnID, DocumentTypeID, DocumentNo,
DocumentDate, CustomerID, CustomerTypeID,
CurrencyID, SubtotalGoodsValue, TotalNetValue,
TotalTaxValue, TotalGrossValue, SourceTypeID,
SourceDocumentNo)
VALUES(?, ?, ?, ?, ?, ?, ?, ?, ?, , ?, ?, ?)
"""
VPScursor = mydbVPS.cursor()
VPScursor.execute(query, SageResult)
MS Access
Use the Office app as a medium between both relational databases. Since you use SQL Server you may have Microsoft Office available with a possibility of its DBMS Office app: MS Access.
Technically, MS Access is like phpMyAdmin, a GUI console to a database (where app is often conflated with its engine), except Access is not restricted to any one database but can supplement its default database, Jet/ACE SQL engine, with any known backend data and database source.
Create two linked tables (using ODBC connections) to the separate RDBMS's.
Create and then run INSERT...SELECT query on linked tables. This query will use Access' SQL dialect which supports TOP clause and bracketed names. Results should propagate immediately in MySQL table.
INSERT INTO SOPOrderReturn_mysql_linked
SELECT TOP 50 [SOPOrderReturnID]
,[DocumentTypeID]
,[DocumentNo]
,[DocumentDate]
,[CustomerID]
,[CustomerTypeID]
,[CurrencyID]
,[SubtotalGoodsValue]
,[TotalNetValue]
,[TotalTaxValue]
,[TotalGrossValue]
,[SourceTypeID]
,[SourceDocumentNo]
FROM SOPOrderReturn_mssql_linked
In fact, have Python run above query after linking tables in a saved database. Likely the MS Access ODBC driver may already be installed with the Office app or alternatively can be installed with downloaded redistributable.
# LIST OF INSTALLED DRIVERS
print(pydbc.drivers())
# MS ACCESS APPEND QUERY
constr = r"DRIVER={Microsoft Access Driver (*.mdb, *.accdb)};DBQ={'C:\\Path\\To\\Database\\File.accdb'};"
accdb = pyodbc.connect(constr)
cur = accdb.cursor()
cur.execute('<APPEND QUERY USING LINKED TABLES>')
I'm currently trying to query a deltadna database. Their Direct SQL Access guide states that any PostgreSQL ODBC compliant tools should be able to connect without issue. Using the guide, I set up an ODBC data source in windows
I have tried adding Set nocount on, changed various formats for the connection string, changed the table name to be (account).(system).(tablename), all to no avail. The simple query works in Excel and I have cross referenced with how Excel formats everything as well, so it is all the more strange that I get the no query problem.
import pyodbc
conn_str = 'DSN=name'
query1 = 'select eventName from table_name limit 5'
conn = pyodbc.connect(conn_str)
conn.setdecoding(pyodbc.SQL_CHAR,encoding='utf-8')
query1_cursor = conn.cursor().execute(query1)
row = query1_cursor.fetchone()
print(row)
Result is ProgrammingError: No results. Previous SQL was not a query.
Try it like this:
import pyodbc
conn_str = 'DSN=name'
query1 = 'select eventName from table_name limit 5'
conn = pyodbc.connect(conn_str)
conn.setdecoding(pyodbc.SQL_CHAR,encoding='utf-8')
query1_cursor = conn.cursor()
query1_cursor.execute(query1)
row = query1_cursor.fetchone()
print(row)
You can't do the cursor declaration and execution in the same row. Since then your query1_cursor variable will point to a cursor object which hasn't executed any query.
I am new in python and I need this query to check my database if it is working. There are lots of queries but it is only one of them.
import pyodbc
db_file = 'C:\\Users\\****\\Desktop\\bbbb.mdb' #define the location of your Access file
odbc_conn_str = 'DRIVER={Microsoft Access Driver (*.mdb)};DBQ=%s' %(db_file) # define the odbc connection parameter
conn = pyodbc.connect(odbc_conn_str)
cursor = conn.cursor() # create a cursor
sql_update_statement = "UPDATE NUMARATAJ SET KAPINUMERIK = IIf (ISNUMERIC (LEFT (REPLACE(KAPINO,'-','/'),4)=-1),LEFT(KAPINO,4))" # edit the SQL statement that you want to execute
cursor.execute(sql_update_statement) # execute the SQL statement
cursor.commit()
cursor.close()
conn.close()
When I try to run this code it says;
File "C:\Users\****\Desktop\aa2a.py", line 9, in <module>
cursor.execute(sql_update_statement) # execute the SQL statement
pyodbc.ProgrammingError: ('42000', "[42000] [Microsoft][ODBC Microsoft Access Sürücüsü] Undefined 'REPLACE' function in the expression. (-3102) (SQLExecDirectW)")
How can I fix it, or get it work, can you help me?
It could be:
sql_update_statement = "UPDATE NUMARATAJ SET KAPINUMERIK = LEFT(KAPINO,4) WHERE ISNUMERIC(LEFT(REPLACE(KAPINO,'-','/'),4)"
However, Replace is a VBA function, not Access SQL, so that is probably why you receive the error.
Try simply:
sql_update_statement = "UPDATE NUMARATAJ SET KAPINUMERIK = LEFT(KAPINO,4) WHERE ISNUMERIC(LEFT(KAPINO),4)"
There are several "VBA functions" that were not directly supported by the older "Jet" ODBC driver (Microsoft Access Driver (*.mdb)) but are directly supported by the newer "ACE" ODBC driver (Microsoft Access Driver (*.mdb, *.accdb)). Replace is one of those functions.
So if you really need to use the Replace function you can download and install the 32-bit version of the Access Database Engine Redistributable and use the newer ODBC driver.
So, after coding with pyodbc for a couple days now, I've run into a road block it seems. My SQL update will not work, even after putting autocommit=True in the connection statement. Nothing changes in the database at all. All my code is provided below. Please help. (I am using the 2016 version of MS Access, code runs with no errors, 32 bit Python and Access.)
import pyodbc
# Connect to the Microsoft Access Database
conn_str = (
r'DRIVER={Microsoft Access Driver (*.mdb, *.accdb)};'
r'DBQ=C:\Users\User_Name\Desktop\Databse\CPLM.accdb'
)
cnxn = pyodbc.connect(conn_str, autocommit=True)
crsr = cnxn.cursor()
crsr2 = cnxn.cursor()
# SQL code used for the for statement
SQL = "SELECT NameProject, Type, Date, Amount, ID FROM InvoiceData WHERE Type=? OR Type=? OR Type IS NULL AND ID > ?"
# Defining variables
date = ""
projectNumber = 12.04
numberDate = []
# Main Code, for each row in the SQL query, update the table
for row in crsr.execute(SQL, "Invoice", "Deposit", "1"):
print (projectNumber)
if row.NameProject is not None:
crsr2.execute("UPDATE Cimt SET LastInvoice='%s' WHERE Num='%s'" % (date, projectNumber))
cnxn.commit()
# Just used to find where to input certain data.
# I also know all the code in this if statement completes due to outside testing
projectNumber = row.NameProject[:5]
numberDate.append([projectNumber, date])
else:
date = row.Date
print(numberDate)
crsr.commit()
cnxn.commit()
cnxn.close()
I am saving MS Access tables as CSV files using Python. There is a table in the MS Access database that is named 'Perm_Site Info'. There is a space in the naming in MS Access. When I run the below snippet, the code blows up. I have tried having single and as well as double quotes in the cursor.execute but no fruition. I request your kind assistance in order to understand how to fix this.
import pyodbc
import csv
conn_string = ("DRIVER={Microsoft Access Driver (*.mdb, *.accdb)};DBQ=C:\\Access\\permissions.accdb")
conn = pyodbc.connect(conn_string)
cursor = conn.cursor()
cursor.execute("select * from Perm_Site Info;")
with open('C:\\Desktop\\Python Files\\Perms_Site_Info.csv','wb') as csvfile:
writer = csv.writer(csvfile)
writer.writerow([i[0] for i in cursor.description])
writer.writerows(cursor)
cursor.close()
conn.close()
print 'All done for now'
The error:
cursor.execute("select * from Perm_Site Info;")
ProgrammingError: ('42S02', "[42S02] [Microsoft][ODBC Microsoft Access Driver] The Microsoft Access database engine cannot find the input table or query 'Perm_Site'. Make sure it exists and that its name is spelled correctly. (-1305) (SQLExecDirectW)")
Try using brackets around the entire table name. It's barking because it doesn't know what to do with the space.
cursor.execute("select * from [Perm_Site Info];")