I am currently developing a program in python that interacts with multiple database. I am using pyodbc to connect, and execute queries. One of the database is an azure database. I noticed sometimes the sent data is not updated in the database although the program run successfully and no error was thrown. Is there any practices that i should follow to make sure this doesn't happen or is this related to my code or db connection issue? I am a beginner. Would appreciate everyone's help thank you!
Also is the .commit() line should be run after every sql run?
The program should be updating a row of data in the database based on a condition, this particular query sometimes doesn't take effect, but no error was thrown. I also executed multiple queries after that, no issue was found for the next queries. It is successfully executed.
the query is a simple query which is
UPDATE DraftVReg SET VRStatus = 'Potential Duplicate Found' WHERE RowID = ?
I tried to reproduce your scenario on my end and was able to update the SQL row in the Azure SQL DB with Pyodbc module.
Yes, Its very necessary to use
conn.commit
to commit your changes inside a database after you perform operations such as Update or Insert inside Azure SQL DB programmatically.
1) Fetch Data with Select statement.
I was able to fetch the Table’s data successfully with Select * from ‘Tablename’ query inside pyodbc code before I try UPDATE statement.
import pyodbc
conn = pyodbc.connect('DRIVER={ODBC Driver 17 for SQL Server};''SERVER=tcp:sqlservernamesql.database.windows.net,1433;''DATABASE=databasename; UID=siliconuser;PWD=Password;')
#conn.commit()
cursor = conn.cursor()
cursor.execute('Select * FROM StudentReviews')
#conn.commit()
for i in cursor:
print(i)
cursor.close()
conn.close()
Result:-
2) UPDATE the rows require conn.commit()
Code :-
import pyodbc
conn = pyodbc.connect('DRIVER={ODBC Driver 17 for SQL Server};''SERVER=tcp:siliconserversql.database.windows.net,1433;''DATABASE=silicondb; UID=userid; PWD=Password;')
cursor = conn.cursor()
#cursor.execute('Select * FROM StudentReviews')
cursor.execute("UPDATE StudentReviews SET ReviewTime = ('7') WHERE ReviewText = ('SQL DB')")
conn.commit()
cursor.close()
conn.close()
Result:-
Update statement Executed successfully and the Table Row was updated in Azure SQL, Refer Below :-
3) With autocommit=true
Thank you #Gord thompson for the comment and suggestion!
Code :-
import pyodbc
conn = pyodbc.connect('DRIVER={ODBC Driver 17 for SQL Server};''SERVER=tcp:siliconserversql.database.windows.net,1433;''DATABASE=silicondb; UID=username; PWD=Password;', autocommit=True)
#conn.commit()
cursor = conn.cursor()
cursor.execute("UPDATE StudentReviews SET ReviewTime = ('8') WHERE ReviewText = ('SQL DB')")
cursor.close()
conn.close()
Results :- With autocommit=true, You do not need to add conn.commit everytime you update the SQL DB.
Related
I want to connect to MySql database using Python through PythonAnywhere, without creating a Flask/Django application.
I have seemingly managed to connect through MySQLdb, using the code below, but I do not receive a response when I run the code. Any solutions?
import MySQLdb
db = MySQLdb.connect(
host = "myuser.mysql.pythonanywhere-services.com",
user = "myuser",
passwd = XXX,
db = "myuser$db_name"
)
cursor = db.cursor()
cursor.execute("SELECT * FROM table_name")
for x in cursor:
print(x)
cursor.close()
db.close()
You retrieve all rows in the table, without error.
cursor.execute("SELECT * FROM table_name")
for x in cursor:
print(x)
Yet you see no output. This is normal for a table that contains zero rows.
Consider doing one or more INSERTs, and a COMMIT,
prior to the query.
I have an issue with inserting data into a database using the python package pyodbc and since I am pretty new to pyodbc & databases in general, I might lack some basic understanding.
I open a connection, and then I want the execute my query.
Actually, in this query I call a stored procedure (which I didn't write and I am not allowed to change!).
This procedure does "one or two" inserts. When I use pyodbc like this
conn = pyodbc.connect(connection_string)
with conn:
c = conn.cursor()
c.execute("{call input_procedure('some','parameters','to','insert')}")
OR
conn = pyodbc.connect(connection_string)
c = conn.cursor()
c.execute("{call input_procedure('some','parameters','to','insert')}")
conn.commit()
I get the following error message:
pyodbc.Error: ('HY000', "[HY000] [MySQL][ODBC 8.0(a) Driver]Commands out of sync; you can't run this command now (2014) (SQLEndTran(SQL_COMMIT))")
As far as I understood, this error message might be due to executing more than one insert within the called procedure. When I print the return of the execute command I become the following: (' ', )
When I instead close the cursor, before doing the commit, everything works fine. Like this:
conn = pyodbc.connect(connection_string)
c = conn.cursor()
c.execute("{call input_procedure('some','parameters','to','insert')}")
c.close()
conn.commit()
I really don't understand what's happening here.
Is there an explanation for this behaviour? Is closing the cursor before doing the commit save?
Thanks a lot for your help!
You seem to have encountered a quirk in MySQL Connector/ODBC's handling of result sets from a stored procedure. For this example procedure:
CREATE DEFINER=`root`#`localhost` PROCEDURE `input_procedure`(IN `p1` VARCHAR(50))
MODIFIES SQL DATA
BEGIN
INSERT INTO table1 (txt) VALUES (p1);
SELECT '' AS foo;
END
this Python code
crsr = cnxn.cursor()
crsr.execute("{call input_procedure('thing')}")
cnxn.commit()
throws the "Commands out of sync" error, whereas this code
crsr = cnxn.cursor()
crsr.execute("{call input_procedure('thing')}")
while crsr.nextset():
pass
cnxn.commit()
does not.
I am trying to execute stored procedure by using pyodbc in databricks, after executing SP I tried to commit the connection but, commit is not happening. Here I am giving my code, please help me out from this issue.
import pyodbc
#### Connecting Azure SQL
def db_connection():
try:
username = "starsusername"
password = "password-db"
server = "server-name"
database_name = "db-name2"
port = "db-port"
conn=pyodbc.connect('Driver={ODBC Driver 17 for SQL server};SERVER=tcp:'+server+','+port+';DATABASE='+ database_name +';UID='+ username +';PWD='+ password)
cursor=conn.cursor()
return cursor, conn
except Exception as e:
print("Faild to Connect AZURE SQL: \n"+str(e))
cursor, conn = db_connection()
# conn1.autocommit=True
cursor.execute("delete from db.table_name")
cursor.execute("insert into db.table_name(BUSINESS_DATE) values('2021-10-02')")
cursor.execute("exec db.SP_NAME '20211023'")
conn.commit()
conn.close()
here I am commiting connection after SP excution. deletion and insertion is not happening at all. and I tried with cursor.execute("SET NOCOUNT ON; exec db.SP_NAME '20211023'") but it's also not working.
Thanks in Advance
If you check this document on pyodbc, you will find that -
To call a stored procedure right now, pass the call to the execute method using either a format your database recognizes or using the ODBC call escape format. The ODBC driver will then reformat the call for you to match the given database.
Note that after connection is set up or done, try doing conn.autocommit = True before calling your SP and it will help. By default it is false.
Executing the Stored Procedure.
You will be able to execute your stored procedure if you follow the below code snippet.
cursor = conn.cursor()
conn.autocommit = True
executesp = """EXEC yourstoredprocedure """
cursor.execute(executesp)
conn.commit()
Delete the Records in SQL Server
You can delete record as shown in the below example.
...#just an example
cursor.execute('''
DELETE FROM product
WHERE product_id in (5,6)
''')
conn.commit()
Don’t forget to add conn.commit() at the end of the code, to ensure that the command would get executed.
Insert record in SQL Server
The below snippet show how we can do the same.
...#just an example
cursor.execute("INSERT INTO EMP (EMPNO, ENAME, JOB, MGR) VALUES (535, 'Scott', 'Manager', 545)")
conn.commit()
I will suggest you to read the for following document for more information.
Delete Record Documentation.
Insert Record Document
I currently have code that executes queries on data stored on a SQL Server database, such as the following:
import pyodbc
conn = pyodbc.connect(
r'DRIVER={SQL Server};'
r'SERVER=SQL2SRVR;'
r'DATABASE=DBO732;'
r'Trusted_Connection=yes;'
)
sqlstr = '''
SELECT Company, Street_Address, City, State
FROM F556
WHERE [assume complicated criteria statement here]
'''
crsr = conn.cursor()
for row in crsr.execute(sqlstr):
print(row.Company, row.Street_Address, row.City, row.State)
I can't find documentation online of whether pyodbc can (or is by default) running my queries on the SQL Server (as passthrough queries), or whether (if pyodbc can't do that) there is another way (maybe sqlalchemy or similar?) of doing that. Any insight?
Or is there a way to execute passthrough queries directly from Pandas?
If you are working with pandas and SQL Server then you should already have created a SQLAlchemy Engine object (usually named engine). To execute a raw DML statement you can use the construct
with engine.begin() as conn:
conn.execute("UPDATE table_name SET column_name ...")
print("table updated")
I would like to run below postgresql function queries in python but I keep getting error message says
cursor = connection.cursor()
cursor.execute("SELECT
ST_Distance_sphere(st_makepoint(32.836956,39.925018)
,st_makepoint(28.990174,41.036857))")
df = cursor.fetchall()
df
InFailedSqlTransaction: current transaction is aborted, commands ignored until end of transaction block
How can I fix this? Thank you.
According to psycopg docs, you probably have an error in your command (SQL).
There was a problem in the previous command to the database, which
resulted in an error. The database will not recover automatically from
this condition: you must run a rollback() before sending new commands
to the session (if this seems too harsh, remember that PostgreSQL
supports nested transactions using the SAVEPOINT command).
I highly recommend using try/except/finally clause in your database connections. Or use with statement.
Here is an example from http://www.postgresqltutorial.com/postgresql-python/transaction/:
conn = psycopg2.connect(dsn)
# transaction 1
with conn:
with conn.cursor() as cur:
cur.execute(sql)
# transaction 2
with conn:
with conn.cursor() as cur:
cur.execute(sql)
conn.close()