I wrote a fairly simple SQL while loop and tried to submit it via pyodbc cursor. But it didn't work, while working perfectly fine in SQL Server Management Studio.
My understanding is that one cannot pass more than one statement with the cursor. But then how does one execute a SQL while loop? I know I can do the below query with the while loop inside the python by cursor.rowcount, but my question is about generic queries with various SQL functions (like while here).
conn = get_output_conn(env=ENVIRONMENT)
conn.autocommit=True
cursor = conn.cursor()
query = """WHILE 1 = 1
BEGIN
BEGIN TRANSACTION;
DELETE TOP(2000)
FROM table with(holdlock)
WHERE ReportDate = '2020-08-23';
IF ##ROWCOUNT < 1 BREAK;
COMMIT TRANSACTION;
END"""
cursor.execute(query)
cursor.commit()
Try testing your rowcount condition after the commit transaction; statement. The following works for me...
import pyodbc
conn = pyodbc.connect(
autoCommit=False,
driver="/usr/local/lib/libtdsodbc.so",
tds_version="7.4",
database="StackOverflow",
port=...,
server="...",
user="...",
password="..."
)
query1 = """drop table if exists dbo.DeleteExample;"""
cursor1 = conn.cursor()
cursor1.execute(query1)
cursor1.commit()
cursor1.close()
query2 = """
select cast('2020-08-23' as date) as ReportDate
into dbo.DeleteExample
from sys.objects a, sys.objects b"""
cursor2 = conn.cursor()
cursor2.execute(query2)
# About 10,000 rows depending on your database
print(cursor2.rowcount, "rows inserted")
cursor2.commit()
cursor2.close()
query3 = """
declare #RowCount int;
while 1=1
begin
begin transaction t1;
delete top (2000)
from dbo.DeleteExample
where ReportDate = '2020-08-23';
set #RowCount = ##RowCount;
commit transaction t1;
if #RowCount < 1 break;
end"""
cursor3 = conn.cursor()
cursor3.execute(query3)
# "2000" which only is the first rowcount...
print(cursor3.rowcount, "rows deleted")
cursor3.commit()
cursor3.close()
Which outputs...
% python ./example.py
(10609, 'rows inserted')
(2000, 'rows deleted')
Executing select count(1) from StackOverflow.dbo.DeleteExample in SSMS returns a count of 0.
Related
import mysql.connector
mydb = mysql.connector.connect(
host = "localhost",
user = "root",
password = "Thsmine2",
database = "_universidad_"
)
cursorrecup = mydb.cursor()
cursorrecup.execute("UPDATE _universidad_.copias SET id_investigadores =8 WHERE id = 10; SELECT * FROM _universidad_.copias")
mydb.commit()
resultadorecup = cursorrecup.fetchall()
for x in resultadorecup:
print(x[0],x[1],x[2],x[3])
I have reduced my problem to these lines, however, there seems to be a problem with the function . I have revised all of my variables and references to the database and they are correct.
You need to iterate over the cursor as it is being executed, rather than over the result. This code works for me:
cur.execute('DROP TABLE IF EXISTS t73955432')
cur.execute('CREATE TABLE t73955432 (a float)')
conn.commit()
cur.executemany('INSERT INTO t73955432 (a) VALUES (%s)', [(1,), (2,), (3,)])
conn.commit()
stmts = 'UPDATE t73955432 set a = RAND(); SELECT a FROM t73955432;'
for result in cur.execute(stmts, multi=True):
print(result.statement)
if result.with_rows:
print(result.fetchall())
else:
print(f'{result.rowcount} rows affected')
conn.commit()
conn.close()
Producing output like
UPDATE t73955432 set a = RAND()
3 rows affected
SELECT a FROM t73955432
[(0.557543,), (0.378451,), (0.21963,)]
I am struggling to establish a connection inside data iteration. Means I am running a select query to postgres and iterating the return data. after some transformation I am writing it to another table. But it is not working. Sample python code is below.
conn = pgconn(------)
cursor = pgconn.Cursor()
query1 = "select * from table"
query2 = "select * from table2 where Id=(%s);"
cursor.execute(query1)
result = query1.fetchall()
for row in result:
If row.a == 2:
cursor.execute(query2, [row.time])
In the above python code I can't able to extract the data by running query2 and passing query1 result as a parameter. It seems cursor is blocked by the query1 so query2 execution is not happening. Please some one help in this issue.
First of all you can write a join statement to do this and can get the data easily
select * from table join table2 where table2.id == table.time
Also why this is not working maybe because the cursor object is getting override inside the for loop and thus the query results get changed.
Use RealDictCursor, and correct the syntax on your inside call to execute():
import psycopg2
import psycopg2.extras
conn = pgconn(------)
cursor = conn.cursor(cursor_factory=psycopg2.extras.RealDictCursor)
query1 = "select * from table"
query2 = "select * from table2 where Id=(%s);"
cursor.execute(query1)
result = query1.fetchall()
for row in result:
If row.a == 2:
cursor.execute(query2, (row['time'],))
1. install psycopg2 and psycopg2.extras. ( pip install)
Then set up your Postgres Connection like:
def Postgres_init(self):
try:
conn = psycopg2.connect(host=os.environ['SD_POSTGRES_SERVER'],
user=os.environ['SD_POSTGRES_USER'],
password=os.environ['SD_POSTGRES_PASSWORD'],
port=os.environ['SD_POSTGRES_PORT'],
database=os.environ['SD_POSTGRES_DATABASE'])
logging.info("Connected to PostgreSQL")
except (Exception, psycopg2.Error) as error:
logging.info(error)
2. Connect your Cursor with the defined connection
cursor = conn.cursor()
3. Execute your query:
cursor.execute("""SELECT COUNT (column1) from tablename WHERE column2 =%s""", (
Value,)) # Check if already exists
result = cursor.fetchone()
Now the value is stored in the "result" variable. Now you can execute the next query like:
cursor.execute("""
INSERT INTO tablename2
(column1, column2, column3)
VALUES
(%s, %s, %s)
ON CONFLICT(column1) DO UPDATE
SET
column2=excluded.column2,
column3=excluded.column3;
""", (result, column2, column3)
)
Now the result of query 1 is stored in the second table in the first column.
Now you can close your connection:
conn.close()
I have a problem getting the query results from my Python-Code. The connection to the database seems to work, but i always get the error:
"InterfaceError: No result set to fetch from."
Can somebody help me with my problem? Thank you!!!
cnx = mysql.connector.connect(
host="127.0.0.1" ,
user="root" ,
passwd="*****",
db="testdb"
)
cursor = cnx.cursor()
query = ("Select * from employee ;")
cursor.execute(query)
row = cursor.fetchall()
If your problem is still not solved, you can consider replacing the python mysql driver package and use pymysql.
You can write code like this
#!/usr/bin/python
import pymysql
db = pymysql.connect(host="localhost", # your host, usually localhost
user="test", # your username
passwd="test", # your password
db="test") # name of the data base
# you must create a Cursor object. It will let
# you execute all the queries you need
cur = db.cursor()
query = ("SELECT * FROM employee")
# Use all the SQL you like
cur.execute(query)
# print all the first cell of all the rows
for row in cur.fetchall():
print(row[0])
db.close()
This should be able to find the result you want
add this to your code
for i in row:
print(i)
you did not print anything which is why that's not working
this will print each row in separate line
first try to print(row),if it fails try to execute using the for the loop,remove the semicolon in the select query statement
cursor = connection.cursor()
rows = cursor.execute('SELECT * FROM [DBname].[dbo].TableName where update_status is null ').fetchall()
for row in rows:
ds = row[0]
state = row[1]
here row[0] represent the first columnname in the database
& row[1] represent the second columnname in the database & so on
I query a table then loop through it to Update another table.
The console Prints shows correct data.
Not sure how to debug the cursor.execute for the UPDATE query.
It is not updating on the table. It's not a permission issue. If I run update command on my SQL workbench it works fine.
cursor = conn.cursor()
cursor.execute("Select Account_Name FROM dsf_CS_WebAppView")
for row in cursor.fetchall():
try:
cursor.execute("Select fullpath FROM customerdesignmap WHERE
fullpath LIKE '%{}%'".format(row.Account_Name))
rows = cursor.fetchall()
print(len(cursor.fetchall()))
if len(rows) > 0:
for rowb in rows:
print(rowb.fullpath)
print(row.Account_Name)
if len(row.Account_Name) > 2:
cursor.execute("UPDATE customerdesignmap SET householdname = {}, msid = {} WHERE fullpath LIKE '{}'".format(row.Account_Name, row.UniqueProjectNumber, rowb.fullpath))
conn.commit()
except:
pass
Consider a pure SQL solution as SQL Server supports UPDATE and JOIN across multiple tables. This avoids the nested loops, cursor calls, and string formatting of SQL commands.
UPDATE m
SET m.householdname = v.Account_Name,
m.msid = v.UniqueProjectNumber
FROM customerdesignmap m
JOIN dsf_CS_WebAppView v
ON m.fullpath LIKE CONCAT('%', v.Account_Name, '%')
In Python, run above in a single cursor.execute() with commit() call.
cursor.execute('''my SQL Query''')
conn.commit()
I'm migrating a script from another language to Python. I watered this down on the specifics of the database calls etc... but this is what the file looks like. I intentionally made some queries fail as I was testing the transaction and it did not rollback() the queries executed prior to the forced error. I am a little confused as how to the transactions work with Python, the example I followed was this one, it was a loop with several queries nested within transactions so I adapted the code according to what I understood from it.
#!/usr/bin/python
import MySQLdb
import thread
import os
# Open database connection
# added local_infile=1 to allow the import to work, otherwise you get an error
db = MySQLdb.connect(CONNECTION ARGS...)
# define our function that will be called from our thread
def import_queued_file(conn,distid):
# prepare a cursor object using cursor() method
cursor = conn.cursor()
# total lines imported for all files for a distributor
total_lines_imported = 0
# current lines imported for each file on each iteration
current_lines_imported = 0
# default this to 0, this will have the total lines for our imports on each iteration
previous_lines_imported = 0
# initialize the file exists flag to 0
file_exists = 0
# sql statement to retrieve the file(s) for a specific distributor
sql = """
SELECT
...
FROM ...
WHERE ...
"""
# execute the sql statement
cursor.execute(sql)
# if we have records, execute the code below
if (cursor.rowcount > 0):
# set the records to the files variable
files = cursor.fetchall()
# set a variable to count iterations
# we'll use this to determine if we need to drop the table
cnt = 0
# keep track of the total number of lines imported per distributor (may be multiple files)
lines_imported = 0
# loop the recordset
for col in files:
# increment the cnt variable
cnt += 1
# set file_exists to 0 at the beginning of the iteration
file_exists = 0
# set some variables to be used in our sql load data statement
var1 = col[1]
var2 = col[2]
....
# this is the path of our file that we will be using for MySQL LOAD DATA also
# TODO: REFACTOR SO THAT THE /home/webex/backup/ IS NOT HARD CODED
inventoryfile = "/path/to/file/%s" % (filepath)
# check to see if we have a file
if (os.path.exists(inventoryfile)):
try:
# set file exists to true
file_exists = 1
# if cnt > 1, it means we have more than 1 file for this distributor
# only drop the table if this is the first iteration
if (cnt == 1):
# drop table sql statement
sql = "DROP TABLE IF EXISTS %s" % (temptable)
# execute the sql command
cur = conn.cursor()
cur.execute(sql)
cur.close()
# assign the create table statement to the sql variable
sql = """
CREATE TABLE IF NOT EXISTS
.......
.......
) ENGINE=MyISAM DEFAULT CHARSET=utf8
""" % (temptable)
# execute the sql statement
cur = conn.cursor()
cur.execute(sql)
cur.close()
# query the temptable to see if we have any records
sql = "SELECT COUNT(0) AS total FROM %s" % (temptable)
cur = conn.cursor()
cur.execute(sql)
cur.close()
# get the count of how many records exist in the database
number_of_line_items = cur.fetchall()
previous_lines_imported = number_of_line_items[0][0]
# load data local infile sql statement
sql = """
LOAD DATA LOCAL INFILE ...
"""
# execute the load data infile sql statement
cur = conn.cursor()
cur.execute(sql)
cur.close()
# clean up the table by removing...
# rows that don't have a part_number,
# rows that have part_number's less than 3 characters
sql = """
DELETE FROM ...
""" % (temptable)
# execute the delete query
cur = conn.cursor()
cur.execute(sql)
cur.close()
# query the temptable to see if we have any records after the import
sql = "SELECT COUNT(0) AS total FROM %s" % (temptable)
# execute the count query
cur = conn.cursor()
cur.execute(sql)
cur.close()
# get the count of how many records exist in the database after the import
number_of_line_items = cur.fetchall()
# get the current lines imported
current_lines_imported = number_of_line_items[0][0] - previous_lines_imported
# add the current lines imported to the total lines imported
total_lines_imported += current_lines_imported
# update distributor_file_settings table last_updated_on field
sql = """
UPDATE ...
""" % (file_id,distributor__id)
print sql
# execute the update query
cur = conn.cursor()
cur.execute(sql)
cur.close()
# close cursor
conn.commit()
except:
conn.rollback()
# no records exists for this distributor
else:
print "dist doesn't exist"
cursor.close()
import_queued_file(db,42)
# prepare a cursor object using cursor() method
cursor = db.cursor()
# select distinct file settings
sql = """
SELECT ...
"""
# disconnect from server
db.close()
After reviewing the code again and again, the issue happened to be the table type. After changing it to INNODB it worked as expected.