Develop Reference

PYODBC Only returning values of the first query in a stored procedure

I am trying to make a script that will run a stored procedure and then return the values. A simple enough script. As I run the script it is only returning the output of the first query in my stored procedure. When the call is made I have the values being appended to a list. Once the list is created I have it printing onto a GUI that I have made. My stored procedure does work and returns the values for all the queries it contains when run on SSMS. My stored procedure is as follows (Sorry if these are poorly made stored procedures, I am fairly new to SQL):
It takes a parameter #username
Select username,user_id,user_type
from db1.users
where username like '%' + TRIM(#username) + '%'
ORDER BY username
Select username,user_id,user_type
from db2.users
where username like '%' + TRIM(#username) + '%'
ORDER BY username
Select username,user_id,user_type
from db3.users
where username like '%' + TRIM(#username) + '%'
ORDER BY username
Select username,user_id,user_type
from db4.users
where username like '%' + TRIM(#username) + '%'
ORDER BY username
Select username,user_id,user_type
from db5.users
where username like '%' + TRIM(#username) + '%'
ORDER BY username
The code that I am using to try and run this procedure and then append it to the list are as follows:
def user_query(user, conn, output_field):
global user
user_results = []
username = user.get()
cursor = conn.cursor()
get_user_stored_proc = "SET NOCOUNT ON; EXEC [dbo].[getUser] #username = ?"
output_field.config(state=NORMAL)
output_field.delete('1.0', END)
cursor.execute(get_user_stored_proc, username)
#TODO Fix this so that it will output everything from the query
columns = [column[0] for column in cursor.description]
for row in cursor:
user_results.append(dict(zip(columns, row)))
print_results(user_results, output_field)
#cursor.close()
#conn.close()
As previously mentioned, the only output I have returned to me when running this is the result of the first query. Any help is appreciated!

I learned that the following link does have the resources needed. Note that they use pandas and import it as pd. They also create a datafield as df but do not show the initial creation of this variable. Additionally, I did make some of my own changes; these are mainly preferential changes and do not really have an impact on the overall output. Thank you to #larnu for pointing this out. Here is my edited code.
def user_query(user, conn, output_field):
global user
user_results = []
username = user.get()
cursor = conn.cursor()
get_user_stored_proc = "EXEC [dbo].[getuser] #username = ?"
output_field.config(state=NORMAL)
output_field.delete('1.0', END)
rows = cursor.execute(get_user_stored_proc, username).fetchall()
columns = [column[0] for column in cursor.description]
user_results.append(pd.DataFrame.from_records(rows, columns=columns))
while True:
rows = cursor.fetchall()
columns = [column[0] for column in cursor.description]
user_results.append(pd.DataFrame.from_records(rows, columns=columns))
if cursor.nextset() == False:
break

SQLAlchemy Retrieving Different Columns

Here is my code:
import pandas as pd
from sqlalchemy import create_engine
db_username = "my_username"
db_pw = "my_password"
db_to_use = "my_database"
#####
engine = create_engine(
"postgresql://" +
db_username + ":" +
db_pw +
"#localhost:5432/" +
db_to_use
)
#####
connection = engine.connect()
fac_id_list = connection.execute (
"""
select distinct
a.name,
replace(regexp_replace(regexp_replace(a.logo_url,'.*/logo','','i'),'production.*','','i'),'/','') as new_logo
from
sync_locations as a
inner join
public.vw_locations as b
on
a.name = b.location_name
order by
new_logo
"""
)
I want to put the results of fac_id_list into two separate lists. One list will contain all of the values from a.name and the other new_logo.
How can I do this?
sql_results = []
for row in fac_id_list:
sql_results.append(row)
This puts every column in my SQL query into a list, but I want them separated.

When you loop over the results, you can spread them into separate variables and append them to the corresponding lists
names = []
logos = []
for name, logo in fac_id_list:
names.append(name)
logos.append(log)

Delete last month based on current month sqlalchemy

I am trying to delete rows in a sql table from python with sqlalchemy based on a condition. Here is the code I have so far
def delete_previous_month():
#creating connection object
connection = engine.connect()
meta = MetaData(engine).reflect()
user_t = meta.tables[f'{table_name}']
del_st = user_t.delete().where(
user_t.c.month == Current_Month - 1)
print('----- deleting previous records ')
connection.execute(del_st)
delete_previous_month()
I need to get the part of the code where it says:
user_t.c.month == Current_Month - 1)
to delete the rows like it would in sql if I did this:
DATEPART(m, month) = DATEPART(m, DATEADD(m, -1, getdate())
What is the equivalent sqlalchemy syntax for sql server datepart function?

Unable to create a second dataframe python pandas

My second data frame is not loading values when i create it. Any help with why it is not working? When i make my cursor a list, it has a bunch of values in it, but for whatever reason when i try to do a normal data frame load with pandas a second time, it does not work.
My code:
conn = pyodbc.connect(constr, autocommit=True)
cursor = conn.cursor()
secondCheckList = []
checkCount = 0
maxValue = 0
strsql = "SELECT * FROM CRMCSVFILE"
cursor = cursor.execute(strsql)
cols = []
SQLupdateNewIdField = "UPDATE CRMCSVFILE SET NEW_ID = ? WHERE Email_Address_Txt = ? OR TELEPHONE_NUM = ? OR DRIVER_LICENSE_NUM = ?"
for row in cursor.description:
cols.append(row[0])
df = pd.DataFrame.from_records(cursor)
df.columns = cols
newIdInt = 1
for row in range(len(df['Email_Address_Txt'])):
#run initial search to figure out the max number of records. Look for email, phone, and drivers license, names have a chance not to be unique
SQLrecordCheck = "SELECT * FROM CRMCSVFILE WHERE Email_Address_Txt = '" + str(df['Email_Address_Txt'][row]) + "' OR TELEPHONE_NUM = '" + str(df['Telephone_Num'][row]) + "' OR DRIVER_LICENSE_NUM = '" + str(df['Driver_License_Num'][row]) + "'"
## print(SQLrecordCheck)
cursor = cursor.execute(SQLrecordCheck)
## maxValue is indeed a list filled with records
maxValue =(list(cursor))
## THIS IS WHERE PROBLEM OCCURS
tempdf = pd.DataFrame.from_records(cursor)

Why not just use pd.read_sql_query("your_query", conn) this will return the result of the query as a dataframe and requires less code. Also you set cursor to cursor.execute(strsql) at the top and then you are trying to call execute on cursor again in your for loop but you can no longer call execute on cursor you will have to set cursor = conn.cursor() again.

Python script returns old MySQL values (until restart) [duplicate]

This question already has answers here:
Why are some mysql connections selecting old data the mysql database after a delete + insert?
(2 answers)
Closed 7 years ago.
I have the following code that I run once and then again when a new value is inserted to refresh.
def getStageAdditives(self, stage):
stagesAdditivesSelectQuery = """SELECT a.id,
a.name,
IFNULL(sa.dose, 0) as dose,
IFNULL(sa.last_dose, 0) as last
FROM additives a
LEFT JOIN stage_additives sa
ON a.id = sa.additive_id
AND sa.stage_id = (
SELECT id
FROM stages
WHERE name = '""" + stage + """')
ORDER BY a.name"""
self.cursor.execute(stagesAdditivesSelectQuery)
data = self.cursor.fetchall()
additives = []
for additive in data:
id = additive[0]
name = additive[1]
dose = additive[2]
last = additive[3]
additives.append({ 'title': name, 'dose': dose, 'last': last })
print stagesAdditivesSelectQuery
return additives
The issue is that after I use the following code to insert a value into 'additives' table I get old values (new value is missing).
def createAdditive(self, name):
additiveInsertQuery = """ INSERT INTO additives
SET name = '""" + name + """'"""
try:
self.cursor.execute(additiveInsertQuery)
self.db.commit()
return "True"
except:
self.db.rollback()
return "False"
I can confirm that values are being inserted into the database by looking at phpMyAdmin. If I restart the script I get the new values as expected. If I run the query with phpMyAdmin that also returns new values. Refreshing the page and waiting 10+ seconds doesn't help and I get old values still.
Both methods are in separate classes/files if it matters. The getStageAdditives is called with ajax after the 'createAdditive' method has returned successfully.
DB initialisation:
import MySQLdb
import time
class Stage:
def __init__(self):
self.db = MySQLdb.connect('192.168.0.100', 'user', 'pass', 'dbname')
self.cursor = self.db.cursor()
Another method that retrieves similar values gets the new values as expected (same class as createAdditives):
def getAdditives(self, additive=None):
where = ''
if additive is not None:
where = "WHERE pa.additive_id = ("
where += "SELECT id FROM additives "
where += "WHERE name = '" + additive + "') "
where += "AND a.name = '" + additive + "'"
additiveSelectQuery = """ SELECT a.name,
pa.pump_id
FROM additives a,
pump_additives pa """ + where + """
ORDER BY a.name"""
self.cursor.execute(additiveSelectQuery)
data = self.cursor.fetchall()
additives = []
for item in data:
additives.append( {'additive': item[0], 'pump': item[1]} )
return additives

For those who find this question: Similar question was solved in Why are some mysql connections selecting old data the mysql database after a delete + insert?
The trick is to add an connection.commit().