i was trying to follow this example from stackoverflow using pg_notify with python. I am not getting this to work. Nothing happens and python does not receive the notification.
Python 3.85 Using Psycopg2. PostgreSQL 13
python & postgresql: reliably check for updates in a specific table
First. I created a python function that listened to the postgreSQL. Then i went to pgadmin and
executed
select pg_notify('process', 'update');
My python function is below
def dblisten(connection):
cur = connection.cursor()
print("inside")
cur.execute("LISTEN process")
while True:
print("listening")
select.select([connection],[],[])
connection.poll()
events = []
while connection.notifies:
notify = connection.notifies.pop().payload
print ("Got NOTIFY:", datetime.datetime.now(), notify.pid, notify.channel, notify.payload)
if __name__ == '__main__': # If it's executed like a script (not imported)
connection = psycopg2.connect(host='host', user='user',
password='password')
connection.set_isolation_level(psycopg2.extensions.ISOLATION_LEVEL_AUTOCOMMIT)
dblisten(connection)
Related
I have Streamlit app that is connected with SQL Server database. I tried to create a function to connect the app but the app crash and display the below error:
name con is not defined
Code:
#st.cache(allow_outup_mutation=True) # this is changed to st.experimantal_singleton
def connect_db():
try:
con=pyodbc.connect(
driver = 'ODBC DRIVER 17 FOR SQL SERVER',
Server = 'localhost',
DATABASE='test_db',
UID = 'test',
PWD ='test',
)
cursor = con.cursor()
df = pd.read_sql_query('select * from test_db',con)
data = df
except Exception as e:
st.write("error is :{}".format(e))
return data
def main()
# call connect_db in order to use it parameters in latter queries
connect_db()
based on the answer of #InsertCheesyLine i added this generator to the function
st.experimantal_singleton
You should use st.experimental_singleton for database connectors instead.
From the docs - Each singleton object is shared across all users connected to the app. Singleton objects must be thread-safe, because they can be accessed from multiple threads concurrently.
st.cache on the other hand: The first time Streamlit runs the function and stores the result in a local cache. next time the function is called, if those three values have not changed Streamlit knows it can skip executing the function altogether. Instead, it just reads the output from the local cache.
I did solve this issue I just had to return con variable instead of data variable and assign the con variable to the function connect_db()
hei guys,
I have an executable python script, say get_data.py (located in project_x/src/) which is working properly, when started by: python get_data.py . It gets data (a list of id's which are necessary for further calculations) from a database via mysql.connector and then processes these data in parallel (via multiprocessing) using pool.map.
BUT it is supposed to be started by an .exe-file (located in project_x/exec/)[EDIT: This .exe uses the php command exec() to directly addresses my python script]. This is not working properly but ending in the try-except-block (in wrapper_fun) when catching the (unknown) mistake and not terminating when deleting the try-except-commands.
Do you have any idea what could be going wrong? I would appreciate any idea. I tried logging but there seems to be a permission problem. My idea is that the connection the db cannot be established and therefore there are no id's.
def calculations():
do_something...
def wrapper_fun(id):
try:
calculations(id)
except Exception:
return(False)
if __name__ == "__main__":
import multiprocessing
import mysql.connector
from mysql.connector import Error
host_name = <secret_1>
user_name = <secret_2>
passt = <secret_3>
connection = None
try:
connection = mysql.connector.connect(
host=host_name,
user=user_name,
passwd=user_password
)
except Error as err:
print(f"Error: '{err}'")
d = pd.read_sql_query(query, connection,coerce_float=False)
connection.close()
id_s = list(d.ids)
results = [pool.map(wrapper_fun,id_s)]
...
I have looked at similar questions but nothing has worked for me so far
So here it is. I want to update my table through a python script. I'm using the module cx_oracle. I can execute a SELECT query but whenever I try to execute an UPDATE query, my program just hangs (freezes). I realize that I need to use cursor.commit() after cursor.execute() if I am updating a table but my code never gets past cursor.commit(). I have added a code snippet below that I am using to debug.
Any suggestions??
Code
import cx_Oracle
def getConnection():
ip = '127.0.0.1'
port = 1521
service_name = 'ORCLCDB.localdomain'
username = 'username'
password = 'password'
dsn = cx_Oracle.makedsn(ip, port, service_name=service_name) # (CONNECT_DATA=(SERVICE_NAME=ORCLCDB.localdomain)))
return cx_Oracle.connect(username, password, dsn) # connection
def debugging():
con = getConnection()
print(con)
cur = con.cursor()
print('Updating')
cur.execute('UPDATE EMPLOYEE SET LATITUDE = 53.540943 WHERE EMPLOYEEID = 1')
print('committing')
con.commit()
con.close()
print('done')
debugging()
**Here is the corresponding output: **
<cx_Oracle.Connection to username#(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=127.0.0.1)(PORT=1521))(CONNECT_DATA=(SERVICE_NAME=ORCLCDB.localdomain)))>
Updating
Solution
After a bit of poking around, I found the underlying cause! I had made changes to the table using Oracle SQL Developer but had not committed them, when the python script tried to make changes to the table it would freeze up because of this. To avoid the freeze, I committed my changes in oracle sql developer before running the python script and it worked fine!
Do you have any option to look in the database ? I mean , in order to understand whether is a problem of the python program or not, we need to check the v$session in the database to understand whether something is blocked.
select sid, event, last_call_et, status from v$session where sid = xxx
Where xxx is the sid of the session which has connected with python.
By the way, I would choose to commit explicitly after cursor execute
cur.execute('UPDATE EMPLOYEE SET LATITUDE = 53.540943 WHERE EMPLOYEEID = 1')
con.commit()
Hope it helps
Best
I am developing a 3D Scanner using a Raspberry Pi and a Canon SLR. To control the reflex and the turntable, I use a simple web interface that I developed.
But I have to make the link between a "Start" or "Pause" buttons between the HTML interface and the Python control program. So I create a database to store the settings for the SLR (ISO, Shutterspeed ...).
For example, is it possible to have a box in the database with a state true or false for the pause button and then read it continuously on the python program?
I made a little test code like this below :
import mysql.connector
import time
mydb = mysql.connector.connect(
host="localhost",
user="scanner",
passwd="valentin",
database="Scanner3D"
)
print(mydb)
mycursor = mydb.cursor()
mycursor.execute("SELECT a FROM test")
myresult = mycursor.fetchall()
print(myresult)
myresult = str(myresult)
print(myresult)
x = myresult[2:4]
print(x)
while x == "10":
print("Hello World")
time.sleep(1)
mycursor.execute("SELECT a FROM test")
myresult = mycursor.fetchall()
myresult = str(myresult)
x = myresult[2:4]
print(x)
Problem is when I change the value of A in the database, the program never displays the new value and never leaves the loop, what is wrong in here?
Consider to use real-time communication like websocket.
The simplest way is: When scanner scans some value and saves it to DB, send via websocket command to client, to pull new data from server. Or you can send new data right in websocket message.
Very nice tutorial is here: https://realpython.com/python-sockets/
I have a program using Python + python mysql connector + Mysql which is installed in a network and uses the same database. How do I refresh a connection without restarting the program on other Machines?
The Program is installed in more than one Machine and connects to the same Mysql database which is located on a Server. The program is working properly, but,
when a new data is entered or modified "SQL INSERT, UPDATE.... Statments" from one machine is not reflected in all other Machines until the program is restarted, which means that a new connection is necessary to show the new database data.
So, I would like to know how to refresh the connection without restarting the program on other machines.
Here is the sample Connection code:
import mysql.connector as mc
conn = mc.connect(host='host', user='user', passwd='passw', db='db_name', port=port_number)
cur = conn.cursor()
How to refresh This Connection while the program is running?
closing and reopening connection may also solve your problem,
see following example in loop
import mysql.connector as mc
conn = mc.connect(host='host', user='user', passwd='passw', db='db_name', port=port_number)
while True:
# check if youre connected, if not, connect again
if (conn.is_connected() == False):
conn = mc.connect(host='host', user='user', passwd='passw', db='db_name', port=port_number)
cur = conn.cursor()
sql = "INSERT INTO db_name.myTable (name) VALUES (%(val)s);"
val = {'val':"Slim Shady"}
cur.execute(sql,val)
conn.commit()
conn.close()
After inserting or updating, you should do a commit to make sure all data it's writed to the DB. Then it will be visible for everyone:
conn.commit()