I want to make a discord bot that saves data to a MySQL Database (currently localhost) but the problem is I don't know when to close the database connection.
Currently, when the user enters the command it always creates a new connection to the database but as you can imagine it's very slow to always connect then execute the query then close the connection, and finally after then returning the data.
Here is an example:
def open_case(case_id):
search_query = f"SELECT `CASE_ID`, `USER_ID`, `LINK_REASON`, `LINK_SCREENSHOT` FROM `Report` WHERE CASE_ID ='{case_id}'"`
mydb = mysql.connector.connect(
host = "localhost",
database = "report",
password = "root",
username = "root"
)
cursor = mydb.cursor()
try:
cursor.execute(search_query)
result = cursor.fetchall()
mydb.close()
return result
except:
return print("Error case not found")
mydb.close()
But I'm afraid if I connect to the DB at the beginning the bot crashers or so and then I've never closed the connection to the database.
Is there a way to make it better?
every connection has an idle timeout, that would detect a unused open connection and close it.
But your approach is very good and clean, as it closes the connection
A much simpler approach is to add finqallyas all try catch and ecept will run through it
Also use prepared statements, when using variabels
def open_case(case_id):
search_query = "SELECT `CASE_ID`, `USER_ID`, `LINK_REASON`, `LINK_SCREENSHOT` FROM `Report` WHERE CASE_ID =%s"
mydb = mysql.connector.connect(
host = "localhost",
database = "report",
password = "root",
username = "root"
)
cursor = mydb.cursor()
try:
cursor.execute(search_query,(case_id,))
result = cursor.fetchall()
return result
except:
result = print("Error case not found")
finally:
mydb.close()
open_case(1)
Related
I am trying to update my mariadb table via python code .While compile the query nothing happen in my database. please check below code and let me know where i made mistake in update function
import mariadb
connection= mariadb.connect(user="user1", database="db1", host="ippp" ,password="pass")
cursor= connection.cursor()
cursor.execute("UPDATE product_options_combinations SET quantity=5944 WHERE item_code ='31628'")
cursor.close()
connection.close()
Hello here I have a clean code example for you. How to update it.
import pymysql
# Create a connection object
# IP address of the MySQL database server
Host = "localhost"
# User name of the database server
User = "user"
# Password for the database user
Password = ""
database = "GFG"
conn = pymysql.connect(host=Host, user=User, password=Password, database)
# Create a cursor object
cur = conn.cursor()
query = f"UPDATE PRODUCT SET price = 1400 WHERE PRODUCT_TYPE = 'broadband'"
cur.execute(query)
#To commit the changes
conn.commit()
conn.close()
You just need to add connection.commit() to your code, but I recommend you use a parametrized SQL preferably with a list of tuples,more of which might be added if needed, along with cursor.executemany() as being more performant for DML statements such as
import mariadb
connection= mariadb.connect(user="user1",
password="pass",
host="ippp",
port=3306,
database="db1")
cursor= connection.cursor()
dml="""
UPDATE product_options_combinations
SET quantity=%s
WHERE item_code =%s
"""
val=[
(5944,'31628')
]
cursor.executemany(dml,val)
connection.commit()
cursor.close()
connection.close()
Are you sure that the connection is working properly?
Have you tried to implement a try and catch routine to print mariadb errors?
Something like this:
# Connect to MariaDB Platform
import mariadb
try:
conn = mariadb.connect(
user="user",
password="password",
host="xx.xx.xx.xx",
port=3306,
database="db_name"
)
except mariadb.Error as e:
print(f"Error connecting to MariaDB Platform: {e}")
sys.exit(1)
I have an SQLite3 database that I want to add to with python, this is the code i have to add a row
def create_connection(db_file):
""" create a database connection to a SQLite database """
conn = None
try:
conn = sqlite3.connect(db_file)
return conn
except Error as e:
print(e)
def add_password(conn, data):
"""
Create an entry into the password database
"""
try:
sql = 'INSERT INTO passwords(added,username,password,website,email) VALUES(?,?,?,?,?)'
cur = conn.cursor()
cur.execute(sql, data)
print('done')
return cur.lastrowid
except Error as e:
print(e)
connection = create_connection('passwords.db')
data = (datetime.now(), 'SomeUsername', 'password123', 'stackoverflow.com', 'some#email.com')
add_password(connection, data)
When I run it prints done and ends, there are no errors. However, when I open the database to view the table, it has no entries.
If I open the database and run the same SQL code
INSERT INTO passwords(added,username,password,website,email)
VALUES('13-5-2020', 'SomeUsername', 'password123', 'stackoverflow.com', 'some#email.com')
it adds to the table. So it must be a problem with my python code. How do I get it to add?
Just make conn.commit() after executing query. It should work
I am trying to copy a file from S3 to redshift table but I am unable to do so. However, I can read from the table so I know that my connection is okay.
Please help me to figure out the problem.
def upload_redshift():
conn_string = passd.redshift_login['login'] //the connection string containing dbname, username etc.
con = psycopg2.connect(conn_string);
sql = """FROM 's3://datawarehouse/my_S3_file' credentials 'aws_access_key_id=***;aws_secret_access_key=***' csv ; ;"""
try:
con = psycopg2.connect(conn_string)
logging.info("Connection Successful!")
except:
raise ValueError("Unable to connect to Redshift")
cur = con.cursor()
try:
cur.execute(sql)
logging.info(" Copy to redshift executed successfully")
except:
raise ValueError("Failed to execute copy command")
con.close()
I am getting Copy to redshift executed successfully message but nothing is happening in my table.
Try the following,
sql = "copy table_name FROM 's3://datawarehouse/my_S3_file' credentials 'aws_access_key_id=***;aws_secret_access_key=***' csv ;"
Also, try creating the connection under "connections tab" and use PostgresHook with aws_access_key_id and key as variables, something like below which enables to store the details encrypted within airflow,
pg_db = PostgresHook(postgres_conn_id='<<connection_id>>')
src_conn = pg_db.get_conn()
src_cursor = src_conn.cursor()
src_cursor.execute(sql)
src_cursor.commit()
src_cursor.close()
Also, you can use s3_to_redshift_operator operator and execute it as a task,
from airflow.operators.s3_to_redshift_operator import S3ToRedshiftTransfer
T1 = S3ToRedshiftTransfer(
schema = ‘’,
table = ‘’,
s3_bucket=‘’,
s3_key=‘’,
redshift_conn_id=‘’, #reference to a specific redshift database
aws_conn_id=‘’, #reference to a specific S3 connection
)
BRIEF DESCRIPTION OF THE PROBLEM: Psycopg2 won't connect to a test DB within docker from unittest, but connects fine from console.
ERROR MESSAGE:
psycopg2.OperationalError: server closed the connection unexpectedly
This probably means the server terminated abnormally before or while processing the request.
DETAILS:
I'm trying to set up a testing database in docker, that will be created and populated before testing and then removed after.
Here's the fuction to set up database:
def set_up_empty_test_db():
client = docker.from_env()
try:
testdb = client.containers.get("TestDB")
testdb.stop()
testdb.remove()
testdb = client.containers.run(
"postgres",
ports={5432: 5433},
detach=True,
name="TestDB",
environment=["POSTGRES_PASSWORD=yourPassword"],
)
except NotFound:
testdb = client.containers.run(
"postgres",
ports={5432: 5433},
detach=True,
name="TestDB",
environment=["POSTGRES_PASSWORD=yourPassword"],
)
while testdb.status != "running":
testdb = client.containers.get("TestDB")
return
If I launch this function from console it works without an error and I can see TestDB container running. I can successfully initiate connection:
conn = psycopg2.connect("host='localhost' user='postgres' password='yourPassword' port='5433'")
But it doesn't work when unit testing. Here's the testing code:
class TestCreateCity(unittest.TestCase):
def setUp(self):
set_up_empty_test_db()
con = psycopg2.connect("host='localhost' user='postgres' password='yourPassword' port='5433'")
self.assertIsNone(con.isolation_level)
cur = con.cursor()
sql_file = open(os.path.join(ROOT_DIR + "/ddl/creates/schema_y.sql"), "r")
cur.execute(sql_file.readline())
con.commit()
con.close()
self.session = Session(creator=con)
def test_create_city(self):
cs.create_city("Test_CITY", "US")
q = self.session.query(City).filter(City.city_name == "Test_CITY").one()
self.assertIs(q)
self.assertEqual(q.city_name, "Test_CITY")
self.assertEqual(q.country_code, "US")
It breaks when trying to initiate connection. Please advise.
I know this is an old question, but I needed to do the same thing today. You try and connect to the postgres server too quickly after starting it - that's why it works in the console.
All you need to do is replace:
set_up_empty_test_db()
con = psycopg2.connect("host='localhost' user='postgres' password='yourPassword' port='5433'")
with:
set_up_empty_test_db()
con = None
while con == None:
try:
con = psycopg2.connect("host='localhost' user='postgres' password='yourPassword' port='5433'")
except psycopg2.OperationalError:
time.sleep(0.5);
Hope this helps someone else. Cheers!
Below Code is my CGI Script, where am trying to do a insert Command.
#! C:\Python27\python.exe -u
import cgi
import MySQLdb
import xml.sax.saxutils
query = cgi.parse()
db = MySQLdb.connect(
host = "127.0.0.1",
user = "root",
passwd = "mysql123",
db = "welcome")
print 'Content-type: text/plain\n\n<?xml version="1.0" encoding="utf-8"?>\n<result>'
try:
c = db.cursor()
c.execute("insert into welcome.registrations values ('test','test',now())")
print '\t<update>true</update>'
except:
print '\t<update>false</update>'
print "</result>"
when i run the go to the url - .com/cgi-bin/reg.cgi, am not finding any insert operation done in mysql DB
You need to do db.commit() after c.execute().
Or you could do:
with db:
c = db.cursor()
c.execute("insert into welcome.registrations values ('test','test',now())")
make sure you do a db.commit() after every insert (or in the end, any will work ):
try:
c = db.cursor()
c.execute("insert into welcome.registrations values ('test','test',now())")
db.commit()
db.close() # Make sure you close the connection else multiple connections will result in hanging of mysql server.