Develop Reference

Postgresql - How to open an Autocommit Connection to handle a Vacuum Full

I am trying to achieve the same thing as in earlier question psycopg2: How to execute vacuum postgresql query in python script; however, the recommendation to open an autocommit connection includes a link which is broken.
The below code runs without error BUT the table is not vacuumed.
How does this need to be written to call the Vacuum Full correctly?
#!/usr/bin/python
import psycopg2
from config import config
def connect():
""" Connect to the PostgreSQL database server """
conn = None
try:
# read connection parameters
params = config()
# connect to the PostgreSQL server
conn = psycopg2.connect(**params)
conn.autocommit=1
# create a cursor
cur = conn.cursor()
# execute Vacuum Full
cur.execute('Vacuum Full netsuite_display')
# close the communication with the PostgreSQL
cur.close()
except (Exception, psycopg2.DatabaseError) as error:
print(error)
finally:
if conn is not None:
conn.close()
print('Database connection closed.')
if __name__ == '__main__':
connect()

Python update query in mariadb

I am trying to update my mariadb table via python code .While compile the query nothing happen in my database. please check below code and let me know where i made mistake in update function
import mariadb
connection= mariadb.connect(user="user1", database="db1", host="ippp" ,password="pass")
cursor= connection.cursor()
cursor.execute("UPDATE product_options_combinations SET quantity=5944 WHERE item_code ='31628'")
cursor.close()
connection.close()

Hello here I have a clean code example for you. How to update it.
import pymysql
# Create a connection object
# IP address of the MySQL database server
Host = "localhost"
# User name of the database server
User = "user"
# Password for the database user
Password = ""
database = "GFG"
conn = pymysql.connect(host=Host, user=User, password=Password, database)
# Create a cursor object
cur = conn.cursor()
query = f"UPDATE PRODUCT SET price = 1400 WHERE PRODUCT_TYPE = 'broadband'"
cur.execute(query)
#To commit the changes
conn.commit()
conn.close()

You just need to add connection.commit() to your code, but I recommend you use a parametrized SQL preferably with a list of tuples,more of which might be added if needed, along with cursor.executemany() as being more performant for DML statements such as
import mariadb
connection= mariadb.connect(user="user1",
password="pass",
host="ippp",
port=3306,
database="db1")
cursor= connection.cursor()
dml="""
UPDATE product_options_combinations
SET quantity=%s
WHERE item_code =%s
"""
val=[
(5944,'31628')
]
cursor.executemany(dml,val)
connection.commit()
cursor.close()
connection.close()

Are you sure that the connection is working properly?
Have you tried to implement a try and catch routine to print mariadb errors?
Something like this:
# Connect to MariaDB Platform
import mariadb
try:
conn = mariadb.connect(
user="user",
password="password",
host="xx.xx.xx.xx",
port=3306,
database="db_name"
)
except mariadb.Error as e:
print(f"Error connecting to MariaDB Platform: {e}")
sys.exit(1)

"AttributeError: __exit__" when importing data in SQL database

I am currently following alongside a book (cpp for quantitative finance) and I am trying to import the symbols for S&P500 from wiki into a sql database I've created. However, I am getting the AttributeError: exit with regards to my "with con" statement (see below). I have read posts from similar errors but I cannot seem to fix mine. I am extremely new to python so perhaps there is some fundamental misunderstanding on my part. I have included the relevant code below, any advice would be hugely appreciated.
"""
Insert the S&P500 symbols into the MySQL database.
"""
# Connect to the MySQL instance
db_host = 'localhost'
db_user = 'sec_user'
db_pass = 'database_password'
db_name = 'database_name'
con = mdb.connect(
host=db_host, user=db_user, passwd=db_pass, db=db_name
)
# Create the insert strings
column_str = """ticker, instrument, name, sector,
currency, created_date, last_updated_date
"""
insert_str = ("%s, " * 7)[:-2]
final_str = "INSERT INTO symbol (%s) VALUES (%s)" % \
(column_str, insert_str)
# Using the MySQL connection, carry out
# an INSERT INTO for every symbol
with con:
cur = con.cursor()
cur.executemany(final_str, symbols)
if __name__ == "__main__":
symbols = obtain_parse_wiki_snp500()
insert_snp500_symbols(symbols)
print("%s symbols were successfully added." % len(symbols))

The error is telling you that the object returned by mdb.connect is not a context manager, that is it cannot be used in a with statement. You'll need to close the connection manually once you've finished with it (con.close()) or use a package that provides a connection that is a context manager.
A quick study of commonly used connectors suggests you want to use pymysql
>>> import MySQLdb
>>> import mysql.connector
>>> import pymysql
>>> params = {'host': 'localhost', 'user': 'root', 'password': '', 'database': 'test'}
>>> for pkg in (MySQLdb, mysql.connector, pymysql):
... conn = pkg.connect(**params)
... try:
... with conn:
... pass
... except AttributeError as ex:
... print(pkg.__name__, 'failed with', ex)
...
MySQLdb failed with __enter__
mysql.connector failed with __enter__
If you have to use a connection that is not a context manager, you can emulate it in a try/except/finally suite:
import MySQLdb
conn = MySQLdb.connect(host='localhost', user='root', password='', database='test')
try:
cursor = conn.cursor()
cursor.execute('SELECT * FROM my_table;')
for row in cursor.fetchall():
print(row)
cursor.close()
conn.commit()
except:
# log the error here
conn.rollback()
finally:
conn.close()
Or you can make your own context manager using the tools provided in contextlib:
import contextlib
import MySQLdb
#contextlib.contextmanager
def managed_connection(conn):
try:
yield
conn.commit()
except:
# log the error here
conn.rollback()
finally:
conn.close()
conn = MySQLdb.connect(host='localhost', user='root', password='', database='test')
with managed_connection(conn) as mc:
cursor = mc.cursor()
cursor.execute('SELECT * FROM my_table;')
for row in cursor.fetchall():
print(row)
cursor.close()
(You can make a cursor context manager too, or have the context manager yield a cursor rather than the connection).

Python - Data for mysql with dataframe

I'm new to Python and I wanted to ask you for help.
I want to put the data of a view in SQL Server into a table of my database in MySQL, when I try to give the following error:
Execution failed on sql: SELECT name FROM sqlite_master WHERE
type='table' AND name=?; not all arguments converted during string
formatting unable to rollback
Using Python version 3.7
Below is the code I use:
import pymysql.cursors
import pyodbc
import pandas as pd
# SQL Server Connection
connection = pyodbc.connect("DSN=SQLServer") #autocommit=True
try:
with connection.cursor() as cursor:
result = "SELECT * FROM dw.dbo.vW_sale"
df = pd.read_sql_query("SELECT * FROM dw.dbo.vW_sale", connection)
cursor.execute(result)
table = cursor.fetchall()
print(table)
finally:
connection.close()
# MySQL connection
cnx = pymysql.connect(host='test',
user='test',
password='test',
db='dw')
try:
with cnx.cursor() as cursor:
mysql = "select *from ft_sale_test"
cursor.execute(mysql)
result = cursor.fetchall()
#print(result)
finally:
cnx.close()
# using if_exists to handle the table that already exists
The error happens right here
df.to_sql(con=cnx, name= 'ft_sale_test', if_exists= 'replace')

About MySQLdb conn.autocommit(True)

I have installed python 2.7 64bit,MySQL-python-1.2.3.win-amd64-py2.7.exe.
I use the following code to insert data :
class postcon:
def POST(self):
conn=MySQLdb.connect(host="localhost",user="root",passwd="mysql",db="dang",charset="utf8")
cursor = conn.cursor()
n = cursor.execute("insert into d_message (mid,title,content,image) values(2,'xx','ccc','fff')")
cursor.close()
conn.close()
if n:
raise web.seeother('/')
This results in printing n as 1, but in mysql client data aren't visible.
google says I must add conn.autocommit(True).
but I don't know why MySQLdb turns it off;

by default MySQLdb autocommit is false,
You can set autocommit to True in your MySQLdb connection like this,
conn=MySQLdb.connect(host="localhost",user="root",passwd="mysql",db="dang",charset="utf8")
conn.get_autocommit() #will return **False**
conn.autocommit(True)
conn.get_autocommit() #Should return **True** now
cursor = conn.cursor()

I don't know if there's a specific reason to use autocommit with GAE (assuming you are using it). Otherwise, you can just manually commit.
class postcon:
def POST(self):
conn=MySQLdb.connect(host="localhost",user="root",passwd="mysql",db="dang",charset="utf8")
cursor = conn.cursor()
n = cursor.execute("insert into d_message (mid,title,content,image) values(2,'xx','ccc','fff')")
conn.commit() # This right here
cursor.close()
conn.close()
if n:
raise web.seeother('/')
Note that you probably should check if the insert happened successfully, and if not, rollback the commit.

Connector/Python Connection Arguments
Turning on autocommit can be done directly when you connect to a database:
import mysql.connector as db
conn = db.connect(host="localhost", user="root", passwd="pass", db="dbname", autocommit=True)
or
import mysql.connector
db = mysql.connector.connect(option_files='my.conf', autocommit=True)
Or call conn.commit() before calling close.