I'm running Ubuntu 16.04 with MySQL. I've opened the MySQL server for remote connections, and my remote Python script can query my database, but all attempts to INSERT fail without any error log entry.
It also looks like my remote INSERTs are being seen, because my AUTO_INCREMENT ID increases without entries being made when I run the Python INSERT code.
Any insight is appreciated!
Simple table schema:
CREATE TABLE test (
ID int NOT NULL AUTO_INCREMENT,
x INT,
PRIMARY KEY (ID)
);
This works directly on the server:
INSERT INTO test (x) VALUES (10);
This is the Python query that's working:
try:
connection = db.Connection(host=HOST, port=PORT, user=USER, passwd=PASSWORD, db=DB)
cursor = connection.cursor()
print("Connected to Server")
cursor.execute("SELECT * FROM test")
result = cursor.fetchall()
for item in result:
print(item)
except Exception as e:
print('exception connecting to server db: ' + str(e))
finally:
print('closing connection...')
connection.close()
And the Python INSERT that's not working:
try:
connection = db.Connection(host=HOST, port=PORT, user=USER, passwd=PASSWORD, db=DB)
cursor = connection.cursor()
print("Connected to Server")
cursor.execute("INSERT INTO test (x) VALUES (10);")
except Exception as e:
print('exception connecting to server db: ' + str(e))
finally:
print('closing connection...')
connection.close()
Thanks
Add this line after the execute() call:
cursor.execute("INSERT INTO test (x) VALUES (10)")
connection.commit()
When making changes to the db, it is required that you commit your changes, no change(s) would take effect.
Related
I am trying to update my mariadb table via python code .While compile the query nothing happen in my database. please check below code and let me know where i made mistake in update function
import mariadb
connection= mariadb.connect(user="user1", database="db1", host="ippp" ,password="pass")
cursor= connection.cursor()
cursor.execute("UPDATE product_options_combinations SET quantity=5944 WHERE item_code ='31628'")
cursor.close()
connection.close()
Hello here I have a clean code example for you. How to update it.
import pymysql
# Create a connection object
# IP address of the MySQL database server
Host = "localhost"
# User name of the database server
User = "user"
# Password for the database user
Password = ""
database = "GFG"
conn = pymysql.connect(host=Host, user=User, password=Password, database)
# Create a cursor object
cur = conn.cursor()
query = f"UPDATE PRODUCT SET price = 1400 WHERE PRODUCT_TYPE = 'broadband'"
cur.execute(query)
#To commit the changes
conn.commit()
conn.close()
You just need to add connection.commit() to your code, but I recommend you use a parametrized SQL preferably with a list of tuples,more of which might be added if needed, along with cursor.executemany() as being more performant for DML statements such as
import mariadb
connection= mariadb.connect(user="user1",
password="pass",
host="ippp",
port=3306,
database="db1")
cursor= connection.cursor()
dml="""
UPDATE product_options_combinations
SET quantity=%s
WHERE item_code =%s
"""
val=[
(5944,'31628')
]
cursor.executemany(dml,val)
connection.commit()
cursor.close()
connection.close()
Are you sure that the connection is working properly?
Have you tried to implement a try and catch routine to print mariadb errors?
Something like this:
# Connect to MariaDB Platform
import mariadb
try:
conn = mariadb.connect(
user="user",
password="password",
host="xx.xx.xx.xx",
port=3306,
database="db_name"
)
except mariadb.Error as e:
print(f"Error connecting to MariaDB Platform: {e}")
sys.exit(1)
I prepared two VM instance with Compute Engine on GCP.
ServerA: Data processing and read/write to SQL(mysql) on ServerB.
ServerB: SQL Server (f1-micro* This is not Cloud SQL, but normal VM instance.)
Trying to access SSH from A to B in order to read/write DB on ServerB with the code below.
error code
error: ERROR | Problem setting SSH Forwarder up: Couldn't open tunnel localhost:3306 <> localhost:3306 might be in use or destination not reachable
sshtunnel.HandlerSSHTunnelForwarderError: An error occurred while opening tunnels.
#SSH connection
with SSHTunnelForwarder(
('PublicIP of ServerA', 22),
ssh_pkey=SSH_PKEY_PATH,
ssh_username=SSH_USER,
remote_bind_address=('localhost', 3306),
local_bind_address=('localhost', 3306)
) as ssh:
try:
#DB connection
connection = mysql.connector.connect(
host='localhost',
port = 3306,
user=MYSQL_USER,
passwd=MYSQL_PASS,
db=MYSQL_DB,
charset='utf8'
)
# print(connection.is_connected())
# Get Cur
cur = connection.cursor()
sql = "use dbname"
cur.execute(sql)
for i in range(len(sqlList)):
print("DB Access：" + str(sqlList[i]))
sql = str(sqlList[i])
# sql = 'create table test (id int, content varchar(32))'
cur.execute(sql)
sqlOUTPUT = cur.fetchall()
# rows = cur.fetchall()
# for row in rows:
# print(row)
except mysql.connector.Error as err:
print("Something went wrong: {}".format(err))
connection.rollback()
raise err
finally:
#Cur close
cur.close()
# Commit
connection.commit()
#DB Connection close
connection.close()
return sqlOUTPUT
But after "local_bind_address=(localhost, MYSQL_PORT)", an error occurs despite it goes through with the same code and same private key on the shell of B or on VSCode local environment.
I don't understand why it goes through with same code using shell and VSCode although it doesn't work on GCE.
Any help?
You might be able to debug this further and discard any issues with sshtunnel if you try to create the tunnel outside of the script from the client VM, with:
$ gcloud compute ssh server-a --zone=your-zone --ssh-flag='-NL 3306:127.0.0.1:3306' &
Then attempt a connection with:
$ mysql -h 127.0.0.1
I have an SQLite3 database that I want to add to with python, this is the code i have to add a row
def create_connection(db_file):
""" create a database connection to a SQLite database """
conn = None
try:
conn = sqlite3.connect(db_file)
return conn
except Error as e:
print(e)
def add_password(conn, data):
"""
Create an entry into the password database
"""
try:
sql = 'INSERT INTO passwords(added,username,password,website,email) VALUES(?,?,?,?,?)'
cur = conn.cursor()
cur.execute(sql, data)
print('done')
return cur.lastrowid
except Error as e:
print(e)
connection = create_connection('passwords.db')
data = (datetime.now(), 'SomeUsername', 'password123', 'stackoverflow.com', 'some#email.com')
add_password(connection, data)
When I run it prints done and ends, there are no errors. However, when I open the database to view the table, it has no entries.
If I open the database and run the same SQL code
INSERT INTO passwords(added,username,password,website,email)
VALUES('13-5-2020', 'SomeUsername', 'password123', 'stackoverflow.com', 'some#email.com')
it adds to the table. So it must be a problem with my python code. How do I get it to add?
Just make conn.commit() after executing query. It should work
I am grabbing json data from a messaging bus and dumping that json into a database. I've had this working pretty well with psycopg2 doing ~3000 entries/sec into postgres. For a number of reason we've since moved to SQL Server 2016, and my entries dropped to around 100 per second.
I've got a functioned called insert_into() that inserts the json to the database. All I've really done to my insert_into() function is change the library to pyodbc and the connection string. It seems that my slow down is coming from setting up then tearing down my connection each time my function is called ('conn' in the code below). If I move the line that setup the connection outside of my insert_into function, my speed comes back. I was just wondering two things:
Whats the proper way to setup connections like this from a SQL Server perspective?
Is this even the best way to do this in postrgres?
For SQL Server, the server is 2016, using ODBC driver 17, SQL authentication.
Slow for SQL Server:
def insert_into():
conn = None
try:
conn = pyodbc.connect('DRIVER={ODBC Driver 17 for SQL Server};SERVER=server1;DATABASE=json;UID=user;PWD=pass')
cur = conn.cursor()
for i in buffer_list:
command = 'INSERT INTO jsonTable (data) VALUES (%s)' % ("'" + i + "'")
cur.execute(command)
cur.close()
conn.commit()
except (Exception, pyodbc.DatabaseError) as error:
print(error)
finally:
if conn is not None:
conn.close()
Fast for SQL Server:
conn = pyodbc.connect('DRIVER={ODBC Driver 17 for SQL Server};SERVER=server1;DATABASE=json;UID=user;PWD=pass')
def insert_into():
#conn = None
try:
cur = conn.cursor()
for i in buffer_list:
command = 'INSERT INTO jsonTable (data) VALUES (%s)' % ("'" + i + "'")
cur.execute(command)
cur.close()
conn.commit()
except (Exception, pyodbc.DatabaseError) as error:
print(error)
This daemon runs 24/7 and any advice on setting up a fast connection to MSSQL will be greatly appreciated.
I'm having a problem where I have a SQL statement that when run in my MySQL Workbench executes properly, but when run with python's mysql package function cursor.execute() doesn't work. The problem SQL statement is:
REPLACE INTO mmm_dev.samp_wp_links SELECT * FROM mmm_master.samp_wp_links;
The statement is supposed to copy all data from mmm_master into mmm_dev. The following the python code that I'm using to execute the query:
cnx = mysql.connector.connect(**config)
cursor = cnx.cursor()
def examine(cursor, cnx):
try:
qry = cursor.execute("REPLACE INTO mmm_dev.samp_wp_links SELECT * FROM mmm_master.samp_wp_links;")
except mysql.connector.Error as err:
print("Failed to select everything")
exit(1)
MySQL Python libraries are PEP 249-compliant:
.commit () Commit any pending transaction to the database.
Note that if the database supports an auto-commit feature, this must
be initially off. An interface method may be provided to turn it back
on.
Database modules that do not support transactions should implement
this method with void functionality.
Call cnx.commit()
Call examine(). Debug with print():
cnx = mysql.connector.connect(**config)
cursor = cnx.cursor()
def examine(cursor):
print("[DEBUG] 1: before query")
try:
qry = cursor.execute("REPLACE INTO mmm_dev.samp_wp_links SELECT * FROM mmm_master.samp_wp_links;")
print("[DEBUG] 2: after query")
except mysql.connector.Error as err:
print("Failed to select everything %s" % err)
exit(1)
print("[DEBUG] 3: success")
examine(cursor)
You have to do a commit.
mydb.commit()
in order to save the changes.