I was writing a proprietary script that queries a company database to pull certain information. I was using Psycopg2. At this point the lines I was using were like:
conn = psycopg2.connect("dbname='somedb' user='usr' host='something.azure.com' password='pswd' port='5432'")
cur = conn.cursor()
query = "something"
cur.execute(query)
results = cur.fetchall()
The script was running fine until a few days ago, after quick successions of runs during debugging, I started getting:
psycopg2.OperationalError: server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
A closer look that I realized I did not properly close the connection. So I modified it to:
with psycopg2.connect("dbname='somedb' user='usr' host='something.azure.com' password='pswd' port='5432'") as conn:
cur = conn.cursor()
query = "something"
cur.execute(query)
results = cur.fetchall()
The error persisted even when no additional connections were present on the server end, so hitting max_connection is unlikely the reason here.
Strangely, I can access the same server through PGAdmin or use sqlalchemy.create_engine with pandas.read_sql on the same machine. Additionally, the script runs fine on another colleague's machine while mimicking my IP address through VPN.
Edit: sqlalchemy worked for my machine for exactly once, after which I started to get the same error.
My sqlalchemy code:
engine = create_engine('postgresql://{0}:{1}#{2}:5432/db'.format(USR, PSWD, HOST))
query = "something"
df = pd.read_sql(sql=query, con=engine)
Related
I am writing and reading from a sql database with tries & excepts. The thought behind the try/except is if for some reason the internet is down or we cannot connect to the server, we will write the sql transactions locally to a text file and then use those statements to update the table. That being said - the try and except only seems to work if there is a connection to the server. We have a table BAR in the DB database on server FOO:
try:
conn = pyodbc.connect('DRIVER={SQL Server};SERVER=FOO;DATABASE=DB;UID=user;PWD=password')
cursor = conn.cursor()
cursor.execute("UPDATE BAR SET Date = '"+time+"' WHERE ID = "+ID)
conn.commit()
except:
f = open("vistorlog.txt", "a")
f.write("UPDATE BAR SET Date = '"+time+"' WHERE ID = "+ID+"\n")
f.close()
the only instance where this try&except works is when there is an issue with the sql statement i.e. "Update BARS..." fails because there is no table named BARS. If I change the server to FOOS (or in a real life scenario unplug the ethernet cord and leave the table/serve names legitimate) the try and except doesn't work - the program freezes with no error.
I am trying to get Python to run a stored procedure in my SQL Server which kicks off a series of procedures which involves importing a file processing it and outputting a couple of files.
So far I have got my code so that it accepts an input to a table but then the whole thing hangs when it calls the stored procedure.
Checking Who2 on the server it is waiting on the preemptive_OS_Pipeops which searching has revealed it is waiting on something outside of SQL Server to finish before proceeding.
Is someone able to shed some light if it is possible to use pyodbc to blind activate a stored procedure then close the connection?
My belief is by just telling the procedure to run then closing out should fix the issue but I am having issues finding the code for this
Python code:
connection2 = pypyodbc.connect('Driver={SQL Server}; Server=server;Database=db', timeout=1)
cursor2 = connection2.cursor()
cursor2.execute("{CALL uspGoBabyGo}")
connection2.close()
return 'file uploaded successfully'
Stored procedure:
BEGIN
SET NOCOUNT ON;
EXECUTE [dbo].[uspCableMapImport]
END
After searching and the script stopped posting the record to the table I found the solution to the issue. I needed to add in the autocommit=True line to the script, now the code is as follows;
connection = pyodbc.connect('Driver={SQL Server};
Server='Server';Database='DB';Trusted_Connection=yes')
connection.autocommit=True
cursor = connection.cursor()
referee = file.filename.rsplit('.')[0]
SQLCommand = ("INSERT INTO RequestTable(Reference, Requested) VALUES ('"+ str(referee) +"', " + datetime.now().strftime('%Y-%m-%d') + ")")
cursor.execute(SQLCommand)
connection.commit
SQLCommand2 = ("{CALL uspGoBabyGo}")
cursor.execute(SQLCommand2)
connection.commit
connection.close()
I have a program using Python + python mysql connector + Mysql which is installed in a network and uses the same database. How do I refresh a connection without restarting the program on other Machines?
The Program is installed in more than one Machine and connects to the same Mysql database which is located on a Server. The program is working properly, but,
when a new data is entered or modified "SQL INSERT, UPDATE.... Statments" from one machine is not reflected in all other Machines until the program is restarted, which means that a new connection is necessary to show the new database data.
So, I would like to know how to refresh the connection without restarting the program on other machines.
Here is the sample Connection code:
import mysql.connector as mc
conn = mc.connect(host='host', user='user', passwd='passw', db='db_name', port=port_number)
cur = conn.cursor()
How to refresh This Connection while the program is running?
closing and reopening connection may also solve your problem,
see following example in loop
import mysql.connector as mc
conn = mc.connect(host='host', user='user', passwd='passw', db='db_name', port=port_number)
while True:
# check if youre connected, if not, connect again
if (conn.is_connected() == False):
conn = mc.connect(host='host', user='user', passwd='passw', db='db_name', port=port_number)
cur = conn.cursor()
sql = "INSERT INTO db_name.myTable (name) VALUES (%(val)s);"
val = {'val':"Slim Shady"}
cur.execute(sql,val)
conn.commit()
conn.close()
After inserting or updating, you should do a commit to make sure all data it's writed to the DB. Then it will be visible for everyone:
conn.commit()
I wanted to know the process of connecting to a MySQL database that is hosted on a web server.
I have a basic free webserver for testing on 000webhost on which I created a MySQL database.
I have the credentials for the database which I will pretend are
host - mysql.webhost000.com
user - dummy_user
password - dummy_password
database - dummy_database
and I have a python script executing from my local computer with internet access
import MySQLdb
db = MySQLdb.connect(host="mysql.webhost000.com",
port=3306,
user="dummy_user",
passwd="dummy_password",
db="dummy_database")
I was hoping it would connect as long as I have the right credentials but when I execute the script it just hangs and once I quit it I see the error
Can't connect to MySQL server on 'mysql.webhost000.com' (4)
Am I missing some steps?
There are two possible problems and im not able to recreate the first one. One is the
host="mysql.webhost000.com"
is incorrect and throwing an error. The connection could be listed as another way. The other I noticed is this is usually how I set up my connection script.
import MySQLdb
def connect():
db = MySQLdb.connect(host="mysql.webhost000.com",
port=3306,
user="dummy_user",
passwd="dummy_password",
db="dummy_database")
c = conn.cursor()
return c, db
I have case :
import pymysql
conn = pymysql.connect(host='127.0.0.1', unix_socket='/opt/lampp/var/mysql/mysql.sock', user='root', passwd=None, db='test')
cur = conn.cursor()
cur.execute("test < /mypath/test.sql")
cur.close()
conn.close()
I always get error :
1064 , "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'test < /mypath/test.sql' at line 1"
I tried to use source and it still failed. Did you know why?
Thank you.
Your error message says that the MySQL server can't understand
test < /mypath/test.sql' at line 1
If you're a long time *nix user, it seems intuitive that you should be able to use commands like this to pass various sorts of data streams to various programs. But that's not the way the Python sql API (or most language-specific) sql APIs works.
You need to pass a valid SQL query to the execute() method in the API, so the API can pass it to the database server. A vaild query will be something like INSERT or CREATE TABLE.
Look, the server might be on a different host machine, so telling the server to read from /mypath/test.sql is very likely a meaningless instruction to that server. Even if it did understand it, it might say File test.sql not found.
The mysql(1) command line client software package can read commands from files. Is that what you want?
>>> import MySQLdb
>>> db = MySQLdb.connect(host = 'demodb', user = 'root', passwd = 'root', db = 'mydb')
>>> cur = db.cursor()
>>> cur.execute('select * from mytable')
>>> rows = cur.fetchall()
Install MySQL-Python package to use MySQLdb.