I'm having a big issue come up with Python (3.4) SqlAlchemy as of late, with a project that very much depends on this.
sqlalchemy.exc.OperationalError: (_mysql_exceptions.OperationalError) (2002, "Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)")
Which is preceded with the following error:
File "/home/dev/venv/project/lib/python3.4/site-packages/sqlalchemy/util/queue.py", line 145, in get
raise Empty
sqlalchemy.util.queue.Empty
I have literally no idea why this issue is happening, and it's becoming quite the burden. If I connect with a SQLLite database, it works.
Here's the base string I'm using for connection:
mysql:///username:password#redacted:3306/inventory
Credentials removed.
The database configuration is loaded from a YAML file on disk, and is verified to be as expected.
All unit tests surrounding the connection have passed, though I only test with SQLLite.
Here is the code that's used to connect & create the database and its elements:
self.database_config = {
'SQLALCHEMY_DATABASE_URI': self.config['databases']['source']['url'],
# 'SQLALCHEMY_BINDS': {
# 'discard': self.config['databases']['discarded-items']['url']
# }
}
try:
db.init(config=self.database_config, Model=Model)
# todo check if db exists
logger.debug("Database Initialized!")
db.create_all()
logger.debug("Database Created")
return True
except SQLAlchemyError as ex:
logger.error(ex)
logger.info("Application config: {0}\nConfig Location: {1}\nDatabase Config: {2}".format(self.config,
self.config_location,
self.database_config))
raise ex
Please, if I'm able to get any help that would be great! It's been quite the headache and would really like to resolve this!
Related
I have a Django app below that uses a proxy to connect to an external Postgres database. I had to replace another package with psycopg2 and it works fine locally, but doesn't work when I move onto our production server which is a Heroku app using QuotaguardStatic for proxy purposes. I'm not sure what's wrong here
For some reason, the psycopg2.connect part returns an error with a different IP address. Is it not inheriting the proxy set in the context manager? What would be
from apps.proxy.socks import Socks5Proxy
import requests
PROXY_URL = os.environ['QUOTAGUARDSTATIC_URL']
with Socks5Proxy(url=PROXY_URL) as p:
public_ip = requests.get("http://wtfismyip.com/text").text
print(public_ip) # prints the expected IP address
print('end')
try:
connection = psycopg2.connect(user=EXTERNAL_DB_USERNAME,
password=EXTERNAL_DB_PASSWORD,
host=EXTERNAL_DB_HOSTNAME,
port=EXTERNAL_DB_PORT,
database=EXTERNAL_DB_DATABASE,
cursor_factory=RealDictCursor # To access query results like a dictionary
) # , ssl_context=True
except psycopg2.DatabaseError as e:
logger.error('Unable to connect to Illuminate database')
raise e
Error is:
psycopg2.OperationalError: FATAL: no pg_hba.conf entry for host "12.345.678.910", user "username", database "databasename", SSL on
Basically, the IP address 12.345.678.910 does not match what was printed at the beginning of the context manager where the proxy is set. Do I need to set a proxy another method so that the psycopg2 connection uses it?
I have a MySQL server installed locally and I have Python code that accesses MySQL Database and executes a simple query:
from mysql.connector import connect
from mysql.connector import ProgrammingError
DB = {
'user':'andrei',
'password':'qwertttyy',
'host':'localhost',
'port':'3306',
'db':'my_database'
}
class Connection:
instance = None
def __new__(cls):
if not cls.instance:
try:
cls.instance = connect(**DB)
except:
raise
return cls.instance
def excuteDQL(query):
cnx = Connection()
cursor = cnx.cursor()
try:
cursor.execute(query)
return cursor.fetchall()
except ProgragrammingError as err:
print('You have an error in your MySQL syntax. Please check and retry')
return []
if __name__ == '__main__':
while True:
query = input('Enter a SQL query: ')
for tuple in executeDQL(query):
print(tuple)
If I go out there and find a cloud MySQL hosting service and pay for it, the access would be as easy as changing the DB mapping with different info?
I think it should be because the connection would still be over standard TCP/IP, except, in this case, it happens to come back the same machine that is emitting. I guess, under the hood, data is packed following TCP/IP rules up to the IP layer, and then these are transferred as IP Packets from the Python process through the OS Networking API to the MySQL Server listening to the port, without further down processing into the Access Layer since the packets never leave the machine, which I understand is the purpose of the Access Layer of the TCP/IP stack, that is, to abstract the physical road the data takes.
Did I say something coherent in my guessing?
If I'm wrong, How can I put a MySQL Server in the cloud?
Yes how you connect to the database would not change. It will be as simple as changing the host name and providing whatever credentials you need ( Access Token , User info, etc). The way you insert data doesn't change once you make a connection to the DB.
Here is a good script which should provide some info: https://gist.github.com/kirang89/7161185
Click to see RDS settings
Im trying to connect to a MYSQL server on Amazons Web Service, specifically the RDS- using SQLAlchemy in python (Pycharm).
I've already installed drivers for pymysql to include in the connection string for the engine (engine = create_engine("mysql+pymysql://...).
I've tried setting the CIDR security group inbound and outbound rules to allow any IP.
I can connect to the AWS-RDS just fine from MYSQL Workbench suing the Endpoint, Port and credentials.
I was able to connect to a local instance of MYSQL using SQLAlchemy and create_engine(...) without any issues too.
I've tried including and excluding the port from the URL.
try: # exception handling for database creation/ existence
engine = create_engine(link_to_db, pool_pre_ping=True, pool_recycle=3600) # Access the DB Engine
connection = engine.connect()
print("Database conn 1 successful")
except Exception as e:
logging.exception(e)
print("error connecting to db")
where link_to_db is "mysql+pymysql://{RDS USERNAME}:{RDS PASSWORD}#
{RDS ENDPOINT}:3306/{DATABASE NAME}"
The expected result is
"Database conn 1 successful"
printed to the output.
Errors :
sqlalchemy.exc.OperationalError: (pymysql.err.OperationalError) (2003, "Can't connect to MySQL server on '{RDS URL}' ([Errno 11001] getaddrinfo failed)")
(Background on this error at: http://sqlalche.me/e/e3q8)
Kindly check the below things.
1) Check your rds is publicly accessible to YES. if its in private subnet check connectivity between your source(office/home/etc) to destination(RDS).
2) open 3306 in the security group.
3) check master user credentials.
# telnet {rds end-point} 3306
I found that it was as simple as changing from a multiline string """ {link} """ to single quotes '{link}'.
I'm getting a 'need to login' error when trying to interact with my MongoHQ database through python console on heroku:
...
File "/app/.heroku/venv/lib/python2.7/site-packages/pymongo/helpers.py", line 128, in _check_command_response
raise OperationFailure(msg % response["errmsg"])
pymongo.errors.OperationFailure: command SON([('listDatabases', 1)]) failed: need to login
My applicable code
app/init.py:
from mongoengine import connect
import settings
db = connect(settings.DB, host=settings.DB_HOST, port=settings.DB_PORT, username=settings.DB_USER, password=settings.DB_PASS)
app/settings.py:
if 'MONGOHQ_URL' in os.environ:
url = urlparse(os.environ['MONGOHQ_URL'])
DB = url.path[1:]
DB_HOST = url.hostname
DB_PORT = url.port
DB_USER = url.username
DB_PASS = url.password
os.environ['MONGOHQ_URL'] looks like:
'mongodb://[username]:[password]#[host]:[port]/[db-name]'
This code works (connects and can read and write to mongodb) both locally and from the heroku web server.
According to the docs (http://www.mongodb.org/display/DOCS/Connections), it should at make a 'login' attempt on connection to the server as long as the username and password params are passed to Connection or parseable from the URI. I couldn't think of a way to see if the login attempt was being made and failing silently.
I've tried bypassing mongoengine and using pymongo.Connection and got the same result. I tried all of the several patterns of using the Connection method. I created a new database user, different from the one mongoHQ creates for heroku's production access -> same same.
It's a flask app, but I don't think any app code is being touched.
Update
I found a solution, but it will cause some headaches. I can manually connect to the database by
conn = connect(settings.DB, host=settings.DB_HOST, port=settings.DB_PORT, username=settings.DB_USER, password=settings.DB_PASS)
db = conn[settings.DB]
db.authenticate(settings.DB_USER, settings.DB_PASS)
Update #2
Mongolab just worked out of the box.
Please use the URI method for connecting and pass the information to via the host kwarg eg:
connect("testdb_uri", host='mongodb://username:password#localhost/mongoenginetest')
MongoHQ add-on uses password hashes not actual passwords and that's perhaps the error.
You should change the environment variable MONGOHQ_URL to a real password with the following command:
heroku config:set MONGOHQ_URL=mongodb://...
Once set, you may restart your applications (heroku apps) so the change gets picked up. If you're in the directory of the failing application, config:seting the config var will restart the application.
I am working with MySQL-python package. I am able to execute MySQL dependent scripts from command line, however doing the same through browser (Apache CGI) yields the following error:
A problem occurred in a Python script. Here is the sequence of function calls leading up to the error, in the order they occurred.
/var/www/html/temp.py in ()
9
10 # Establich a connection
11 db = MySQLdb.connection(host="127.0.0.1", user="root", passwd="", db="inserv")
12
13 # Run a MySQL query from Python and get the result set
db undefined, MySQLdb = <module 'MySQLdb' from
'/usr/lib64/python2.6/site-packages
/MySQLdb/__init__.pyc'>, MySQLdb.connection = <type '_mysql.connection'>,
host undefined, user undefined, passwd undefined
<class '_mysql_exceptions.OperationalError'>:
(2003, "Can't connect to MySQL server on '127.0.0.1' (13)")
args = (2003, "Can't connect to MySQL server on '127.0.0.1' (13)")
message = ''
I have been stuck in this situation for past few days. MySQL commands issued through php based sites execute appropriately and I can, also, login to MySQL from command line. The problem seem to be with Python CGI only.
I have also tried the same with oursql package and there seems to be a similar problem. How can I address this situation?
Edit
As per #Real's answer I have edited my code to use MySQLdb.connect() but the problem still persist and traceback ends with:
2003, "Can't connect to MySQL server on '127.0.0.1' (13)"
You should be using connect, not connection. Mysqldb.connect() returns a connection object but you appear to be calling mysqldb.connection(). See The docs for an example of how to do it.