I am having issues with connecting Amazon AWS MySQL with SQLAlchemy. According to the instruction, I have connected.
app.config['SQLALCHEMY_DATABASE_URI'] = 'mysql://<user>:<password>#<host>/<dbname>
But there is an error:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/ec2-user/venv/lib/python3.7/site-packages/flask_sqlalchemy/__init__.py", line 514, in __get__
return type.query_class(mapper, session=self.sa.session())
File "/home/ec2-user/venv/lib/python3.7/site-packages/sqlalchemy/orm/scoping.py", line 74, in __call__
return self.registry()
File "/home/ec2-user/venv/lib/python3.7/site-packages/sqlalchemy/util/_collections.py", line 1001, in __call__
return self.registry.setdefault(key, self.createfunc())
File "/home/ec2-user/venv/lib/python3.7/site-packages/sqlalchemy/orm/session.py", line 2950, in __call__
return self.class_(**local_kw)
File "/home/ec2-user/venv/lib/python3.7/site-packages/flask_sqlalchemy/__init__.py", line 143, in __init__
bind = options.pop('bind', None) or db.engine
File "/home/ec2-user/venv/lib/python3.7/site-packages/flask_sqlalchemy/__init__.py", line 877, in engine
return self.get_engine()
File "/home/ec2-user/venv/lib/python3.7/site-packages/flask_sqlalchemy/__init__.py", line 896, in get_engine
return connector.get_engine()
File "/home/ec2-user/venv/lib/python3.7/site-packages/flask_sqlalchemy/__init__.py", line 559, in get_engine
self._engine = rv = sqlalchemy.create_engine(info, **options)
File "/home/ec2-user/venv/lib/python3.7/site-packages/sqlalchemy/engine/__init__.py", line 424, in create_engine
return strategy.create(*args, **kwargs)
File "/home/ec2-user/venv/lib/python3.7/site-packages/sqlalchemy/engine/strategies.py", line 81, in create
dbapi = dialect_cls.dbapi(**dbapi_args)
File "/home/ec2-user/venv/lib/python3.7/site-packages/sqlalchemy/dialects/mysql/mysqldb.py", line 102, in dbapi
return __import__('MySQLdb')
ModuleNotFoundError: No module named 'MySQLdb'
I am using Python3.7 version. Even according to the stackoverflow, I have installed pymysql, but still facing the problem.
Thank you for your reply.
If you use the PyMySQL client to connect to MySQL, your SQLAlchemy connection string needs to start with
mysql+pymysql://
instead of mysql://.
If you connect with mysql:// then you will need to install the MySQLdb library as shown in the Traceback.
Above issues has been solved with the following:
import pymysql
pymysql.install_as_MySQLdb()
Nothing to change anywhere.
Step 1
If you want to use MySQLDB you need to use one of the following commands. Which one depends on what OS and software you have and use.
easy_install mysql-python (mix os)
pip install mysql-python (mix os/ python 2)
pip install mysqlclient (mix os/ python 3)
apt-get install python-mysqldb (Linux Ubuntu, ...)
cd /usr/ports/databases/py-MySQLdb && make install clean (FreeBSD)
yum install MySQL-python (Linux Fedora, CentOS ...)
For Windows, see this answer: Install mysql-python (Windows)
Step 2:
Create Engine
engine = create_engine('mysql+mysqldb://...', pool_recycle=3600)
use the create_engine.pool_recycle option which ensures that a connection will be discarded and replaced with a new one if it has been present in the pool for a fixed number of seconds:
Create Connection object
conn = engine.connect()
Execute SQL queries
conn.execute("SELECT * From table;")
Related
I'm using Debian "Linux mint 19.1" and python 3.7.4 and system have already 2.7 & 3.6 versions. I create an env but when I start to create database file with db.create_all() I have "...
ModuleNotFoundError: No module named '_sqlite3' "
I have try by installing this packages [ quick-sqlite , pysqlite3 ]
db.create_all()
Traceback (most recent call last):
File "/media/gabalawy/projetcs/training/Flask_Blog/env3.7/lib/python3.7/site-packages/sqlalchemy/dialects/sqlite/pysqlite.py", line 338, in dbapi
from pysqlite2 import dbapi2 as sqlite
ModuleNotFoundError: No module named 'pysqlite2'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "", line 1, in
File "/media/gabalawy/projetcs/training/Flask_Blog/env3.7/lib/python3.7/site-packages/flask_sqlalchemy/init.py", line 1033, in create_all
self._execute_for_all_tables(app, bind, 'create_all')
File "/media/gabalawy/projetcs/training/Flask_Blog/env3.7/lib/python3.7/site-packages/flask_sqlalchemy/init.py", line 1025, in _execute_for_all_tables
op(bind=self.get_engine(app, bind), **extra)
File "/media/gabalawy/projetcs/training/Flask_Blog/env3.7/lib/python3.7/site-packages/flask_sqlalchemy/init.py", line 956, in get_engine
return connector.get_engine()
File "/media/gabalawy/projetcs/training/Flask_Blog/env3.7/lib/python3.7/site-packages/flask_sqlalchemy/init.py", line 561, in get_engine
self._engine = rv = self._sa.create_engine(sa_url, options)
File "/media/gabalawy/projetcs/training/Flask_Blog/env3.7/lib/python3.7/site-packages/flask_sqlalchemy/init.py", line 966, in create_engine
return sqlalchemy.create_engine(sa_url, **engine_opts)
File "/media/gabalawy/projetcs/training/Flask_Blog/env3.7/lib/python3.7/site-packages/sqlalchemy/engine/init.py", line 435, in create_engine
return strategy.create(*args, **kwargs)
File "/media/gabalawy/projetcs/training/Flask_Blog/env3.7/lib/python3.7/site-packages/sqlalchemy/engine/strategies.py", line 87, in create
dbapi = dialect_cls.dbapi(**dbapi_args)
File "/media/gabalawy/projetcs/training/Flask_Blog/env3.7/lib/python3.7/site-packages/sqlalchemy/dialects/sqlite/pysqlite.py", line 343, in dbapi
raise e
File "/media/gabalawy/projetcs/training/Flask_Blog/env3.7/lib/python3.7/site-packages/sqlalchemy/dialects/sqlite/pysqlite.py", line 341, in dbapi
from sqlite3 import dbapi2 as sqlite # try 2.5+ stdlib name.
File "/usr/local/lib/python3.7/sqlite3/init.py", line 23, in
from sqlite3.dbapi2 import *
File "/usr/local/lib/python3.7/sqlite3/dbapi2.py", line 27, in
from _sqlite3 import *
ModuleNotFoundError: No module named '_sqlite3'
I have Installed "pysqlite3"
pip install pysqlite3
and modify
"/site-packages/sqlalchemy/dialects/sqlite/pysqlite.py", line 341
from sqlite3 import dbapi2 as sqlite # try 2.5+ stdlib name.
to
from pysqlite3 import dbapi2 as sqlite # try 2.5+ stdlib name.
I believe that the _sqlite3 import is an internal import and should be available from your standard python installation. The fact that it's missing suggests a problem with your python build (did you compile it yourself).
I was able to fix this on suse by first ensuring that the ncurses were installed and then installing:
zypper install python-curses
On ubunti I guess the equivalent is (although I'm not sure its absence would give the same error):
sudo apt-get install libncursesw5-dev
(my complete build pre-requisites are:
sudo apt-get install libreadline-gplv2-dev libncursesw5-dev libssl-dev libsqlite3-dev tk-dev libgdbm-dev libc6-dev libbz2-dev
)
I'm trying to open a mysql database on windows 10, with anaconda (python 3.7 64 bit) and the pymysql package.
From windows command prompt, I do the following :
import pymysql
db = pymysql.connect(host='localhost', user='tester', passwd='xxxx', port=3306, db='mydb')
Username, password and port are correct; the host should be localhost (I get the same error if I pass 127.0.0.1); but I get the following error, which at the end states that it can't find the ffi package:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\programs\Anaconda3\lib\site-packages\pymysql\__init__.py", line 94, in Connect
return Connection(*args, **kwargs)
File "C:\programs\Anaconda3\lib\site-packages\pymysql\connections.py", line 327, in __init__
self.connect()
File "C:\programs\Anaconda3\lib\site-packages\pymysql\connections.py", line 598, in connect
self._request_authentication()
File "C:\programs\Anaconda3\lib\site-packages\pymysql\connections.py", line 873, in _request_authentication
auth_packet = _auth.caching_sha2_password_auth(self, auth_packet)
File "C:\programs\Anaconda3\lib\site-packages\pymysql\_auth.py", line 252, in caching_sha2_password_auth
data = sha2_rsa_encrypt(conn.password, conn.salt, conn.server_public_key)
File "C:\programs\Anaconda3\lib\site-packages\pymysql\_auth.py", line 132, in sha2_rsa_encrypt
rsa_key = serialization.load_pem_public_key(public_key, default_backend())
File "C:\programs\Anaconda3\lib\site-packages\cryptography\hazmat\backends\__init__.py", line 15, in default_backend
from cryptography.hazmat.backends.openssl.backend import backend
File "C:\programs\Anaconda3\lib\site-packages\cryptography\hazmat\backends\openssl\__init__.py", line 7, in <module>
from cryptography.hazmat.backends.openssl.backend import backend
File "C:\programs\Anaconda3\lib\site-packages\cryptography\hazmat\backends\openssl\backend.py", line 53, in <module>
from cryptography.hazmat.bindings.openssl import binding
File "C:\programs\Anaconda3\lib\site-packages\cryptography\hazmat\bindings\openssl\binding.py", line 13, in <module>
from cryptography.hazmat.bindings._openssl import ffi, lib
ImportError: DLL load failed: The specified module could not be found.
I've seen that conda navigator (nor pip) doesn't see an ffi package from its standard channels.
I have already tried this suggestion (adding the conda-forge channel) :
conda config --add channels conda-forge
conda install libffi
but conda takes forever in "solving environment" and doesn't give answers in reasonable time.
What am I supposed to do to install ffi and being able to connect to the db ? Am I the first in the world in calling mysql from python/Anaconda on windows ? I suppose there must be a solution.
Thanks, Marco
I ran into the same problem. Here's what solved it for me:
1. pip uninstall scrapy
2. conda uninstall scrapy
3. pip install --force --upgrade scrapy
4. scrapy startproject dingding
It works.
I got similar error:
File "/root/ncs/bootloader/mcuboot/scripts/imgtool.py", line 22, in <module>
main.imgtool()
File "/usr/lib/python3.7/site-packages/lib/python3.7/site-packages/click/core.py", line 829, in __call__
return self.main(*args, **kwargs)
File "/usr/lib/python3.7/site-packages/lib/python3.7/site-packages/click/core.py", line 782, in main
rv = self.invoke(ctx)
File "/usr/lib/python3.7/site-packages/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/lib/python3.7/site-packages/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/lib/python3.7/site-packages/lib/python3.7/site-packages/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/root/ncs/bootloader/mcuboot/scripts/imgtool/main.py", line 111, in getpub
key = load_key(key)
File "/root/ncs/bootloader/mcuboot/scripts/imgtool/main.py", line 74, in load_key
key = keys.load(keyfile)
File "/root/ncs/bootloader/mcuboot/scripts/imgtool/keys/__init__.py", line 52, in load
backend=default_backend())
File "/usr/lib/python3.7/site-packages/cryptography/hazmat/backends/__init__.py", line 14, in default_backend
from cryptography.hazmat.backends.openssl.backend import backend
File "/usr/lib/python3.7/site-packages/cryptography/hazmat/backends/openssl/__init__.py", line 6, in <module>
from cryptography.hazmat.backends.openssl.backend import backend
File "/usr/lib/python3.7/site-packages/cryptography/hazmat/backends/openssl/backend.py", line 113, in <module>
from cryptography.hazmat.bindings.openssl import binding
File "/usr/lib/python3.7/site-packages/cryptography/hazmat/bindings/openssl/binding.py", line 14, in <module>
from cryptography.hazmat.bindings._openssl import ffi, lib
ImportError: this module was compiled for Python 3.5
Solution:
Search _cffi_backend.cpython*.so in /usr/lib and /usr/local/lib
Kill these _cffi_backend.cpython*.so
Search cffi directories in /usr/lib and /usr/local/lib
Kill these directories
pip3 install --force-reinstall cryptography
This image will help you to find the issue I have configured Openstack newton on ubunu 16.04 LTS. It works fine
Now i have planned to integrate Murano in to this.
All progressed fine..When i am running dashboard using "tox -e venv -- python manage.py runserver <IP:PORT>" command. I am able to run the murano dashboard.
When i am accessing environment tab it displaying error
Error: There was an error communicating with server.
And log message displays an error
No module named memcache
For your reference error message below:
Traceback (most recent call last):
File "/root/murano/murano/murano/api/middleware/fault.py", line 130, in
process_request
return req.get_response(self.application)
File "/root/murano/murano/.tox/venv/local/lib/python2.7/site-packages/webob/request.py",
line 1299, in send
application, catch_exc_info=False)
File "/root/murano/murano/.tox/venv/local/lib/python2.7/site-packages/webob/request.py",
line 1263, in call_application
app_iter = application(self.environ,
start_response)
File "/root/murano/murano/.tox/venv/local/lib/python2.7/site-packages/webob/dec.py",
line 130, in __call__
resp = self.call_func(req, *args,
**self.kwargs)
File "/root/murano/murano/.tox/venv/local/lib/python2.7/site-packages/webob/dec.py",
line 195, in call_func
return self.func(req, *args, **kwargs)
File "/root/murano/murano/.tox/venv/local/lib/python2.7/site-packages/keystonemiddleware/auth_token/__init__.py",
line 320, in __call__
response = self.process_request(req)
File "/root/murano/murano/.tox/venv/local/lib/python2.7/site-packages/keystonemiddleware/auth_token/__init__.py",
line 552, in process_request
resp = super(AuthProtocol,
self).process_request(request)
File "/root/murano/murano/.tox/venv/local/lib/python2.7/site-packages/keystonemiddleware/auth_token/__init__.py",
line 348, in process_request
data, user_auth_ref =
self._do_fetch_token(request.user_token)
File "/root/murano/murano/.tox/venv/local/lib/python2.7/site-packages/keystonemiddleware/auth_token/__init__.py",
line 388, in _do_fetch_token
data = self.fetch_token(token)
File "/root/murano/murano/.tox/venv/local/lib/python2.7/site-packages/keystonemiddleware/auth_token/__init__.py",
line 661, in fetch_token
cached =
self._cache_get_hashes(token_hashes)
File "/root/murano/murano/.tox/venv/local/lib/python2.7/site-packages/keystonemiddleware/auth_token/__init__.py",
line 644, in _cache_get_hashes
cached =
self._token_cache.get(token)
File "/root/murano/murano/.tox/venv/local/lib/python2.7/site-packages/keystonemiddleware/auth_token/_cache.py",
line 214, in get
with self._cache_pool.reserve() as cache:
File "/usr/lib/python2.7/contextlib.py", line 17, in __enter__
return self.gen.next()
File "/root/murano/murano/.tox/venv/local/lib/python2.7/site-packages/keystonemiddleware/auth_token/_cache.py",
line 78, in reserve
import memcache
ImportError: No module named memcache
Can someone help on this?
Thanks in Advance.
Just installing memcached is not enough. You will also need to install python memcache module which is provided separately by python-memcache and python-pymemcache. I am not sure which one is used here.
You can try installing them with apt-get and see which one works for you.
Now working.. I have installed murano api and murano dashboard in different venv.
I have missed to install memcache in munano api venv. When i'm tried after install memcache in murano api venv..Now working fine..
#tutuDajuju Thanks for the Help...
This question was asked by paul_h, but I'm too new to comment with (hopefully) relevant information. I'm having the same problem with imapClient, and it's obviously not Python/Windows version specific. Paul H is running Python 2.7.11 on win 10. imapclient error on Windows
I was running 3.4.3, and even upgraded to 3.5.1. on both win7, and even tried an old Vista machine.
import imapclient
imapObj = imapclient.IMAPClient('imap.gmail.com', ssl=True)
It returns:
Traceback (most recent call last):
File "<pyshell#8>", line 1, in <module>
imapObj = imapclient.IMAPClient('imap.gmail.com', ssl=True)
File "C:\Python\lib\site-packages\imapclient\imapclient.py", line 152, in __init__
self._imap = self._create_IMAP4()
File "C:\Python\lib\site-packages\imapclient\imapclient.py", line 164, in _create_IMAP4
self._timeout)
File "C:\Python\lib\site-packages\imapclient\tls.py", line 153, in __init__
imaplib.IMAP4.__init__(self, host, port)
File "C:\Python\lib\imaplib.py", line 189, in __init__
self.open(host, port)
File "C:\Python\lib\site-packages\imapclient\tls.py", line 159, in open
self.sock = wrap_socket(sock, self.ssl_context, host)
File "C:\Python\lib\site-packages\imapclient\tls.py", line 126, in wrap_socket
ssl_context = create_default_context()
File "C:\Python\lib\site-packages\imapclient\tls.py", line 109, in create_default_context
context.load_verify_locations(cadata=certs)
File "C:\Python\lib\site-packages\backports\ssl\core.py", line 654, in load_verify_locations
self._ctx.load_verify_locations(cafile, capath)
File "C:\Python\lib\site-packages\OpenSSL\SSL.py", line 528, in load_verify_locations
_raise_current_error()
File "C:\Python\lib\site-packages\OpenSSL\_util.py", line 48, in exception_from_error_queue
raise exception_type(errors)
OpenSSL.SSL.Error: []
Meanwhile the stock imaplib connects with ssl fine.
M = imaplib.IMAP4_SSL('imap.gmail.com')
And sorry about starting a new question, but I didn't want to post this as an answer.
I had this problem as well , I was waiting for someone to answer this but took to long.
We were using IMAPclient 1.0.1 and got this error , so workaround is to install imapclient 0.13
pip uninstall imapclient
pip install imapclient==0.13
Here's a workaround that worked for me (Python 3.5, Windows 10):
from backports import ssl
from imapclient import IMAPClient
context = ssl.SSLContext(ssl.PROTOCOL_TLSv1_2)
imapObj = IMAPClient('imap.gmail.com', ssl=True, ssl_context=context)
The above code was derived from the developer's workaround here, but I found I only needed the one line defining context to make it work. Specifying other SSL/TLS protocols also worked.
I'm trying to run python setup.py a small script on a Linode instance where I'm running Flask + sqlalchemy (mysql).
#filename - setup.py
from daaru import db
def init_db():
db.drop_all()
db.create_all()
init_db()
This is the error I'm getting -
Traceback (most recent call last):
File "setup.py", line 21, in <module>
init_db()
File "setup.py", line 9, in init_db
db.drop_all()
File "/home/captain/public/daaru/v1/local/lib/python2.7/site-packages/flask_sqlalchemy.py", line 830, in drop_all
self._execute_for_all_tables(app, bind, 'drop_all')
File "/home/captain/public/daaru/v1/local/lib/python2.7/site-packages/flask_sqlalchemy.py", line 814, in _execute_for_all_tables
op(bind=self.get_engine(app, bind), tables=tables)
File "/home/captain/public/daaru/v1/local/lib/python2.7/site-packages/flask_sqlalchemy.py", line 763, in get_engine
return connector.get_engine()
File "/home/captain/public/daaru/v1/local/lib/python2.7/site-packages/flask_sqlalchemy.py", line 443, in get_engine
self._engine = rv = sqlalchemy.create_engine(info, **options)
File "/home/captain/public/daaru/v1/local/lib/python2.7/site-packages/sqlalchemy/engine/__init__.py", line 338, in create_engine
return strategy.create(*args, **kwargs)
File "/home/captain/public/daaru/v1/local/lib/python2.7/site-packages/sqlalchemy/engine/strategies.py", line 64, in create
dbapi = dialect_cls.dbapi(**dbapi_args)
File "/home/captain/public/daaru/v1/local/lib/python2.7/site-packages/sqlalchemy/connectors/mysqldb.py", line 52, in dbapi
return __import__('MySQLdb')
ImportError: No module named MySQLdb
The funny part is that when I log into the server, activate the virtual env (named v1), and run init_db in ipython everything works as expected. But when I run it as a script, it gives me an ImportError for the module named MySQLdb. MySQLdb has a systemwide installation using apt-get install python-mysqldb as this package is not available via pip.
What am I doing wrong here?
Running pip freeze on the server shows me this:
Flask==0.9
Flask-SQLAlchemy==0.16
Jinja2==2.6
SQLAlchemy==0.7.9
Werkzeug==0.8.3
argparse==1.2.1
wsgiref==0.1.2
Is the error coming because mysqldb package is not found in the venv? Why isn't then the systemwide installation of that package being used?
I used this blog post to install mysql-python in my venv. Everything works likes a charm now. Thanks.
I've also added a fabfile to automate deployment -
from fabric.api import *
env.user = "host"
env.hosts = ["hostname"]
env.directory = "/home/captain/public/daaru"
env.activate = "source /home/captain/public/daaru/v1/bin/activate"
def deploy():
with cd(env.directory):
run("git pull")
run("sudo service apache2 reload")
def virtualenv(command):
with cd(env.directory), prefix(env.activate):
run(command)
def populate_db():
virtualenv("python setup.py")
def freeze():
""" command for testing virtualenv """
virtualenv("pip freeze")
I just ran into this, and the simple, dirty workaround for me was to cp -r /usr/lib/python2.7/dist-packages/*mysql* $VIRTUALENV/lib/python2.7/dist-packages/ and cp -r /usr/lib/python2.7/dist-packages/*MySQL* $VIRTUALENV/lib/python2.7/dist-packages/, which picked up all needed parts.
Looking into it more it turned out that pip install MySQL-python failed in the virtualenv due to libmysqlclient-dev not being installed via apt. Hope this helps.