I want to import a dataframe into a access database but I got an error NoSuchModuleError: Can't load plugin: sqlalchemy.dialects:access.pyodbc
from sqlalchemy import create_engine
import urllib
import pyodbc
conec = (r"Driver={Microsoft Access Driver (*.mdb, *.accdb)};"
r"DBQ=C:\Users\redim\Desktop\18_marzo\Libr2.accdb"
)
con = f"access+pyodbc:///?odbc_connect={urllib.parse.quote_plus(conec)}"
acc_engine = create_engine(con)
df.to_sql('hola', acc_engine)
For some reason this only works in Jupyter notebooks, not in PyCharm which I was having the same problems for quite some time in both until I upgraded SQLalchemy
I use conda install with most of my libraries:
conda update sqlalchemy
But if you are using pip, then below is command:
pip install --upgrade sqlalchemy
I went from sqlalchemy 1.3.7 to 1.4.39 and the error went away, at least in Jupyter Notebooks anyways.
Related
I am trying to import psycopg2 in a python file I created to establish a connection with my Postgres database. Issue is that my system shows I have psycopg2 installed when I run the following in my terminal pip freeze | grep psycopg2 but in my .py file I have the following code
import psycopg2
conn = psycopg2.connect('dbname=test user=postgres')
cur = conn.cursor()
cur.execute("SELECT * FROM my_data")
records = cur.fetchall()
and it fails at the import. The problem I get is Import "psycopg2" count not be resolved from source Pylance(reportMissingModuleSource)
What am I doing wrong. I have psycopg2-binary installed as well. My Mac version is Big Sur version 11.6
My psycopg2 version is 2.8.6
I am using AWS Glue Spark with python job to sync the data from s3 to on-prem Sql Server and using AWS Wrangler and attached pyodbc wheel file along with it. when I ran my job I am getting this error "ModuleNotFoundError: You need to install pyodbc respectively the AWS Data Wrangler package with the sqlserver extra for using the sqlserver module".
Need help on how to install the pyodbc respectively with AWS Data Wrangler package.
Another way I tried to import the pyodbc is from importlib.util.find_spec("pyodbc") but getting same error as well.
import sys
from awsglue.transforms import *
from awsglue.utils import getResolvedOptions
from pyspark.context import SparkContext
from awsglue.context import GlueContext
from awsglue.job import Job
import importlib
import awswrangler as wr
from awswrangler import _data_types
from awswrangler import _databases as _db_utils
from awswrangler import exceptions
from awswrangler import sqlserver
__all__ = ["connect", "read_sql_query", "read_sql_table", "to_sql"]
_pyodbc_found = importlib.util.find_spec("pyodbc")
if _pyodbc_found:
import pyodbc
args = getResolvedOptions(sys.argv, ['JOB_NAME'])
sc = SparkContext()
glueContext = GlueContext(sc)
spark = glueContext.spark_session
job = Job(glueContext)
job.init(args['JOB_NAME'], args)
con = wr.sqlserver.connect(connection="glue_to_onprem_test_1", jdbc_driver_version=3.0)
df = wr.sqlserver.read_sql_query(sql="SELECT TOP 10 * FROM dbo.GlueDataSync", con=con)
con.close()
Notes for Microsoft SQL Server
awswrangler is using the pyodbc for interacting with Microsoft SQL Server. For installing this package you need the ODBC header files, which can be installed, for example, with the following commands:
sudo apt install unixodbc-dev
yum install unixODBC-devel
After installing these header files you can either just install pyodbc or awswrangler with the sqlserver extra, which will also install pyodbc:
pip install pyodbc
pip install awswrangler[sqlserver]
Finally you also need the correct ODBC Driver for SQL Server. You can have a look at the documentation from Microsoft to see how they can be installed in your environment.
If you want to connect to Microsoft SQL Server from AWS Lambda, you can build a separate Layer including the needed OBDC drivers and pyobdc.
If you maintain your own environment, you need to take care of the above steps. Because of this limitation usage in combination with Glue jobs is limited and you need to rely on the provided functionality inside Glue itself.
Source: https://aws-data-wrangler.readthedocs.io/en/stable/install.html
Trying to create a bigquery connector using sqlalchemy
from sqlalchemy import create_engine
engine = create_engine('bigquery://<project_id>/<project_name>',
credentials_path=GCP_KEY)
conn = engine.connect()
Error:
sqlalchemy.exc.NoSuchModuleError: Can't load plugin: sqlalchemy.dialects:bigquery
Your error is generally related to the lack of some required module to use SQLAlchemy.
Therefore, after going though the documentation, I found out that you should install the requirements in you environment using:
pip3 install pybigquery
In addition, within your script you should import the following modules:
SQLAchemy
from sqlalchemy import *
from sqlalchemy.engine import create_engine
from sqlalchemy.schema import *
API Client
from pybigquery.api import ApiClient
Afterwards, you should have all the necessary packages to execute your code.
If you have any more question about using the SQLAlchemy and API client for BigQuery, you can consult the provided documentation above or I would also be glad to help.
I am getting that error when trying to import SQLAlchemy from flask_sqlalchemy:
from flask_sqlalchemy import SQLAlchemy
The complete error message is:
File "C:\Users\..\lib\site-packages\sqlalchemy\pool\base.py", line 63, in Pool
#util.deprecated_params(
AttributeError: module 'sqlalchemy.util' has no attribute 'deprecated_params'
The version of my libraries are sqlalchemy = 1.1.13 and flask_sqlalchemy = 2.4.0
Any suggestions on how to solve this?
Thanks
Had the same error with my python env which is managed via pipenv.
I ran pipenv update sqlalchemy which strangely enough rolled back sqlalchemy's version to 1.2.19 from what I had before, which was 1.3.5.
pip freeze | grep chemy
Flask-SQLAlchemy==2.4.0
SQLAlchemy==1.2.19
This now works for me without error:
python -c "from flask_sqlalchemy import SQLAlchemy"
I guess that without pipenv, one can manually use pip to re-install sqlalchemy for this specific version.
I am trying to get a simple test program working on my machine that connects to a SQL DB. I pip installed and then uninstalled and then installed with pip3: pymysql. the issue I'm getting:
import PyMySQL
ModuleNotFoundError: No module named 'PyMySQL'.
it can be found when i run the command pip list, but not found when running the program itself. I was perusing over other SO Q&A's but nothing helped.
Thanks
First insall PyMySQL using:
pip install PyMySQL
In python 3 it is pymysql, all in lower case
import pymysql
Module names are case-sensitive, and if you take a look at this example in the GitHub repo, you'll see that the module should be imported as pymysql, which is consistent with the all-lowercase convention for modules spelled out in the PEP 8 style guide.
import pymysql.cursors
# Connect to the database
connection = pymysql.connect(host='localhost',
user='user',
password='passwd',
db='db',
charset='utf8mb4',
cursorclass=pymysql.cursors.DictCursor)
# ...