I wrote a python script that communicate with MySQL server on OSX 10.10, it runs fine Mac until I put it on an VPS running Ubuntu 14.04
The problem mainly lies on the pyMySQL module. :I can't even run their example script on the git page here
for running the following code:
import pymysql.cursors
# Connect to the database
connection = pymysql.connect(host='localhost',
user='user',
password='passwd',
db='db',
charset='utf8mb4',
cursorclass=pymysql.cursors.DictCursor)
try:
with connection.cursor() as cursor:
# Create a new record
sql = "INSERT INTO `users` (`email`, `password`) VALUES (%s, %s)"
cursor.execute(sql, ('webmaster#python.org', 'very-secret'))
# connection is not autocommit by default. So you must commit to save
# your changes.
connection.commit()
with connection.cursor() as cursor:
# Read a single record
sql = "SELECT `id`, `password` FROM `users` WHERE `email`=%s"
cursor.execute(sql, ('webmaster#python.org',))
result = cursor.fetchone()
print(result)
finally:
connection.close()
The result is
Traceback (most recent call last):
File "1.py", line 9, in <module>
cursorclass=pymysql.cursors.DictCursor)
File "build/bdist.linux-x86_64/egg/pymysql/__init__.py", line 93, in Connect
TypeError: __init__() got an unexpected keyword argument 'password'
Environment:
Ubuntu 14.04 x86_64
Python 2.7.10
PyMySQL 0.6.7
Mac is running the above script fine but not the ubuntu.
Thanks in advance.
I didn't do much checking on the error message. Instead I uninstall all the packages on the unbuntu server and follow the package list on the Mac to install them again one by one to the same version. Amazingly, the problem is solved. Unfortunately I can't tell which module is causing the problem
You might want to give it a try if you don't want to spend much time on updating modules one by one.
It seems that this is a ubuntu problem, which is solved if you install the latest pypi version.
I removed the package from ubuntu in dist-packages and installed it in site-packages:
sudo apt-get purge python3-pymysql
sudo pip3 install --upgrade pymysql
You don't (really) want to do this, especially on a server. Alternatively you use a virtualenv:
virtualenv -p python3.4 venv3
source venv3/bin/activate
pip3 install --update pip
pip3 install --update pymysql
You need to learn a bit more about virtualenv's to use them, but the idea is that you have a complete and separate python3.4 environment installed in venv3 which you activate with source venv3/bin/activate. This way it is harder to break your ubuntu (which heavily depends on a working python). Moreover, you can install different versions of packages in different virtualenv's. Also, when debugging, you can add print-statements to python-code in the site-packages folder of the virtualenv, without risking breaking your system severely. This is great, as many good python libraries have poor error reporting.
Faced this error with Python:3.6, PyMySQL3:0.5.
solution: change 'password' parameter to 'passwd'
Below is the parameters list with default values
host="localhost", user=None, passwd="", db=None, port=3306, unix_socket=None, charset='', sql_mode=None, read_default_file=None, conv=decoders, use_unicode=None, client_flag=0, cursorclass=Cursor, init_command=None, connect_timeout=None, ssl=None, read_default_group=None, compress=None, named_pipe=None
Related
I want to use the Python mariadb connector in a python container. Yet, until now, I've met only troubles and now I segfault when trying to pass arguments to my SQL queries.
here are my dockerfile and a script that highlight the problem.
FROM python:3.11-buster
RUN apt-get install gcc wget
# https://stackoverflow.com/questions/74429209/mariadb-in-docker-mariadb-connector-python-requires-mariadb-connector-c-3-2
RUN wget https://dlm.mariadb.com/2678579/Connectors/c/connector-c-3.3.3/mariadb-connector-c-3.3.3-debian-buster-amd64.tar.gz -O - | tar -zxf - --strip-components=1 -C /usr
WORKDIR /appli
RUN pip3 install pynetdicom>=2.0 mariadb==1.1.4
COPY . .
CMD ["python3", "AEServer.py"]
import mariadb
conn = mariadb.connect(user='myself', password='mypass', host=ip, database='db') # Successfully created connection
cursor = conn.cursor()
data = ('1.2.392', 'NAME^SURNAME^', 'BDAY', 'DX', 1) # Placeholder
insert_stmt = ("INSERT Examens VALUES ('%s', '%s', '%s', '%s', %i, 'COMING')" % data)
segv_stmt = ("INSERT Examens VALUES (?, ?, ?, ?, ?, 'COMING')", data)
segv = "INSERT Examens VALUES (?, ?, ?, ?, ?, 'COMING')"
cursor.execute(insert_stmt) # GOOD
cursor.execute(segv, data) # Segmentation fault
First, I tried adding mariadb in my pip install in the dockerfile. It failed, because I didn't have the mariadb/c connector.
Then, I tried adding it by installing libmariadb3 and libmariadb-dev, according to this page of the documentation : https://mariadb.com/docs/ent/connect/programming-languages/c/install/ but I failed again, because the package fetched by the manager weren't up to date, so I had a version conflit when pip installing.
Then, I tried the following reference : Mariadb in Docker: MariaDB Connector/Python requires MariaDB Connector/C >= 3.2.4, found version 3.1.16, to wget the connector and compiling it myself in the Dockerfile. I succeeded to create my container and I could create mariadb.coonnection and cursor so I thought it was over.
Finally when trying to use insert statement with placeholder (either '%s' or '?'), I am met with a segfault. I tried various associations of sources images (3.11:bullseye, 3.11:buster), connectors (version 3.3.3, 3.2.7) and mariadb (version 1.1.0, 1.1.2, 1.1.4), but all of them segfault.
What I want is a container with python3, my dependancy pynetdicom, and the ability to interract safely with a mariadb Database. A Dockerfile fulfilling the prevous prerequisites should be enough for me to move forward, but I also wish to know why things did happen this way.
Edit : Posted some credentials used in dev, removed them.
Even if you installed a newer MariaDB Connector/C version, the preinstalled 3.1.13 version from 3.11-buster is still installed.
After installation of MariaDB Connector/C 3.3.3 you have 2 versions installed:
/usr/lib/x86_64-linux-gnu/libmariadb.so.3
/usr/lib/mariadb/libmariadb.so.3
Now when running your python script, it crashes in /usr/lib/x86_64-linux-gnu/libmariadb.so.3.
To force Python to use the newer MariaDB Connector/C library just add
ENV LD_PRELOAD=/usr/lib/mariadb/libmariadb.so or ENV LD_LIBRARY_PATH=/usr/lib/mariadb
to your Dockerfile.
I have a Django app working perfectly on my local Python 3.6 version and want to make sure that it will do so when installed elsewhere.
For this reason I created a virtualenv using precisely the same Python version which works fine globally, but without any packages:
virtualenv --no-site-packages --python=$(which python3.6) clear_env
source clear_env/bin/activate
Then I installed the requirements locally:
pip install -r requirements.txt
When I try to run the server, or even when I used the admin panel and make changes to the DB, everything works.
However, when I run the tests:
python manage.py test --nomigrations
I get the following error:
django.db.utils.ProgrammingError: (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '(6) NOT NULL, `user_id` integer NOT NULL, `content_type_id` integer NULL, `objec' at line 1")
which tracebacks to:
File "/home/niki/basic-django-ecommerce-site/clear_env/lib/python3.6/site-packages/django/db/backends/utils.py", line 83, in _execute
return self.cursor.execute(sql)
File "/home/niki/basic-django-ecommerce-site/clear_env/lib/python3.6/site-packages/django/db/backends/mysql/base.py", line 71, in execute
return self.cursor.execute(query, args)
File "/home/niki/basic-django-ecommerce-site/clear_env/lib/python3.6/site-packages/MySQLdb/cursors.py", line 206, in execute
res = self._query(query)
File "/home/niki/basic-django-ecommerce-site/clear_env/lib/python3.6/site-packages/MySQLdb/cursors.py", line 312, in _query
db.query(q)
File "/home/niki/basic-django-ecommerce-site/clear_env/lib/python3.6/site-packages/MySQLdb/connections.py", line 224, in query
_mysql.connection.query(self, query)
MySQLdb._exceptions.ProgrammingError: (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '(6) NOT NULL, `user_id` integer NOT NULL, `content_type_id` integer NULL, `objec' at line 1")
Note: I am using the --nomigrations flag in the testing to avoid this issue. This again works perfectly well in my global Python environment.
Initially I thought that the issue may be related to some missing linux MySQL/Python packages, similar to that issue. I remember that once I had to install something like libmysqlclient-dev python-dev and recompile the Python version to get it working.
However, provided that my virutalenv is using the same Python version which works globally, what could be the reason? And even more strange, why only the tests fail with that error and the runserver and everything else related to DB is working?
Solved. It was due to a possible bug in pipreqs, which I use to build requirements.txt. My global app version was using Django==2.0 and I use Mysql 5.5. For some reason, pipreqs has specified as a requirement Django==2.1.5 which supports only MySQL > 5.6, as stated here
As a result, when doing the fresh requirements installation in the virtualenv, the newer Django version was installed, which was not communicating properly with my local MySQL version.
I am trying to;
Run the python code triggered by Cosmos DB when cosmos DB receives the data..
The python code in Azure Functions has code to ingest data from Azure MySQL.
What I have done are;
. Wrote python in Azure Functions and run it with triggered by Cosmos
DB. This was successful.
. Installed mysql.connector referred to
https://prmadi.com/running-python-code-on-azure-functions-app/ and
run the code to connect to Azure MySQL, but It does not work.
Do you know how to install mysql module for Python to Azure Functions and connect to the database?
Thanks!
According to your description ,I think your issue is about how to install the Python third-party module in the Azure function app.
Please refer to the steps as below :
Step 1 :
login kudu : https://Your_APP_NAME.scm.azurewebsites.net/DebugConsole.
Run Below command in d:/home/site/wwwroot/<your function name> folder.(will take some time)
python -m virtualenv myvenv
Step 2 :
Load the env via the below command in env/Scripts folder.
activate.bat
Step 3 :
Your shell should be now prefixed by (env).
Update pip
python -m pip install -U pip
Install what you need
python -m pip install MySQLdb
Step 4 :
In your code, update the sys.path to add this venv:
import sys, os.path
sys.path.append(os.path.abspath(os.path.join(os.path.dirname( __file__ ), 'env/Lib/site-packages')))
Then connect to mysql db via the snippet of code below
#!/usr/bin/python
import MySQLdb
# Connect
db = MySQLdb.connect(host="localhost",
user="appuser",
passwd="",
db="onco")
cursor = db.cursor()
# Execute SQL select statement
cursor.execute("SELECT * FROM location")
# Commit your changes if writing
# In this case, we are only reading data
# db.commit()
# Get the number of rows in the resultset
numrows = cursor.rowcount
# Get and display one row at a time
for x in range(0, numrows):
row = cursor.fetchone()
print row[0], "-->", row[1]
# Close the connection
db.close()
Hope it helps you.
I'm sure this issue has been raised an uncountable number of times before but perhaps, someone could still help me.
I am using pymssql v2.1.3 with Python 2.7.12 and the code that I used several times until yesterday to write data to my Azure SQL DB has somehow decided not to work anymore - for no apparent reason.
The firewall is set, my IP is in the whitelist, I can connect to the database using SQL Server Management Studio and query the data but I still keep getting this error when attempting to connect using pymssql.
The app is a Flask web-app and following is how I connect to the DB:
conn = pymssql.connect(server='myserver.database.windows.net', user='myusername#mydatabase', password='mypassword', database='mydatabase')
This is likely due to the pymssql version. Did you upgrade pymssql? If yes, try reverting back to 2.1.1
sudo pip install pymssql==2.1.1
Not really a solution to the issue I raised, but using pypyodbc instead of pymssql works.
conn = pypyodbc.connect(driver='{SQL Server}',server='tcp:myserver.database.windows.net,1433',database='mydatabase', uid='myusername', pwd='mypassword')
freetds-dev might be missing on linux:
apt-get update && apt-get install freetds-dev
unbelievable the bug is still present...
ENV: WSL2 on Windows 10
Fix -> switch to pyodbc:.
sudo apt-get install unixodbc-dev && pip3 install pyodbc
follow instruction for ubuntu
https://learn.microsoft.com/de-de/sql/connect/odbc/linux-mac/installing-the-microsoft-odbc-driver-for-sql-server?view=sql-server-ver15
import pyodbc
server = 'tcp:myserver.database.windows.net'
database = 'mydb'
username = 'myusername'
password = 'mypassword'
cnxn = pyodbc.connect('DRIVER={ODBC Driver 17 for SQL Server};SERVER='+server+';DATABASE='+database+';UID='+username+';PWD='+ password)
cursor = cnxn.cursor()
I uninstall and install fresh pymssql and it works for me now.
Currently, I'm running a simple python script to connect to a database:
import pyodbc
cnxn = pyodbc.connect('DRIVER={Teradata};DBCNAME=(MYDB);UID=(MYUSER); PWD=(MYPASS);QUIETMODE=YES')
With the server and credentials substituted in obviously. However, when running this script, I get the following error:
pyodbc.Error: ('200', '[200] [unixODBC][eaaa[DCTrdt rvr o nuhifraint o n (0) (SQLDriverConnectW)')
The only help I've been able to find is here installed the Teradata ODBC drivers, but I just don't understand why I can't connect. Anyone have any ideas on this?
You have to use like following to connect:
TDCONN = pyodbc.connect('DSN=yourDSNname;',ansi=True, autocommit=True)
You can replace DSN=yourDSNname; with what you have already.
I hit this "non english" error message problem. I think it was due to the wrong versions of libodbc.so and libodbcinst.so being used. Changing the links in /usr/lib/... to point to the teradata installed versions worked for me. Commands using the default install directories in Ubuntu 12.04, 64bit were:
cd /usr/lib/x86_64-linux-gnu
ls -lha | grep odbc (To should see the files below which are to be replaced).
sudo mv libodbc.so.1.0.0 Xlibodbc.so.1.0.0
sudo ln -s /opt/teradata/client/14.10/odbc_64/lib/libodbc.so libodbc.so.1.0.0
sudo mv libodbcinst.so.1.0.0 Xlibodbcinst.so.1.0.0
sudo ln -s /opt/teradata/client/14.10/odbc_64/lib/libodbcinst.so libodbcinst.so.1.0.0
I had also previously installed (through apt-get) odbcinst, so I redirected both files. But possibly only the first is needed.