I have a Django app working perfectly on my local Python 3.6 version and want to make sure that it will do so when installed elsewhere.
For this reason I created a virtualenv using precisely the same Python version which works fine globally, but without any packages:
virtualenv --no-site-packages --python=$(which python3.6) clear_env
source clear_env/bin/activate
Then I installed the requirements locally:
pip install -r requirements.txt
When I try to run the server, or even when I used the admin panel and make changes to the DB, everything works.
However, when I run the tests:
python manage.py test --nomigrations
I get the following error:
django.db.utils.ProgrammingError: (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '(6) NOT NULL, `user_id` integer NOT NULL, `content_type_id` integer NULL, `objec' at line 1")
which tracebacks to:
File "/home/niki/basic-django-ecommerce-site/clear_env/lib/python3.6/site-packages/django/db/backends/utils.py", line 83, in _execute
return self.cursor.execute(sql)
File "/home/niki/basic-django-ecommerce-site/clear_env/lib/python3.6/site-packages/django/db/backends/mysql/base.py", line 71, in execute
return self.cursor.execute(query, args)
File "/home/niki/basic-django-ecommerce-site/clear_env/lib/python3.6/site-packages/MySQLdb/cursors.py", line 206, in execute
res = self._query(query)
File "/home/niki/basic-django-ecommerce-site/clear_env/lib/python3.6/site-packages/MySQLdb/cursors.py", line 312, in _query
db.query(q)
File "/home/niki/basic-django-ecommerce-site/clear_env/lib/python3.6/site-packages/MySQLdb/connections.py", line 224, in query
_mysql.connection.query(self, query)
MySQLdb._exceptions.ProgrammingError: (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '(6) NOT NULL, `user_id` integer NOT NULL, `content_type_id` integer NULL, `objec' at line 1")
Note: I am using the --nomigrations flag in the testing to avoid this issue. This again works perfectly well in my global Python environment.
Initially I thought that the issue may be related to some missing linux MySQL/Python packages, similar to that issue. I remember that once I had to install something like libmysqlclient-dev python-dev and recompile the Python version to get it working.
However, provided that my virutalenv is using the same Python version which works globally, what could be the reason? And even more strange, why only the tests fail with that error and the runserver and everything else related to DB is working?
Solved. It was due to a possible bug in pipreqs, which I use to build requirements.txt. My global app version was using Django==2.0 and I use Mysql 5.5. For some reason, pipreqs has specified as a requirement Django==2.1.5 which supports only MySQL > 5.6, as stated here
As a result, when doing the fresh requirements installation in the virtualenv, the newer Django version was installed, which was not communicating properly with my local MySQL version.
Related
I have a question need your help~
I have a django program run in a vps(centos7,django2.2),it works well with Nginx+Uwsgi.
I edit three files(like a.py b.py c.py),and upload to the vps by winscp.exe,the program can't work now.
I found these logs in the uwsgi.log file.
File "/mnt/datasource/<privacy_hidden>/venv/lib/python3.6/site-packages/django/db/backends/sqlite3/base.py", line 68, in <module>
check_sqlite_version()
File "/mnt/datasource/<privacy_hidden>/venv/lib/python3.6/site-packages/django/db/backends/sqlite3/base.py", line 65, in check_sqlite_version
raise ImproperlyConfigured('SQLite 3.8.3 or later is required (found %s).' % Database.sqlite_version)
django.core.exceptions.ImproperlyConfigured: SQLite 3.8.3 or later is required (found 3.7.17).
unable to load app 0 (mountpoint='') (callable not found or import error)
I wrote "sqlite3 --version" in centos7 shell, it show version = 3.30.1
I wrote "python >>> import sqlite3 >>>print sqlite3.sqlite_version" it show version=3.30.1
I wrote "python manage.py runserver --noreload 0.0.0.0:80",it works well, no info show sqlite error.
But the program can't work in uwsgi,I think the uwsgi.ini is correct.
What can I do ?
Thank you!
I think I have solved this question.
In centos shell:
> mv /usr/lib64/libsqlite3.so.0.8.6 /usr/lib64/libsqlite3.so.0.8.6_old
> cp /usr/local/lib/libsqlite3.so.0.8.6 /usr/lib64/libsqlite3.so.0.8.6
then it's ok
uwsgi has RPATH set. and its' /usr/lib64, and the old sqlite library is in this directory, you need to use the newer sqlite library in /usr/local/lib.
you could recompile uwsgi like this: sudo LDFLAGS="-L/usr/local/lib -Wl,-rpath,/usr/local/lib" pip3 install uwsgi.
refrence: https://sourceexample.com/en/1c4153d4c3d4e65dac45/
I've installed the django-dbbackup package and from what the Documentation tells, i need to run python manage.py dbbackup
but it generated error pg_dump: error: too many command-line arguments
from what i have seen in the logs
dbbackup.db.exceptions.CommandConnectorError: Error running: pg_dump database_name --host=127.0.0.1 --port=5432 --username=postgres --no-password --clean
From what i have known, the correct command for pg_dump is to include the database name in the last part but the dbbackup include the database name first.
Anyone know the fix for the Django-dbbackup?
I had the same error. What I did was
Made sure pg_dump was in the environment variable in my system.
pip uninstall dbbackup
pip install django-dbbackup --upgrade
And it worked!
I have same issue with you, this is my env
Window 10
postgres10
django 2.2.9
django-dbbackup 3.2.0
I can manual run successfully like below, add "--dbname" parm name.
pg_dump --dbname=database_name --host=127.0.0.1 --port=5432 --username=postgres --no-password --clean
I don't know how to override the command by create a new method, so I changed the source code in dbbackup package directly, it works.
file "\Lib\site-packages\dbbackup\db\postgresql.py"
from:
cmd = '{} --dbname={}'.format(self.dump_cmd, self.settings['NAME'])
to:
cmd = '{} {}'.format(self.dump_cmd, self.settings['NAME'])
I would like to connect to Hive on our kerberized Hadoop cluster and then run some hql queries (obviously haha :)) from machine, which already has its own Kerberose Client and it works, keytab has been passed and tested.
Our Hadoop runs HWS 3.1 and CentOS7, my machine als runs CentOS7
I'm using Python 3.7.3 and PyHive (0.6.1).
I have installed bunch of libraries (and I also tried to uninstall them), as I was going through different forums (HWS, Cloudera, here SO...)
I installed through pip sasl libraries
pure-sasl (0.6.1)
pysasl (0.4.1)
sasl (0.2.1)
thrift-sasl (0.3.0)
I installed through yum
cyrus-sasl-2.1.26-23.el7.x86_64
cyrus-sasl-lib-2.1.26-23.el7.x86_64
cyrus-sasl-plain-2.1.26-23.el7.x86_64
saslwrapper-devel-0.16-5.el7.x86_64
saslwrapper-0.16-5.el7.x86_64
cyrus-sasl-lib-2.1.26-23.el7.i686
cyrus-sasl-devel-2.1.26-23.el7.x86_64
Below lies my connection to the hive
return hive.Connection(host=self.host, port=self.port,
database=self.database, auth=self.__auth,
kerberos_service_name=self.__kerberos_service_name)
This is part of my yaml
hive_interni_hdp:
db_type: hive
host: domain.xx.lan
database: database_name
user: user_name
port: 10000
auth: KERBEROS
kerberos_service_name: hive
When I try to run the code, I'm getting following error.
File "/opt/Python3.7.3/lib/python3.7/site-packages/dfpy/location.py", line 1647, in conn
self.__conn = self._create_connection()
File "/opt/Python3.7.3/lib/python3.7/site-packages/dfpy/location.py", line 1633, in _create_connection
kerberos_service_name=self.__kerberos_service_name)
File "/opt/Python3.7.3/lib/python3.7/site-packages/pyhive/hive.py", line 192, in __init__
self._transport.open()
File "/opt/Python3.7.3/lib/python3.7/site-packages/thrift_sasl/__init__.py", line 79, in open
message=("Could not start SASL: %s" % self.sasl.getError()))
thrift.transport.TTransport.TTransportException: Could not start SASL: b'Error in sasl_client_start (-4) SASL(-4): no mechanism available: No worthy mechs found'
Did anyone had luck? Where is the obstacle? Pyhive libs, wrong Kerberos connection settings?
I found an solution, I checked out this documentation https://www.cyrusimap.org/sasl/sasl/sysadmin.html
where is GSSAPI mentioned (with Kerberos 5, which I'm using) and I have checked, that I have no support for gssapi on my machine using
sasl2-shared-mechlist
It stated
GSS-SPNEGO,LOGIN,PLAIN,ANONYMOUS
but after installing gssapi library
yum install cyrus-sasl-gssapi
mechlist states
GSS-SPNEGO,GSSAPI,LOGIN,PLAIN,ANONYMOUS
Than I run the code again and Hooray!
P.S. Don't forget to autentificate and verify your keytab is valid
kinit -kt /root/user.keytab user#domain.com
klist
I wrote a python script that communicate with MySQL server on OSX 10.10, it runs fine Mac until I put it on an VPS running Ubuntu 14.04
The problem mainly lies on the pyMySQL module. :I can't even run their example script on the git page here
for running the following code:
import pymysql.cursors
# Connect to the database
connection = pymysql.connect(host='localhost',
user='user',
password='passwd',
db='db',
charset='utf8mb4',
cursorclass=pymysql.cursors.DictCursor)
try:
with connection.cursor() as cursor:
# Create a new record
sql = "INSERT INTO `users` (`email`, `password`) VALUES (%s, %s)"
cursor.execute(sql, ('webmaster#python.org', 'very-secret'))
# connection is not autocommit by default. So you must commit to save
# your changes.
connection.commit()
with connection.cursor() as cursor:
# Read a single record
sql = "SELECT `id`, `password` FROM `users` WHERE `email`=%s"
cursor.execute(sql, ('webmaster#python.org',))
result = cursor.fetchone()
print(result)
finally:
connection.close()
The result is
Traceback (most recent call last):
File "1.py", line 9, in <module>
cursorclass=pymysql.cursors.DictCursor)
File "build/bdist.linux-x86_64/egg/pymysql/__init__.py", line 93, in Connect
TypeError: __init__() got an unexpected keyword argument 'password'
Environment:
Ubuntu 14.04 x86_64
Python 2.7.10
PyMySQL 0.6.7
Mac is running the above script fine but not the ubuntu.
Thanks in advance.
I didn't do much checking on the error message. Instead I uninstall all the packages on the unbuntu server and follow the package list on the Mac to install them again one by one to the same version. Amazingly, the problem is solved. Unfortunately I can't tell which module is causing the problem
You might want to give it a try if you don't want to spend much time on updating modules one by one.
It seems that this is a ubuntu problem, which is solved if you install the latest pypi version.
I removed the package from ubuntu in dist-packages and installed it in site-packages:
sudo apt-get purge python3-pymysql
sudo pip3 install --upgrade pymysql
You don't (really) want to do this, especially on a server. Alternatively you use a virtualenv:
virtualenv -p python3.4 venv3
source venv3/bin/activate
pip3 install --update pip
pip3 install --update pymysql
You need to learn a bit more about virtualenv's to use them, but the idea is that you have a complete and separate python3.4 environment installed in venv3 which you activate with source venv3/bin/activate. This way it is harder to break your ubuntu (which heavily depends on a working python). Moreover, you can install different versions of packages in different virtualenv's. Also, when debugging, you can add print-statements to python-code in the site-packages folder of the virtualenv, without risking breaking your system severely. This is great, as many good python libraries have poor error reporting.
Faced this error with Python:3.6, PyMySQL3:0.5.
solution: change 'password' parameter to 'passwd'
Below is the parameters list with default values
host="localhost", user=None, passwd="", db=None, port=3306, unix_socket=None, charset='', sql_mode=None, read_default_file=None, conv=decoders, use_unicode=None, client_flag=0, cursorclass=Cursor, init_command=None, connect_timeout=None, ssl=None, read_default_group=None, compress=None, named_pipe=None
So I have a centos 6.6 on ec2.
Installed python 2.7, virtualenv for 2.7, pip-2.7 and created a virtualenv
install with my pip inside by virtualenv (2.7 all, promise) django 1.7.1
startproject works. I have my project. but:
python manage.py runserver 0.0.0.0:8080 / python manage.py migrate and everything that use manage.py gets this error:
Performing system checks...
System check identified no issues (0 silenced).
Unhandled exception in thread started by <function wrapper at 0x1c43848>
Traceback (most recent call last):
File "/opt/webapps/env/lib/python2.7/site-packages/django/utils/autoreload.py", line 222, in wrapper
fn(*args, **kwargs)
....
File "/opt/webapps/env/lib/python2.7/site-packages/django/db/backends/utils.py", line 63, in execute
return self.cursor.execute(sql)
File "/opt/webapps/env/lib/python2.7/site-packages/django/db/backends/sqlite3/base.py", line 483, in execute
return Database.Cursor.execute(self, query)
django.db.utils.OperationalError: near "N": syntax error
I have no idea why this keeps happening. uninstalled & re-installed all but this error would stay anyway.
Google keeps suggesting south install / vagrant but I don't use them now.
Someone know what this thing is about?
BTW:
which python: /opt/webapps/env/bin/python
which pip: /opt/webapps/env/bin/pip
all in virtualenv in the right place and --version should 2.7
django.db.utils.OperationalError occures when validation of one of migrations failed. So, you should check all migrations of your project for incorrect SQL-syntax ("N").