I am trying to run my Django app on Google App Engine SDK (locally) inside a virtualenv with MySQL as a database. Everything in my requirements.txt file is installing perfectly. When I start the Google App Engine SDK environment, it throws me a nasty error that seems common, but Stack Overflow doesn't have any examples of solving this within Google App Engine SDK yet.
Here is my work flow from the root dir of my project...
virtualenv venv && mkdir lib
source venv/bin/activate
pip install -r requirements.txt -t lib/ && pip install -r requirements.txt
When I run the following command to start up the SDK....
dev_appserver.py app.yaml
I get the following error in my traceback...
File "/Users/username/Repositories/projectname/lib/django/db/utils.py", line 211, in __getitem__
backend = load_backend(db['ENGINE'])
File "/Users/username/Repositories/projectname/lib/django/db/utils.py", line 115, in load_backend
INFO 2018-06-26 20:09:30,812 module.py:846] default: "GET /_ah/start HTTP/1.1" 500 -
return import_module('%s.base' % backend_name)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/importlib/__init__.py", line 37, in import_module
__import__(name)
File "/Users/username/Repositories/projectname/lib/django/db/backends/mysql/base.py", line 30, in <module>
'Did you install mysqlclient or MySQL-python?' % e
ImproperlyConfigured: Error loading MySQLdb module: No module named _mysql.
Did you install mysqlclient or MySQL-python?
My requirements.txt
Django==1.11.8
djangorestframework==3.8.2
facebook-sdk
oauth2client==2.0.1
google-api-python-client==1.6.2
facebookads==2.11.1
httplib2==0.10.3
enum==0.4.6
requests-toolbelt==0.8.0
google-cloud-storage==1.6.0
google-resumable-media==0.3.1
google-auth
requests==2.18.0
lxml==3.8.0
pycrypto==2.6.1
MySQL-python==1.2.5
Contents of lib/
I am calling MySQLdb in my app.yaml as well...
libraries:
- name: MySQLdb
version: "latest"
Contents of appengine_config.py
# [START vendor]
from google.appengine.ext import vendor
vendor.add('lib')
# [END vendor]
Some things I have checked off the list while debugging...
1) MySQL-python==1.2.5 is installed when I do a pip freeze in my virtual environment.
2) MySQL is installed and works perfectly on my local computer.
3) I've looked through the lionshare of Stack Overflow's questions thus far and none of them seem to help.
The installation was set up according to Google recommendations and it contained no visible errors. The problem could not be reproduced on external environment (a similar Django setup worked fine), so we just came to a workaround, see it below (set PYTHONPATH).
GAE requests installing third-party libraries into the lib/ directory, making it a poor man's virtual environment. Then lib/ is being added to a module search path by calling vendor.add('lib') in the appengine_config.py.
That worked correctly in the author's setup, so GAE managed to import Django from lib/django/. Then this working config failed misteriously for importing MySQLdb, which was surely installed into the same lib/.
During the investigation we have checked the idea if PYTHONPATH environment variable could affect anything. It did not, but setting it fixed the issue:
export PYTHONPATH=/Users/username/Repositories/projectname/lib/
Note: avoid using relative directories in PATH-like environment variables (./lib in Erik's comment), that creates a security flaw.
The MySQL-Python which is a C extension,will probably fail(No module named _mysql).
you can give a try with pymysql module
I am using googleAppEngineLauncher to try mysql connection.
It gives the log
File "/Users/kakshilshah/Desktop/hope/skeduleBackend/django/utils/importlib.py", line 40, in import_module
__import__(name)
File "/Users/kakshilshah/Desktop/hope/skeduleBackend/django/db/backends/mysql/base.py", line 17, in <module>
raise ImproperlyConfigured("Error loading MySQLdb module: %s" % e)
ImproperlyConfigured: Error loading MySQLdb module: No module named _mysql
I have done pip install MySQL-python
Even commands like python manage.py dbshell works, and connects me to the cloudsql backend.
I can access all the tables there.
But, running it gives the same error.
I have mysql 5.6 installed.
Adding the following to app.yaml
- name: MySQLdb
version: "latest"
also does not help, because I checked the libraries directory and there was no mysqldb.
My settings -
import os
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'HOST': '173.194.xxx.xxx',
'NAME': 'my_database',
'USER': 'root',
'PASSWORD': 'xxxxxxxx',
}
}
Stuck here, please help.
You cannot have MySQL-python while uploading. Remove that and then upload.
Just the YAML file's
- name: MySQLdb
version: "latest"
Will do, if you add both, it causes a clash.
It runs locally because it does not consider the YAML's sqldb connector.
Hope this helps!
I'm setting up a simple db using Django and I got the above error when running 'python manage.py syncdb'
Is this a problem in manage.py or my .db file? Suggestions for how to resolve?
EDIT: Adding full traceback
Traceback (most recent call last):
File "manage.py", line 14, in <module>
execute_manager(settings)
File "/Library/Python/2.7/site-packages/django/core/management/__init__.py", line 438, in execute_manager
utility.execute()
File "/Library/Python/2.7/site-packages/django/core/management/__init__.py", line 379, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/Library/Python/2.7/site-packages/django/core/management/__init__.py", line 252, in fetch_command
app_name = get_commands()[subcommand]
File "/Library/Python/2.7/site-packages/django/core/management/__init__.py", line 101, in get_commands
apps = settings.INSTALLED_APPS
File "/Library/Python/2.7/site-packages/django/utils/functional.py", line 276, in __getattr__
self._setup()
File "/Library/Python/2.7/site-packages/django/conf/__init__.py", line 42, in _setup
self._wrapped = Settings(settings_module)
File "/Library/Python/2.7/site-packages/django/conf/__init__.py", line 139, in __init__
logging_config_func(self.LOGGING)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/logging/config.py", line 776, in dictConfig
dictConfigClass(config).configure()
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/logging/config.py", line 562, in configure
'filter %r: %s' % (name, e))
ValueError: Unable to configure filter 'require_debug_false': Cannot resolve 'django.utils.log.CallbackFilter': No module named CallbackFilter
I had the same problem
Go to your settings.py and find the LOGGING settings
LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'filters': {
'require_debug_false': {
'()': 'django.utils.log.CallbackFilter',
'callback': lambda r: not DEBUG
}
},
See the CallbackFilter there? Remove the whole 'filters' key. This happened because I tried to use the same settings.py file across 2 different Django versions. If you use djangoadmin.py startproject you get a valid settings.py file.
Please double check if the django version 1.3.1 you have is the most recent one, because this is a bug in Django.
To avoid such situations, I recommend to use virtualenv: How to create multiple Django environments using virtualenv
If you have the most recent version and still facing the issue, please update this bug: Ticket #16568 require_debug_false does not work as intended (backward incompatible)
The bug also has a simple solution:
Remove require_debug_false from global_settings.py (since it does
not work) and force everyone to copy/paste the default LOGGING snippet
to their settings
Answer posted by django developer andreas_pelme here, this is not a bug and if correct version of django is installed, it should work fine.
~ $ mkvirtualenv t16568-regression
New python executable in t16568-regression/bin/python
Installing setuptools............done.
Installing pip...............done.
virtualenvwrapper.user_scripts creating /Users/andreas/.virtualenvs/t16568-regression/bin/predeactivate
virtualenvwrapper.user_scripts creating /Users/andreas/.virtualenvs/t16568-regression/bin/postdeactivate
virtualenvwrapper.user_scripts creating /Users/andreas/.virtualenvs/t16568-regression/bin/preactivate
virtualenvwrapper.user_scripts creating /Users/andreas/.virtualenvs/t16568-regression/bin/postactivate
virtualenvwrapper.user_scripts creating /Users/andreas/.virtualenvs/t16568-regression/bin/get_env_details
[t16568-regression] ~ $ pip install django==1.3.1
Downloading/unpacking django==1.3.1
Downloading Django-1.3.1.tar.gz (6.5Mb): 6.5Mb downloaded
Running setup.py egg_info for package django
Installing collected packages: django
Running setup.py install for django
changing mode of build/scripts-2.7/django-admin.py from 644 to 755
changing mode of /Users/andreas/.virtualenvs/t16568-regression/bin/django-admin.py to 755
Successfully installed django
Cleaning up...
[t16568-regression] ~ $ cd code/
[t16568-regression] ~ $ django-admin.py startproject foo
[t16568-regression] ~ $ cd foo/
[t16568-regression] ~/foo $ python manage.py shell
In [1]: import django; django.get_version()
Out[1]: '1.3.1'
I have found the prolem. If we have worked with eclipse PyDev and create project into eclipse - we have incorrect project structure.
To solve this problem we need to create project in command line with manage.py startproject myprojectname and in eclipse create new Django project and specify older with already created project via command line tool.
Best regards,
Anton.
Have you tried re-installing django? If not, I think this is a great oportunity to start using virtual env.
There is a class called CallbackFilter in
/usr/local/lib/python2.6/dist-packages/django/utils/dictconfig.py
The fact that you can't read it, means either is not there or your python_path is not well configured, but since you can read other django stuff that is not probably the case.
I had the same problem, and in my case it was due to running manage.py using sudo, which ignores virtual env. There is a solution for this at this thread.
I had the same problem. I would say that the source of the problem is that your "setting.py" was created with a different Django version that your current version. If you dont know what version you have, I recommend you to start a new project in order to have a setting.py with your current Django version.
If you know the Django version that you used to create the project, you can easily install it again with the next command ( example Django version 1.4.5)
pip install Django==1.4.5
I hope this can help.
I have Pythong2.6, psycopg2 and pgAdmin3 installed using Macports. My settings.py is:
DATABASE_ENGINE = 'postgresql_psycopg2' # 'postgresql_psycopg2', 'postgresql', 'mysql', 'sqlite3' or 'oracle'.
DATABASE_NAME = 'mysite' # Or path to database file if using sqlite3.
DATABASE_USER = 'postgres' # Not used with sqlite3.
DATABASE_PASSWORD = '' # Not used with sqlite3.
DATABASE_HOST = '' # Set to empty string for localhost. Not used with sqlite3.
DATABASE_PORT = '' # Set to empty string for default. Not used with sqlite3.
The error I get when I run python manage.py syncdb is:
Traceback (most recent call last):
File "manage.py", line 11, in <module>
execute_manager(settings)
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/django/core/management/__init__.py", line 362, in execute_manager
utility.execute()
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/django/core/management/__init__.py", line 303, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/django/core/management/base.py", line 195, in run_from_argv
self.execute(*args, **options.__dict__)
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/django/core/management/base.py", line 221, in execute
self.validate()
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/django/core/management/base.py", line 249, in validate
num_errors = get_validation_errors(s, app)
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/django/core/management/validation.py", line 22, in get_validation_errors
from django.db import models, connection
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/django/db/__init__.py", line 41, in <module>
backend = load_backend(settings.DATABASE_ENGINE)
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/django/db/__init__.py", line 17, in load_backend
return import_module('.base', 'django.db.backends.%s' % backend_name)
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/django/utils/importlib.py", line 35, in import_module
__import__(name)
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/django/db/backends/postgresql_psycopg2/base.py", line 22, in <module>
raise ImproperlyConfigured("Error loading psycopg2 module: %s" % e)
django.core.exceptions.ImproperlyConfigured: Error loading psycopg2 module: No module named psycopg2
Please note, I am a complete beginner in this stuff. I am originally a PHP-guy and trying out Python for a small personal project. Do I need to "turn on" Postgres?
Also, when I sudo python manage.py runserver 8080
I get this error:
Validating models...
Unhandled exception in thread started by <function inner_run at 0x1242670>
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/django/core/management/commands/runserver.py", line 48, in inner_run
self.validate(display_num_errors=True)
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/django/core/management/base.py", line 249, in validate
num_errors = get_validation_errors(s, app)
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/django/core/management/validation.py", line 22, in get_validation_errors
from django.db import models, connection
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/django/db/__init__.py", line 41, in <module>
backend = load_backend(settings.DATABASE_ENGINE)
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/django/db/__init__.py", line 17, in load_backend
return import_module('.base', 'django.db.backends.%s' % backend_name)
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/django/utils/importlib.py", line 35, in import_module
__import__(name)
File "/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/django/db/backends/postgresql_psycopg2/base.py", line 22, in <module>
raise ImproperlyConfigured("Error loading psycopg2 module: %s" % e)
django.core.exceptions.ImproperlyConfigured: Error loading psycopg2 module: No module named psycopg2
Please guide me. Any reply will be appreciated.
Thanks,
Wenbert!
There seems to be a problem with your psycopg2 installation – Python does not find it. This is a Python installation problem, not a Django issue.
You can try to load it manually using the Python interpreter and see if it works:
$ python
>>> import psycopg2
If you get an ImportError exception, your installation is erroneous. To get a list of all directories Python looks for modules, use sys.path:
$ python
>>> import sys
>>> print sys.path
You can also add custom directories to Python's module search path by modifying the sys.path variable. Do this somewhere before the respective import statement(s):
import sys
sys.path.append("my-path")
# ...
import psycopg2
If you have pip installed, simply install the missing extension by running:
$ pip install psycopg2
For the record I got the same error for a different reason:
I had put
'ENGINE': 'django.db.backends.postgresql'
instead of
'ENGINE': 'django.db.backends.postgresql_psycopg2'
in settings.py
Follow the steps:
Install python libraries on your OS:
python-dev
libpq-dev
Execute the command to install the psycopg2 library:
easy_install psycopg2
source: http://initd.org/psycopg/install/
I realized I didn't have psycopg2 installed
aptitude install python-psycopg2
Worked like a charm
I had this issue recently after updating homebrew on OSX.
psycopg2 was already listed in my virtualenv.
I just reinstalled psycopg2 and it worked again:
pip install --force-reinstall psycopg2
Although you installed it, Python can apparently not find the module psycopg2. This is usually due to the module not being in Python's path. See if you can find a folder named psycopg2 in /Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages. If it's not there, did MacPorts tell you where it put psycopg2? If you can locate it, just move it to the site-packages directory and you should be fine.
For me, psycopg2 was indeed installed, but not into the virtualenv in which Django was running. These two steps fixed it:
sudo apt-get build-dep python-psycopg2
sudo /opt/myenv/bin/pip install psycopg2
Yes Tim's answer works for me too.It works without prefix 'django.db.backends.' also. But remember to create database or schema you mentioned in settings.py:
DATABASE_NAME = 'your_db_name'
manually using your database client so when you run 'python manage.py syncdb' you don't get the same problem. I was stuck because i didn't create it manually. I got the same problem may be because I used buildout.
I got the same error, but it was because I was using python26 ./manage.py runserver when my virtualenv only had python and python2.6 executables (thus the system python26 was being used, which didn't have psycopg2 installed
Tim's answer worked for me too.
By default, settings.py shows options like 'postgresql_psycopg2', 'mysql', etc, without a package name. Prefixing with 'django.db.backends.' worked for me (for postgresql_psycopg2, at least).
I'm on Windows and had installed psycopg2, but the 64 bit version. So my fix was to download the 32 bit one from here then in PowerShell with my virtual environment activated, my fix was:
pip uninstall psycopg2
easy_install C:\WHEREVER_I_DOWNLOADED_IT\psycopg2-2.6.1.win32-py3.4-pg9.4.4-release.exe
(The Windows Python 3.4 installer automatically installs easy_install and pip, and pip is the easiest way to remove the package, even if originally installed using easy_install.)
THIS HAS HELPED ME:
I just added PostgreSQL bin path to ENV and it was able to fine the needed dll:
C:\Program Files (x86)\PostgreSQL\9.4\bin
https://groups.google.com/forum/#!topic/robotframework-users/qLq9KvHe1wE
same here, but maybe it was missed in current project, so I navigated to the main project directory and installed it.
pip install psycopg2
and it worked
You can get this error in heroku deployment with django postgresql as backend simply add psycopg2 to your requirements.txt file
This error also occurs when you forgot to activate your virtual environment.
Using Django 3.2.9 with Python 3.9.1, after pip3 install psycopg2, this is what I had to do in order to get the initial python manage.py migrate to work:
In my Django site's settings.py file, I added these lines to the top of the file (after the from pathlib import Path line):
import sys
sys.path.append('Users/MY_USER_NAME/Sites/SITE_NAME/env/lib/python3.9/site-packages')
Sorry... a ~/Sites/... path setting doesn't work (I tried), it needs to be an absolute, full path.
My Database settings are as follows:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': 'DATABASE_NAME',
'USER': os.getenv('PG_USER'),
'PASSWORD': os.getenv('PG_PASS'),
'PORT': '5432',
'USE_TZ': True,
}
}
Once I added the two lines to the top of the settings.py file, the migration ran successfully :)
For the module not found errors, usually a good
pip install module_name
Will solve the problem. If it's not the case look for the binary of the module you are trying to install. Install the binary then install the module again. Example for this case the module is
psycopg2
Binary is psychopg2-binary so
Install them like this
pip install psycopg2-binary
pip install psycopg2