Develop Reference

ProgrammingError when switching from Sqlite3 to Postgresql in django admin console

In django, I have attempted to switch from using an sqlite3 database to postgresql. settings.py has been switched to connect to postgres. Both python manage.py makemigrations and python manage.py migrate run without errors. makemigrations says that it creates the models for the database, however when running migrate, it says there is no changes to be made.
The django server will run, however when clicking on a specfic table in the database in the /admin webpage, it throws the error:
ProgrammingError at /admin/app/tablename/
relation "app_tablename" does not exist
LINE 1: SELECT COUNT(*) AS "__count" FROM "app_tablename"
With the same code (other than settings.py database connection) this worked when using sqlite3.

Happened same thing to me. i deleted the table from database then created the migration again using python manage.py makemigrations . Then i ran that particular migration again using python manage.py migrate myapp 00123 assuming that appname is myapp and migration name is 00123.py

What I did was create a new database and switched to it through the settings.py file. After that, migrations should happen with no problems

ImproperlyConfigured: Requested settings DATABASES

I have seen 10 different ways of configuring the MySQL database with Django, and none of them work.
I am running MySQL 8.0 and
Django 2.0
I updated the Project/settings.py with the database settings as followed:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'name_of_db',
'USER': 'root_user',
'PASSWORD': 'not_telling_u_my_pass_:)',
'HOST': 'localhost',
'PORT': '3306',
}
}
When I run the command to test the connectivity:
python manage.py dbshell
I get error:
django.core.exceptions.ImproperlyConfigured: Requested setting DATABASES, but settings are not configured. You must either define the environment variable DJANGO_SETTINGS_MODULE or call settings.configure() before accessing settings.
No other settings have been made, however, I have tried to follow a few guides which say that I must declare:
set DJANGO_SETTINGS_MODULE=mysite.settings
I have also issued:
python manage.py shell
With no luck. I am running Windows 7 as well. I have all of the mysql dependencies installed for connecting from Python to MySQL. I know this question has been asked a number of times, but a thread should exist with clear instructions. Many of the other threads do not have clear and concise instructions, and I'm using a later version of MySQL so hopefully this will help solve others issues experiencing the same.
Project Tree:
Server
|
+-- Server
| |
| +-- _init_.py
| +-- settings.py
| +-- urls.py
| +-- wsgi.py
+-- manage.py
Update 1 (10 mins after posting):
I ran the python manage.py migrate and this installed all of the tables into my database schema. However, when I run the python manage.py dbshell command again, I still get the same error above saying the settings are not configured.
Update 2:
After running:
python manage.py shell --settings=Server.settings
I get:
python manage.py dbshell
CommandError: You appear not to have the 'mysql' program installed or on your path.

The ImproperlyConfigured often masks other errors. As you note, your database configuration is correct, which means the problem is most likely one of two problems.
Some file imported during setup is causing an exception
There is an import in settings.py which is in turn trying to use a setting (i.e., settings is used while being loaded)
First, if you have any imports in your settings.py, make sure those aren't going to in turn import django.conf.settings (imports like import os) are fine.
Next, inspect files from your project that might be imported during setup (apps, urls, custom middleware, template contexts, etc.) for possible errors that might arise during import. Make sure you can import all files on their own (some may complain because they require setup to be run first, you'll just have to inspect those extra carefully, or remove references to settings so that you can test them on their own).

I met the same problem and I have fixed the problem finally. The reason is that MySql will treat different when you using the 'localhost' parameter. It will connect to the local server using a Unix socket file.
Check this document:
https://dev.mysql.com/doc/refman/5.7/en/connecting.html
So you should use '127.0.0.1' instead of 'localhost' in DATABASES.
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'name_of_db',
'USER': 'root_user',
'PASSWORD': 'not_telling_u_my_pass_:)',
'HOST': '127.0.0.1', #This string will works!
'PORT': '3306',
#If you want to use 'localhost', you should provide the path
# of the UNIX socket. You should check /etc/my.cnf(the mysql
# config file) to get this parameter.
'OPTIONS': {'unix_socket': '/tmp/mysql.sock'}
}
}
That will work in MySql 5.7, I am not test in MySql 8.0. Another suggestion: check user privileges in MySql, may be something wrong.

Changing Django development Database from the default SQLite to PostgreSQL

What are the steps I need to take to migrate from the default SQLite database to Postgres database?
I'm doing this to get my local development environment as close to my live server (which uses postrgres).
Or is there a reason why local development uses SQLite? Is it not recommended to use Postgres for local development?

You can try the following steps:
1. Install psycopg2 to configure the database:
pip install psycopg2
2. Inside the default settings.py
Change original values:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),
}
}
To:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'NAME_OF_DB',
'USER': 'DB_USER_NAME',
'PASSWORD': 'DB_PASSWORD',
'HOST': 'localhost',
'PORT': 'PORT_NUMBER',
}
}
3. Migrate the DB:
python manage.py makemigrations
python manage.py migrate
EDIT:
Thanks #robotHamster comment. Here is the method to sync the existing data:
Backup the data first:
python manage.py dumpdata > datadump.json
After changing the DB setting:
python manage.py loaddata datadump.json
Source: What's the best way to migrate a Django DB from SQLite to MySQL?

when you are changing the database you might get a UNICODEERRO:'utf-8'
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 0: invalid start byte
After wasting more than 5 days I finally got the solution .. you will never got that accurate error in internet, I figured it out by myself .
python manage.py dumpdata > datadump.json
then change the database settings.py as per your desired database and then apply the following commands ...
python manage.py makemigrations
python manage.py migrate
python manage.py loaddata datadump.json
and then if u got the error I have mentioned earlier, please follow step by step guide :
1.Install notepad ++
2.open your datadum.json file in notepad++
3.on the bottom right corner you will get the encoding will be anything else than utf-8
4.on the top bar select encoding to UTF-8
you are good to go ..then again
python manage.py load data datadump.json
I have suffered a lot for this ...so please upvote, and shares are also appreciated.
Thank you!
and for more clearance, you can watch this video:https://youtu.be/RBtEr3TXNwg

Here is a great tutorial on how to do this from Django Girls
It shows you the installation as well as the required changes in settings.py.

Hope I am not late. So to my experience if you already have data in your sqlite db, you might face some challenges because some fields in sqlite don't directly match with fields in postgres. For example datetime, and boolean fields.
I found a library that helped me to do this:
https://github.com/Hitman23/pgloader
The library does any needed conversions.

Unable to config django for mysql

I'm trying to connect Django app to MySQL on windows 10. I have installed mysqlclient as an interface between my Django app and MySQL. Whenever I try to open dbshell I'm getting error something like this:-
django.core.exceptions.ImproperlyConfigured: Requested setting USE_I18N, but settings are not configured
You must either define the environment variable DJANGO_SETTINGS_MODULE or call settings.configure() before accessing settings.
Here is my setting.py file
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'blog',
'USER':'root',
'PASSWORD':'root',
'HOST':'127.0.0.1',
'PORT':'3306'
}
}
I have tried other solutions on StackOverflow but it didn't work for me.
I also tried deleting the dbsqlite3 file from the project directory.
Versions for different elements are as follows:-
Python (3.6)
Django (1.11.4)
mysqlclient (1.3.12)
pip (9.0.1)
MySQL(5.7)

DJANGO_SETTINGS_MODULE variable is not available. run
python manage.py shell
if you linux user and run
python3 manage.py shell
as you use python 3.6

Finally, I solved this problem. I was using django-admin dbshell command instead of python manage.py dbshell
In case of windows, set env variable for MySQL in user variables section, otherwise you will get CommandError saying that mysql is not installed.
For Ubuntu use python3 manage.py dbshell

Django migration relation does not exist

So I'm trying to run the initial migrations on a django app and when I try to run the migrate command (python manage.py migrate or makemigrations) I get the following error:
psycopg2.ProgrammingError: relation "dotworks_server_internship" does not exist
LINE 1: ...s", "dotworks_server_internship"."questions" FROM "dotworks_...
^
I'm on a Windows environment using Django 1.9.6 and my database is postgres. Plus, I'm using PGAdmin to manage my database.
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': 'dotworks',
'USER': 'postgres',
'PASSWORD': 'mypasswordgoeshere',
'HOST': 'localhost',
'PORT': '5432',
}
}

I had this problem and I had to comment out everything in urls.py that referenced views.py, then run makemigrations. Hope this helps.

Make sure that you don't have any class variables in your code that are calling Django manager
For example:
class SomeViewSet(viewsets.ViewSet):
se = SomeEntity.objects.first() # fetching some entity on the class level
def list(self, request):
# the rest of the code
So, when you try to create/apply migrations, this variable will also try to initialise, and will try to access SomeEntity, but at that moment that entity doesn't even exist, and the error occurs.

If all other solutions mentioned fail, if you are still in development probably the easiest solution is dropping the database (in pgAdmin 4 2.0, right-click on database) and then run makemigrations and migrate.

Try to migrate particular app using following process. Refer Django migrations
python manage.py makemigrations
Initial migration created then run migrate command with app name
python manage.py migrate appname1, appname2

When you run a query before applying migrations this error appears.
If you got this error during python manage.py makemigrations or python manage.py migrate you must consider that makemigrations and migrate commands run after successful django bootstrap! So this error happens when you run a query during django bootstrap! So you must find the place you run this query during bootstrap progress.
For example, during bootstrap, django reads root {project}/urls.py and its nested imports. If you use views or viewsets in urls.py and they are run a query during initializing (in their __init__ method or __init__.pyfile or somewhere etc.), it happens!
In this situation and similars, you must comment out any entry in urls.py and similar files which cause running a query during bootstrap and prevent them from running by raising of exception during bootstrap! makemigrations and migrate need successful bootstrap to be run!
If your commented out code needs to makemigrations and migrate handcooks :D, it needs to be patient and be silent for a cycle or a while ;), and after a successful migrations it could be active and verbose ;D.

Your app is trying to call some DB entries that does not exist.
If you are trying to migrate it to a new database, one of your options is to export a dump of old database and import it to your new DB.
For example in PostgreSQL, import the database using below command then migration will work!
sudo -u postgres -i psql mydb < mydb-export.sql

If you're running in local,
For each Django app (maybe you have only one), erase the content of the migrations folder.
Then, run python manage.py makemigrations app1 app2 app3 (if you have 3 Django apps named app1, app2, app3). This will (re)create the migrations files required to migrate your database
Then, run python manage.py migrate. It will apply the migration files you just created.

This error may have related to previous database error.so if you created new database and you also face that type of error ,you can simply run the command with the app name:
1)python manage.py makemigrations <"app name">
2)python manage.py migrate <"app name">

I've solved this error with this solution.
first remove all url in urls.py .
create simple function view for viewing nothing.
def simple(request):
context = {}
return render(request, 'base.html', context)
and add url to urs.py
do migrate
python manage.py migrate
after migrate,
recover the deleted urls.py contents
:)

For me the error came from some initialization code I put into the app.ready() method. Commenting that part of code allowed me to run the command makemigrations without any issue.
I believe app.ready is called at some point by manage.py even for the makemigrations command, which caused my code to query my database before any migration.
I found the problematic code thanks to the traceback.