I am trying to python manage.py makemigrations for a django app in postgres, but I am getting the follow error:
django.db.utils.OperationalError: connection to server at "localhost" (::1), port 5432 failed: FATAL: database "backend_db" does not exist
Before this, I am doing docker compose up with the following docker-compose and .env file:
version: '3.2'
services:
postgres:
image: postgres:13.4
environment:
POSTGRES_DB: backend_db
POSTGRES_USER: postgres
POSTGRES_PASSWORD: postgres
volumes:
- database-data:/var/lib/postgresql/data/
ports:
- 5432:5432
networks:
- postgres
volumes:
database-data:
driver: local
networks:
postgres:
driver: bridge
DB_NAME='backend_db'
DB_USER='postgres'
DB_PASSWORD='postgres'
# DB_HOST is localhost or the IP of the machine running postgres
DB_HOST='localhost'
DB_PORT='5432'
The part of the settings.py that I define the postgres is the following:
DATABASES = {
'default': get_config(
'DATABASE_URL',
'sqlite:///' + BASE_DIR.child('db.sqlite3'),
cast=db_url
),
'postgres': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': get_config('DB_NAME', 'backend_db'),
'USER': get_config('DB_USER', 'postgres'),
'PASSWORD': get_config('DB_PASSWORD', 'postgres'),
'HOST': get_config('DB_HOST', 'postgres-service'),
'PORT': get_config('DB_PORT', '5432')
}
}
Any idea of what causes the error?
I believe you can use:
docker-compose run web python manage.py makemigrations
You may need to change web to your appropriate service.
I encountered the same problem, and it turns out that in my case, there was a PostgreSQL server (via Postbird) running on my local machine and the python server was inadvertently connecting to it instead of to the one in the Docker container. Context: I normally run my python app natively (i.e., locally with python manage.py runserver) and database services in a Docker container, connected to via port forwarding.
The solution was to kill the local Postgres instance and let the python server connect to the Docker instance
Related
This question already has answers here:
django cannot connect mysql in docker-compose
(3 answers)
How to share localhost between two different Docker containers?
(2 answers)
Closed 8 months ago.
I built 2 containers, one for project with python-3.4.10 and another for MySQL (with mysql-5.7-debian).
When I try to run any command for example:
python manage.py makemigrations {name} or python manage.py runserver 0.0.0.0:8000 it gives me the following error:
django.db.utils.OperationalError: (2003, 'Can\'t connect to MySQL server on \'127.0.0.1\' (111 "Connection refused")')
I have literally tried everything and went through each stack issues related to it. But the issue still persists.
When I connect on MySQL Workbench (which is installed on my base OS: Windows 11) with the running MySQL container, it gets connected but when I run the command of python manage.py ... on the different container it shows me the above error.
Could you please help me out here? Thank you.
settings.py
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 's_db',
'USER': 'root',
'PASSWORD': 'root',
'HOST': '127.0.0.1',
'PORT': '3306',
}
}
docker-compose.yaml
version: '3.8'
services:
web:
image: s-compose:latest
ports:
- 8000:8000
command: >
sh -c "python manage.py runserver 0.0.0.0:8000"
volumes:
- ./s:/home/app/webapp
depends_on:
- database
database:
image: mysql:5.7-debian
ports:
- 3306:3306
environment:
MYSQL_ROOT_PASSWORD: root
.Dockerfile
FROM python:3.4.10
ENV Dockerhome=/home/app/webapp
RUN mkdir -p $Dockerhome
WORKDIR $Dockerhome
COPY ./sansthaonline $Dockerhome
RUN pip install -r req.txt; exit 0
EXPOSE 8000
## testing server
CMD python manage.py runserver 0.0.0.0:8000
django.db.utils.OperationalError: could not translate host name "db" to address: Name or service not known
That is the error I get when running docker-compose exec web python manage.py migrate
my docker-compose.yml contains:
version: '3.8'
services:
web:
build: .
command: python /code/manage.py runserver 0.0.0.0:8000
volumes:
- .:/code
ports:
- 8000:8000
depends_on:
- db
db:
image: postgres:11
This Is what I put for DATABASE in settings.py:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': 'postgres',
'USER': 'postgres',
'PASSWORD': 'postgres',
'HOST': 'db',
'PORT': 5432
}
}
I have tried running docker-compose up -d --build and then docker-compose exec web python manage.py migrate
But that doesn't work.
You will need to add an environment for your db service:
services:
# ...
db:
# ...
environment:
POSTGRES_PASSWORD: postgres
as POSTGRES_PASSWORD is the only non-optional env variable needed to run the image as stated in the docs.
I'm trying to start the server through docker-compose up
I'm get an error:
2002, "Can't connect to local MySQL server through socket '/run/mysqld/mysqld.sock' (2)"
docker-compose.yml
version: '3'
services:
db:
image: mysql:5.7
environment:
MYSQL_DATABASE: 'slack_bot1'
MYSQL_USER: 'root'
MYSQL_PASSWORD: ''
MYSQL_ROOT_PASSWORD: '****'
volumes:
- /opt/slack_bot/mysql_data:/var/lib/mysql
redis:
image: "redis:alpine"
volumes:
- /opt/slack_bot/redis_data:/var/lib/redis
web:
build: .
command: python manage.py runserver 0.0.0.0:8001
ports:
- "8001:8001"
depends_on:
- db
Dockerfile
FROM python:3.7-alpine
ENV PYTHONUNBUFFERED 1
WORKDIR /home/slack_bot
ADD requirements.txt /home/slack_bot/
RUN set -e; \
apk add --no-cache --virtual .build-deps \
gcc \
libc-dev \
linux-headers \
mariadb-dev \
python3-dev \
;
RUN pip install -r requirements.txt
ADD . /home/slack_bot/
EXPOSE 8001
CMD ["python", "manage.py", "runserver", "0.0.0.0:8001"]
docker ps log
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
68b61ca0ce74 slack_bot_web "python manage.py ru…" 8 minutes ago Up 32 seconds 0.0.0.0:8001->8001/tcp slack_bot_web_1
c5f254a527b0 mysql:5.7 "docker-entrypoint.s…" 8 minutes ago Up 34 seconds 3306/tcp, 33060/tcp slack_bot_db_1
4cbc1fa3765e redis:alpine "docker-entrypoint.s…" 15 minutes ago Up 33 seconds 6379/tcp slack_bot_redis_1
Django settings of database
'ENGINE': 'django.db.backends.mysql',
'NAME': 'slack_bot1',
'USER': 'root',
'PASSWORD': '',
'HOST': '',
'PORT': '',
In your django settings, you leave database host as EMPTY, then the default value would be localhost. When use localhost, mysql client driver will connect mysql server with unix socket not tcp.
So, for your case you need to export unix socket in your mysql container to volume, then your django app container utilize this volume to share the unix socket.
Next is what you needed to do:
Manual new a folder which later as volume to share your unix socket file
(IMPORTANT: you could not rely on docker-compose to new this folder, you will encountered permission error)
mkdir -p /tmp/slack_bot/mysqld && chmod -R 777 /tmp/slack_bot/mysqld
In your docker-compose.yaml, add one more volume to mysql service:
db:
image: mysql:5.7
environment:
MYSQL_DATABASE: 'slack_bot1'
MYSQL_USER: 'root'
MYSQL_PASSWORD: ''
MYSQL_ROOT_PASSWORD: '****'
volumes:
- /opt/slack_bot/mysql_data:/var/lib/mysql
- /tmp/slack_bot/mysqld:/var/run/mysqld
In your docker-compose.yaml, add one more volume to web service:
web:
build: .
command: python manage.py runserver 0.0.0.0:8001
ports:
- "8001:8001"
depends_on:
- db
volumes:
- /tmp/slack_bot/mysqld:/run/mysqld
NOTE: here you say you get the error /run/mysqld/mysqld.sock as next, I'm not sure if you paste the all log as most offen it could be /var/run/mysqld/mysqld.sock, nertheless, if the error is /var/run/mysqld/mysqld.sock then you should modify above example volume to - /tmp/slack_bot/mysqld:/var/run/mysqld
2002, "Can't connect to local MySQL server through socket '/run/mysqld/mysqld.sock' (2)"
With above, now your web service could share the unix socket in mysql container.
Additional, configure HOST as db I think also could be another solution, just unix socket is much faster compared to use tcp.
I was able to connect django and db docker containers which are in the same machine but aren't in the same docker-compose file by the following steps:
Use host.docker.internal as host in settings.py instead of localhost as following:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'dbName',
'USER': 'dbUser',
'PASSWORD': 'dbUserPassword',
'HOST': 'host.docker.internal',
'PORT': '3306'
}
}
Expose the port in db compose-file
ports:
- 3306:3306
I'm learning how to work with Compose + Django using this manual https://docs.docker.com/compose/django/
Here are my configuration files
docker-compose.yml
version: '3'
services:
db:
image: postgres
web:
build: .
command: python3 manage.py runserver 0.0.0.0:8000
volumes:
- .:/code
ports:
- "8000:8000"
depends_on:
- db
setting.py
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': 'postgres',
'USER': 'postgres',
'HOST': 'db',
'PORT': 5432,
}
}
When I run using docker-compose up everything is fine.
But when I run using python manage.py runserver I got this error
psycopg2.OperationalError: could not translate host name "db" to address: Name or service not known
Guess, I have wrong cofiguration
You forgot set hostname in db service.
version: '3'
services:
db:
image: postgres
hostname: db
web:
build: .
command: python3 manage.py runserver 0.0.0.0:8000
volumes:
- .:/code
ports:
- "8000:8000"
depends_on:
- db
Tip: Don't use runserver when you deploy production server. Check this doc
The solution is described in comments
Edit)
separate settings example. Make your original settings.py to base.py in settings folder.
<your_app>/settings/local.py
from .base import *
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': 'postgres',
'USER': 'postgres',
'HOST': 'localhost',
'PORT': 5432,
}
}
<your_app>/settings/deploy.py
from .base import *
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': 'postgres',
'USER': 'postgres',
'HOST': 'db',
'PORT': 5432,
}
}
So finally, your settings folder contains three python file. base.py, local.py, deploy.py
And when you run server with local settings, python manage.py runserver --settings <your_app>.settings.local.
This is a common problem in docker/django setup , postgres container is initialized but the engine is not ready yet.
You must wait for the postgresql server to start before running the web container.
You can do this in several ways , have a look at dockerize
dockerize -timeout=20s -wait ${POSTGRES_PORT}
I guess you trying to run manage.py runserver on host machine. Error means, that you have no access to host 'db' from it (try ping db in console). There are several causes of this:
Your docker container db is down. Try run docker-compose up -d db
You are using Windows as host system. In Windows there is no access to containers from host, you must install postgres locally.
You have networking problems. Try ping 127.0.0.11 (it's docker's DNS server)
P.S. Do you really want local Django use Postgres in container?
I'm setting up an application in Docker that uses Django and Postgres. I'm unable to connect to Postgres from the app. When I run either:
docker-compose run web python manage.py runserver
docker-compose run web python manage.py migrate
I get the following error:
django.db.utils.OperationalError: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
Here's my Dockerfile:
FROM python:2.7
ADD requirements.txt /app/requirements.txt
WORKDIR /app/
RUN pip install -r requirements.txt
My docker-compose.yml:
version: '2'
services:
db:
image: postgres:9.4
hostname: db
environment:
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
- POSTGRES_DB=postgres
ports:
- "5432:5432"
web:
build:
context: .
dockerfile: Dockerfile
hostname: web
volumes:
- .:/app
ports:
- "8000:8000"
links:
- db
depends_on:
- db
command:
python manage.py runserver 0.0.0.0:8000
and the database settings in Django:
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'postgres',
'USER': 'postgres',
'HOST': 'db',
'PORT': 5432,
}
}
docker ps -a shows:
e9df7e1644ce web "python manage.py ..." About a minute ago Up About a minute 0.0.0.0:8000->8000/tcp web_1
60801d3256e4 postgres:9.4 "docker-entrypoint..." About a minute ago Up About a minute 0.0.0.0:5432->5432/tcp db_1
Django v1.10.5
Docker for Mac v1.13.1
docker-compose v 1.11.1
Here are the logs:
web: https://gist.github.com/steven-mercatante/912ca964fd05bee2c835b1ac5586c154
db: https://gist.github.com/steven-mercatante/4c9a03bfc4f2e90f4b9931bdc185652f
Turns out the issue was that Postgres hadn't fully spun up before starting the Django app. This was easily solved by waiting a few seconds before starting the web server. I set the command for my web service to ./bin/boot.sh, and the contents of that file are:
#!/bin/sh
# wait for Postgres to start
sleep 10
python manage.py migrate
python manage.py runserver 0.0.0.0:8000
Edit
This method might not be good enough depending on your application - sleeping for 10 seconds works for me, most of the time, but it's possible that postgres will take 11 seconds to spin up. Here's an improved bash file, with a function I took from this example.
#!/bin/bash
# wait for Postgres to start
function postgres_ready(){
python << END
import sys
import psycopg2
try:
conn = psycopg2.connect(dbname="postgres", user="postgres", password="postgres", host="postgres")
except psycopg2.OperationalError:
sys.exit(-1)
sys.exit(0)
END
}
until postgres_ready; do
>&2 echo "Postgres is unavailable - sleeping"
sleep 1
done
python manage.py migrate
gunicorn pft.wsgi:application -w 2 -b 0.0.0.0:8000 --reload
I had this problem and it was only because I was using a VPN. If you are using something like sshuttle just turn it off.