Develop Reference

How to dump and restore correctly a postgresql db from docker

I stuck with this error when trying to backup and restore my database from a docker django app environment :
I first did this command to backup my whole DB
docker exec -t project_final-db-1 pg_dumpall -c -U fred2020 > ./db/dump.sql
And then trying to restory with this command
cat dump.sql | docker exec -i --user fred2020 catsitting-db-1 psql -U fred2020 -d postgres
I have two containers, one for my django app named catsitting-web-1 and one for my postgresql named catsitting-db-1.
I don't understand why it gaves me that error, my db user is the same that I specified on the Dockerfile.
Any clue ?
For purpose help, here is my docker files configuration :
Dockerfile
FROM python:3.9
ENV PYTHONUNBUFFERED=1
WORKDIR /code
COPY requirements.txt /code/
RUN pip install -r requirements.txt
RUN pip install Pillow
COPY . /code/
docker-compose.yml
version: "3.9"
services:
db:
image: postgres
environment:
- POSTGRES_DB=postgres
- POSTGRES_USER=fred2020
- POSTGRES_PASSWORD=p*******DD
expose:
- "5432"
ports:
- 5432:5432
web:
build: .
command: python manage.py runserver 0.0.0.0:8000
volumes:
- .:/code
ports:
- "8000:8000"
depends_on:
- db
requirements.txt
Django>=3.0,<4.0
psycopg2-binary>=2.8
Pillow==8.1.0
And that's my process to migrate from laptop1 to laptop2 :
Installation
Run a command line go into a root directory and run:
git clone https://github.com/XXXXXXXXXXXXXXXX
In the command line go into the root directory:
cd catsitting
In the same command line window, run:
docker-compose build --no-cache
In the command line window you need first to migrate the database for Django, run :
docker-compose run web python manage.py migrate
In the command line window then you need to apply the migrations, run :
docker-compose run web python manage.py makemigrations
In the command line window then you need to import database, run :
cat dump.sql | docker exec -i --user fred2020 catsitting-db-1 psql -U fred2020 -d postgres
(for dumping my DB I used docker exec -t project_final-db-1 pg_dumpall -c -U fred2020 > ./db/dump.sql)
You can now run:
docker-compose up
Is there something I get wrong ?

I solved !
It was a problem in misconfiguration in the pg_hba.conf inside my docker postgresql
I changed the value from scram-sha-256 to md5 and it works now I can display my webapp with the current db !!
Do you know how to specifie md5 when I build my docker environnement ? by default it puts scram-sha-256

Do you know why when I restore my dump in the new environnement by default in the container the pg_hba.conf set the authentification methode to scram-sha-256 and to do my connection working I need to edit that file and to put the authentification method set to md5 ?
# TYPE DATABASE USER ADDRESS METHOD
local all all md5

Ok sorry folks I found the solution.
I've put that line in my docker-compose.yml:
environment:
- POSTGRES_HOST_AUTH_METHOD=trust

Django app while dockerizing gives "Starting development server at http://0.0.0.0:8000/" but doesn't show up on browser

So I am a beginner in docker and Django. What I have here is a django app which I am trying to dockerize and run. My requirements.txt has only django and gunicorn as the packages.
I am getting the below in terminal after building and running the docker image:
Watching for file changes with StatReloader
Performing system checks...
System check identified no issues (0 silenced).
August 26, 2021 - 06:57:22
Django version 3.2.6, using settings 'myproject.settings'
Starting development server at http://0.0.0.0:8000/
Quit the server with CONTROL-C.
Below is my Dockerfile:
FROM python:3.6-slim
ENV PYTHONUNBUFFERED=1
RUN mkdir /Django
WORKDIR /Django
ADD . /Django
RUN pip install -r requirements.txt
EXPOSE 8000
CMD python manage.py runserver 0.0.0.0:8000
The commands I am using are:
docker build . -t myproj
docker run -p 8000:8000 myproj
I have tried adding allowedhosts = ['127.0.0.1'] in settings.py but still I am getting "The site can't be reached. 127.0.0.1 refused to connect.
Not able to see the "Congratulations" screen.
Please help me out with this.
P.s: I am using windows machine
Updates
I tried running the below line and got the following output:
docker exec 8e6c4e4a58db curl 127.0.0.1:8000
OCI runtime exec failed: exec failed: container_linux.go:349: starting container process caused "exec: \"curl\": executable file not found in $PATH": unknown

Without your settings.py, this can be hard to figure out. You say you have allowedhosts = ['127.0.0.1'] in there and that should definitely not be necessary. It might actually be what's blocking your host, since requests from your host come from a different IP address.
I've made the following Dockerfile that creates a starter project and runs it
FROM python:latest
RUN python -m pip install Django
RUN django-admin startproject mysite
WORKDIR /mysite
EXPOSE 8000
CMD python manage.py runserver 0.0.0.0:8000
If I build it and run it with
docker build -t mysite .
docker run -d -p 8000:8000 mysite
I can connect to http://localhost:8000/ on my machine and get the default page (I'm on Windows too).
I hope that helps you to locate your issue.
PS: Your curl command fails because curl isn't installed in your image, so it failing has nothing to do with your issue.

Docker compose executable file not found in $PATH": unknown

but I'm having a problem.
Dockerfile:
FROM python:3
ENV PYTHONUNBUFFERED 0
RUN mkdir /code
WORKDIR /code
COPY requirements.txt /code/
RUN pip install -r requirements.txt
COPY . /code/
compose.yml :
version: '3'
services:
db:
image: postgres
volumes:
- ./docker/data:/var/lib/postgresql/data
environment:
- POSTGRES_DB=sampledb
- POSTGRES_USER=sampleuser
- POSTGRES_PASSWORD=samplesecret
- POSTGRES_INITDB_ARGS=--encoding=UTF-8
django:
build: .
environment:
- DJANGO_DEBUG=True
- DJANGO_DB_HOST=db
- DJANGO_DB_PORT=5432
- DJANGO_DB_NAME=sampledb
- DJANGO_DB_USERNAME=sampleuser
- DJANGO_DB_PASSWORD=samplesecret
- DJANGO_SECRET_KEY=dev_secret_key
ports:
- "8000:8000"
command:
- python3 manage.py runserver
volumes:
- .:/code
error :
ERROR: for django Cannot start service django: OCI runtime create failed: container_linux.go:346: starting container process caused "exec: \"python3 manage.py runserver\": executable file not found in $PATH": unknown
At first, I thought Python Manage was wrong.
But i tried command ls , To my surprise, I succeeded.
Then I tried the ls -al command, but it failed.
I think the addition of a command to write space is causing a problem.
how can i fix it ?

When you use list syntax in the docker-compose.yml file, each item is taken as a word. You're running the shell equivalent of
'python3 manage.py runserver'
You can either break this up into separate words yourself
command:
- python3
- manage.py
- runserver
or have Docker Compose do it for you
command: python3 manage.py runserver
In general fixed properties of the image like this should be specified in the Dockerfile, not in the docker-compose.yml. Every time you run this image you're going to want to run this same command, and you're going to want to run the code built into the image. There are two syntaxes, with the same basic difference:
# Explicitly write out the words
CMD ["python3", "manage.py", "runserver"]
# Docker wraps in sh -c '...' which splits words for you
CMD python3 manage.py runserver
With the code built into the image and a reasonable default command defined there, you can delete the volumes: and command: from your docker-compose.yml file.

Docker compose - can't find manage.py file

I want to create with docker-compose 2 Docker containers. 1 for DB (Mongo) and 1 for web (Django).
Here are my files
docker-compose.yml
version: '3'
services:
db:
image: mongo
command: mongod
web:
build: code/
command: python manage.py runserver 0.0.0.0:8000
volumes:
- .:/code
ports:
- "8000:8000"
depends_on:
- db
code/Dockerfile
FROM python:3
ENV PYTHONUNBUFFERED 1
RUN mkdir /code
WORKDIR /code
COPY requirements.txt /code/
RUN pip install -r requirements.txt
COPY cleverInvestWeb/ /code/
In the directory code/cleverInvestWeb/ is my manage.py file from django.
when i run docker-compose up it throws me the following error:
web_1 | python: can't open file 'manage.py': [Errno 2] No such file or directory
When i start the container via Docker itself with docker exec -it dockerdiplom_web bash and do an ls there it shows the manage.py file.
Do you have an idea why docker-compose doesn't find my file when starting?

The directory structure in your Dockerfile and docker-compose seems confusing. Another thing that is strange is you are able to see to file after docker exec.
You copy your code to COPY cleverInvestWeb/ /code/ in Dockerfile, and then mount the volume in Docker-compose to .:/code so everything will be overide in the existing image on /code location.
I assume your python file place inside local directory cleverInvestWeb so docker-compose boot up it will override the existing image code and your file update location will be code/cleverInvestWeb/manage.py or /code/manage.py
To debug:
remove mounting in docker-compose and check if it works
command: tail -f /dev/null set this command in docker-compose and verify your file location using docker exec

change your command to
command: python code/manage.py runserver 0.0.0.0:8000

This is a common error among the Django beginners
The docker file 'docker-compose.yml' configuration (shown at the beginning of this request) is alrigh... Except for the django project folder missing when calling the manage.py file!
The Line should have been:
command: python nameOfDjangoFolder/manage.py runserver 0.0.0.0:8000
Explaination about Volumes named "/code"
The Volume "/code" is building your application using the root of the Docker project folder:
/home/user/Documents/youDockerProjectFolder = (Code)
Inside the "/home/user/Documents/youDockerProjectFolder" you should find the Django project folder: yourDjangoProjectFolder/
Inside that folder you will find the manage.py... So it's normal to have to use yourDjangoProjectFolder/manage.py in order to call manage.py
MORE but not important for this solution
Inside the Docker Project Folder another important file is found
yourDjangoProjectFolder/youDjangoProjectFolder/settings.py

django runserver hangs in docker-compose up but runs correctly in docker-compose run

Edit
Adding --ipv6 to the command, while not properly configured for, seem to surpass the point where the process hangs.
Problem
Calling docker-compose up executes runserver but hangs at some point after printing the current time.
Calling docker-compose run -p 8000:8000 web python manage.py runserver 0.0.0.0:8000 also execute the server, but does so succesfully and can be reached at 192.168.99.100:8000.
Questions
How come I can run the server directly from docker-compose in my shell but not from the .yml file?
To me, the content of the .yml file and the docker-compose run line from the shell are strikingly similar.
The only difference I can think of would perhaps be permissions at some level required to properly start a django server, but I don't know how to address that. Docker runs on a windows 8.1 machine. The shared folder for my virtual machine is the default c:\Users.
Files
My folder contain a fresh django project as well as these docker files. I've tampered with different versions of python and django but the result is the same. I've cleaned up my images and containers between attempts using
docker rm $(docker ps -a -q)
docker rmi $(docker images -q)
docker-compose.yml
version: '3'
services:
web:
build: .
command: python manage.py runserver 0.0.0.0:8000
volumes:
- .:/code
ports:
- "8000:8000"
Dockerfile
FROM python:3.6-alpine
ENV PYTHONUNBUFFERED 1
RUN mkdir /code
WORKDIR /code
ADD requirements.txt /code/
RUN pip install -r requirements.txt
ADD . /code/
requirements.txt
Django>=1.8,<2.0
System
My operative system is windows 8.1

I was hit by this issue myself and it seems that you need to allocate a tty and a stdin to your container in order to make runserver work:
python:
image: my-image:latest
stdin_open: true # docker run -i
tty: true # docker run -t
build:
context: ..
dockerfile: docker/Dockerfile

I had the same issue and could not get it to do anything else. However when i went to the ip of the docker machine docker-machine ip it returned 192.168.99.100, then by going to 192.168.99.100:8000 my docker container started receiving the requests