Develop Reference

How to dump and restore correctly a postgresql db from docker

I stuck with this error when trying to backup and restore my database from a docker django app environment :
I first did this command to backup my whole DB
docker exec -t project_final-db-1 pg_dumpall -c -U fred2020 > ./db/dump.sql
And then trying to restory with this command
cat dump.sql | docker exec -i --user fred2020 catsitting-db-1 psql -U fred2020 -d postgres
I have two containers, one for my django app named catsitting-web-1 and one for my postgresql named catsitting-db-1.
I don't understand why it gaves me that error, my db user is the same that I specified on the Dockerfile.
Any clue ?
For purpose help, here is my docker files configuration :
Dockerfile
FROM python:3.9
ENV PYTHONUNBUFFERED=1
WORKDIR /code
COPY requirements.txt /code/
RUN pip install -r requirements.txt
RUN pip install Pillow
COPY . /code/
docker-compose.yml
version: "3.9"
services:
db:
image: postgres
environment:
- POSTGRES_DB=postgres
- POSTGRES_USER=fred2020
- POSTGRES_PASSWORD=p*******DD
expose:
- "5432"
ports:
- 5432:5432
web:
build: .
command: python manage.py runserver 0.0.0.0:8000
volumes:
- .:/code
ports:
- "8000:8000"
depends_on:
- db
requirements.txt
Django>=3.0,<4.0
psycopg2-binary>=2.8
Pillow==8.1.0
And that's my process to migrate from laptop1 to laptop2 :
Installation
Run a command line go into a root directory and run:
git clone https://github.com/XXXXXXXXXXXXXXXX
In the command line go into the root directory:
cd catsitting
In the same command line window, run:
docker-compose build --no-cache
In the command line window you need first to migrate the database for Django, run :
docker-compose run web python manage.py migrate
In the command line window then you need to apply the migrations, run :
docker-compose run web python manage.py makemigrations
In the command line window then you need to import database, run :
cat dump.sql | docker exec -i --user fred2020 catsitting-db-1 psql -U fred2020 -d postgres
(for dumping my DB I used docker exec -t project_final-db-1 pg_dumpall -c -U fred2020 > ./db/dump.sql)
You can now run:
docker-compose up
Is there something I get wrong ?

I solved !
It was a problem in misconfiguration in the pg_hba.conf inside my docker postgresql
I changed the value from scram-sha-256 to md5 and it works now I can display my webapp with the current db !!
Do you know how to specifie md5 when I build my docker environnement ? by default it puts scram-sha-256

Do you know why when I restore my dump in the new environnement by default in the container the pg_hba.conf set the authentification methode to scram-sha-256 and to do my connection working I need to edit that file and to put the authentification method set to md5 ?
# TYPE DATABASE USER ADDRESS METHOD
local all all md5

Ok sorry folks I found the solution.
I've put that line in my docker-compose.yml:
environment:
- POSTGRES_HOST_AUTH_METHOD=trust

Setting up a dockerized python server on local machine gives Session data corrupted

I'm trying to set up a dockerized Python server named Bullet Train on my local machine:
It has 3 components:
A Postgres database
A Python server
A React frontend
All of these 3 need to work together to get the server up and running, so this is the docker-compose file which sits at the top level of both the frontend and api-server:
version: '3'
services:
db:
image: postgres
environment:
POSTGRES_PASSWORD: password
POSTGRES_DB: bullettrain
ports:
- "5432:5432"
api:
build:
context: ./bullet-train-api
dockerfile: docker/Dockerfile
command: bash -c "pipenv run python manage.py migrate --noinput
&& pipenv run python manage.py collectstatic --noinput
&& pipenv run gunicorn --bind 0.0.0.0:8000 -w 3 app.wsgi
&& pipenv run python src/manage.py createsuperuser"
environment:
DJANGO_DB_NAME: bullettrain
DJANGO_DB_USER: postgres
DJANGO_DB_PASSWORD: password
DJANGO_DB_PORT: 5432
DJANGO_ALLOWED_HOSTS: localhost
ports:
- "8000:8000"
depends_on:
- db
links:
- db:db
frontend:
build:
context: ./bullet-train-frontend
dockerfile: Dockerfile
ports:
- "8080:8080"
This way, all the 3 components run in parallel. So far so good! Now to initialize it, I run the createsuperuser as stated here by following these steps:
docker exec -it research_api_1 bash ## go to the context of the API server terminal
run python manage.py createsuperuser ## run the createsuperuser command
The command runs successfully and I get this output:
To confirm, I went to the database:
docker exec -it research_db_1 bash ## go to the database instance
psql bullettrain postgres ## connect to the bullettrain database
select * from public.users_ffadminuser; ## check if the super user is created
The results show that the user is indeed created:
Now, if I go to the admin panel as per the docs, nothing happens and the server logs always throw Session data corrupted:

Docker compose executable file not found in $PATH": unknown

but I'm having a problem.
Dockerfile:
FROM python:3
ENV PYTHONUNBUFFERED 0
RUN mkdir /code
WORKDIR /code
COPY requirements.txt /code/
RUN pip install -r requirements.txt
COPY . /code/
compose.yml :
version: '3'
services:
db:
image: postgres
volumes:
- ./docker/data:/var/lib/postgresql/data
environment:
- POSTGRES_DB=sampledb
- POSTGRES_USER=sampleuser
- POSTGRES_PASSWORD=samplesecret
- POSTGRES_INITDB_ARGS=--encoding=UTF-8
django:
build: .
environment:
- DJANGO_DEBUG=True
- DJANGO_DB_HOST=db
- DJANGO_DB_PORT=5432
- DJANGO_DB_NAME=sampledb
- DJANGO_DB_USERNAME=sampleuser
- DJANGO_DB_PASSWORD=samplesecret
- DJANGO_SECRET_KEY=dev_secret_key
ports:
- "8000:8000"
command:
- python3 manage.py runserver
volumes:
- .:/code
error :
ERROR: for django Cannot start service django: OCI runtime create failed: container_linux.go:346: starting container process caused "exec: \"python3 manage.py runserver\": executable file not found in $PATH": unknown
At first, I thought Python Manage was wrong.
But i tried command ls , To my surprise, I succeeded.
Then I tried the ls -al command, but it failed.
I think the addition of a command to write space is causing a problem.
how can i fix it ?

When you use list syntax in the docker-compose.yml file, each item is taken as a word. You're running the shell equivalent of
'python3 manage.py runserver'
You can either break this up into separate words yourself
command:
- python3
- manage.py
- runserver
or have Docker Compose do it for you
command: python3 manage.py runserver
In general fixed properties of the image like this should be specified in the Dockerfile, not in the docker-compose.yml. Every time you run this image you're going to want to run this same command, and you're going to want to run the code built into the image. There are two syntaxes, with the same basic difference:
# Explicitly write out the words
CMD ["python3", "manage.py", "runserver"]
# Docker wraps in sh -c '...' which splits words for you
CMD python3 manage.py runserver
With the code built into the image and a reasonable default command defined there, you can delete the volumes: and command: from your docker-compose.yml file.

Docker compose - can't find manage.py file

I want to create with docker-compose 2 Docker containers. 1 for DB (Mongo) and 1 for web (Django).
Here are my files
docker-compose.yml
version: '3'
services:
db:
image: mongo
command: mongod
web:
build: code/
command: python manage.py runserver 0.0.0.0:8000
volumes:
- .:/code
ports:
- "8000:8000"
depends_on:
- db
code/Dockerfile
FROM python:3
ENV PYTHONUNBUFFERED 1
RUN mkdir /code
WORKDIR /code
COPY requirements.txt /code/
RUN pip install -r requirements.txt
COPY cleverInvestWeb/ /code/
In the directory code/cleverInvestWeb/ is my manage.py file from django.
when i run docker-compose up it throws me the following error:
web_1 | python: can't open file 'manage.py': [Errno 2] No such file or directory
When i start the container via Docker itself with docker exec -it dockerdiplom_web bash and do an ls there it shows the manage.py file.
Do you have an idea why docker-compose doesn't find my file when starting?

The directory structure in your Dockerfile and docker-compose seems confusing. Another thing that is strange is you are able to see to file after docker exec.
You copy your code to COPY cleverInvestWeb/ /code/ in Dockerfile, and then mount the volume in Docker-compose to .:/code so everything will be overide in the existing image on /code location.
I assume your python file place inside local directory cleverInvestWeb so docker-compose boot up it will override the existing image code and your file update location will be code/cleverInvestWeb/manage.py or /code/manage.py
To debug:
remove mounting in docker-compose and check if it works
command: tail -f /dev/null set this command in docker-compose and verify your file location using docker exec

change your command to
command: python code/manage.py runserver 0.0.0.0:8000

This is a common error among the Django beginners
The docker file 'docker-compose.yml' configuration (shown at the beginning of this request) is alrigh... Except for the django project folder missing when calling the manage.py file!
The Line should have been:
command: python nameOfDjangoFolder/manage.py runserver 0.0.0.0:8000
Explaination about Volumes named "/code"
The Volume "/code" is building your application using the root of the Docker project folder:
/home/user/Documents/youDockerProjectFolder = (Code)
Inside the "/home/user/Documents/youDockerProjectFolder" you should find the Django project folder: yourDjangoProjectFolder/
Inside that folder you will find the manage.py... So it's normal to have to use yourDjangoProjectFolder/manage.py in order to call manage.py
MORE but not important for this solution
Inside the Docker Project Folder another important file is found
yourDjangoProjectFolder/youDjangoProjectFolder/settings.py

django runserver hangs in docker-compose up but runs correctly in docker-compose run

Edit
Adding --ipv6 to the command, while not properly configured for, seem to surpass the point where the process hangs.
Problem
Calling docker-compose up executes runserver but hangs at some point after printing the current time.
Calling docker-compose run -p 8000:8000 web python manage.py runserver 0.0.0.0:8000 also execute the server, but does so succesfully and can be reached at 192.168.99.100:8000.
Questions
How come I can run the server directly from docker-compose in my shell but not from the .yml file?
To me, the content of the .yml file and the docker-compose run line from the shell are strikingly similar.
The only difference I can think of would perhaps be permissions at some level required to properly start a django server, but I don't know how to address that. Docker runs on a windows 8.1 machine. The shared folder for my virtual machine is the default c:\Users.
Files
My folder contain a fresh django project as well as these docker files. I've tampered with different versions of python and django but the result is the same. I've cleaned up my images and containers between attempts using
docker rm $(docker ps -a -q)
docker rmi $(docker images -q)
docker-compose.yml
version: '3'
services:
web:
build: .
command: python manage.py runserver 0.0.0.0:8000
volumes:
- .:/code
ports:
- "8000:8000"
Dockerfile
FROM python:3.6-alpine
ENV PYTHONUNBUFFERED 1
RUN mkdir /code
WORKDIR /code
ADD requirements.txt /code/
RUN pip install -r requirements.txt
ADD . /code/
requirements.txt
Django>=1.8,<2.0
System
My operative system is windows 8.1

I was hit by this issue myself and it seems that you need to allocate a tty and a stdin to your container in order to make runserver work:
python:
image: my-image:latest
stdin_open: true # docker run -i
tty: true # docker run -t
build:
context: ..
dockerfile: docker/Dockerfile

I had the same issue and could not get it to do anything else. However when i went to the ip of the docker machine docker-machine ip it returned 192.168.99.100, then by going to 192.168.99.100:8000 my docker container started receiving the requests