Develop Reference

Failure to connect to Docker Postgresql instance from Python

I am using Docker to "containerize" a PostgreSQL deployment. I can spin up the container and connect to PostgreSQL via the command line as shown below:
minime2#CEBERUS:~/Projects/skunkworks$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
dc176901052a df:pg "docker-entrypoint..." About an hour ago Up About an hour 5432/tcp vigilant_agnesi
minime2#CEBERUS:~/Projects/skunkworks$ CONTAINER_ID=dc176901052a
minime2#CEBERUS:~/Projects/skunkworks$ IP=$(docker inspect -f '{{.NetworkSettings.Networks.bridge.IPAddress}}' $CONTAINER_ID)
minime2#CEBERUS:~/Projects/skunkworks$ echo $IP
172.17.0.2
minime2#CEBERUS:~/Projects/skunkworks$ docker exec -it vigilant_agnesi psql -U postgres -W cookiebox
Passwod for user postgres:
psql (9.6.5)
Type "help" for help
cookiebox#
Now attempting connection with Python:
Python 3.5.2 (default, Sep 14 2017, 22:51:06)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import psycopg2
>>> conn = psycopg2.connect("dbname='cookiebox' user='postgres' host='172.17.0.2' password='nunyabiznes'") Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/minime2/Projects/skunkworks/archivers/env/lib/python3.5/site-packages/psycopg2/__init__.py", line 130, in connect
conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
psycopg2.OperationalError: could not connect to server: Connection refused
Is the server running on host "172.17.0.2" and accepting
TCP/IP connections on port 5432?
>>>
Can anyone explain why I can't connect to PostgreSQL using Python - even though I'm using the same arguments/parameters that enable a successful connection at the command line (using docker exec?).
[[Additional Info]]
As suggested by #Itvhillo, I tried to use a desktop application to connect to the PG service. I run the docker service using the following command:
docker run -i -p 5432:5432 --name $CONTAINER_NAME $DOCKER_IMAGE
I am using Db Visualizer to connect to the database, and I have set the hostname to 'localhost'. I can successfully ping the port, but still get an error message when I try to connect to the database (possible permissions related error):
An error occurred while establishing the connection:
Long Message:
The connection attempt failed.
Details:
   Type: org.postgresql.util.PSQLException
   SQL State: 08001
Incidentally, this is the tail end of the output for the PG service instance:
PostgreSQL init process complete; ready for start up.
LOG: could not bind IPv6 socket: Cannot assign requested address
HINT: Is another postmaster already running on port 5432? If not, wait a few seconds and retry.
LOG: database system was shut down at 2018-01-30 16:21:59 UTC
LOG: MultiXact member wraparound protections are now enabled
LOG: database system is ready to accept connections
LOG: autovacuum launcher started
[[Additional Info2]]
Here is the tail end of my Dockerfile:
# modified target locations (checked by login onto Docker container)
# show hba_file;
# show config_file;
#################################################################################
# From here: https://docs.docker.com/engine/examples/postgresql_service/
# Adjust PostgreSQL configuration so that remote connections to the
# database are possible.
RUN echo "host all all 0.0.0.0/0 md5" >> /var/lib/postgresql/data/pg_hba.conf
# And add ``listen_addresses`` to ``/var/lib/postgresql/data/postgresql.conf``
RUN echo "listen_addresses='*'" >> /var/lib/postgresql/data/postgresql.conf
#################################################################################
EXPOSE 5432
# Add VOLUMEs to allow backup of config, logs and databases
VOLUME ["/etc/postgresql", "/var/log/postgresql", "/var/lib/postgresql", "/usr/lib/postgresql/"]

If you are are running
$ docker run -i -p 5432:5432 --name $CONTAINER_NAME $DOCKER_IMAGE
Then you should be able to connect to localhost:5432 from the host. The easiest way to check whether something is listening on port 5432 is using netcat. In case of success you should get:
$ nc -zv localhost 5432
Connection to localhost 5432 port [tcp/postgresql] succeeded!
In this case, you should be able to connect using:
>>> psycopg2.connect("dbname='cookiebox' user='postgres' host='localhost' password='nunyabiznes'")
If, on the other hand, you get something like:
$ nc -zv localhost 5432
nc: connect to localhost port 5432 (tcp) failed: Connection refused
Then it means that PostgreSQL is not listening, and hence something is wrong in your Dockerfile, and you'll need to post more details on your Dockerfile to diagnose it.

It seems that PostgreSQL couldn't bind socket to listen for TCP connections for some reason. It still listens the default UNIX socket inside the container, though, so you could connect to it via docker exec -it $CONTAINER_NAME psql.

In the postgresql.conf file change
listen_addresses = 'localhost' to listen_addresses = '*'
Then try to connect via psql
psql -h docker-ip -u username -w

Try to isolate the issue: run clean postgres instance without any extensions on default port:
docker run -d --name tst-postgres-01 -p 5432:5432 -e POSTGRES_PASSWORD=mysecretpassword postgres:9.6
And try to connect to it. If you can, then you have to review your Dockerfile accordingly: just remove everything from there and then add new things one by one. Otherwise, if it doesn't connect, then try to run it on other port:
docker run -d --name tst-postgres-01 -p 45432:5432 -e POSTGRES_PASSWORD=mysecretpassword postgres:9.6
And try to connect. If it works this time, then the issue your network configuration on host machine, somehow 5432 port is blocked or in use by other app.

docker container connect local postgresql

I run python container, I want to connect localhost postegresql. And I try some method. But not work. Please talk me. How can I do ? Thanks.
I have run postegresql on port 5432, create datatbase and grant user.
run docker comand
docker run --name=python3 -v ${pwd}:/code -w /code python
python code
import psycopg2
def main():
#Define our connection string
conn_string = "host='localhost' dbname='testDB' user='test' password='test'"
# print the connection string we will use to connect
print ("Connecting to database\n ->{}".format(conn_string))
# get a connection, if a connect cannot be made an exception will be raised here
conn = psycopg2.connect(conn_string)
# conn.cursor will return a cursor object, you can use this cursor to perform queries
cursor = conn.cursor()
print ("Connected!\n")
if __name__ == "__main__":
main()
error message
Is the server running on host "localhost" (::1) and accepting
TCP/IP connections on port 5432?
could not connect to server: Connection refused
Is the server running on host "localhost" (127.0.0.1) and accepting
TCP/IP connections on port 5432?

It depends on your host OS and your docker version.
I assume here your database is not running in a container itself, but rather on the host itself (localhost).
For instance, as mentioned in "From inside of a Docker container, how do I connect to the localhost of the machine?", with Docker for Mac v 17.06 and above (June 2017), you can connect to the special Mac-only DNS name docker.for.mac.localhost which will resolve to the internal IP address used by the host.
On Linux directly, you would use the host mode, with an image having ifconfig installed:
docker run --network="host" -id <Docker image ID>

Reading that you're on Windows 10 and running postgresql on the host, I advise you to run postgresql in a container. It makes this way easier.
To connect the python container to the postgres container you'll need a docker network though. Let's call it postgres_backend.
docker network create postgres_backend
You can create the postgresql container with the following command. Just change the /path/to/store/data to a local directory in which you'd like to store the postgres data:
docker run --name postgres \
-e POSTGRES_PASSWORD=test \
-e POSTGRES_USER=test \
-d --restart always \
-v /path/to/store/data:/var/lib/postgresql/data \
--net postgres_backend \
postgres:9.6
Now your postresql container should be up and running :)
To connect your python container to it, you'll have to add a --net postgres_backend to your docker run command and change the host in your script to "postgres" (it's the name we gave the postgres container with --name).
If the python container can't find the host "postgres", try it with the IP shown when entering the command docker exec -ti postgres ip addr show.

to fix this bug:
First, it is normal that it does not work, because postgresql does not run in the same container as the application so the host localhost: 5432 does not exist.
to fix it :
on the properties file isntead localhost:5432 use your IPadresse like IP:5432 and in the pg_hba.conf add this
host all all 0.0.0.0/0 md5

How to connect to myql in docker container from CLI or python script

I want to connect to MySQL inside Docker container. I have a running instance of MySQL in a docker container. Since I already have 3306 port busy on my host, I decide to use port 8081 to start my MySQL container. Basically, I started my container with docker run -p 8080:80 -p 8081:3306 --name test test. When I connect to my container, I can connect to my MySQL without error. On the other hand, I have a web app that is able to connect to MySQL with the exact same port (8081) from my host. This means that MySQL is working properly and is reachable from outside. But now in my python script, I cannot connect. I am unable to connect to MySQL with CLI either. It seems like the port number is simply not interpreted. I see that if use for example mysql -P 8081 -u root -p. This is just trying to connect to host MySQL (port 3306) instead of container MySQL on port 8081(when I enter host MySQL credentials, it connect to host MySQL). In my python script, I used this: conn = MySQLdb.connect( host = 'localhost', port=8081, user='root', passwd=''). But this is not working either. In the MySQL man page, I see this :
· --port=port_num, -P port_num
The TCP/IP port number to use for the connection.
What am I doing wrong, please?
mysql --version:
mysql Ver 14.14 Distrib 5.7.18, for Linux (x86_64) using EditLine wrapper
update
here is my docker ps:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
b370c91594d3 test "/bin/sh -c /start.sh" 14 hours ago Up 14 hours 8080/tcp, 0.0.0.0:8080->80/tcp, 0.0.0.0:8081->3306/tcp test

Running a Self-Developed Python Server on Docker Container

I have this python server:
import SocketServer
class TCPHandler(SocketServer.BaseRequestHandler):
def handle(self):
print 'client is connected'
data = self.request.recv(1024)
print data
self.request.sendall('Message received!')
HOST, PORT = '0.0.0.0', 5004
server = SocketServer.TCPServer((HOST, PORT), TCPHandler)
print 'Listening on port {}'.format(PORT)
server.serve_forever()
and this client:
import socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(('192.168.56.101', 5004))
s.sendall('Hello my name is John')
response = s.recv(1024)
print 'Response: {}'.format(response)
My OS is macOS and I have Ubuntu 14.04 installed on a virtual machine using VirtualBox. In VirtualBox, I setup a NAT network and I gave Ubuntu this IP address: 192.168.56.101. I put the server program on Ubuntu and added a rule in IPTables to allow incoming connections from port 5004. I started the server on Ubuntu and I tried to connect to the server using the client above on my macOS. The connection went through and the data exchange finished successfully.
Now to the problem. I installed Docker on my virtualized Ubuntu. Docker itself uses another version of Ubuntu 14.04. What I want is to run the server inside the Dockrized version of Ubuntu, so I wrote this Dockerfile:
FROM bamos/ubuntu-opencv-dlib-torch:ubuntu_14.04-opencv_2.4.11-dlib_19.0-torch_2016.07.12
ADD . /root
EXPOSE 5004
CMD ["python2", "/root/server.py"]
I build it using this command: sudo docker build -t boring91/mock and it was built successfully. I ran the Docker container using this command: sudo docker run -p 5004:5004 -t boring91/mock and it showed that it is started listening on port 5004. When I tried to connect to it using my client on my macOS, the socket connected but no data exchange happened. Same thing happens when I run the client on the virtualized Ubuntu. Can anyonw tell me whats the issue here?

Docker container with Python Web App - Connect to Postgres on Host machine (OSX)

I can connect to my local Postgres DB in my web app, but NOT if I am running the web app inside of a Docker container.
Web app running inside of a Docker container
Postgres running in the Host machine
I am not sure if it is related to the Postgres connection settings or to the Docker network settings.
Follow my settings and commands:
Host:
OSX 10.11.6
PostgreSQL 9.6
Docker container
Docker 1.13.1
Docker-machine 0.9.0
Docker container OS: python:3.6.0-alpine
Python 3.6 + psycopg2==2.7
postgresql.conf:
listen_addresses = '*'
pg_hba.conf
host all all 127.0.0.1/32 trust
host all all 0.0.0.0/0 trust
host all all 172.17.0.0/24 trust
host all all 192.168.99.0/24 trust
With Docker network in HOST mode
docker run -i --net=host -h 127.0.0.1 -e POSTGRES_URI=postgresql://127.0.0.1:5432/db my/image
Error:
could not connect to server: Connection refused
Is the server running
on host "127.0.0.1" and accepting TCP/IP connections on port 5432?
With Docker network in BRIDGE mode
docker run -i --add-host=dockerhost:`docker-machine ip ${DOCKER_MACHINE}` -e POSTGRES_URI=postgresql://dockerhost:5432/db -p 8000:8000 -p 5432:5432 my/image
Error:
server closed the connection unexpectedly
This probably means the
server terminated abnormally before or while processing the request.
Any ideas?

There is a note about doing this in the docs
I want to connect from a container to a service on the host
The Mac has a changing IP address (or none if you have no network access). Our current recommendation is to attach an unused IP to the lo0 interface on the Mac; for example: sudo ifconfig lo0 alias 10.200.10.1/24, and make sure that your service is listening on this address or 0.0.0.0 (ie not 127.0.0.1). Then containers can connect to this address.