Docker: Finding Container That Binds A Port In The Host Network Mode
During deployment, the other day, running an HTTP server application in a Docker container in the host network mode kept resulting in an “Address already in use” error. I knew which port was in the conflict but couldn’t tell which container already bound that port. I was already running quite a set of containers and didn’t want to stop them one by one. I figured out how to quickly identify the conflicting container and would like to share this knowledge with whoever finds it useful.
OS Error (48) Address already in use
If you ever struggled with “Address already in use” being shown in your console and weren’t able to find out what other process already binds that port, then you are probably not aware of the lsof command:
Lsof revision N lists on its standard output file information about files opened by processes for the following UNIX dialects
Consider the following example:
# lsof -Pi TCP:8080
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
python 25676 root 3u IPv4 39156234 0t0 TCP *:8080 (LISTEN)
From the output, we can tell that port TCP:8080 is bound by the process with id 25676 that is running command python.
Docker network_mode: “host”
Docker allows processes within containers to bind to the host’s network interfaces using the host network mode.
Let’s create a simple Dockerfile:
FROM python:latest
CMD python -m http.server 0
And a simple Compose file:
version: "3.9"
services:
http-server:
build: .
network_mode: host
Running the following command will spawn 5 instances of a python-based HTTP server each of which will bind to and listen at a random system-allocated dynamic port:
# docker-compose up --scale http-server=5 -d
Now let’s run lsof (still on the host machine) to see what ports these are:
# lsof -i
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
...
python 26731 root 3u IPv4 39179287 0t0 TCP *:40447 (LISTEN)
python 26733 root 3u IPv4 39176617 0t0 TCP *:39271 (LISTEN)
python 26739 root 3u IPv4 39171825 0t0 TCP *:42439 (LISTEN)
python 26760 root 3u IPv4 39174565 0t0 TCP *:36945 (LISTEN)
python 26761 root 3u IPv4 39176031 0t0 TCP *:36787 (LISTEN)
Notice that in the output there is no sign of the actual docker containers running these processes.
“docker ps” doesn’t show the ports
Let’s list the running containers first using the docker ps command:
# docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
9748dde410b0 root_http-server "/bin/sh -c 'python …" 10 minutes ago Up 9 minutes root_http-server_4
18a40f0e9ddf root_http-server "/bin/sh -c 'python …" 10 minutes ago Up 9 minutes root_http-server_5
666b6eb1a0e8 root_http-server "/bin/sh -c 'python …" 10 minutes ago Up 9 minutes root_http-server_1
e85a079e75d6 root_http-server "/bin/sh -c 'python …" 10 minutes ago Up 9 minutes root_http-server_2
c7f7db8d2d3f root_http-server "/bin/sh -c 'python …" 10 minutes ago Up 9 minutes root_http-server_3
Notice that each container has a NAME and a CONTAINER ID, but there is no information regarding the port that the processes bind to. That’s specific for the host network mode.
Finding out the container to blame
Let us say that we want to find the container that binds to TCP/36787. From the lsof output above we know the process id (PID) is 26761.
To find the ID of the container, that spawns a subprocess with PID 26761, we will use the ps command with the --forest (or -f) flag that will print the “ASCII art process tree”. We will also use the grep command so that only relevant lines from the output are displayed (that is two lines before the searched string):
# ps axf | grep -B 2 26761
...
--
26658 ? Sl 0:00 /usr/bin/containerd-shim-runc-v2 -namespace moby -id 18a40f0e9ddffb6d4f6caf33eb13a9e4a047cf44114c775e1e75ffc5f445ce89 -address /run/containerd/containerd.sock
26697 ? Ss 0:00 \_ /bin/sh -c python -m http.server 0
26761 ? S 0:00 \_ python -m http.server 0
Voilà!!! We can see that the container with CONTAINER ID that starts with 18a40f0e9ddf is the one that spawns subprocess with PID 26761, and thus the container we’re looking for is named root_http-server_5
Conclusion
This post describes how to find a Docker container in the host network mode that binds to a particular port using a combination of lsof -i, docker ps, and ps axf commands.
