Solution
It was found (here, and here) that Podman uses its own DNS server, aardvark-dns
which is bound to port 53 (this explains why I was able to bind to 53 with nc
on the host while the container would still fail). So the solution is to bridge the network for that port. So, in the compose file, the ports section would become:
ports:
- "<host-ip>:53:53/tcp"
- "<host-ip>:53:53/udp"
- "80:80/tcp"
where <host-ip>
is the ip of the machine running the container — e.g. 192.168.1.141
.
Original Post
I so desperately want to bash my head into a hard surface. I cannot figure out what is causing this issue. The full error is as follows:
Error: cannot listen on the UDP port: listen udp4 :53: bind: address already in use
This is my compose file:
version: "3"
services:
pihole:
container_name: pihole
image: docker.io/pihole/pihole:latest
ports:
- "53:53/tcp"
- "53:53/udp"
- "80:80/tcp"
environment:
TZ: '<redacted>'
volumes:
- './etc-pihole:/etc/pihole'
- './etc-dnsmasq.d:/etc/dnsmasq.d'
restart: unless-stopped
and the result of :
Netid State Recv-Q Send-Q Local Address:Port Peer Address:Port Process
udp UNCONN 0 0 [fe80::e877:8420:5869:dbd9]:546 *:* users:(("NetworkManager",pid=377,fd=28))
tcp LISTEN 0 128 0.0.0.0:22 0.0.0.0:* users:(("sshd",pid=429,fd=3))
tcp LISTEN 0 128 [::]:22 [::]:* users:(("sshd",pid=429,fd=4))
I have looked for possible culprit services like systemd-resolved
. I have tried disabling Avahi. I have looked for other potential DNS services. I have rebooted the device. I am running the container as sudo (so it has access to all ports). I am quite at a loss.
- Raspberry Pi Model 1 B Rev 2
- Raspbian (bookworm)
- Kernel v6.6.20+rpt-rpi-v6
- Podman v4.3.1
- Podman Compose v1.0.3
EDIT (2024-03-14T22:13Z)
For the sake of clarity, shows nothing on 53, and
shows nothing listening to port 53 — the only listening service is SSH on 22, as expected.
Also, as suggested here, I tried manually binding to port 53, and I was able to without issue.
Ports below 1024 are by default reserved for root. So unless you use sudo or change this you wont be able to use port 80 and 53 without root
This article covers the solution https://access.redhat.com/solutions/7044059
That is not the solution. As I have already mentioned a number of times, I am running the container in priveleged mode — I am running the container as root. It has access to all ports.
In the /etc/systemd/resolved.conf
Set the DNSStubListener=no
like below then restart the network resolvd service this should allow you to run an alternate service on port 53
# This file is part of systemd.
#
# systemd is free software; you can redistribute it and/or modify it under the
# terms of the GNU Lesser General Public License as published by the Free
# Software Foundation; either version 2.1 of the License, or (at your option)
# any later version.
#
# Entries in this file show the compile time defaults. Local configuration
# should be created by either modifying this file, or by creating "drop-ins" in
# the resolved.conf.d/ subdirectory. The latter is generally recommended.
# Defaults can be restored by simply deleting this file and all drop-ins.
#
# Use 'systemd-analyze cat-config systemd/resolved.conf' to display the full config.
#
# See resolved.conf(5) for details.
[Resolve]
# Some examples of DNS servers which may be used for DNS= and FallbackDNS=:
# Cloudflare: 1.1.1.1#cloudflare-dns.com 1.0.0.1#cloudflare-dns.com 2606:4700:4700::1111#cloudflare-dns.com 2606:4700:4700::1001#cloudflare-dns.com
# Google: 8.8.8.8#dns.google 8.8.4.4#dns.google 2001:4860:4860::8888#dns.google 2001:4860:4860::8844#dns.google
# Quad9: 9.9.9.9#dns.quad9.net 149.112.112.112#dns.quad9.net 2620:fe::fe#dns.quad9.net 2620:fe::9#dns.quad9.net
DNS=127.0.0.1
#FallbackDNS=
#Domains=
#DNSSEC=no
#DNSOverTLS=no
#MulticastDNS=no
#LLMNR=no
#Cache=no-negative
#CacheFromLocalhost=no
DNSStubListener=no
#DNSStubListenerExtra=
#ReadEtcHosts=yes
#ResolveUnicastSingleLabel=no
systemd-resolved is not running — it isn’t even installed on the device. I also already mentioned that I have looked into this fact within the body of the post.
This is the correct solution. OP’s issue is a common one which shows up on the PiHole discussion forums regularly. I frequently forget to perform this step on new installs, too.
I’ve not done much with podman, but my first thought is that port 53 is privileged and usually podman runs as a non-privileged user, right? Do you have some mechanism in place that would allow podman to use port 53?
This may sound silly, but have you tried restarting? I feel like that worked for me when I had a similar issue in the past. Something was holding onto the port, but it wasn’t showing up anywhere. Restart got it to let go and it worked afterwards.
Yeah, I have already tried rebooting the device. To no avail, unfortunately.
Darn.
I suppose you could also try using the lsof command to see if it shows anything different?
Some examples: https://www.cyberciti.biz/faq/unix-linux-check-if-port-is-in-use-command/
I appreciate the suggestion, but I have already tried essentially all alternative network commands to see if one might yield a different result. They, of course, all show the same things — nothing is listening on 53. That command specifically only shows that sshd is listening on 22, which is expected.
Check with “sudo ss -tupan | grep 53” to see if there is a process already using port 53. It might be systemd’s resolver or something like that.