[BUG] k3d registry create + cluster create --network host --registry use don't seem to work together

[akshaya-a]

What did you do

Following the docs,

k3d registry create...
k3d cluster create --registry-use ...

works well together. While trying to debug a connectivity issue, I added --network host to the cluster create command. This fails to start the cluster due to:

Command '/usr/local/bin/k3d cluster create XYZ --kubeconfig-update-default=0 --image rancher/k3s:v1.25.3-k3s1 --wait --timeout=240s --registry-use ABC --network host' returned non-zero exit status 1.
/usr/local/bin/k3d cluster create XYZ --kubeconfig-update-default=0 --image rancher/k3s:v1.25.3-k3s1 --wait --timeout=240s --registry-use ABC --network host
b"\x1b[36mINFO\x1b[0m[0000] [SimpleConfig] Hostnetwork selected - disabling injection of docker host into the cluster, server load balancer and setting the api port to the k3s default \n\x1b[36mINFO\x1b[0m[0000] [ClusterConfig] Hostnetwork selected - disabling injection of docker host into the cluster, server load balancer and setting the api port to the k3s default \n\x1b[36mINFO\x1b[0m[0000] Prep: Network \n\x1b[36mINFO\x1b[0m[0000] Re-using existing network 'host' (1da4a9f896cefba85d1d35694b84122b832079332322eea6d19796d528de2972) \n\x1b[36mINFO\x1b[0m[0000] Created image volume XYZ \n\x1b[36mINFO\x1b[0m[0000] Deleting cluster 'XYZ' \n"
b"\x1b[33mWARN\x1b[0m[0000] Failed to connect to network 'host': Connection failed \n\x1b[33mWARN\x1b[0m[0000] Error response from daemon: container cannot be disconnected from host network or connected to host network \n\x1b[31mERRO\x1b[0m[0000] Failed Cluster Preparation: Failed to connect registry node 'ABC' to cluster network: Failed to connect to one or more networks \n\x1b[31mERRO\x1b[0m[0000] Failed to create cluster >>> Rolling Back \n\x1b[31mERRO\x1b[0m[0000] failed to get cluster: No nodes found for given cluster \n\x1b[31mFATA\x1b[0m[0000] Cluster creation FAILED, also FAILED to rollback changes! \n"

• How was the cluster created?
  /usr/local/bin/k3d cluster create XYZ --kubeconfig-update-default=0 --image rancher/k3s:v1.25.3-k3s1 --wait --timeout=240s --registry-use ABC --network host

• What did you do afterwards?

  • k3d commands?
  • docker commands?
  • OS operations (e.g. shutdown/reboot)?

What did you expect to happen

k3d cluster create --network host respects previously created k3d registry create

Screenshots or terminal output

If applicable, add screenshots or terminal output (code block) to help explain your problem.

Which OS & Architecture

ubuntu x64

Which version of k3d

k3d version v5.6.0
k3s version v1.27.4-k3s1 (default)

Which version of docker

Client: Docker Engine - Community
Version: 25.0.3
API version: 1.44
Go version: go1.21.6
Git commit: 4debf41
Built: Tue Feb 6 21:13:09 2024
OS/Arch: linux/amd64
Context: default

Server: Docker Engine - Community
Engine:
Version: 25.0.3
API version: 1.44 (minimum version 1.24)
Go version: go1.21.6
Git commit: f417435
Built: Tue Feb 6 21:13:09 2024
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.6.28
GitCommit: ae07eda36dd25f8a1b98dfbf587313b99c0190bb
runc:
Version: 1.1.12
GitCommit: v1.1.12-0-g51d5e94
docker-init:
Version: 0.19.0
GitCommit: de40ad0

Client: Docker Engine - Community
Version: 25.0.3
Context: default
Debug Mode: false
Plugins:
buildx: Docker Buildx (Docker Inc.)
Version: v0.12.1
Path: /usr/libexec/docker/cli-plugins/docker-buildx
compose: Docker Compose (Docker Inc.)
Version: v2.24.5
Path: /usr/libexec/docker/cli-plugins/docker-compose

Server:
Containers: 2
Running: 1
Paused: 0
Stopped: 1
Images: 56
Server Version: 25.0.3
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Using metacopy: false
Native Overlay Diff: true
userxattr: false
Logging Driver: json-file
Cgroup Driver: systemd
Cgroup Version: 2
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
Swarm: inactive
Runtimes: io.containerd.runc.v2 runc
Default Runtime: runc
Init Binary: docker-init
containerd version: ae07eda36dd25f8a1b98dfbf587313b99c0190bb
runc version: v1.1.12-0-g51d5e94
init version: de40ad0
Security Options:
apparmor
seccomp
Profile: builtin
cgroupns
Kernel Version: 6.5.0-25-generic
Operating System: Ubuntu 22.04.4 LTS
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 9.704GiB
Name: xxx
ID: db1d1237-4a37-49d5-a777-e17f0e010bec
Docker Root Dir: /var/lib/docker
Debug Mode: false
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false

[akshaya-a]

I should also add that I found the undocumented --default-network flag for registry create that seems pretty much what I want: https://github.com/k3d-io/k3d/blob/main/cmd/registry/registryCreate.go#L111

but then cluster create on use fails with

b"\x1b[36mINFO\x1b[0m[0000] [SimpleConfig] Hostnetwork selected - disabling injection of docker host into the cluster, server load balancer and setting the api port to the k3s default \n\x1b[36mINFO\x1b[0m[0000] [ClusterConfig] Hostnetwork selected - disabling injection of docker host into the cluster, server load balancer and setting the api port to the k3s default \n\x1b[36mINFO\x1b[0m[0000] Prep: Network \n\x1b[36mINFO\x1b[0m[0000] Deleting cluster XYZ \n"
b'\x1b[31mERRO\x1b[0m[0000] Failed Cluster Preparation: Failed Network Preparation: failed to create cluster network: failed to check for duplicate docker networks: failed to parse IP Prefix of network "host"'s member <>: ParseAddr(""): unable to parse IP \n\x1b[31mERRO\x1b[0m[0000] Failed to create cluster