System VMs' Cloudstack agent is grey out & Cloud0 is auto creating

[ramthota27]

Hello everyone,

I've installed Apache CloudStack 4.20 and added the KVM host. I see two system VMs running on the CloudStack UI, but the agent state appears greyed out. After following Copilot for troubleshooting, I found that the management server isn't able to communicate with the system VMs.

I've set the traffic labels in the physical network as follows:

Management traffic = cloudbr0
Private network = cloudbr0
Public/shared network = cloudbr0
Guest (VM data traffic) = cloudbr1

My KVM host is operational with virt services and has two NICs: ens192 and ens224. Both are configured as bridge networks—ens192 is linked to cloudbr0 and ens224 to cloudbr1. Both bridges are up, with cloudbr0 assigned IP 10.80.148.30, while cloudbr1 has no IP. The network YAML file is shown in the attached image.

The output of "ip a" on the KVM host is also in the image.

I've noticed an extra NIC named cloud0 keeps getting created automatically, even after deleting it (ip link delete cloud0) multiple times. This cloud0 interface is interfering with my KVM and Apache CloudStack setup. How can I permanently remove or stop it from being auto-generated, so the management server communicates with cloudbr0 instead?

Since the agent state on the system VMs is disabled/greyed out, the Upload ISO option is also disabled in the UI. Any guidance on resolving this issue would be appreciated.

Thank you
With Regards
Ram

[boring-cyborg[bot]]

Thanks for opening your first issue here! Be sure to follow the issue template!

[weizhouapache]

it is expected
cloud0 is the bridge for link local IPs (169.254.0.0/16) of system vms and virtual routers. the IP is configured as 169.254.0.1/16

CloudStack agent connects throught it to the system vms and virtual routers running on the host
you can ssh into the systemvm from the host via

ssh -p 3922 -i ~/.ssh/id_rsa.cloud root@169.254.x.x

[ramthota27]

@weizhouapache Thank yoiu for the inputs. As you suggets, I tried ssh to the system vms, I am getting "port 3922: No route to host". I disabled the firewalld. I am also attaching my networking settings for more insight,

[weizhouapache]

@weizhouapache Thank yoiu for the inputs. As you suggets, I tried ssh to the system vms, I am getting "port 3922: No route to host". I disabled the firewalld. I am also attaching my networking settings for more insight,

does ping work ?

[ramthota27]

No, I the system vms are not pinging from either Apache cloudstack server or kvm.

[weizhouapache]

No, I the system vms are not pinging from either Apache cloudstack server or kvm.

can you destroy the system vms and check if the new system vms are still unreachable ?

[ramthota27]

Yes, I deleted them and they were recreated. I did it several times, but the issue persists.