Why IOS-XR 9Kv devices are booting constantly??

Moderator: mike

Post Reply
hugodantas2909
Posts: 5
Joined: Sun May 01, 2022 1:25 am

Why IOS-XR 9Kv devices are booting constantly??

Post by hugodantas2909 » Sun May 01, 2022 1:32 am

Hi,
This is an issue that I saw in other EVE servers too. IOS-XR devices constantly boot. Sometimes, I am in the middle of a config and it suddenly boots making me to lose whole thing.
I practice in my own lab, I have enough resources to run my lab (mem, CPU, disk), but I saw it in other EVE Servers that I try to practice.

Is there a solution for this?

thanks,

Uldis (UD)
Posts: 5067
Joined: Wed Mar 15, 2017 4:44 pm
Location: London
Contact:

Re: Why IOS-XR 9Kv devices are booting constantly??

Post by Uldis (UD) » Sun May 01, 2022 10:55 am

No resource mate all simple
even you see your CPU is not overloaded, but in reality every CRv9K need 4 x real CPU cores...
count
I think your eve will hold half of your lab maybe

hugodantas2909
Posts: 5
Joined: Sun May 01, 2022 1:25 am

Re: Why IOS-XR 9Kv devices are booting constantly??

Post by hugodantas2909 » Thu Jun 02, 2022 3:14 pm

Hi,

I have a 48 Physical CPU and 768 GB on my dedicated server for EVE-NG. Even if I have 3 or 4 XR9Kv, I face these problems. It doesn't happen with XRv or any CSR. Here are the issues that happen:

1) My default config is loaded normally on CSRs and XRv, but on XR9Kv the interfaces constantly comes without config and I have to make manually.
2) after sometime I make the interface configs manually on XR9Kv and I am working on my configs, the device reboot after a message saying that "VM lost multiple heartbeats".
3) After 10+ minutes reboot time, it comes without interfaces. a "show interface brief" can only show mgmt 0 and loopback interfaces, but no GigabitEthernet interfaces. I need to boot multiple times wiping the config in some cases, but I start to the same problem cycle from item 1.

I think there is nothing to do with performance or resources, because I have much more than needed. I think EVE-NG cannot handle XR9Kv VMs, maybe because it is a 64-bit VM. All others 32-bit can work without any issues.

It doesn't happen only in my server. I rent labs sometimes, with same problems (this is the reason I have my own server)

hugodantas2909
Posts: 5
Joined: Sun May 01, 2022 1:25 am

Re: Why IOS-XR 9Kv devices are booting constantly??

Post by hugodantas2909 » Thu Jun 02, 2022 10:07 pm

Just as a feedback. I disabled UKSM and it looks like the problem is solved. I ran 25 XR-9Kv on version 6.5.3. I made lab for 4 hours without any reboot or "VM losing heartbeat" error.

from my 320GB dedicated to EVE-NG, the level of memory was around 51%. I turned on more 10 XR-9Kv and 12 CSR1Kv. Level of memory was up to 72%, but CPU raised to 97% and the lab started to become slow. After 30 minutes, I dropped these last 22 devices, but I didn't have any reboot during the time that I had 49 devices up.

Conclusion is that disabling UKSM is the key to solve this issue. You can disable from GUI and Cookbook shows how to do it, although, recommend to keep enabled.

The unique issue that I still had was about the startup configuration. Some devices didn't bring interface configuration from the startup and I had to make it manually.

Uldis (UD)
Posts: 5067
Joined: Wed Mar 15, 2017 4:44 pm
Location: London
Contact:

Re: Why IOS-XR 9Kv devices are booting constantly??

Post by Uldis (UD) » Fri Jun 10, 2022 12:49 pm

you forgot about cpu dear
RAM quantity is not driver, but CPU cores
such unstability is only of CPU weakness

Post Reply