Need help: Bare metal install interface names change after reboot

Before posting something, READ the changelog, WATCH the videos, howto and provide following:
Your install is: Bare metal, ESXi, what CPU model, RAM, HD, what EVE version you have, output of the uname -a and any other info that might help us faster.

Moderator: mike

Post Reply
philip.chang
Posts: 4
Joined: Wed Sep 28, 2022 12:32 pm

Need help: Bare metal install interface names change after reboot

Post by philip.chang » Wed Sep 28, 2022 1:10 pm

Hi guys,

We bought and installed the eve-ng pro on a bare metal server using the official ISO file. The installer had a few issues throughout the install:

Issue 1:
The bare metal server we used to install Eve-NG has to be behind a proxy server. The initial Ubuntu install went ok. The installation used our proxy server setting to complete the install. However after that, during the boot up process there were plenty of error messages about cloud-init and unable to download packages from au.archive.ubuntu.com/ubuntu focal-updates. I suspect that the proxy server set during the Ubuntu install didn't persist into the cloud-init stage. After a few failed attempts, cloud-init gave up and rebooted the server into the login command prompt. After logging in, I followed the installation guide. It turned out the server was missing a few packages required by the eveng-pro package. So I manually installed the packages with proxy server settings.

Issue 2:
After reboot, this time finally I could see the First Time Configuration screen. I configured the required settings such as hostname, domain name, subnets, NTP and proxy servers. After reboot, all network connectivities to the server were broken. I logged into the server from IPMI interface and checked the network configurations in /etc/network/interfaces. It turns out all the mappings between physical and logical interfaces changed. For example pnet0(eth0) used to be mapped to physical interface enp5s0f0. After first time configuration completed, now pnet0(eth0) is enp5s0f1 which is a different physical interface. After reboot, the network mapping changed again so pnet0(eth0) is linked to another physical interface. This is a huge problem because each time the server reboots, we lose all network connectivities. We fixed the issue by manually changing the /etc/default/grub setting GRUB_CMDLINE_LINUX_DEFAULT="net.ifnames=1. My colleague came up with this workaround which make the interface names consistent after reboot. And he changed the /etc/network/interfaces configuration so pnet maps directly to the physical interface names. Finally after reboot, all network settings are consistent. However this workaround also leads to Issue 3.

iface enp5s0f0 inet manual
auto pnet1
iface pnet1 inet static
pre-up ip link set dev enp5s0f0 up
address x.x.x.x
netmask x.x.x.x
gateway x.x.x.x
dns-domain x.x.x
dns-nameservers x.x.x.x
bridge_ports enp5s0f0
bridge_stp off

Issue 3:
Now that we are no longer using interface names such as eth0,1,2,3. The unl_wrapper -a fixpermissions script stopped working (error message below). So we can no longer install new devices. Please help and provide advice on how to get around these problems. (I know we were supposed to have direct internet connection for the install, but in our environment, we can't do that.)

/opt/unetlab/wrappers/unl_wrapper -a fixpermissions
Sep 28 12:25:31 Sep 28 12:25:31 Online Check state: Valid
eth0: error fetching interface information: Device not found
eth0: error fetching interface information: Device not found


Issue 4:
When we clone a lab, the nodes inside the cloned lab interfere with the original lab. For example when a node is stopped in the cloned lab, the same node from the original lab is also stopped. This causes a problem because we don't want the VMs to interfere with each other after cloning. I tried export/import labs from zip files and the interference is the same.
How can we make a perfect clean clone of a lab so they are independent of each other?

Sorry about the many questions. Please help.

Uldis (UD)
Posts: 5067
Joined: Wed Mar 15, 2017 4:44 pm
Location: London
Contact:

Re: Need help: Bare metal install interface names change after reboot

Post by Uldis (UD) » Wed Sep 28, 2022 4:22 pm

First advice is do not use management interface on pnet1, it must be pnet0
only first interface (pnet0) is configured to communicate with EVE license server !
After this will be sorted we can look further..

philip.chang
Posts: 4
Joined: Wed Sep 28, 2022 12:32 pm

Re: Need help: Bare metal install interface names change after reboot

Post by philip.chang » Thu Sep 29, 2022 2:40 am

Uldis (UD) wrote:
Wed Sep 28, 2022 4:22 pm
First advice is do not use management interface on pnet1, it must be pnet0
only first interface (pnet0) is configured to communicate with EVE license server !
After this will be sorted we can look further..
Thanks Uldis. I've changed the network configuration to below config. So the internet facing interface is pointed to pnet0.

Running the /opt/unetlab/wrappers/unl_wrapper -a fixpermissions still generates the error message. We don't have a eth0, what would the next step be?
Sep 29 02:38:33 Sep 29 02:38:33 Online Check state: Valid
eth0: error fetching interface information: Device not found
eth0: error fetching interface information: Device not found
auto lo
iface lo inet loopback

iface enp5s0f0 inet manual
auto pnet0
iface pnet0 inet static
pre-up ip link set dev enp5s0f0 up
address x.x.x.x
netmask x.x.x.x
gateway x.x.x.x
dns-nameservers x.x.x.x
bridge_ports enp5s0f0
bridge_stp off

iface enp5s0f1 inet manual
auto pnet1
iface pnet1 inet manual
bridge_ports enp5s0f1
bridge_stp off

iface eno1 inet manual
auto pnet2
iface pnet2 inet manual
bridge_ports eno1
bridge_stp off

iface eno2 inet manual
auto pnet3
iface pnet3 inet manual
bridge_ports eno2
bridge_stp off

iface eno3 inet manual
auto pnet4
iface pnet4 inet manual
bridge_ports eno3
bridge_stp off

iface eno4 inet manual
auto pnet5
iface pnet5 inet manual
bridge_ports eno4
bridge_stp off

Uldis (UD)
Posts: 5067
Joined: Wed Mar 15, 2017 4:44 pm
Location: London
Contact:

Re: Need help: Bare metal install interface names change after reboot

Post by Uldis (UD) » Thu Sep 29, 2022 5:16 pm

I feel that need rename interfaces to ethX

philip.chang
Posts: 4
Joined: Wed Sep 28, 2022 12:32 pm

Re: Need help: Bare metal install interface names change after reboot

Post by philip.chang » Fri Sep 30, 2022 3:39 am

Uldis (UD) wrote:
Thu Sep 29, 2022 5:16 pm
I feel that need rename interfaces to ethX
Thank you Uldis. We have renamed the physical interface to eth0. The fixpermission script is working now : )

Just about the last part of the questions, thank you for your time.
Issue 4:
When we clone a lab, the nodes inside the cloned lab interfere with the original lab. For example when a node is stopped in the cloned lab, the same node from the original lab is also stopped. This causes a problem because we don't want the VMs to interfere with each other after cloning. I tried export/import labs from zip files and the interference is the same.
Also when cloning labs, sometimes remote console connection (telnet/vnc) stops working on the cloned nodes. When clicking on the broken node, the screen says connection error and give three options: Home, reconnect, exit.
How can we make a perfect clean clone of a lab so they are independent of each other?

Uldis (UD)
Posts: 5067
Joined: Wed Mar 15, 2017 4:44 pm
Location: London
Contact:

Re: Need help: Bare metal install interface names change after reboot

Post by Uldis (UD) » Fri Sep 30, 2022 7:57 am

Is that node vEOS Arista? , only on these images were observed such bugs. When you stop any node connected to cloud interface, it stops all other Arista vEOS nodes in all labs what are connected to this cloud.
Try create some simple lab with cloud interface and connect some vIOS Cisco node to it, and then re-test this behave with it

philip.chang
Posts: 4
Joined: Wed Sep 28, 2022 12:32 pm

Re: Need help: Bare metal install interface names change after reboot

Post by philip.chang » Sun Oct 02, 2022 3:29 am

Is that node vEOS Arista? , only on these images were observed such bugs. When you stop any node connected to cloud interface, it stops all other Arista vEOS nodes in all labs what are connected to this cloud.
Try create some simple lab with cloud interface and connect some vIOS Cisco node to it, and then re-test this behave with it
It is Cumulus VX we are observing this issue. I'll run some tests like you recommended. Thank you.

Post Reply