[SOLVED] EVE-NG pro cluster : node shutdown issue

Before posting something, READ the changelog, WATCH the videos, howto and provide following:
Your install is: Bare metal, ESXi, what CPU model, RAM, HD, what EVE version you have, output of the uname -a and any other info that might help us faster.

Moderator: mike

Post Reply
mamakos
Posts: 48
Joined: Thu Dec 19, 2019 4:49 pm

[SOLVED] EVE-NG pro cluster : node shutdown issue

Post by mamakos » Mon Jan 31, 2022 4:11 pm

Hello,

EDIT : this is solved since version 4.0.1-86 (the 1st of may 2022)

Here is our cluster setup :
  • Bare Metal installation
  • EVE Pro 4.0.1-65
  • 1 master, 4 satellites (3 are really in use)
I conducted the tests in a situation where the different cluster members were not overcrowded, I swear I did :) I give the figures below :)

We have nodes (Nokia) that share the same UUID for their license is attached to one specific UUID. So nodes are using the same UUID in order to be able to use the same license. This is working fine. Nodes start properly and can be used without any major issue except when we try to stop one node :
  • On master we can stop each node independently without any issue.
  • On satellite member, when stopping one node, all the nodes that share the same UUID get stopped in all labs of all users. This is quite annoying although we can play trick to our colleagues :-D
I moved the nodes to one satellite to another, this makes no difference.

In the /opt/unetlab/data/Logs/php_errors.txt file we can see that the unl_wrapper CLI is properly specifying the device ID (14 for instance here)
[28-Jan-2022 13:37:35 Europe/Berlin] stop node with sudo /opt/unetlab/wrappers/unl_wrapper -a stop -m 3 -T 1 -D 14 -F "/opt/unetlab/labs/Users/ICC/LOUIS/test.unl" 2>> /opt/unetlab/data/Logs/unl_wrapper.txt

For the record, the hosts/vCPU/RAM distributionbefore I started 8 nodes.

Code: Select all

HOSTS distribution
user  |  cluster  | muztc628  | muztc629  | muztc620  |
total |  11 hosts |   3 hosts |   8 hosts |
  2   |  10 hosts |   2 hosts |   8 hosts |
  1   |   1 hosts |   1 hosts |           |


vCPU distribution
user  |    cluster    | muztc628/112  | muztc629/112  |  muztc620/96  |
total |       19 vCPU |        5 vCPU |       14 vCPU |
  2   |       18 vCPU |        4 vCPU |       14 vCPU |
  1   |        1 vCPU |        1 vCPU |               |


RAM distribution
user  |   cluster       | muztc628/385574 | muztc629/385574 | muztc620/257395 |
total |        43008 Mb |        10240 Mb |        32768 Mb |
  2   |        40960 Mb |         8192 Mb |        32768 Mb |
  1   |         2048 Mb |         2048 Mb |                 |
I can share the logs of starting/stopping nodes on master or satellite if you want.

Is there a solution to avoid such a behavior (except using different UUID of course).

Thank you !
Last edited by mamakos on Sat May 07, 2022 8:03 pm, edited 1 time in total.

ecze
Posts: 533
Joined: Wed Mar 15, 2017 1:54 pm

Re: EVE-NG pro cluster : node shutdown issue

Post by ecze » Mon Jan 31, 2022 7:34 pm

Thanks for reporting.

This issue will be resolved in the future.
As uuid should be unique normally, this usecase is a bit special.
However, the code will be modified to permit such.
Could you send us log from sats ? ( journalctl -u 'eveng*' ) and share uuid to verify in log...

Kind Regards,

E.

mamakos
Posts: 48
Joined: Thu Dec 19, 2019 4:49 pm

Re: EVE-NG pro cluster : node shutdown issue

Post by mamakos » Mon Jan 31, 2022 8:01 pm

Great. I won't be slashed by my colleagues anymore then :)
Thank you ! We look forward to upgrading to the next release


Edit : May the 6th
I realized I had forgotten to send you the logs you requested :cry: :cry:
So I sent them via PM

mamakos
Posts: 48
Joined: Thu Dec 19, 2019 4:49 pm

Re: [SOLVED] EVE-NG pro cluster : node shutdown issue

Post by mamakos » Sat May 07, 2022 8:54 pm

Marked the topic as solved.
Version 4.0.1-86 solved the issue

Post Reply