ssh connection to EVE-NG with large delay around 36 sec

Before posting something, READ the changelog, WATCH the videos, howto and provide following:
Your install is: Bare metal, ESXi, what CPU model, RAM, HD, what EVE version you have, output of the uname -a and any other info that might help us faster.

Moderator: mike

Post Reply
dkeita
Posts: 1
Joined: Mon Oct 03, 2022 5:36 pm

ssh connection to EVE-NG with large delay around 36 sec

Post by dkeita » Wed Oct 05, 2022 9:51 pm

When I ssh to my eve-ng server, there is a large delay to get logged in. The delay is around 36sec which is not related to the network.
On eve-ng server I see qemu-wrapper and quemu-system-x86 running at constant CPU above 100%. Is this utilization might cause a large ssh delay to my server?

See my top output showing high cpu utilization with qemu-wrapper and quemu-system-x86 processes.

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
3396 root 20 0 3434724 29500 7156 S 115.1 0.0 5529:44 process-scanner
2837929 root 20 0 2855328 2.0g 36800 R 113.4 0.4 13045:00 qemu-system-x86
22526 root 20 0 3023812 1.9g 36628 S 105.9 0.4 3570:20 qemu-system-x86
746100 root 20 0 2798248 1.9g 36696 S 100.3 0.4 4436:11 qemu-system-x86
2837927 root 20 0 4572 92 0 R 100.0 0.0 11959:22 qemu_wrapper
28893 root 20 0 3012496 1.9g 36480 S 95.7 0.4 3590:04 qemu-system-x86

Thank you for your support on this.

Uldis (UD)
Posts: 5067
Joined: Wed Mar 15, 2017 4:44 pm
Location: London
Contact:

Re: ssh connection to EVE-NG with large delay around 36 sec

Post by Uldis (UD) » Thu Oct 06, 2022 5:17 pm

Only cause could be for this, is wrongly configured your network

frogman
Posts: 2
Joined: Wed Oct 05, 2022 8:57 pm

Re: ssh connection to EVE-NG with large delay around 36 sec

Post by frogman » Thu Oct 13, 2022 12:53 pm

The network is not the issue. When we ssh to the server, we get the authentication prompt immediately. After we type the password, you can see in the logs that the password is accepted, also immediately. Then the SSH session is stuck at (removed the IP):

***output removed for brevity***
debug1: Next authentication method: password
root@x.x.x.x's password:
debug1: Authentication succeeded (password).
Authenticated to x.x.x.x ([x.x.x.x]:22).
debug1: channel 0: new [client-session]
debug1: Requesting no-more-sessions@openssh.com
debug1: Entering interactive session.
debug1: pledge: network. ### <--- Stuck here for 30+ seconds.

I have found other people running into the same issue in a google search. The problem is we don't know what we can try without killing EVE. We have a pro license with multiple user licenses and some of the stuff running is better off not interrupted.

Thoughts?

frogman
Posts: 2
Joined: Wed Oct 05, 2022 8:57 pm

Re: ssh connection to EVE-NG with large delay around 36 sec

Post by frogman » Thu Oct 20, 2022 9:28 pm

The reboot we did helped for a while. Now we are right back to the delay. Any thoughts on what we can do here?

Genokrad
Posts: 5
Joined: Tue May 12, 2020 9:36 pm

Re: ssh connection to EVE-NG with large delay around 36 sec

Post by Genokrad » Sun Feb 05, 2023 12:00 pm

Probably, i found a solution.
Every time, when ssh login, its spawn landscape-sysinfo - motd information about system, like a cpu/ram/disk usage.
EVE-NG use a lot of file system mount points for every runned node, so my guess - depends on how many nodes are runned, daemon landscape-sysinfo scan filesystem for display in motd and its couse huge delay (in my case about 3 mintue).
So, for fix it:
1. Open landscape config file (was empty)

Code: Select all

nano /etc/landscape/client.conf
2. Write as below for disable Disk and Network plugin

Code: Select all

[sysinfo]
exclude_sysinfo_plugins = Disk,Network

Post Reply