EVE-NG Juniper vMX - VMWARE Workstation versus GCP

Before posting something, READ the changelog, WATCH the videos, howto and provide following:
Your install is: Bare metal, ESXi, what CPU model, RAM, HD, what EVE version you have, output of the uname -a and any other info that might help us faster.

Moderator: mike

Post Reply
Sipart
Posts: 59
Joined: Sun Mar 19, 2017 9:20 pm

EVE-NG Juniper vMX - VMWARE Workstation versus GCP

Post by Sipart » Tue Feb 11, 2020 4:36 pm

Hi - I got a new work laptop recently and as it has a core i7 and 32G RAM I thought I would have a go with EVE-NG (community version) in VMWARE workstation again.

The newer Juniper vMX images (17.3R3-S2.2 VCP and VFP) I have works really well and boots up quick in VMWARE EVE-NG (laptop running Win10 with VMWARE WS PRO v15 - core i7 32G RAM) - takes about 5-7 mins to get all GE ports available.

So I spun up EVE-NG in GCP as well to see how it performs. More RAM, decent processor spec and 100G persistent SSD. I have used EVE-NG in GCP before - but due to trial running out I spun a new one up again in another account - using the guide on the EVE-NG site to do the install (using step '2.2 For EVE-NG Community start EVE-NG Community installation' as I do not have EVE-NG Pro).

But when running the same vMX in GCP is runs markedly slower - and I can't work out why as the specs should be ‘better’ on GCP – vMX chugs away and it takes about 15-20 mins before GE ports available.

So I have gathered some info and posting here to see if anyone can spot anything amiss and help – and if any more info is needed then I can add.

Many thanks in advance

### LAPTOP VMWARE info ###

root@eve-ng:~# free
total used free shared buff/cache available
Mem: 24942052 1836684 22617336 25544 488032 22634180
Swap: 8388604 0 8388604

root@eve-ng:~# lscpu
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 8
On-line CPU(s) list: 0-7
Thread(s) per core: 1
Core(s) per socket: 1
Socket(s): 8
NUMA node(s): 1
Vendor ID: GenuineIntel
CPU family: 6
Model: 142
Model name: Intel(R) Core(TM) i7-8665U CPU @ 1.90GHz
Stepping: 12
CPU MHz: 2112.001
BogoMIPS: 4224.00
Virtualization: VT-x
Hypervisor vendor: VMware
Virtualization type: full
L1d cache: 32K
L1i cache: 32K
L2 cache: 256K
L3 cache: 8192K
NUMA node0 CPU(s): 0-7
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon nopl xtopology tsc_reliable nonstop_tsc cpuid pni pclmulqdq vmx ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single ssbd ibrs ibpb stibp ibrs_enhanced tpr_shadow vnmi ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 invpcid rdseed adx smap xsaveopt arat flush_l1d arch_capabilities

root@eve-ng:~# uname -r
4.20.17-eve-ng-ukms+

root@eve-ng:~# df
Filesystem 1K-blocks Used Available Use% Mounted on
udev 12448192 0 12448192 0% /dev
tmpfs 2494208 14848 2479360 1% /run
/dev/mapper/eve--ng--vg-root 135611700 13446664 116306512 11% /
tmpfs 12471024 0 12471024 0% /dev/shm
tmpfs 5120 0 5120 0% /run/lock
tmpfs 12471024 0 12471024 0% /sys/fs/cgroup
/dev/sda1 482922 152248 305740 34% /boot


## GCP INFO -some info masked ##

$ gcloud compute disks describe eve --zone europe-west2-c
creationTimestamp: '2020-02-10T07:59:00.449-08:00'
guestOsFeatures:
- type: VIRTIO_SCSI_MULTIQUEUE
id: '*****************'
kind: compute#disk
labelFingerprint: **********
lastAttachTimestamp: '2020-02-10T07:59:00.450-08:00'
licenseCodes:
- '1000201'
- '1002001'
licenses:
- https://www.googleapis.com/compute/v1/p ... 604-xenial
- https://www.googleapis.com/compute/v1/p ... enable-vmx
name: eve
physicalBlockSizeBytes: '4096'
selfLink: https://www.googleapis.com/compute/v1/p ... /disks/eve
sizeGb: '100'
sourceImage: https://www.googleapis.com/compute/v1/p ... ntu-xenial
sourceImageId: '**********'
status: READY
type: https://www.googleapis.com/compute/v1/p ... pes/pd-ssd
users:
- https://www.googleapis.com/compute/v1/p ... tances/eve
zone: https://www.googleapis.com/compute/v1/p ... pe-west2-c


GCP Machine type
custom (8 vCPUs, 48 GB memory)

GCP CPU platform
Intel Skylake

GCP Boot disk
eve nested-ubuntu-xenial 100 eve SSD persistent disk

root@eve:~# free
total used free shared buff/cache available
Mem: 49454660 2199108 38028036 19740 9227516 46441016
Swap: 0 0 0

root@eve:~# lscpu
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 8
On-line CPU(s) list: 0-7
Thread(s) per core: 2
Core(s) per socket: 4
Socket(s): 1
NUMA node(s): 1
Vendor ID: GenuineIntel
CPU family: 6
Model: 85
Model name: Intel(R) Xeon(R) CPU @ 2.00GHz
Stepping: 3
CPU MHz: 2000.120
BogoMIPS: 4000.24
Virtualization: VT-x
Hypervisor vendor: KVM
Virtualization type: full
L1d cache: 32K
L1i cache: 32K
L2 cache: 1024K
L3 cache: 39424K
NUMA node0 CPU(s): 0-7
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology nonstop_tsc cpuid tsc_known_freq pni pclmulqdq vmx ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single pti ssbd ibrs ibpb stibp tpr_shadow flexpriority ept vpid fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 erms invpcid rtm mpx avx512f avx512dq rdseed adx smap clflushopt clwb avx512cd avx512bw avx512vl xsaveopt xsavec xgetbv1 xsaves arat arch_capabilities

root@eve:~# uname -r
4.20.17-eve-ng-ukms+

root@eve:~# df
Filesystem 1K-blocks Used Available Use% Mounted on
udev 24705136 0 24705136 0% /dev
tmpfs 4945468 9004 4936464 1% /run
/dev/sda1 101445900 12471844 88957672 13% /
tmpfs 24727328 0 24727328 0% /dev/shm
tmpfs 5120 0 5120 0% /run/lock
tmpfs 24727328 0 24727328 0% /sys/fs/cgroup
/dev/sda15 106858 3700 103159 4% /boot/efi
tmpfs 4945468 0 4945468 0% /run/user/0

Sipart
Posts: 59
Joined: Sun Mar 19, 2017 9:20 pm

Re: EVE-NG Juniper vMX - VMWARE Workstation versus GCP

Post by Sipart » Wed Feb 12, 2020 4:18 pm

Had some help from the live helpdesk - I tried using the legacy templates (in /opt/unetlab/html/ - swapping the folder names) - but this did not improve the boot time.

I have captured the full boot of the vMX in each environment - the GCP example sees some "jlock hog"s.
Obviously you can’t 'see' the delays. The GCP boot is much slower generally and where both boots stop at "Trying to mount root from cd9660:/dev/md0.uzip []..." and "Loading BCMSDK module....." the GCP example is a much longer delay.

Cheers
vMX boots.zip
You do not have the required permissions to view the files attached to this post.

Uldis (UD)
Posts: 5067
Joined: Wed Mar 15, 2017 4:44 pm
Location: London
Contact:

Re: EVE-NG Juniper vMX - VMWARE Workstation versus GCP

Post by Uldis (UD) » Wed Feb 12, 2020 7:39 pm

My friend, I just did test,
EVE-PRO 2.0.6-34, and with Community will be the same.

GCP vCPU 8/ 32Gb RAM
skylake, 100Gb SSD

and all running just flawless !!! sped and boot etc, all works !!! All flys.
Boot time of those 3 nodes took 3 mins till GE interfaces!!

On 8 cores absolutely fast and nicely running 12 cores lab
actually x3 vMX 19.1

more than 3 vMX this GCP simply does not hold, CPU cores are short...

Uldis
You do not have the required permissions to view the files attached to this post.

Sipart
Posts: 59
Joined: Sun Mar 19, 2017 9:20 pm

Re: EVE-NG Juniper vMX - VMWARE Workstation versus GCP

Post by Sipart » Wed Feb 12, 2020 8:04 pm

Thank you. I will try 19.1 vMX. I had stuck with 17.3 originally as that is JTAC recommended so assumed it would be ideal.

Thank you for your time and efforts :-) 👍

Sipart
Posts: 59
Joined: Sun Mar 19, 2017 9:20 pm

Re: EVE-NG Juniper vMX - VMWARE Workstation versus GCP

Post by Sipart » Thu Feb 13, 2020 8:08 pm

19.1R1.6 is loads better.

In VMware WS pro 15 the vMX now boots with GE ints available after 2.5-3 mins :-D

In GCP it is also vastly improved - time is now 6 -7 mins. Not 3 mins like Uldis though - this **might** be because I am using community eve - but Uldis says community should be the same - so still a little confused. I did multiple boots and clean shut downs of just one vMX and tried with UKSM enabled and disabled.

Only thing I haven't tried is the legacy template - so will try that.

I am 100% sure I have exactly the same spec as Uldis did in GCP: 8 vCPU / 32Gb RAM - skylake and 100Gb SSD. Using the install process from site so correct nested Ubuntu image.

But loads loads better now with help from Uldis :D

Post Reply