A bit of background, I have a Windows 11 desktop with 32GB of RAM and a Ryzen 7 3700X. I run VMware Workstation Player 16 (latest public release) on it and I usually use the EVE-NG community edition OVF. For quite a long time now (eve-ng 4.x days) I have labbed without issue on this desktop. Over the period I generally try keep EVE-NG updated as I do with VMWare Workstation Player. As such, I am running EVE-NG community edition 5.0.1-13. I have ensured VT-X/CPU virtualisation is enabled as required also (as I mentioned, I regularly lab without issue).
This week I have attempted to start up VMWare Workstation Player, start the EVE-NG machine, create four XRv9k nodes (7.3.3) for my new lab I was going to work on, and then I noticed after I booted them that XRv9k seemed to be kernel panicking at loading the Calvados VM. The logs are within this post in a code block.
What's interesting is I've never had this issue before, never encountered any major problem using EVE-NG until now. So originally before this issue began, I was running EVE-NG community edition 5.0.1-10 and I logged into my EVE-NG machine and ran the apt-update/upgrade which brought everything up to 5.0.1-13. Thinking that maybe I had broken something during the upgrade, I elected to power everything off, try rebooting my desktop and going again without any luck. Same kernel panic occurred. I began to dig deeper, I had also ran a VMWare Workstation Player update on the same day from 16.0.0 to their latest release. Thinking this may be the cause, I elected to backup my EVE-NG labs, delete my entire EVE VM and create a brand new one, downgraded VMWare Workstation to 16.0.0 and using my old install files I installed EVE-NG Community 5.0.1-10 instead of -13. I added brand new XRv9k nodes before I copied any of my labs back, booted one up just by itself on a completely fresh install and sure enough, I still have the same kernel panic now. This leads me to believe whilst I did an upgrade of EVE-NG and VMWare Workstation Player they're not the root cause of the issues, as I haven't labbed for about 3-4 weeks now so possibly something else has gone wrong since then.
I have spent the better part of two days troubleshooting, upgrading, downgrading, clean installs of EVE and VMWare Workstation Player and no luck at all. I have tried XRv9k 7.5.2, 7.4.2, 7.3.3 and 6.6.3. I know with 100% certainty that 6.6.3 and 7.3.3 were working on my deployment only a few weeks ago.
This is starting to feel a lot like maybe windows updates, microcode CPU updates or some other updates have gone out that are causing issues. As such, I wanted to test to see if EVE-NG itself still works with other vendor nodes. I tested Junos 21.1 (VMX) and it works perfectly fine, so I at least know my EVE-NG does work, just not for XRv9k machines on my hardware/software combination anymore.
Based on the below kernel panic/trace logs from the XRv9k not booting, I'm hoping someone may be able to assist troubleshooting further.
Code: Select all
Mon Aug 1 08:30:59 UTC 2022 (/proc/self/fd/9): Hardware profile: vrr
Mon Aug 1 08:30:59 UTC 2022 (/proc/self/fd/9): Host has 15.51GB RAM / 4 vCPUs
Mon Aug 1 08:30:59 UTC 2022 (/proc/self/fd/9): Management plane: 1024MB RAM
Mon Aug 1 08:30:59 UTC 2022 (/proc/self/fd/9): XR control plane: 7168MB RAM
Mon Aug 1 08:30:59 UTC 2022 (/proc/self/fd/9): XR packet memory: 128MB RAM
Mon Aug 1 08:30:59 UTC 2022 (/proc/self/fd/9): Centralized LC: 7168MB RAM
Mon Aug 1 08:30:59 UTC 2022 (/proc/self/fd/9): Data plane core assignment: 2-3
Mon Aug 1 08:30:59 UTC 2022 (/proc/self/fd/9): Control plane core assignment: 0 -1
################################################################################
# #
# Welcome to the Cisco IOS XRv9k platform #
# #
# Please wait for Cisco IOS XR to start. #
# #
# Copyright (c) 2014-2019 by Cisco Systems, Inc. #
# #
################################################################################
Cisco IOS XR console will start on the 1st serial port
Cisco IOS XR aux console will start on the 2nd serial port
Cisco Calvados console will start on the 3rd serial port
[ 18.120400] reboot: Restarting system
[ 18.123388] PANIC: double fault, error_code: 0x0
[ 18.123388] Kernel panic - not syncing: Machine halted.
[ 18.123388] CPU: 0 PID: 7032 Comm: reboot Tainted: G O 3.14.23-WR7.0.0.2_standard #1
[ 18.123388] Hardware name: cisco Cisco IOS XRv 9000, BIOS rel-1.8.2-0-g33fbe13 by qemu-project.org 04/01/2014
[ 18.123388] ffff88043fc04f18 ffff88043fc04e88 ffffffff875b3daf ffffffff877c16e2
[ 18.123388] ffff88043fc04f08 ffffffff875b2a8c 0000000000000008 ffff88043fc04f18
[ 18.123388] ffff88043fc04eb0 0000000000000092 0000000000000046 0000000000000007
[ 18.123388] Call Trace:
[ 18.123388] 3 <#DF>
[ 18.123388] [ 18.123388] [<ffffffff875b3daf>] dump_stack+0x45/0x56
[ 18.123388] [<ffffffff875b2a8c>] panic+0xc6/0x1f0
[ 18.123388] [<ffffffff87041b51>] df_debug+0x31/0x40
[ 18.123388] [<ffffffff8700340d>] do_double_fault+0x5d/0x80
[ 18.123388] [<ffffffff875c4588>] double_fault+0x28/0x30
[ 18.123388] 3 <<EOE>> [ 0.000000] Initializing cgroup subsys cpuset
[ 0.000000] Initializing cgroup subsys cpu
[ 0.000000] Initializing cgroup subsys cpuacct
[ 0.000000] Linux version 3.14.23-WR7.0.0.2_standard (hetsoi@calcium-99.cisco.com) (gcc version 4.9.1 (Wind River Linux 4.9.1-7) ) #1 SMP Sat Nov 16 15:17:13 PST 2019
[ 0.000000] Command line: root=/dev/ram intel_iommu=on pcie_aspm=off platform=xrv9k isolcpus=6-7 default_hugepagesz=1G hugepagesz=1G hugepages=6 elevator=noop __hw_profile=vrr __reboot_on_xr_bake=true boardtype=RP vmtype=hostos console=ttyS0 prod=1 pci=hpmemsize=0M,hpiosize=0M slot=RP irqpoll maxcpus=1 reset_devices cgroup_disable=memory dumpdisk=/dev/mapper/panini_vol_grp-host_data_scratch_lv0 elfcorehdr=900468K
[ 0.000000] e820: BIOS-provided physical RAM map:
[ 0.000000] BIOS-e820: [mem 0x0000000000000000-0x0000000000000fff] reserved
[ 0.000000] BIOS-e820: [mem 0x0000000000001000-0x000000000009fbff] usable
[ 0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved
[ 0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved
[ 0.000000] BIOS-e820: [mem 0x000000002b000000-0x0000000036f5cfff] usable
[ 0.000000] BIOS-e820: [mem 0x0000000036fffc00-0x0000000036ffffff] usable
[ 0.000000] BIOS-e820: [mem 0x00000000bffdf000-0x00000000bfffffff] reserved
[ 0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved
[ 0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved
[ 0.000000] NX (Execute Disable) protection: active
[ 0.000000] SMBIOS 2.8 present.
[ 0.000000] Hypervisor detected: KVM
[ 0.000000] No AGP bridge found
[ 0.000000] e820: last_pfn = 0x37000 max_arch_pfn = 0x400000000
[ 0.000000] x86 PAT enabled: cpu 0, old 0x7010600070106, new 0x7010600070106
[ 0.000000] x2apic enabled by BIOS, switching to x2apic ops
[ 0.000000] found SMP MP-table at [mem 0x000f6470-0x000f647f] mapped at [ffff8800000f6470]
[ 0.000000] Scanning 1 areas for low memory corruption
[ 0.000000] init_memory_mapping: [mem 0x00000000-0x000fffff]
[ 0.000000] init_memory_mapping: [mem 0x36800000-0x369fffff]
[ 0.000000] init_memory_mapping: [mem 0x2b000000-0x367fffff]
[ 0.000000] init_memory_mapping: [mem 0x36a00000-0x36f5cfff]
[ 0.000000] RAMDISK: [mem 0x36b96000-0x36f4efff]
[ 0.000000] ACPI: RSDP 00000000000f6420 000014 (v00 BOCHS )
[ 0.000000] ACPI: RSDT 00000000bffe14eb 000030 (v01 BOCHS BXPCRSDT 00000001 BXPC 00000001)
[ 0.000000] ACPI: FACP 00000000bffe095a 000074 (v01 BOCHS BXPCFACP 00000001 BXPC 00000001)
[ 0.000000] ACPI: DSDT 00000000bffdfdc0 000B9A (v01 BOCHS BXPCDSDT 00000001 BXPC 00000001)
[ 0.000000] ACPI: FACS 00000000bffdfd80 000040
[ 0.000000] ACPI: SSDT 00000000bffe09ce 000A8D (v01 BOCHS BXPCSSDT 00000001 BXPC 00000001)
[ 0.000000] ACPI: APIC 00000000bffe145b 000090 (v01 BOCHS BXPCAPIC 00000001 BXPC 00000001)
[ 0.000000] Setting APIC routing to cluster x2apic.
[ 0.000000] No NUMA configuration found
[ 0.000000] Faking a node at [mem 0x0000000000000000-0x0000000036ffffff]
[ 0.000000] Initmem setup node 0 [mem 0x00000000-0x36ffffff]
[ 0.000000] NODE_DATA [mem 0x36b6f000-0x36b95fff]
[ 0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00
[ 0.000000] kvm-clock: cpu 0, msr 0:36aef001, boot clock
[ 0.000000] Zone ranges:
[ 0.000000] DMA [mem 0x00001000-0x00ffffff]
[ 0.000000] DMA32 [mem 0x01000000-0xffffffff]
[ 0.000000] Normal empty
[ 0.000000] Movable zone start for each node
[ 0.000000] Early memory node ranges
[ 0.000000] node 0: [mem 0x00001000-0x0009efff]
[ 0.000000] node 0: [mem 0x2b000000-0x36f5cfff]
[ 0.000000] ACPI: PM-Timer IO Port: 0x608
[ 0.000000] ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] enabled)
[ 0.000000] ACPI: LAPIC (acpi_id[0x03] lapic_id[0x03] enabled)
[ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] dfl dfl lint[0x1])
[ 0.000000] ACPI: IOAPIC (id[0x00] address[0xfec00000] gsi_base[0])
[ 0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23
[ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
[ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)
[ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
[ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)
[ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)
[ 0.000000] Using ACPI (MADT) for SMP configuration information
[ 0.000000] smpboot: Allowing 4 CPUs, 0 hotplug CPUs
[ 0.000000] e820: [mem 0x37000000-0xbffdefff] available for PCI devices
[ 0.000000] Booting paravirtualized kernel on KVM
[ 0.000000] setup_percpu: NR_CPUS:8192 nr_cpumask_bits:4 nr_cpu_ids:4 nr_node_ids:1
[ 0.000000] PERCPU: Embedded 26 pages/cpu @ffff880036800000 s77440 r8192 d20864 u524288
[ 0.000000] kvm-clock: cpu 0, msr 0:36aef001, primary cpu clock
[ 0.000000] KVM setup async PF for cpu 0
[ 0.000000] kvm-stealtime: cpu 0, msr 3680c000
[ 0.000000] Built 1 zonelists in Node order, mobility grouping on. Total pages: 48453
[ 0.000000] Policy zone: DMA32
[ 0.000000] Kernel command line: root=/dev/ram intel_iommu=on pcie_aspm=off platform=xrv9k isolcpus=6-7 default_hugepagesz=1G hugepagesz=1G hugepages=6 elevator=noop __hw_profile=vrr __reboot_on_xr_bake=true boardtype=RP vmtype=hostos console=ttyS0 prod=1 pci=hpmemsize=0M,hpiosize=0M slot=RP irqpoll maxcpus=1 reset_devices cgroup_disable=memory dumpdisk=/dev/mapper/panini_vol_grp-host_data_scratch_lv0 elfcorehdr=900468K
[ 0.000000] Intel-IOMMU: enabled
[ 0.000000] PCIe ASPM is disabled
[ 0.000000] hugepagesz: Unsupported page size 1024 M
[ 0.000000] Misrouted IRQ fixup and polling support enabled
[ 0.000000] This may significantly impact system performance
[ 0.000000] Disabling memory control group subsystem
[ 0.000000] PID hash table entries: 1024 (order: 1, 8192 bytes)
[ 0.000000] Checking aperture...
[ 0.000000] No AGP bridge found
[ 0.000000] Memory: 167268K/196588K available (5915K kernel code, 1250K rwdata, 2652K rodata, 1928K init, 2604K bss, 29320K reserved)
[ 0.000000] Hierarchical RCU implementation.
[ 0.000000] RCU restricting CPUs from NR_CPUS=8192 to nr_cpu_ids=4.
[ 0.000000] RCU: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4
[ 0.000000] NR_IRQS:524544 nr_irqs:712 16
[ 0.000000] Console: colour VGA+ 80x25
[ 0.000000] console [ttyS0] enabled
[ 0.000000] tsc: Detected 3593.254 MHz processor
[ 0.000000] tsc: Marking TSC unstable due to TSCs unsynchronized
[ 0.001000] Calibrating delay loop (skipped) preset value.. 7186.50 BogoMIPS (lpj=3593254)
[ 0.001000] pid_max: default: 32768 minimum: 301
[ 0.001000] ACPI: Core revision 20131218
[ 0.001000] ACPI: All ACPI Tables successfully acquired
[ 0.001000] Security Framework initialized
[ 0.001000] SELinux: Initializing.
[ 0.001000] Dentry cache hash table entries: 32768 (order: 6, 262144 bytes)
[ 0.001000] Inode-cache hash table entries: 16384 (order: 5, 131072 bytes)
[ 0.001000] Mount-cache hash table entries: 512 (order: 0, 4096 bytes)
[ 0.001000] Mountpoint-cache hash table entries: 512 (order: 0, 4096 bytes)
[ 0.001000] Initializing cgroup subsys debug
[ 0.001000] Initializing cgroup subsys memory
[ 0.001000] Initializing cgroup subsys devices
[ 0.001000] Initializing cgroup subsys freezer
[ 0.001000] Initializing cgroup subsys net_cls
[ 0.001000] Initializing cgroup subsys blkio
[ 0.001000] Initializing cgroup subsys perf_event
[ 0.001000] Initializing cgroup subsys hugetlb
[ 0.001000] Initializing cgroup subsys vm
[ 0.001000] mce: CPU supports 10 MCE banks
[ 0.001000] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0
[ 0.001000] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0, 1GB 0
[ 0.001000] tlb_flushall_shift: -1
[ 0.001000] ftrace: allocating 22644 entries in 89 pages
[ 0.001000] Switched APIC routing to physical x2apic.
[ 0.001000] Default APIC routing: physical x2apic. BootCmdLine:root=/dev/ram intel_iommu=on pcie_aspm=off platform=xrv9k isolcpus=6-7 default_hugepagesz=1G hugepagesz=1G hugepages=6 elevator=noop __hw_profile=vrr __reboot_on_xr_bake=true boardtype=RP vmtype=hostos console=ttyS0 prod=1 pci=hpmemsize=0M,hpiosize=0M slot=RP irqpoll maxcpus=1 reset_devices cgroup_disable=memory dumpdisk=/dev/mapper/panini_vol_grp-host_data_scratch_lv0 elfcorehdr=900468K.
[ 0.001000] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
[ 0.001008] smpboot: CPU0: AMD QEMU Virtual CPU version 2.4.0 (fam: 06, model: 06, stepping: 03)
[ 0.104208] Performance Events: AMD PMU driver.
[ 0.104994] ... version: 0
[ 0.105001] ... bit width: 48
[ 0.106001] ... generic registers: 4
[ 0.106660] ... value mask: 0000ffffffffffff
[ 0.107001] ... max period: 00007fffffffffff
[ 0.108001] ... fixed-purpose events: 0
[ 0.109001] ... event mask: 000000000000000f
[ 0.111405] x86: Booted up 1 node, 1 CPUs
[ 0.112002] smpboot: Total of 1 processors activated (7186.50 BogoMIPS)
[ 0.113583] NMI watchdog: enabled on all CPUs, permanently consumes one hw-PMU counter.
[ 0.114105] devtmpfs: initialized
[ 0.115105] EVM: security.selinux
[ 0.115775] EVM: security.ima
[ 0.116001] EVM: security.capability
[ 0.117524] NET: Registered protocol family 16
[ 0.118063] kworker/u8:0 (15) used greatest stack depth: 14576 bytes left
[ 0.119115] cpuidle: using governor ladder
[ 0.120011] cpuidle: using governor menu
[ 0.120935] ACPI: bus type PCI registered
[ 0.121009] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5
[ 0.122137] PCI: Using configuration type 1 for base access
[ 0.123020] Adding vbridge under 0:0.1
[ 0.124003] Adding vbridge under 0:0.2
[ 0.124734] Adding vbridge under 0:0.3
[ 0.125004] Adding vbridge under 0:0.4
[ 0.126016] Adding vbridge under 0:0.5
[ 0.126584] Adding vbridge under 0:0.6
[ 0.127005] Adding vbridge under 0:0.7
[ 0.128401] kworker/u8:0 (52) used greatest stack depth: 14200 bytes left
[ 0.132972] bio: create slab <bio-0> at 0
[ 0.133116] ACPI: Added _OSI(Module Device)
[ 0.134014] ACPI: Added _OSI(Processor Device)
[ 0.135004] ACPI: Added _OSI(3.0 _SCP Extensions)
[ 0.136003] ACPI: Added _OSI(Processor Aggregator Device)
[ 0.138083] ACPI: Interpreter enabled
[ 0.138686] ACPI: (supports S0 S5)
[ 0.139003] ACPI: Using IOAPIC for interrupt routing
[ 0.140073] PCI: Ignoring host bridge windows from ACPI; if necessary, use "pci=use_crs" and report a bug
[ 0.144030] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])
[ 0.145016] acpi PNP0A03:00: _OSC: OS supports [Segments MSI]
[ 0.146005] acpi PNP0A03:00: _OSC failed (AE_NOT_FOUND); disabling ASPM
[ 0.147018] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge.
[ 0.148151] acpiphp: Slot [2] registered
[ 0.149044] acpiphp: Slot [3] registered
[ 0.149881] acpiphp: Slot [4] registered
[ 0.150047] acpiphp: Slot [5] registered
[ 0.151044] acpiphp: Slot [6] registered
[ 0.152052] acpiphp: Slot [7] registered
[ 0.153043] acpiphp: Slot [8] registered
[ 0.154033] ACPI Error: No installed handler for fixed event - PM_Timer (0), disabling (20131218/evevent-286)
[ 0.155000] ACPI Error: No installed handler for fixed event - PowerButton (2), disabling (20131218/evevent-286)
[ 0.155000] ACPI Error: No installed handler for fixed event - SleepButton (3), disabling (20131218/evevent-286)
[ 0.155000] ACPI Error: No installed handler for fixed event - RealTimeClock (4), disabling (20131218/evevent-286)
[ 0.155096] acpiphp: Slot [9] registered
[ 0.156046] acpiphp: Slot [10] registered
[ 0.157043] acpiphp: Slot [11] registered
[ 0.158041] acpiphp: Slot [12] registered
[ 0.158808] acpiphp: Slot [13] registered
[ 0.159042] acpiphp: Slot [14] registered
[ 0.160041] acpiphp: Slot [15] registered
[ 0.161041] acpiphp: Slot [16] registered
[ 0.161744] acpiphp: Slot [17] registered
[ 0.162000] acpiphp: Slot [18] registered
[ 0.162043] acpiphp: Slot [19] registered
[ 0.163042] acpiphp: Slot [20] registered
[ 0.163894] acpiphp: Slot [21] registered
[ 0.164041] acpiphp: Slot [22] registered
[ 0.165040] acpiphp: Slot [23] registered
[ 0.166052] acpiphp: Slot [24] registered
[ 0.167040] acpiphp: Slot [25] registered
[ 0.168040] acpiphp: Slot [26] registered
[ 0.168821] acpiphp: Slot [27] registered
[ 0.169000] acpiphp: Slot [28] registered
[ 0.169045] acpiphp: Slot [29] registered
[ 0.169644] acpiphp: Slot [30] registered
[ 0.170042] acpiphp: Slot [31] registered
[ 0.171017] PCI host bridge to bus 0000:00
[ 0.171796] pci_bus 0000:00: root bus resource [bus 00-ff]
[ 0.172002] pci_bus 0000:00: root bus resource [io 0x0000-0xffff]
[ 0.173004] pci_bus 0000:00: root bus resource [mem 0x00000000-0xffffffffff]
[ 0.176145] pci 0000:00:01.3: quirk: [io 0x0600-0x063f] claimed by PIIX4 ACPI
[ 0.177012] pci 0000:00:01.3: quirk: [io 0x0700-0x070f] claimed by PIIX4 SMB
[ 0.185299] ACPI: PCI Interrupt Link [LNKA] (IRQs 5 10 11) *0, disabled.
[ 0.186622] ACPI: PCI Interrupt Link [LNKB] (IRQs 5 10 11) *0, disabled.
[ 0.188396] ACPI: PCI Interrupt Link [LNKC] (IRQs 5 10 11) *0, disabled.
[ 0.189628] ACPI: PCI Interrupt Link [LNKD] (IRQs 5 10 11) *0, disabled.
[ 0.191474] ACPI: PCI Interrupt Link [LNKS] (IRQs *9)
[ 0.192539] ACPI: Enabled 16 GPEs in block 00 to 0F
[ 0.193103] vgaarb: loaded
[ 0.194098] SCSI subsystem initialized
[ 0.195158] ACPI: bus type USB registered
[ 0.196063] usbcore: registered new interface driver usbfs
[ 0.197031] usbcore: registered new interface driver hub
[ 0.198037] usbcore: registered new device driver usb
[ 0.199067] pps_core: LinuxPPS API ver. 1 registered
[ 0.200005] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
[ 0.201031] PTP clock support registered
[ 0.201596] PCI: Using ACPI for IRQ routing
[ 0.202369] NetLabel: Initializing
[ 0.203010] NetLabel: domain hash size = 128
[ 0.204001] NetLabel: protocols = UNLABELED CIPSOv4
[ 0.205028] NetLabel: unlabeled traffic allowed by default
[ 0.206081] Switched to clocksource kvm-clock
[ 0.211715] Warning: Zero PT_NOTE entries found
[ 0.212568] Kdump: vmcore not initialized
[ 0.213326] FS-Cache: Loaded
[ 0.213885] pnp: PnP ACPI init
[ 0.215036] pnp: PnP ACPI: found 6 devices
[ 0.262077] PM-Timer failed consistency check (0xffffff) - aborting.
[ 0.263334] pci 0000:00:03.0: BAR 6: assigned [mem 0x38000000-0x3803ffff pref]
[ 0.264812] pci 0000:00:03.1: BAR 6: assigned [mem 0x38040000-0x3807ffff pref]
[ 0.266288] pci 0000:00:03.2: BAR 6: assigned [mem 0x38080000-0x380bffff pref]
[ 0.267761] pci 0000:00:03.3: BAR 6: assigned [mem 0x380c0000-0x380fffff pref]
[ 0.268957] pci 0000:00:03.4: BAR 6: assigned [mem 0x38100000-0x3813ffff pref]
[ 0.270450] pci 0000:00:03.5: BAR 6: assigned [mem 0x38140000-0x3817ffff pref]
[ 0.271936] pci 0000:00:03.6: BAR 6: assigned [mem 0x38180000-0x381bffff pref]
[ 0.273325] pci 0000:00:02.0: BAR 1: assigned [mem 0x381c0000-0x381c0fff]
[ 0.274634] pci 0000:00:03.0: BAR 1: assigned [mem 0x381c1000-0x381c1fff]
[ 0.275971] pci 0000:00:03.1: BAR 1: assigned [mem 0x381c2000-0x381c2fff]
[ 0.277341] pci 0000:00:03.2: BAR 1: assigned [mem 0x381c3000-0x381c3fff]
[ 0.278685] pci 0000:00:03.3: BAR 1: assigned [mem 0x381c4000-0x381c4fff]
[ 0.279887] pci 0000:00:03.4: BAR 1: assigned [mem 0x381c5000-0x381c5fff]
[ 0.281260] pci 0000:00:03.5: BAR 1: assigned [mem 0x381c6000-0x381c6fff]
[ 0.282627] pci 0000:00:03.6: BAR 1: assigned [mem 0x381c7000-0x381c7fff]
[ 0.284022] pci 0000:00:02.0: BAR 0: assigned [io 0x1000-0x103f]
[ 0.285301] pci 0000:00:03.0: BAR 0: assigned [io 0x1040-0x105f]
[ 0.286481] pci 0000:00:03.1: BAR 0: assigned [io 0x1060-0x107f]
[ 0.287697] pci 0000:00:03.2: BAR 0: assigned [io 0x1080-0x109f]
[ 0.288928] pci 0000:00:03.3: BAR 0: assigned [io 0x10a0-0x10bf]
[ 0.290161] pci 0000:00:03.4: BAR 0: assigned [io 0x10c0-0x10df]
[ 0.291338] pci 0000:00:03.5: BAR 0: assigned [io 0x10e0-0x10ff]
[ 0.292595] pci 0000:00:03.6: BAR 0: assigned [io 0x1400-0x141f]
[ 0.293654] pci 0000:00:01.1: BAR 4: assigned [io 0x1420-0x142f]
[ 0.294960] NET: Registered protocol family 2
[ 0.296048] TCP established hash table entries: 2048 (order: 2, 16384 bytes)
[ 0.297504] TCP bind hash table entries: 2048 (order: 3, 32768 bytes)
[ 0.298768] TCP: Hash tables configured (established 2048 bind 2048)
[ 0.300061] TCP: reno registered
[ 0.300726] UDP hash table entries: 256 (order: 1, 8192 bytes)
[ 0.301866] UDP-Lite hash table entries: 256 (order: 1, 8192 bytes)
[ 0.303179] NET: Registered protocol family 1
[ 0.304087] IOMEM min address 0x0 max_address 0x0
[ 0.304836] pci 0000:00:00.0: Limiting direct PCI/PCI transfers
[ 0.305856] pci 0000:00:01.0: PIIX3: Enabling Passive Release
[ 0.307025] pci 0000:00:01.0: Activating ISA DMA hang workarounds
[ 0.308205] Trying to unpack rootfs image as initramfs...
[ 0.353714] Freeing initrd memory: 3812K (ffff880036b96000 - ffff880036f4f000)
[ 0.355350] microcode: AMD CPU family 0x6 not supported
[ 0.356549] Scanning for low memory corruption every 60 seconds
[ 0.357796] futex hash table entries: 1024 (order: 4, 65536 bytes)
[ 0.358901] Initialise system trusted keyring
[ 0.359796] audit: initializing netlink subsys (disabled)
[ 0.360943] audit: type=2000 audit(1659342667.331:1): initialized
[ 0.375176] bounce pool size: 64 pages
[ 0.375886] HugeTLB registered 2 MB page size, pre-allocated 6 pages
[ 0.377214] VFS: Disk quotas dquot_6.5.2
[ 0.377990] Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[ 0.379371] fuse init (API version 7.22)
[ 0.380241] msgmni has been set to 334
[ 0.381179] NET: Registered protocol family 38
[ 0.382334] Key type asymmetric registered
[ 0.383179] Asymmetric key parser 'x509' registered
[ 0.384238] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 251)
[ 0.385732] io scheduler noop registered (default)
[ 0.386654] io scheduler deadline registered
[ 0.387517] io scheduler cfq registered
[ 0.388413] pci_hotplug: PCI Hot Plug PCI Core version: 0.5
[ 0.389622] pciehp: PCI Express Hot Plug Controller Driver version: 0.4
[ 0.391001] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4
[ 0.392556] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
[ 0.416088] 00:04: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A
[ 0.439904] 00:05: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A
[ 0.441369] Non-volatile memory driver v1.3
[ 0.442202] Linux agpgart interface v0.103
[ 0.444922] brd: module loaded
[ 0.446047] loop: module loaded
[ 0.446815] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[ 0.448209] ehci-pci: EHCI PCI platform driver
[ 0.448914] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12
[ 0.451045] ACPI Error: No installed handler for fixed event - PM_Timer (0), disabling (20131218/evevent-286)
[ 0.451486] ACPI Error: No installed handler for fixed event - PowerButton (2), disabling (20131218/evevent-286)
[ 0.451486] ACPI Error: No installed handler for fixed event - SleepButton (3), disabling (20131218/evevent-286)
[ 0.451486] ACPI Error: No installed handler for fixed event - RealTimeClock (4), disabling (20131218/evevent-286)
[ 0.459838] ACPI Error: No installed handler for fixed event - PM_Timer (0), disabling (20131218/evevent-286)
[ 0.460467] ACPI Error: No installed handler for fixed event - PowerButton (2), disabling (20131218/evevent-286)
[ 0.460467] ACPI Error: No installed handler for fixed event - SleepButton (3), disabling (20131218/evevent-286)
[ 0.460467] ACPI Error: No installed handler for fixed event - RealTimeClock (4), disabling (20131218/evevent-286)
[ 0.468173] serio: i8042 KBD port at 0x60,0x64 irq 1
[ 0.469183] serio: i8042 AUX port at 0x60,0x64 irq 12
[ 0.470336] mousedev: PS/2 mouse device common for all mice
[ 0.471775] input: AT Raw Set 2 keyboard as /devices/platform/i8042/serio0/input/input0
[ 0.473430] ACPI Error: Could not disable RealTimeClock events (20131218/evxfevnt-267)
[ 0.475122] rtc_cmos 00:00: RTC can wake from S4
[ 0.477959] rtc_cmos 00:00: rtc core: registered rtc_cmos as rtc0
[ 0.479155] rtc_cmos 00:00: alarms up to one day, 114 bytes nvram
[ 0.483051] drop_monitor: Initializing network drop monitor service
[ 0.484343] u32 classifier
[ 0.484891] Actions configured
[ 0.485658] ipip: IPv4 over IPv4 tunneling driver
[ 0.492315] TCP: cubic registered
[ 0.492870] Initializing XFRM netlink socket
[ 0.493604] NET: Registered protocol family 10
[ 0.495372] mip6: Mobile IPv6
[ 0.495977] NET: Registered protocol family 17
[ 0.497102] Loading compiled-in X.509 certificates
[ 0.498069] registered taskstats version 1
[ 0.498988] Key type trusted registered
[ 0.499853] Key type encrypted registered
[ 0.500595] IMA: No TPM chip found, activating TPM-bypass!
[ 0.502292] console [netcon0] enabled
[ 0.502987] netconsole: network logging started
[ 0.504008] rtc_cmos 00:00: setting system clock to 2022-08-01 08:31:07 UTC (1659342667)
[ 0.505516] ACPI Error: No installed handler for fixed event - PM_Timer (0), disabling (20131218/evevent-286)
[ 0.506221] ACPI Error: No installed handler for fixed event - PowerButton (2), disabling (20131218/evevent-286)
[ 0.506221] ACPI Error: No installed handler for fixed event - SleepButton (3), disabling (20131218/evevent-286)
[ 0.506221] ACPI Error: Could not disable RealTimeClock events (20131218/evxfevnt-267)
[ 0.512972] ACPI Error: No installed handler for fixed event - PM_Timer (0), disabling (20131218/evevent-286)
[ 0.513880] ACPI Error: No installed handler for fixed event - PowerButton (2), disabling (20131218/evevent-286)
[ 0.513880] ACPI Error: No installed handler for fixed event - SleepButton (3), disabling (20131218/evevent-286)
[ 0.513880] ACPI Error: Could not disable RealTimeClock events (20131218/evxfevnt-267)
[ 0.521586] Freeing unused kernel memory: 1928K (ffffffff81b3a000 - ffffffff81d1c000)
[ 0.523121] Write protecting the kernel read-only data: 10240k
[ 0.524635] Freeing unused kernel memory: 216K (ffff8800355ca000 - ffff880035600000)
[ 0.528667] Freeing unused kernel memory: 1444K (ffff880035897000 - ffff880035a00000)
[ 0.531055] mkdir (661) used greatest stack depth: 14192 bytes left
Running INIT process in Crash Kernel[ 0.533905] ACPI Error: No installed handler for fixed event - PM_Timer (0), disabling (20131218/evevent-286)
[ 0.534270] ACPI Error: No installed handler for fixed event - PowerButton (2), disabling (20131218/evevent-286)
[ 0.534270] ACPI Error: No installed handler for fixed event - SleepButton (3), disabling (20131218/evevent-286)
[ 0.534270] ACPI Error: Could not disable RealTimeClock events (20131218/evxfevnt-267)
[ 0.541539] ACPI Error: No installed handler for fixed event - PM_Timer (0), disabling (20131218/evevent-286)
[ 0.542418] ACPI Error: No installed handler for fixed event - PowerButton (2), disabling (20131218/evevent-286)
[ 0.542418] ACPI Error: No installed handler for fixed event - SleepButton (3), disabling (20131218/evevent-286)
[ 0.542418] ACPI Error: Could not disable RealTimeClock events (20131218/evxfevnt-267)
/etc/init.d/pd-functions: line 743: on_baremetal: command not found[ 0.563168] ACPI Error: No installed handler for fixed event - PM_Timer (0), disabling (20131218/evevent-286)
[ 0.564122] ACPI Error: No installed handler for fixed event - PowerButton (2), disabling (20131218/evevent-286)
[ 0.564122] ACPI Error: No installed handler for fixed event - SleepButton (3), disabling (20131218/evevent-286)
[ 0.564122] ACPI Error: Could not disable RealTimeClock events (20131218/evxfevnt-267)
[ 0.570982] ACPI Error: No installed handler for fixed event - PM_Timer (0), disabling (20131218/evevent-286)
[ 0.571869] ACPI Error: No installed handler for fixed event - PowerButton (2), disabling (20131218/evevent-286)
[ 0.571869] ACPI Error: No installed handler for fixed event - SleepButton (3), disabling (20131218/evevent-286)
[ 0.571869] ACPI Error: Could not disable RealTimeClock events (20131218/evxfevnt-267)
/etc/init.d/pd-functions: line 564: on_baremetal: command not found[ 0.610513] ACPI Error: No installed handler for fixed event - PM_Timer (0), disabling (20131218/evevent-286)
[ 0.611467] ACPI Error: No installed handler for fixed event - PowerButton (2), disabling (20131218/evevent-286)
[ 0.611467] ACPI Error: No installed handler for fixed event - SleepButton (3), disabling (20131218/evevent-286)
[ 0.611467] ACPI Error: Could not disable RealTimeClock events (20131218/evxfevnt-267)
[ 0.617522] ACPI Error: No installed handler for fixed event - PM_Timer (0), disabling (20131218/evevent-286)
[ 0.618409] ACPI Error: No installed handler for fixed event - PowerButton (2), disabling (20131218/evevent-286)
[ 0.618409] ACPI Error: No installed handler for fixed event - SleepButton (3), disabling (20131218/evevent-286)
[ 0.618409] ACPI Error: Could not disable RealTimeClock events (20131218/evxfevnt-267)
/etc/init.d/pd-functions: line 571: is_xrv9k_aws: command not found[ 0.625989] ACPI Error: No installed handler for fixed event - PM_Timer (0), disabling (20131218/evevent-286)
[ 0.626107] ACPI Error: No installed handler for fixed event - PowerButton (2), disabling (20131218/evevent-286)
[ 0.626107] ACPI Error: No installed handler for fixed event - SleepButton (3), disabling (20131218/evevent-286)
[ 0.626107] ACPI Error: Could not disable RealTimeClock events (20131218/evxfevnt-267)
[ 0.633369] ACPI Error: No installed handler for fixed event - PM_Timer (0), disabling (20131218/evevent-286)
[ 0.634261] ACPI Error: No installed handler for fixed event - PowerButton (2), disabling (20131218/evevent-286)
[ 0.634261] ACPI Error: No installed handler for fixed event - SleepButton (3), disabling (20131218/evevent-286)
[ 0.634261] ACPI Error: Could not disable RealTimeClock events (20131218/evxfevnt-267)
/etc/init.d/pd-functions: line 673: blkid: command not found[ 0.645639] ACPI Error: No installed handler for fixed event - PM_Timer (0), disabling (20131218/evevent-286)
[ 0.646596] ACPI Error: No installed handler for fixed event - PowerButton (2), disabling (20131218/evevent-286)
[ 0.646596] ACPI Error: No installed handler for fixed event - SleepButton (3), disabling (20131218/evevent-286)
[ 0.646596] ACPI Error: Could not disable RealTimeClock events (20131218/evxfevnt-267)
[ 0.652613] ACPI Error: No installed handler for fixed event - PM_Timer (0), disabling (20131218/evevent-286)
[ 0.653481] ACPI Error: No installed handler for fixed event - PowerButton (2), disabling (20131218/evevent-286)
[ 0.653481] ACPI Error: No installed handler for fixed event - SleepButton (3), disabling (20131218/evevent-286)
[ 0.653481] ACPI Error: Could not disable RealTimeClock events (20131218/evxfevnt-267)
/init: line 119: get_board_type: command not found[ 0.660687] ACPI Error: No installed handler for fixed event - PM_Timer (0), disabling (20131218/evevent-286)
[ 0.661646] ACPI Error: No installed handler for fixed event - PowerButton (2), disabling (20131218/evevent-286)
[ 0.661646] ACPI Error: No installed handler for fixed event - SleepButton (3), disabling (20131218/evevent-286)
[ 0.661646] ACPI Error: Could not disable RealTimeClock events (20131218/evxfevnt-267)
[ 0.667915] ACPI Error: No installed handler for fixed event - PM_Timer (0), disabling (20131218/evevent-286)
[ 0.668793] ACPI Error: No installed handler for fixed event - PowerButton (2), disabling (20131218/evevent-286)
[ 0.668793] ACPI Error: No installed handler for fixed event - SleepButton (3), disabling (20131218/evevent-286)
[ 0.668793] ACPI Error: Could not disable RealTimeClock events (20131218/evxfevnt-267)
[ 0.678802] device-mapper: ioctl: 4.27.0-ioctl (2013-10-30) initialised: dm-devel@redhat.com
[ 0.680588] insmod (714) used greatest stack depth: 14120 bytes left
[ 0.689255] insmod (719) used greatest stack depth: 14072 bytes left
[ 0.696362] virtio-pci 0000:00:02.0: enabling device (0000 -> 0003)
[ 0.707834] ACPI: PCI Interrupt Link [LNKB] enabled at IRQ 11
[ 0.711767] blk-mq: CPU -> queue map
[ 0.712531] CPU 0 -> Queue 0
[ 1.079561] input: ImExPS/2 Generic Explorer Mouse as /devices/platform/i8042/serio1/input/input2