PSCP Backup failed

Before posting something, READ the changelog, WATCH the videos, howto and provide following:
Your install is: Bare metal, ESXi, what CPU model, RAM, HD, what EVE version you have, output of the uname -a and any other info that might help us faster.

Moderator: mike

eveaccount
Posts: 8
Joined: Tue Sep 26, 2023 3:06 pm

PSCP Backup failed

Post by eveaccount » Wed Sep 27, 2023 1:26 pm

Hi,
I have been using eve for a couple of years now and yesterday I did a fresh install of version 5.0.1-19.
I was previously on version 2.0.3-112 and migrated all the backups files to the new installation before creating a new lab like the procedure recommends.
My labs are working fine but I just realized that my daily backup fails.
I have a powershell script that backups pretty much all the folder listed in the FAQ with PSCP.
The issue seems to be that since version 5, eve-ng uses (I think) a container solution that creates symbolic links.
When PSCP, WINSCP or filezilla try to copy these links, the transfer crash.
As an example, the following folders are not in the backup and I believe that they are the reason why pscp crashes.
opt/unetlab/tmp/0/LABID/7/lib
opt/unetlab/tmp/0/LABID/7/usr
opt/unetlab/tmp/0/LABID/7/dev/net/tun
opt/unetlab/tmp/0/LABID/7/lib64
opt/unetlab/tmp/0/LABID/7/dev/kvm

Anyone have this issue, how do you deal with backups with version 5?

winscp Error message :
Bad message (badly formatted packet or protocol incompatibility).
Error code: 5
Error message from server: Bad message

FileZilla error message :
https://imgur.com/YVTjX9D

Content on the folder :
https://imgur.com/a/UWmfB1o

Thank you!

rusty725
Posts: 127
Joined: Thu Mar 29, 2018 4:19 pm

Re: PSCP Backup failed

Post by rusty725 » Wed Sep 27, 2023 7:51 pm

nodes need to be off when backing up

eveaccount
Posts: 8
Joined: Tue Sep 26, 2023 3:06 pm

Re: PSCP Backup failed

Post by eveaccount » Wed Sep 27, 2023 7:55 pm

Hi,
They are powered off and there's no lab opened.

rusty725
Posts: 127
Joined: Thu Mar 29, 2018 4:19 pm

Re: PSCP Backup failed

Post by rusty725 » Thu Sep 28, 2023 2:56 am

you can try using rsync.

eveaccount
Posts: 8
Joined: Tue Sep 26, 2023 3:06 pm

Re: PSCP Backup failed

Post by eveaccount » Thu Sep 28, 2023 12:32 pm

I will try this today, thank you for the suggestion.

I am under the impression that while the routers are off, there should be no symbolic link or some dev/ or usr/ folder or all the stuff that I've showed in one of the picture in the initial post.
I might have a bug with my setup..
Maybe I should attempt to migrate my stuff to another installation and see how it goes.

eveaccount
Posts: 8
Joined: Tue Sep 26, 2023 3:06 pm

Re: PSCP Backup failed

Post by eveaccount » Thu Sep 28, 2023 7:12 pm

I just tested with a fresh installed and after migrating my backup and booting a lab, I am still stuck with symbolic links that crash file transfer.

Here's the content of /opt/unetlab/tmp/0/5a8a5722-f9d9-4026-849b-485fe0f52479/3 after powering a lab and shutting it down.

Code: Select all

drwxr-sr-x  3 root root    4096 Feb 22  2023 dev
-rw-rw-r--  1 root unl     4013 Sep 22 19:55 ios_config.txt
drwxr-sr-x  2 root root    4096 May  9  2022 jail
lrwxrwxrwx  1 root root       8 May  3  2022 lib -> jail/lib
lrwxrwxrwx  1 root root      10 May  3  2022 lib64 -> jail/lib64
-rw-rw-r--  1 root unl  9801728 Sep 22 19:55 minidisk
drwxr-sr-x 14 root root    4096 Feb 22  2023 opt
-rw-rw-r--  1 root unl     4013 Sep 22 19:55 startup-config
lrwxrwxrwx  1 root root       8 May  3  2022 usr -> jail/usr
-rw-rw-r--  1 root unl  1245184 Sep 28 18:54 virtioa.qcow2
-rw-rw-r--  1 root unl      371 Sep 28 19:09 wrapper.txt
Is this normal?
The behavior was different with version 2 and it looks like it break backup copy of the /opt/unetlab/tmp/0/ folder.

Uldis (UD)
Posts: 5050
Joined: Wed Mar 15, 2017 4:44 pm
Location: London
Contact:

Re: PSCP Backup failed

Post by Uldis (UD) » Thu Sep 28, 2023 7:48 pm

Normal behave on Ubuntu 18 and later, it is harmless

eveaccount
Posts: 8
Joined: Tue Sep 26, 2023 3:06 pm

Re: PSCP Backup failed

Post by eveaccount » Thu Sep 28, 2023 7:56 pm

Okay thank you, and can you confirm that the procedure to backup haven't changed ?
https://www.eve-ng.net/index.php/backup-eve-ng-content/

rusty725
Posts: 127
Joined: Thu Mar 29, 2018 4:19 pm

Re: PSCP Backup failed

Post by rusty725 » Thu Sep 28, 2023 10:18 pm

not changed

eveaccount
Posts: 8
Joined: Tue Sep 26, 2023 3:06 pm

Re: PSCP Backup failed

Post by eveaccount » Fri Sep 29, 2023 1:55 pm

For anyone finding this post in the future, SFTP and SCP do not treat symbolic links well and has you've probably seen, it breaks backup copies.
example with pscp :

Code: Select all

PS C:\Users> C:\"Program Files"\PuTTY\pscp.exe -v -scp -r -q -pw eve root@192.168.1.37:/opt/unetlab/tmp C:\temp\
[...]
Entering directory: D2755 0 qemu
Sending file modes: C0644 9801728 minidisk
Entering directory: D2755 0 dev
scp: /opt/unetlab/tmp/0/5a8a5722-f9d9-4026-849b-485fe0f52479/3/dev/kvm: not a regular file
Entering directory: D2755 0 net
Protocol error: Illegal file descriptor format
Session sent command exit status 1
Main session channel closed
All channels closed
So far, I found that invoking scp from powershell with the -r argument seems to work.
Something like :

Code: Select all

PS C:\Users\> scp -r root@192.168.1.37:/opt/unetlab/tmp/0/ C:/temp/
Maybe filtering only the 2 or 3 files needed for each node by name would do the job just well ; parse the folders recursively, create text file containing a list of all the files that you want to copy and SCP them individually based on this dictionary.

Post Reply