Backup and Restore All Mint Settings

[Mint_BackupRestore]

Introduction: For what are these scripts good for?

Linux Mint is a great operating system, but since updates are frequent, maintaining the system can cause a lot of work. In
contrast to Ubuntu, Linux Mint is more in favor of reinstalling the entire system with every new release, rather then upgrading.
But I slightly prefer Linux Mint, because I can use flash, Java and playback DVD right out of the box. Installing a new release
is quite quick, but getting the settings (such as fstab, cronjobs, etc.) back to how I like them takes me usually 1-2 days.

So I decided to write two scripts:
a.) Backup_Linux_Settings:
Saves all the current settings, installed packages, data stored in home folder and tweaks you have done to the system.
b.) Restore_Linux_Settings
Script which restores these "settings" into your new freshly installed Linux Mint. With different options to chose (e.g., all settings, only selected settings, only firefox, only Thunderbird...)

For much more detailed information, please take a look at the attached >>README<< file on Github, by following the link below.

UPDATE: The files are now available through https://github.com/MintBackupRestore/MintBackupRestore
______________________________________________________________________________________________________________________

[xenopeek]

Just a warning to potential users; as always, don't download software and run it on your computer unless you trust the source.

OP: it would be good if you publish the software on something like GitHub.

[Mint_BackupRestore]

Dear xenopeek,

thanks for warning others not to use software of untrusted sources. Thanks also for the hint with GitHub. I was at first thinking of uploading to Sourceforge, but since I've never done that before I found it quite complicated for a simple bash script. Thus I just posted on the Mint forum. Since the scripts are completely open source, everyone can look at the code in the files. They are not compiled and it is not closed source software. I would also never download a closed source software from a forum. But these are just script files that the Mint community can check here openly. I was hoping that together we can further improve the scripts over the time in this forum. They are easily understandable and I put in a lot of comments. Everyone who has entered commands into a terminal before will understand the code. It will take maybe 15min to understand the backup file and about 1 hour to understand the restore file. IF ANYONE SEES ANYTHING SUSPICIOUS IN THE FILES PLEASE DIRECTLY POST IT AND WE WILL EITHER CORRECT IT OR PUT THEM COMPLETELY OFF LINE. So please look into the files without executing them first! I want to have your feedback, especially of experienced users.

xenopeek, do you think it would be better if I post the text in the files rather than the files themselves? Then anyone who wants can put the script into a file. I just thought this way would be more convenient.

Have a nice day,
Bastian

[xenopeek]

If these are just bash scripts yes it would be very good if you could put them on github instead. It makes it easier for people to preview/review, as github has source code highlighting and such. All the Linux Mint developed software is on github also; it also makes it easier for others to contribute improvements to your script! Doing that here on the forums is going to be tedious and error prone work, let's use the tools available. You don't even need to use git to work with github; you can edit files online.

[Mint_BackupRestore]

OK, got it. I'll move them next weekend to github.

Cheers,
Bastian

[Mint_BackupRestore]

Dear Linux Mint Users,

thanks to the hint of xenopeek, the files are now available through "github". You can see the contents of the files before downloading them if you follow the link below. Thus increasing security, user safety and transparency. Additionally it is easier for us to work together when maintaining and updating the files.
If anyone has good (or bad) experiences with the scripts please let me know in this forum. We'll improve them together.


The files are now available through https://github.com/MintBackupRestore/MintBackupRestore
______________________________________________________________________________________________________________________

[Mint_BackupRestore]

Version 10 is out. Which now uses *.tar.7zip format as replacement for *.zip. The backup can thus still be opened manually. The advantage is that the new format now stors rights and owners of the files.
Zip could only store rights.

Best regards,
Mint_BackupRestore

[Mint_BackupRestore]

Release version 3.0 is out.

Current version includes "backup script version 15" and "restore script version 25stable". This is the first version that is released as stable out of the following reasons:
+ tested all options many times when moving from Mint 16 to 17
+ tested many times within one version (Mint 16 or Mint 17)
+ no bug reports from users worldwide

Download at:
https://github.com/MintBackupRestore/MintBackupRestore

Besr regards,
MBR

[expat_tony]

-> Mint_BackupRestore :
Dear Bastian
I'm a relative newbie, wanting to upgrade from LMC16 to LMC17.
I just ran your MintBackup script v15, which worked exactly as described in the many comment lines and echos within it (a best practice lesson that some posters, who only ever cite bare command ines, could learn).
I am extremely impressed so far. This is the kind of utility and the level of help that would wean a lot more Windows victims off their drug. It gives me a sense of confidence about changing the Linux system. If MintBackup&Restore is not already in the Software Centre, I hope people will help you take the necessary validation steps to make it so

Before I do the install of Qiana, I hope you can tell me if I've understood the following correctly:
- My home folder, with all subdirectories e.g. Downloads, is one of the things now safely inside that tar.7zip file on my external HDD, so I can happily let 17 overwrite the original.
- This is the code line you use in the Backup script to ensure people don't have to spend hours restoring software they added post-installation of 16 :
tar -upvf "$destination$version$Date$ArchiveFormat" /Installed_Programs/

And now a few questions about the restore process :
- In the LM16 terminal, I see just 2 UUIDs in my current /etc/fstab file that I would have to change. Is this what you would expect?
- I should expect to be able to use commands apt, 7za, tar, xdg-open and blkid as normal on the vanilla LMC 17 terminal, is that right?
- I have changed some default fonts on my Office Writer, but not added or removed any, nor in the OS. Do I need to reactivate this line?
#cp -rv --preserve=ownership,timestamps,mode $temp_dir/etc/fonts/ /etc/ # activate if you want to restore fonts settings
- Much more important re: Office. I removed LibreOffice and now use OpenOffice4.1. I know it's on the backup (saw the filenames) but must I remove Libre Office from LM17 before I restore, to avoid a failure?
- I note there's a debate about whether to activate the following or not:
#cp -rv --preserve=ownership,timestamps,mode $temp_dir/etc/apt/sources.list /etc/apt/sources.list # some people write this is important (I would not do it because you get packages from former distro). I'm minded to follow your advice here, but what kind of Petra content might be disabled in Quiana and how would I restore it if need to?

I amazed at myself that I can ask you stuff at this level of detail, but it's really a tribute to your scriptwriting skills. In theory I'd love to help your project, but it'd have to be something that requires almost no knowledge of linux command syntax.
Tony

PS. In backup v15 you refer to it as v13, and restore v25 is still called 24beta. More a hint for future updating than changing text in these versions, as I've now made it public

[Mint_BackupRestore]

Dear Tony,

first of all thanks a lot for your nice feedback. I really appreciate it.

Let me try to answer your questions between your lines:

>> 1.) My home folder, with all subdirectories e.g. Downloads, is one of the things now safely inside >> that tar.7zip file on my external HDD, so I can happily let 17 overwrite the original.

Yes, the entire home folder is in the backed up file with all subdirectories. You can open the *tar.zip file by double clicking on it to see if all files are there (Linux Mint has a GUI to look at 7zip as well as tar files).
Note that there is never a 100% safety for any kind of backup. To be even more safe you can try the following additional steps.
a.) I always have two partitions on my hard-drive which are formatted with EXT4. On one I keep my old Linux (e.g. 14) and on the second one the next version (e.g. 15). When version 16 comes out I then overwrite version 14 and so on. This way I have an additional backup and can always jump back one version
b.) You can make a backup of your old Linux version by using Clonezilla (http://www.clonezilla.org)

>> 2.) This is the code line you use in the Backup script to ensure people don't have to spend hours restoring software they added post-installation of 16 :
tar -upvf "$destination$version$Date$ArchiveFormat" /Installed_Programs/

No, that is not correct. The command tar -upvf just tells tar to add the “Installed_Programs” folder to the archive (the u option means update, the p option means preserve all users and owners, the v option stands for verbose output, the f option tells tar that it should use the “$destination$version$Date$ArchiveFormat” as filename to store the tar archive).
What actually backs up all the installed packages (just the names for later reinstallation and not the entire packages) is the following command some lines before in the script:

$ dpkg --get-selections > /Installed_Programs/Package.txt

>> 3.) In the LM16 terminal, I see just 2 UUIDs in my current /etc/fstab file that I would have to change. Is this what you would expect?

In principle each partition or drive in your computer has an UUID. The fstab just tells linux were to mount a specific UUID. Since I always edit the fstab quit strongly and adjust everything to the way I like it a backup of the fstab file is very important for me. In your case it seems that you only have two partitions but it is difficult to tell from your comment above. Explaining how to edit fstab from scratch is a bit too tideous for this post. Please check this site for more details and if you still have questions come back to me: https://wiki.archlinux.org/index.php/fstab
Note, usually you can keep all settings and all UUIDs the same in the fstab when moving from one Linux version to the other. But if you have formatted a partition during installation the UUID of this single partition will change. Thus you have to update this singe UUID in the fstab.

BTW, you can list all UUID's of all drives on your computer by typing $ sudo blkid
into the terminal.


>>4.) I should expect to be able to use commands apt, 7za, tar, xdg-open and blkid as normal on the vanilla LMC 17 terminal, is that right?

In all versions of Linux Mint (and Ubuntu) I have tried the scripts and all of them have the packages you mentioned preinstalled. You should be able to run the commands directly on the command line. But it is more laborious and you have to set all variables (e.g., name of the backup file) by hand.

>> 5.) I have changed some default fonts on my Office Writer, but not added or removed any, nor in the OS. Do I need to reactivate this line?
#cp -rv --preserve=ownership,timestamps,mode $temp_dir/etc/fonts/ /etc/ # activate if you want to restore fonts settings

Since I don't edit the default fonts in Libroffice. Frankly speaking I don't know 100%. But this is why your feedback is so important to me. I would assume that you will have to reactivate (romove #) in the line you cite above (PLEASE BE SURE TO BE IN THE CORRECT RESTORE OPTION IN THE SCRIPT).

Could you please explain to me the exact steps you did when changing the default fonts (e.g., downloaded font from http://, then copied the font to folder XYZ, then changed settings in Libreoffice to 123). Then I can check the script is restore that or not. From other users I have also been asked to add this or that to the script. Thus if you can explain to me what you did, I can try to add these steps to the script if necessary.

>> 6.) Much more important re: Office. I removed LibreOffice and now use OpenOffice4.1. I know it's on the backup (saw the filenames) but must I remove Libre Office from LM17 before I restore, to avoid a failure?

No you don't. The script only reinstalls packages you had on the last Linux Version and updates them to the current version. But it does not delete any packages. Thus you will end up having both LibreOffice and Openoffice. You will then have to uninstall Libreoffice by hand.

Note that Openoffice is no longer being developed, thus the migration to Libreoffice might make sense.


>> 7.) #cp -rv --preserve=ownership,timestamps,mode $temp_dir/etc/apt/sources.list /etc/apt/sources.list # some people write this is important (I would not do it because you get packages from former distro). I'm minded to follow your advice here, but what kind of Petra content might be disabled in Quiana and how would I restore it if need to?

If it is really disabled in in Quiana, then I would do the following: 1.) Install and update all packages using the Quiana repositories as far as possible. Then restore your old “sources.list” file by hand (just doubleclick on the *.tar.7zip archive and look for the file). Put that file into your /etc/apt/ directory. And install that one package using “apt install packagename”. Then of coure restore the sources.list to Quiana.

However, there are usually good reasons that a distribution chooses to exclude a certain package. Thus I would look for an alternative software rather then installing obsolete packages. With the scripts I want you to quickly restore the most important settings but not “downgrade” back to an obsolete Linux version

>> 8.) In theory I'd love to help your project, but it'd have to be something that requires almost no knowledge of linux command syntax.

Well Tony, you have already helped, since you gave me some insights in your needs as a user. I will pick that up in newer versions. Please tell me how to modify the fonts. I would be interested to pick that up. Of course some things you just cant migrate between two distros. But we'll see.
What you can (if you want) really help me with is testing the scripts and reporting issues. If you have spare time it would be great if you can make three partitions. In partition 1 you install an old Mint version and in partition 2 you install a new version. Then you make images of both operating systems using Clonezilla (http://www.clonezilla.org) on partition 3. Then try migrating settings back and forth using the beta scripts. If you find that something is not working as it should or have an idea for improvements please write them to me. When something gets messed up or you want to go back to the original frehsly installed Linux version, restore it using Clonezilla, Then you can further try the scripts.


>> 9.) PS. In backup v15 you refer to it as v13, and restore v25 is still called 24beta. More a hint for future updating than changing text in these versions, as I've now made it public

Yes, I already noticed that too. It is embarrassing, but I never change the content of a file when I have already published it as a certain version and don't want to hide my mistakes. I just forgot to update the "version text" in the script when uploading. It is fixed in the current beta versions. You can try those as well. They are more colorful now

P.S. Since I just bought a house and am renovating + working hard on the job, I have to make a programing break on the scripts for the next 2 Months. But I'll be back soon and hopefully with some support from your side

Again thanks for the friendly feedback and hope to talk to you soon,
MBR

[expat_tony]

Hi Bastian
Great tips in your reply, thanks

I always have two partitions on my hard-drive which are formatted with EXT4. On one I keep my old Linux (e.g. 14) and on the second one the next version (e.g. 15). When version 16 comes out I then overwrite version 14 and so on. This way I have an additional backup and can always jump back one version
b.) You can make a backup of your old Linux version by using Clonezilla (http://www.clonezilla.org)

The command tar -upvf just tells tar to add the “Installed_Programs” folder to the archive (the u option means update, the p option means preserve all users and owners, the v option stands for verbose output, the f option tells tar that it should use the “$destination$version$Date$ArchiveFormat” as filename to store the tar archive).

First time I've seen all 4 options explained

BTW, you can list all UUID's of all drives on your computer by typing $ sudo blkid
into the terminal.

I used blkid and found there are UUIDs for 3 partitions, one being for the Win dual install. Odd that I didn't see that one when I typed cat /etc/fstab the time before. Maybe 'sudo' makes the difference.

I would assume that you will have to reactivate (romove #) in the line you cite above (PLEASE BE SURE TO BE IN THE CORRECT RESTORE OPTION IN THE SCRIPT).

I will do that, it can't harm anything and the changes are quick to do manually if it turns out they were not stored by OpenOffice.

Could you please explain to me the exact steps you did when changing the default fonts (e.g., downloaded font from http://, then copied the font to folder XYZ, then changed settings in Libreoffice to 123). Then I can check the script is restore that or not.

I did no file management operations at all, I simply changed default font settings within the Office software.

You will then have to uninstall Libreoffice by hand.

Last time I used apt-get remove from the terminal. Would it be better to include --purge as well?

Note that Openoffice is no longer being developed, thus the migration to Libreoffice might make sense.

That's news to me. Apache only just released 4.1 at the end of April 2014. Where did you read that?

If it is really disabled in in Quiana, then I would do the following: 1.) Install and update all packages using the Quiana repositories as far as possible. Then restore your old “sources.list” file by hand (just doubleclick on the *.tar.7zip archive and look for the file). Put that file into your /etc/apt/ directory. And install that one package using “apt install packagename”. Then of coure restore the sources.list to Quiana.

Again, really clear and with enough detail

If you have spare time it would be great if you can make three partitions. In partition 1 you install an old Mint version and in partition 2 you install a new version. Then you make images of both operating systems using Clonezilla (http://www.clonezilla.org) on partition 3.

What partition sizes would be needed? I'm a bit short of space to shrink much more. Also what sort of partition, not all primary I hope? Also got a big issue with my formerly bootable USB disk. Needs a total reformat and repopulating with ISOs. Would you recommend unetbootin? Haven't used that before

[Mint_BackupRestore]

Dear expat_tony,

again see my response between your lines:

I did no file management operations at all, I simply changed default font settings within the Office software.

Then according to this site: https://forum.openoffice.org/en/forum/v ... hp?t=12426
the scripts will also restore your openoffice settings (as well as libreoffice settings). The scripts restore your entire home folder.

Last time I used apt-get remove from the terminal. Would it be better to include --purge as well?

Why not just use the Mint Package Manager (Menu --> Package Manager)?

That's news to me. Apache only just released 4.1 at the end of April 2014. Where did you read that?

According to Wikipedia it was discontinued in 2011: http://en.wikipedia.org/wiki/OpenOffice.org.

What partition sizes would be needed? I'm a bit short of space to shrink much more. Also what sort of partition, not all primary I hope? Also got a big issue with my formerly bootable USB disk. Needs a total reformat and repopulating with ISOs. Would you recommend unetbootin? Haven't used that before

All Linux installations need EXT partitions. I would recommend EXT4. Linux doesn't require primary partitions. Extended can also be used. You need around 10 Gigs for each installation. Sorry, I don't know what "unetbootin" is. Never used it. So I guess you don't need it

Best regards and thanks for offering your help,
MBR

[expat_tony]

Bastian, hi.

You need to read this: http://en.wikipedia.org/wiki/Apache_OpenOffice. It's a far better product now, e.g. it is docx / xlsx / etc compatible.

That openoffice.org wiki is the first piece of misleading information I've ever seen on Wikipedia, only ever heard about things like that before.
Okay, there are two links to Apache Open Office inside the article, but the fact that you overlooked them makes me wonder how many other users think it's dead and gone. I think I'll write a post on the Apache OO forum to let them know.

unetbootin is for creating bootable USB drives on memory sticks, for those of us who need to dual boot to Windows software that we can't get 'Wine" to handle. What's your favourite one of those? I have used easy2boot so far, which is good, but I'd like to try sth else.

Thanks for the rest of the info.
Tony

[expat_tony]

HI Bastian

After some puzzling and searching in blogs, I finally made a change that enabled me to save my current settings of 17 Qiana.
It seems that I was missing p7zip-full in order to run MintBackup successfully. Because that's what happened after I added it to p7zip.

So now I have tar.7zip files for both LM 16 & 17, and I can restore the 16 Petra settings to see what happens, without fear of having to re-do the whole installation.
Won't happen tonight, I'll keep you updated.

This post is dated after the PM I sent you.
Tony

[aalemann]

Dear Mint_BackupRestore,

First of all, I should say that I am not a too experienced user.
I very much like your script but I have problems with the 2nd step of the backup procedure: backing up the etc settings is not working, your script is just waiting with the message

Code: Select all

"Now backing up your etc-settings (all of them)"

So I tried to manually do this via

Code: Select all

tar -upvf etcBackup /etc/

and I get the error

Code: Select all

tar: /etc/passwd-: Cannot open: Permission denied
tar: Exiting with failure status due to previous errors

Using sudo, it works, so I tried "sudoing" your script, but still have the same error.

I have somehow the feeling this is a very basis problem I am facing....?

Thanks,
aalemann

edit: I am using v15 of your backup script

[aalemann]

Ok, it was obviously my fault: I had a relatively large home folder (~40 GB) and adding the etc-setting to the tar file took a long time, something on the order of 10 min.

I therefore have one minor suggestion for "improvement" of the backup script:
Instead of just echoing

Code: Select all

echo "Now backing up your etc-settings (all of them)"

you could echo something like

Code: Select all

echo "Now backing up your etc-settings (all of them) and adding them to the tar-archive"

.

As for the restore script v25stable, I have two comments:
1) in line 3 it says

Code: Select all

# Restore_Linux_Settings V.24beta: Restore Linux Mint settings from >>Backup_Linux_Settings<< script. #

which is obviously a typo
2) the check, if 7zip is installed on the system did not work in my case for some reason: it was not installed but the script detected it to be installed (on a clean, fresh installed Linux Mint 17 Qiana). Unfortunately, I don't know why this has happened...

Finally: thanks a lot for you scripts!

[Znap!]

For some reason, I can't get your script to work. After downloading and editing the backup.sh file with gedit, I then get the following error after trying to run it in the terminal:

Backup_Linux_Settings_v15.sh: line 5: syntax error near unexpected token `newline'
Backup_Linux_Settings_v15.sh: line 5: `<!DOCTYPE html>'

What am I doing wrong here? I'd love to get it work properly.

[edit] Removing or editing the first line of code that says <!DOCTYPE html> doesn't help. After removing it, it gives the same syntax error message, using the second line of code in < and > brackets.

[Znap!]

Okay, I get it. I downloaded straight from https://github.com/MintBackupRestore/MintBackupRestore by right-clicking and 'saving as', instead of clicking the file and downloading the raw version. That means the whole html code was included. Stupid, but since it was my first time using github, I imagine it's a mistake that other noobs might make too. Anyhoo, it seems to be working fine now. Thanks!

[Znap!]

Mierda. Just took hours for the whole backup program to finish. Lo and behold, what do I end up with? Nothing. No archive, no backup, nothing.

Does this explain it?

Code: Select all

Backup_Linux_Settings_v15.sh: line 163: 7za: command not found
Compression is finished.
Deleting the temporary folder for storing software lists.
Deleting the temporary tape archive for storing all files, while preserving all owners and rights.
Backup script has finished.
If there were no errors, you can now find a *.tar.7zip-file in the folder /media/snaap/Backup/ that includes all your backed up information.
Have a nice day.

[CyS]

Hello,

I am rather new at this: I wish to change my HDD for a SSD in my laptop and thinking of doing a fresh install with encryption. then restoring everything with your script (i am in the process of backing all up)

I am running LM 17 Cinnamon x64 - will be installing the same version.

I am also a inexperienced user in Linux... to make that clear.

Since everything will be saved in the back-up, I am copying all relevant folders too: docs, pics, downloads - i have some program kits there.

Any head-ups? After reading all documentation and the thread, all should be fine.

Thank you!