bruno/test-repo
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Add 'boot-disk.org'

Browse Source
This commit is contained in:
Bruno Raoult
2022-05-08 10:13:55 +02:00
parent 61138a2294
commit 719d7ef376
1 changed files with 771 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

771
boot-disk.org Normal file
View File

@@ -0,0 +1,771 @@
#+include: "assets/setup-org.org"
#+title: Second Ubuntu desktop boot disk
#+subtitle: This document is (C) 2014-2020 Bruno Raoult and licensed under the [[https://creativecommons.org/licenses/by-sa/4.0/][Creative Commons Attribution Share Alike 4.0 International license]].
#+TODO: TODO STUDY ONGOING TRANSLATING TESTING | DONE
#+TODO: | CANCELED
* Introduction
It maybe happened to you already : A disk failure.
Or you are unsure of a new distribution upgrade, and you prefer to keep an operational one in case of.
Or you want to try a new distribution while keeping the old one, or simply test a new kernel, such as [[https://www.kernel.org/doc/man-pages/linux-next.html][linux-next]].
Of course, I have already a daily backup (on a different machine), but it does not allow to easily recover in case of serious system issue.
I considered having a mirrored system disk, but it is useful only in case of disk failure, surely not in case of human mistakes.
This document is a step-by-step explanation of the creation of such second (or third) bootable disk, from the copy to the kernel generation. It should work as-is on any debian-based distribution.
* Initial configuration and goal
My desktop configuration is described below :
** DONE Disks
I have 3 500Gb disks, ~/dev/sda~, ~/dev/sdb~, ~/dev/sdc~. Only one is initially used, ~/dev/sdb~, which contains the following partitions :
#+caption: Active system partitions (example)
#+begin_src text
$ sudo sgdisk -p /dev/sdb
Disk /dev/sdb: 976773168 sectors, 465.8 GiB
Model: SAMSUNG HD501LJ
Sector size (logical/physical): 512/512 bytes
Disk identifier (GUID): 6BAC802E-87B0-4DA6-9E60-E008558C6948
Partition table holds up to 128 entries
Main partition table begins at sector 2 and ends at sector 33
First usable sector is 34, last usable sector is 976773134
Partitions will be aligned on 2048-sector boundaries
Total free space is 2029 sectors (1014.5 KiB)
Number Start (sector) End (sector) Size Code Name
1 2048 1015807 495.0 MiB EF00 efi1
2 1026048 42969087 20.0 GiB 8300 root1
3 42969088 51357695 4.0 GiB 8200 swap1
4 51357696 976773119 441.3 GiB 8300 export1
5 1015808 1026047 5.0 MiB EF02 bios1
$ lsblk -l -o PATH,TYPE,PARTFLAGS,SIZE,FSTYPE,PTTYPE,LABEL /dev/sdb
PATH TYPE PARTFLAGS SIZE FSTYPE PTTYPE LABEL
/dev/sdb disk 465.8G gpt
/dev/sdb1 part 495M vfat gpt EFI1
/dev/sdb2 part 20G ext4 gpt root1
/dev/sdb3 part 4G swap gpt swap1
/dev/sdb4 part 441.3G ext4 gpt export1
/dev/sdb5 part 5M gpt
#+end_src
What is important here:
- Partitioning is GPT
- All partitions have a name (/LABEL/ column) which ends with "1". This number means simply "my first disk". It is not linked to an hardware position
- Partitions look /strange/ : (/EFI1/ and /partition 5/) exist as I currently boot with BIOS, but plan to migrate to UEFI boot when I have time (this is not as simple as I originally thought).
Looking at ~/etc/fstab~, corresponding lines are :
#+CAPTION: /etc/fstab (example)
#+begin_src text
$ grep '[[:alpha:]+]1[[:space:]]' /etc/fstab
LABEL=EFI1 /boot/efi.old vfat noauto,defaults 0 2
LABEL=root1 / ext4 errors=remount-ro 0 1
LABEL=export1 /export ext4 defaults 0 2
LABEL=swap1 none swap nofail,sw,pri=0 0 0
#+end_src
** DONE Automount
:PROPERTIES:
:CUSTOM_ID: automount
:END:
The /automounter/ (see ~autofs(8)~), when configured, allows to automagically mount/umount file systems according to rules.
As an example, we can decide that a directory ~/mnt/hdd~ could handle all partitions with a given label (you may choose different rules).
For example, if a given partition label is ~FOO~, you could just do a ~ls /mnt/hd/foo~ to view its contents. In this case, the automounter will mount the partition before the execution of the ~ls~ command. This mount willl be released (un-mounted) later, after some time.
As an example, my ~autofs(8)~ configuration is :
#+CAPTION: /etc/auto.master
#+begin_src text
/mnt/hd /etc/auto.hd --timeout=120 --ghost
#+end_src
This line says :
1. /mnt/hd is managed by autofs,
1. The automounter rules for this directory are in ~/etc/auto.hd~,
1. The mounts will be automatically unmounted if not accessed for 120 seconds,
1. the ~--ghost~ option says that the automounter will create directories in ~/mnt/hd~, without having to access their contents.
See ~auto.master(5)~ for full details.
#+CAPTION: /etc/auto.hd
#+begin_src text
,* -fstype=auto,defaults :LABEL=&
#+end_src
Here, we say that for any X ('~*~') access in /mnt/hd, the partition with a label X will be mounted with the options specified.
See ~autofs(5)~ for the description of automounter maps.
*Practical example* : If a partition's LABEL is ~export2~, when a processus will try to access ~/mnt/hd/export2~, that partition will be automagically mounted. The partition will be un-mounted 2 minutes after being not used.
*Demonstration* : Let say, with the above configuration, that I have a partition ~/dev/sdb4~, with the label ~export1~.
#+CAPTION: automount demonstration
#+begin_src text
$ ls -l /mnt/hd
total 0
drwxr-xr-x 2 root root 0 Oct 13 21:39 EFI1
drwxr-xr-x 2 root root 0 Oct 13 21:39 EFI2
drwxr-xr-x 2 root root 0 Oct 13 14:05 export1
drwxr-xr-x 2 root root 0 Oct 13 01:08 export2
drwxr-xr-x 2 root root 0 Oct 13 02:00 porsche1
drwxr-xr-x 2 root root 0 Oct 13 01:01 root1
drwxr-xr-x 2 root root 0 Oct 13 01:01 root2
$ mount -l | grep sdb4
/dev/sdb4 on /export type ext4 (rw,relatime) [export1]
$ ls -l /mnt/hd/export1
total 32
drwxr-xr-x 8 root root 4096 Oct 2 06:47 home
drwx------ 2 root root 16384 May 20 2018 lost+found
drwxr-xr-x 4 root root 4096 Aug 16 08:50 to-classify
$ mount -l | grep sdb4
/dev/sdb4 on /export type ext4 (rw,relatime) [export1]
/dev/sdb4 on /mnt/hd/export1 type ext4 (rw,relatime) [export1]
$ sleep 120
$ mount -l | grep sdb4
/dev/sdb4 on /export type ext4 (rw,relatime) [export1]
#+end_src
** DONE Goal
I will here duplicate ~/dev/sdb~ (my *disk 1*) on ~/dev/sda~ (*disk 2*), and install ~grub~ on ~sda~ (to make it bootable). Eventually, a script will allow to automate subsequent updates (for instance to run it on a daily basis).
** DONE Ensure destination disk is not in use
In this document, ~SRC~ and ~DST~ variables will be the source and destination disks. In my case : ~/dev/sdb~, and ~/dev/sda~.
#+caption: Setting $SRC and $DST
#+begin_src text
export SRC=/dev/sdb
export DST=/dev/sda
#+end_src
It is necessary to check that ~DST~ is not used. It may be for instance used for:
- A mount point (try ~sudo mount | grep ${DST}~)
- A swap device (try ~swapon | grep ${DST}~)
- By automount
- By another operating system if you have a multi-boot systemavez un multi-boot
- By a database in raw mode, or more generally by any application using directly a raw device
- etc...
Be sure to understand these dependancies *before goint further*.
Once done, you have to umount/"swapoff" (see ~umount(8)~ and ~swapoff(8)~ for details) anything related to ~DST~, and remove any reference to this device in ~fstab~, database configuration, etc.
* *Step 1:* Destination disk partitioning
** DONE Basic concepts
*** DONE Disk partitioning (partition table)
This is a table, stored on disk, which describes the different partitions. Most common nowadays are :
- [[https://fr.wikipedia.org/wiki/GUID_Partition_Table][GPT]] : GTP is used by default on most systems nowadays (Linux, MacOS, Windows).
- [[https://fr.wikipedia.org/wiki/Master_boot_record][DOS/MBR]] : Used on end-of-century operating systems (DOS, Windows). MBR could be found on old machines, and maybe on new disks. You probably remember the "primary/secondary partition" jargon ?
- And [[https://en.wikipedia.org/wiki/Disk_partitioning][many others]]...
*** DONE Partition types
This is the format of an individual [[https://fr.wikipedia.org/wiki/Partition_(informatique)][partition]], within the disk itself. On GNU/Linux, you will use mostly :
- [[https://en.wikipedia.org/wiki/Extended_file_system][Extended File System]] : The most common nowadays. The last version of this family, [[https://en.wikipedia.org/wiki/Ext4][EXT4]], if often the default on many distributions, such as [[https://en.wikipedia.org/wiki/Debian][Debian]], [[https://en.wikipedia.org/wiki/Ubuntu][Ubuntu]] etc...
- [[https://en.wikipedia.org/wiki/Btrfs][BTRFS]] : Designed by Oracle, and relatively recent, this file system has some interesting features, such as snapshots, integration of some RAID configurations, atomic file cloning, etc... It is the default file system on [[https://en.wikipedia.org/wiki/Fedora_(operating_system)][Fedora]] since version 33 (October 2020).
- [[https://en.wikipedia.org/wiki/Hierarchical_File_System][HFS]] and [[https://en.wikipedia.org/wiki/HFS%2B][HFS+]] : If you have a MaxOS machine, for instance dual-booting MacOS/Linux, you will probably have have some HFS+ partitions.
- [[https://en.wikipedia.org/wiki/File_Allocation_Table][FAT]], [[https://en.wikipedia.org/wiki/File_Allocation_Table#VFAT][VFAT]], [[https://en.wikipedia.org/wiki/File_Allocation_Table#FAT32][FAT32]], [[https://en.wikipedia.org/wiki/ExFAT][exFAT]] etc... : The different types/extensions of Microsoft's FAT file system. You will often find them by default on external disks, as they are well supported by most Operating systems. FAT is also used in [[https://en.wikipedia.org/wiki/Unified_Extensible_Firmware_Interface][EFI]] specifications to host the boot system.
- [[https://fr.wikipedia.org/wiki/Espace_d%27%C3%A9change][SWAP]] : As its name says.
- etc...
*** DONE Partitions label
Partitions often have a name (you can see it for instance when you insert a CD, an external disk), etc... This naming can be confusing, as for example, if you use ~gparted~, you will see one ~Label~ column and one ~Name~ column.
- *Label* : This is a name which resides *inside* the partition itself, and depends on the support of such names within this partition type. All partitions types described above (EXT, BTRFS, FAT, SWAP, etc...) have support for some naming, with different counstraints : case sensitivity, label lenght, etc... This label can be created independently of partitition table type (GPT, MBR, APM, etc...), as it is part of the partition itself. As this label is partition-type specific, we can set/modify it while creating the partition, or with different commands (~e2label~ for extX, ~swaplabel~ for swap etc...
- *Name/PARTLABEL* : Only for GPT disks, unsupported on MBR disks. This name is handled at partition table level, therefore it can be set with partitioning tools such as ~parted(8)~, ~sgdisk(8)~, etc...
Practical use ot these names is mostly in ~/etc/fstab~ and for the ~automounter~. As you may know, it is *bad^{tm}* to use physical device names (such as ~/dev/sda1~) as reference, as nothing guarantees they will keep the same name between reboots. We can use instead four different naming schemes : ~UUID~, ~PARTUUID~, ~LABEL~, and ~PARTLABEL~.
I originally switched from UUID/PARTUID (/long/ and /impossible/ to remember) to PARTLABEL, but now I simply use LABEL, reserving PARTLABEL for more descriptive information, such as "/disk 2 root partition/".
Example:
#+caption: Labels in fstab
#+begin_src text
LABEL=foo /foo ext4 defaults 0 2 # GPT or MBR
PARTLABEL=exp /export ext4 defaults 0 2 # Only on GPT
#+end_src
As a sidenote, the ~blkid(8)~ and ~lsblk(8)~ commands can show all above information. Here with blkid :
#+caption: blkid output
#+begin_src text
$ blkid /dev/sdb*
/dev/sdb1: LABEL="EFI1" UUID="25E3-E3C5" TYPE="vfat" PARTLABEL="efi1" PARTUUID="9ce00989-9e23-4866-8c3b-8926604505a0"
/dev/sdb2: LABEL="root1" UUID="13243a74-031c-41b5-8a72-6c03d339e857" TYPE="ext4" PARTLABEL="root1" PARTUUID="6b32e47d-faaa-4fb6-8766-601e471e2305"
/dev/sdb3: LABEL="swap1" UUID="e9aa408c-33a4-41d9-aec3-afca45e5f9bd" TYPE="swap" PARTLABEL="swap1" PARTUUID="8ab9aba3-9331-4e4b-94e7-62ab22f9aeed"
/dev/sdb4: LABEL="export1" UUID="85b3c324-d374-4718-afd9-f6110eb2fcfa" TYPE="ext4" PARTLABEL="export1" PARTUUID="4ab6583a-9417-4c8c-a4bb-10770820471c"
#+end_src
** DONE partition table creation
We will create here a GPT disk able to boot both with BIOS and UEFI methods.
The sizes used will be, for my 500Gb disk:
#+caption: partition scheme example
| # | Partition | Partition type | FS type | Size |
|---+----------------+---------------------------+---------+------------|
| - | protective MBR | - | - | Whole disk |
| 1 | BIOS boot | 0xef02 - BIOS boot | - | 1 MiB |
| 2 | UEFI boot | 0xef00 - EFI boot | fat | 100 MiB |
| 3 | \slash | 0x8300 - Linux filesystem | ext4 | 30 GiB |
| 4 | swap | 0x8200 - Linux swap | swap | 4 GiB |
| 5 | export | 0x8300 - Linux filesystem | ext4 | Remaining |
*Note* : to get a summary of partition types, you may use ~sgdisk -L~.
*** Method 1: General case
Many commands can create a partition table. Here, I will create a ~GPT~ scheme, with ~sgdisk(8)~ :
#+caption: partition table creation
#+begin_src text
$ sudo sgdisk --clear \
--new 1::+1M --typecode=1:ef02 --change-name=1:'BIOS boot partition' \
--new 2::+100M --typecode=2:ef00 --change-name=2:'EFI System' \
--new 3::+30G --typecode=3:8300 --change-name=3:'Linux root partition' \
--new 4::+4G --typecode=4:8200 --change-name=4:'Linux swap' \
--new 5::-0 --typecode=5:8300 --change-name=5:'Linux export partition' \
${DST}
#+end_src
Source : [[https://blog.heckel.io/2017/05/28/creating-a-bios-gpt-and-uefi-gpt-grub-bootable-linux-system/#Creating-a-GPT-with-a-BIOS-boot-partition-and-an-EFI-System-Partition][gpt bios efi boot]]
*** Method 2: Duplicate existing partition table
If ~${SRC}~ and ~${DST}~ disks are identical, and if you wish to keep exactly the same partitioning, you can (1) replicate the partition table (2) generate new random UIDs for the new disk.
#+caption: partition table duplication
#+BEGIN_SRC text
$ sudo sgdisk ${SRC} --replicate=${DST}
The operation has completed successfully.
$ sudo sgdisk --randomize-guids ${DST}
The operation has completed successfully.
#+END_SRC
*** Basic partitions check
We can use ~lsblk(8)~ (or blkid(8) as above) to check the new disk situation :
#+caption: DST still has something wrong...
#+begin_src text
$ sudo lsblk -l -o NAME,SIZE,PARTLABEL,PARTUUID,FSTYPE,LABEL ${DST}
NAME SIZE PARTLABEL PARTUUID FSTYPE LABEL
sda 465.8G
sda1 1M BIOS boot partition 59e81b6b-26cf-40b3-ba8e-2d0366b52a6b ext4 root2
sda2 100M EFI System 843d4f41-d774-44fb-9285-ba310786720a
sda3 30G Linux root partition e14f484f-337a-44f2-8775-dcbe45a69b1f
sda4 4G Linux swap c1bc4b83-2861-4bb8-93de-e1279c58b58c
sda5 431.7G Linux export partition 41397dd5-6e98-4fdb-82af-c0da1afccbb1
#+end_src
We can note that the two last columns (file system type and file system label) are still meaningless for destination disk, as we did not initialize them.
For instance, ~sda1~ contains information about a previous ext4 filesystem. We can wipe this with a ~dd(1)~, or ~wipefs(8)~ :
#+caption: wiping a partition signature
#+begin_src text
$ sudo wipefs /dev/sda1
DEVICE OFFSET TYPE UUID LABEL
sda1 0x438 ext4 38fcc5a5-2faf-4fa3-a8b7-512007938938 root2
$ sudo wipefs --all /dev/sda1
/dev/sda1: 2 bytes were erased at offset 0x00000438 (ext4): 53 ef
#+end_src
#+caption: The final (clean) DST partitions
#+begin_src text
$ sudo lsblk -l -o NAME,SIZE,PARTLABEL,PARTUUID,FSTYPE,LABEL ${DST}
NAME SIZE PARTLABEL PARTUUID FSTYPE LABEL
sda 465.8G
sda1 1M BIOS boot partition 59e81b6b-26cf-40b3-ba8e-2d0366b52a6b
sda2 100M EFI System 843d4f41-d774-44fb-9285-ba310786720a
sda3 30G Linux root partition e14f484f-337a-44f2-8775-dcbe45a69b1f
sda4 4G Linux swap c1bc4b83-2861-4bb8-93de-e1279c58b58c
sda5 431.7G Linux export partition 41397dd5-6e98-4fdb-82af-c0da1afccbb1
#+end_src
Voilà !
** DONE Filesystems creation
If you remember, we will format DST partitions as following:
#+caption: My own mission
| Part # | usage / mount point | FS type | Label/FS name (for fstab) |
|---------+---------------------+---------+---------------------------|
| ${DST}1 | BIOS boot | - | - |
| ${DST}2 | EFI boot | fat | EFI2 |
| ${DST}3 | \slash | ext4 | root2 |
| ${DST}4 | swap | swap | swap2 |
| ${DST}5 | \slash{}export | ext4 | export2 |
Let's do it:
#+caption: ${DST}2 : FAT filesystem, named 'EFI2'
#+begin_src text
$ sudo mkfs.vfat -n EFI2 ${DST}2
mkfs.fat 4.1 (2017-01-24)
#+end_src
#+caption: ${DST}3 : ext4 filesystem, named 'root2'
#+begin_src text
$ sudo mkfs.ext4 -L root2 ${DST}3
mke2fs 1.45.5 (07-Jan-2020)
Creating filesystem with 7864320 4k blocks and 1966080 inodes
Filesystem UUID: 3e921e46-19a7-4c3f-b72a-0169e573bc4f
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
4096000
Allocating group tables: done
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done
#+end_src
#+caption: ${DST}4 : swap partition, named 'swap2'
#+begin_src text
$ sudo mkswap -L swap2 ${DST}4
Setting up swapspace version 1, size = 4 GiB (4294963200 bytes)
LABEL=swap2, UUID=6613ebaf-3000-46ee-917f-c577054b5861
#+end_src
#+caption: ${DST}5 : ext4 filesystem, named 'export2'
#+begin_src text
$ sudo mkfs.ext4 -L export2 ${DST}5
mke2fs 1.45.5 (07-Jan-2020)
Creating filesystem with 113157633 4k blocks and 28295168 inodes
Filesystem UUID: fa670dfa-0a24-4c58-8013-f55a9cca6383
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968,
102400000
Allocating group tables: done
Writing inode tables: done
Creating journal (262144 blocks): done
Writing superblocks and filesystem accounting information: done
#+end_src
Here is the final result :
#+caption: disks labels mapping
#+BEGIN_SRC text
$ sudo lsblk -l -o NAME,SIZE,FSTYPE,LABEL,PARTLABEL,UUID,PARTUUID ${DST}
NAME SIZE FSTYPE LABEL PARTLABEL UUID PARTUUID
sda 465.8G
sda1 1M ext4 root2 BIOS boot partition 38fcc5a5-2faf-4fa3-a8b7-512007938938 59e81b6b-26cf-40b3-ba8e-2d0366b52a6b
sda2 100M vfat EFI2 EFI System 67DC-2F82 843d4f41-d774-44fb-9285-ba310786720a
sda3 30G ext4 root2 Linux root partition 3e921e46-19a7-4c3f-b72a-0169e573bc4f e14f484f-337a-44f2-8775-dcbe45a69b1f
sda4 4G swap swap2 Linux swap 6613ebaf-3000-46ee-917f-c577054b5861 c1bc4b83-2861-4bb8-93de-e1279c58b58c
sda5 431.7G ext4 export2 Linux export partition b4db512d-02fb-448f-af07-64c4ede7e135 41397dd5-6e98-4fdb-82af-c0da1afccbb1
#+END_SRC
** DONE /etc/fstab preparation
:PROPERTIES:
:CUSTOM_ID: fstab
:END:
~/etc/fstab~ will differ between SRC and DST. I prefer to keep both versions on all disks, and copy the correct one as running version later.
First, I prepare it with the following changes :
#+caption: /etc/fstab before changes
#+begin_src text
[...]
LABEL=EFI1 /boot/efi.old vfat noauto,defaults 0 2
LABEL=root1 / ext4 errors=remount-ro 0 1
LABEL=export1 /export ext4 defaults 0 2
LABEL=swap1 none swap nofail,sw,pri=0 0 0
[...]
#+end_src
I create a new section for disk2, and merge the swap information (better to use all available swap).
#+caption: /etc/fstab after adding new information
#+begin_src text
[...]
######################## When booting on disk1
LABEL=EFI1 /boot/efi.old vfat noauto,defaults 0 2
LABEL=root1 / ext4 errors=remount-ro 0 1
LABEL=export1 /export ext4 defaults 0 2
######################## When booting on disk2
#LABEL=EFI2 /boot/efi.old vfat noauto,defaults 0 2
#LABEL=root2 / ext4 errors=remount-ro 0 1
#LABEL=export2 /export ext4 defaults 0 2
######################################## swap
LABEL=swap1 none swap nofail,sw,pri=0 0 0
LABEL=swap2 none swap nofail,sw,pri=0 0 0
[...]
#+end_src
We copy the fstab file to two versions: ~fstab.disk1~ and ~fstab.disk2~ :
#+caption: create /etc/fstabs for different boot disks
#+begin_src text
$ sudo cp -p /etc/fstab /etc/fstab.disk1
$ sudo cp -p /etc/fstab /etc/fstab.disk2
#+end_src
And now, we immediately /fix/ the version for the new disk, commenting-out the disk1 part, and uncommenting the disk2 one :
#+caption: /etc/fstab.disk2
#+begin_src text
[...]
######################## When booting on disk1
#LABEL=EFI1 /boot/efi.old vfat noauto,defaults 0 2
#LABEL=root1 / ext4 errors=remount-ro 0 1
#LABEL=export1 /export ext4 defaults 0 2
######################## When booting on disk2
LABEL=EFI2 /boot/efi.old vfat noauto,defaults 0 2
LABEL=root2 / ext4 errors=remount-ro 0 1
LABEL=export2 /export ext4 defaults 0 2
######################################## swap
LABEL=swap1 none swap nofail,sw,pri=0 0 0
LABEL=swap2 none swap nofail,sw,pri=0 0 0
[...]
#+end_src
* *Step 2* : First copies
** Double-double check everything
GNU/Linux (to be more precise: the magic comes from ~udev(7)~) offers a very convenient naming system for devices. I mean : partitions can be directly be accessed by their name, UUID, etc...
If, like me, you prefer to base your naming on partition labels, you can just have a look in =/dev/disk/by-label= :
#+caption: /dev/disk/by-label example
#+begin_src text
$ ls -l /dev/disk/by-label
total 0
lrwxrwxrwx 1 root root 10 Oct 12 22:07 EFI1 -> ../../sdb1
lrwxrwxrwx 1 root root 10 Oct 12 22:07 EFI2 -> ../../sda2
lrwxrwxrwx 1 root root 10 Oct 12 22:07 export1 -> ../../sdb4
lrwxrwxrwx 1 root root 10 Oct 12 22:07 export2 -> ../../sda5
lrwxrwxrwx 1 root root 10 Oct 12 22:07 porsche1 -> ../../sdc1
lrwxrwxrwx 1 root root 10 Oct 12 22:07 root1 -> ../../sdb2
lrwxrwxrwx 1 root root 10 Oct 12 22:07 root2 -> ../../sda3
lrwxrwxrwx 1 root root 10 Oct 12 22:07 root3 -> ../../sdd1
lrwxrwxrwx 1 root root 10 Oct 12 22:07 swap1 -> ../../sdb3
lrwxrwxrwx 1 root root 10 Oct 12 22:07 swap2 -> ../../sda4
#+end_src
Please ensure that these labels correspond to what you expect, and point to the correct devices.
** Do the filesystems copies
We will mount the new disk partitions we wish to duplicate (in my case: the root and export partitions) in ~/mnt~.
First ensure that in the /disk labels mapping/ table above, the partitions labels correctly
*** DONE Mount destination filesystems (automounter version)
With a correct automount system, such as [[#automount][explained above]], we just create symlinks in /mnt to the corresponding /labels/ in ~/dev/hd~. This is not necessary, but it is simpler to access directly ~/mnt/export2/~ instead of ~/mnt/hd/export2/~.
#+caption: with automounter
#+begin_src text
$ cd /mnt/
$ for d in root export EFI; do ln -s hd/${i}2; done
#+end_src
*** DONE Mount destination filesystems (manual version)
We need to create the same directories than above in ,/mnt and mount the corresponding filesystems :
#+caption: without automounter
#+begin_src text
$ cd /mnt
$ sudo mkdir root2 export2 EFI2
$ sudo mount -t ext4 LABEL=root2 /mnt/root2
$ sudo mount -t ext4 LABEL=export2 /mnt/export2
#+end_src
I do not mount EFI2 partition here, it is useless until migration to UEFI boot is complete.
*** DONE Last checks
We double check that all mount points are correct. We should have something like :
#+caption: last check of destination partitions
#+begin_src text
$ ls /mnt/{root,export}2
/mnt/export2:
lost+found
/mnt/root2:
lost+found
$ mount -l | grep $DST
/dev/sda5 on /mnt/hd/export2 type ext4 (rw,relatime) [export2]
/dev/sda3 on /mnt/hd/root2 type ext4 (rw,relatime) [root2]
#+END_SRC
*** The copies
We eventually make the copies. As it will be long, I run them in background and send me an email. You may prefer to send the output to a file, for instance if your email system is not configured. Or run the commands in different terminals.
*TAKE CARE DESTINATION DIRECTORIES* : Use your own names/mount points.
#+caption: rsync copies. CHANGE DEVICES !!
#+begin_src text
$ sudo rsync -axH / /mnt/root2/ 2>&1 | mail -s "copy root" me@domain.com
$ sudo rsync -axH /export/ /mnt/export2/ 2>&1 | mail -s "copy export" me@domain.com
#+end_src
*Note :* The option ~-x~ above (same as ~--one-file-system~) prevents disallow crossing source filesystems boundaries. ~-H~ preserves hard links. If you use ACLs, or extended attributes, you may consider ~-A~ and/or ~-X~ options.
If you have the automount setup, you may find easier to use something like that, if you have many partitions :
#+caption: rsync copies with automount.
#+BEGIN_SRC text
$ for i in root export; do
> sudo rsync -axH /mnt/${i}1/ /mnt/${i}2/ 2>&1 | mail -s "copy ${i}" me@domain.com &
> done
#+END_SRC
*** basic checks
When copies are done, you can roughly check that DST partitions occupation is /similar/ to SRC :
#+caption: after-copy check
#+begin_src text
$ df -h / /mnt/root2
Filesystem Size Used Avail Use% Mounted on
/dev/sdb2 20G 17G 1.8G 91% /
/dev/sda3 30G 17G 12G 61% /mnt/hd/root2
$ df -h /export /mnt/export2
Filesystem Size Used Avail Use% Mounted on
/dev/sdb4 434G 165G 247G 41% /export
/dev/sda5 424G 166G 237G 42% /mnt/hd/export2
#+end_src
** Reminder table
I strongly suggest you write down a table such as this one, as a reminder :
#+caption: SRC/DEST reminder
| Description | SRC Label | DST Label | SRC dir | DST dir |
|------------------+-----------+-----------+---------------+--------------|
| root partition | root1 | root2 | \slash | /mnt/root2 |
| export partition | export1 | export2 | /export | /mnt/export2 |
| EFI partition | EFI1 | EFI2 | /boot/efi.old | /mnt/EFI2 |
* *Step 3* : Destination disk setup after initial copy
We will need some changes on destination disks. This includes fstab adjustment,
crontabs (if they contain specific device references), grub, etc...
** DONE fstab fix
We simply copy the fstab we prepared earlier [[#fstab][I suggested above]] :
#+caption: fix new disk fstab
#+begin_src text
$ sudo cp /mnt/root2/fstab.disk2 /mnt/root2/fstab
#+end_src
** DONE Make the new disk bootable
I suggest to read some information about ~chroot(8)~ and ~grub~ before going further.
I will mount the new filesystems to a full subtree so that we can create a bootable disk on it. Note that if you have a separate ~boot~ or ~usr~ partition, they must be mounted relatively to new root mount-point. Nothing below should be done if you do not understand the commands. *NOTHING*.
#+caption: Prepare a chroot environment for new disk
#+begin_src text
$ sudo mkdir /mnt/disk2
$ sudo mount LABEL=root2 /mnt/disk2
# if you have separate boot (or /usr) partition do something similar to :
$ sudo mount LABEL=boot2 /mnt/disk2/boot
#+end_src
Now, we bind-mount the virtual filesystems from the running system :
#+caption: Virtual filesystems mount
#+begin_src text
$ sudo mount -o bind /sys /mnt/disk2/sys
$ sudo mount -o bind /proc /mnt/disk2/proc
$ sudo mount -o bind /dev /mnt/disk2/dev
#+end_src
We can now change our root reference with the new disk partitions. I strongly suggest to use a terminal dedicated for that :
#+caption: change root reference to new disk, create its grub configuration
#+begin_src text
$ cd /mnt/disk2
$ sudo chroot .
# update-grub
Sourcing file `/etc/default/grub'
Sourcing file `/etc/default/grub.d/init-select.cfg'
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-5.4.0-48-generic
Found initrd image: /boot/initrd.img-5.4.0-48-generic
Found linux image: /boot/vmlinuz-5.4.0-47-generic
Found initrd image: /boot/initrd.img-5.4.0-47-generic
Found linux image: /boot/vmlinuz-5.4.0-45-generic
Found initrd image: /boot/initrd.img-5.4.0-45-generic
Found memtest86+ image: /boot/memtest86+.elf
Found memtest86+ image: /boot/memtest86+.bin
Found Ubuntu 20.04.1 LTS (20.04) on /dev/sdb2
done
#+end_src
You probably noticed the last line: A boot system was found on /dev/sdb2, which was the original disk.
Let's install grub now, to finish the procedure :
#+caption: install grub on new disk
#+begin_src text
# grub-install ${DST}
Installing for i386-pc platform.
Installation finished. No error reported.
#+end_src
** Testing
You can now reboot your system, and change BIOS settings to boot from new disk. This step is necessary (choosing second disk kernel from first disk grub menu is not sufficient, as it does not confirm grub is correct on second disk).
* Some problems with what was done
There are a couple of issues with what we did.
** fstab
As fstab is part of new disk, we can either :
- Prevent it to be copied, in rsync rules
- Copy the correct fstab on target disk after the rsync. I prefer the latter method.
** Useless dirs
We may want to avoid copying some directories, possibly huge: log files, cache files, virtual file systems, etc...
** Databases
Obviously, copying a database while it is live is a bad^{TM} idea. Again, we have three solutions :
1. Stop it on source disk before the copy, if your system allows it.
1. Separate the database directory sync from the container filesystem one, to allow a shorter database downtime. Something like :
1. copy container filesystem without database directory
2. stop database
3. copy database directory
4. restart database
1. Make a database dump, and include them in rsync, while excluding database directory itself.
The difficulty here is to reload the dumps on target disk :
If we do it after the copy, it probably means we need a second database instance.
If we delay this restore operation, for instance to do it when we boot on target disk, there are risks we simply forget, and we need some changes for database server startup, /that should work also on source disk, as next copies will also include this configuration/.
Solutions 2 and 3 look messy, and are prone to errors.
My preference is for first solution (DB server shutdown before sync, and restart after the copy is done).
*** Mariadb
*TODO *: For mariadb/mysql, check importance of innodb_fast_shutdown global variable (should it be 0 or 1 ?)
As explained above, I prefer to stop/restart database : as it is a desktop, database uptime is not really a priority for me.
#+caption: stopping/starting mariadb before rsync, using systemctl
#+begin_src text
$ sudo systemctl stop mariadb
$ sudo systemctl start mariadb
#+end_src
* TESTING script to ease subsequent copies
Below is a tentative bash script, which will automate the processus *after the first copy*.
We also assume that automounter is configured as [[#automount][explained above]].
And finally, it is assumed that ,/boot is not a separate partition, and that source disk have same structure (1 partition on source disk corresponds to 1 partition on destination disk), and that "/" partitions labels are "rootX", such as root1, or root2.
** TESTING The bash script
#+caption: TODO - all-in one script (with automount)
#+begin_src bash
#!/bin/bash
#
# dup-boot-disk.sh - duplicate live system partitions and install grub on new disk.
#
# This script will not work for all situations, I strongly suggest you don't use it
# if you don't fully understand it.
# dest device (used for grub)
DSTDISK=/dev/sda
# partitions suffixes, for source and destination partitions.
# For example, if we want to copy XX partition, source partition will be
# /mnt/XX${SRCNUM}, and destination will be /mnt/XX${DSTNUM}
SRCNUM=2
DSTNUM=1
# array of partitions to copy
TO_COPY=(root export)
# An alternative to SRCNUM, DSTNUM, and TO_COPY variables would be to have
# an array containing src and destination partitions:
# (partsrc1 partdst1 partsrc2 partdst2 etc...)
# example:
# TO_COPY=(root2 root1 export2 export1)
# declare -i i
# for ((i=0; i<${#TO_COPY[@]}; i+=2)); do
# SRC=${#TO_COPY[$i]}
# DST=${#TO_COPY[$i + 1]}
# etc...
# where we will configure/install grub: mount point, device
GRUB_ROOT=/mnt/root${DSTNUM}
GRUB_DEV=/dev/$(lsblk -no pkname /dev/disk/by-label/root${DSTNUM})
# we will use ".rsync-disk-copy" files to exclude files/dirs
RSYNCOPTS=(-axH --delete)
FILTER="--filter=dir-merge .rsync-disk-copy"
# stop what could be problematic (databases, etc...)
echo systemctl stop mariadb
# partitions copy
for part in ${TO_COPY[@]}; do
SRCPART=/mnt/${part}${SRCNUM}
DSTPART=/mnt/${part}${DSTNUM}
echo copy from $SRCPART to $DSTPART
echo sudo rsync ${RSYNCOPTS} "$FILTER" "$SRCPART" "$DSTPART"
done
# grub install
# mount virtual devices
echo mount -o bind /sys ${GRUB_ROOT}/sys
echo mount -o bind /proc ${GRUB_ROOT}/proc
echo mount -o bind /dev ${GRUB_ROOT}/dev
echo chroot ${GRUB_ROOT} update-grub
echo chroot ${GRUB_ROOT} grub-install ${GRUB_DEV}
# restart stopped process (db, etc...)
echo systemctl start mariadb
#+end_src
** DONE rsync filter files
An example of files that can be excluded. Note the last line, ~/mnt/*/~ : It is linked to automounter, to avoid having all directories to be mounted ruring rsync copy.
#+caption: /.rsync-disk-copy example
#+begin_src text
- /dev
- /media
- /lost+found
- /proc
- /run
- /srv
- /sys
- /tmp
- core
- *~
- #*#
- /etc/fstab
- /mnt/*/
#+end_src
* Links
1. grub
+ /[Stack Exchange]/ : [[https://unix.stackexchange.com/questions/191218/how-to-install-grub-on-a-new-drive][How to install GRUB on a new drive ?]]
+ /[Ask Ubuntu]/ : [[https://askubuntu.com/questions/1001311/how-to-manually-install-grub-with-two-hard-disks][How to manually install grub with two hard disks]]
+ /[Linux Mint Forums]/ : [[https://forums.linuxmint.com/viewtopic.php?t=245872][Linux Mint Fresh Install on Second DISK]]
+ /[TLDP]/ [[https://tldp.org/HOWTO/Hard-Disk-Upgrade/copy.html][Hard Disk Upgrade Mini How-To]]
1. UEFI
+ [[https://blog.getreu.net/projects/legacy-to-uefi-boot/#_modify_the_hard_disk_partitions][Switch Debian from legacy to UEFI boot mode]]
+ [[https://tecadmin.net/copy-a-gpt-partition-table-to-new-disk/][How To Copy a GPT Partition Table to Another Disk using sgdisk]]
1. Others
+ https://www.supergrubdisk.org/