DevHeads.net

Merging two linux partitions

I'm asking for advice so that I don't create additional problems for myself.

I've cloned my 120GB SSD to a 1TB SSD and now I want to extend the Linux
partition so that it uses all of the extra space of the drive.

I have formatted (using gparted) the remaining 800GB to ext4 so now I
have three partitions Windows, Linux and the new partition. So, the
question is how do I merge the Linux partition with the new partition
without losing any data?

The easiest solution would be to delete my Linux partition and create a
new larger / partition and then reinstall. That would be a lot of effort
even though I have a backup of my home directory.

One other question. Towards the end of the Clonezilla options was a
question asking whether to copy 400 odd bytes to the first sector of the
new drive. I selected yes, but now a reboot brings up a menu with two
options. Selecting either does not reboot the computer. Shutting down
and restarting works as usual.

Comments

Re: Merging two linux partitions

By Ken D'Ambrosio at 05/14/2019 - 09:58

IDLE MUSINGS AHEAD -- NO PRACTICAL APPLICATION

So, I got to thinking about how one *would* merge two partitions... and
I don't really have a good answer. I mean, for example, how do you deal
with identically-named files/directories? Which made me think the whole
thing was simply impossible, when it occurred to me that there were
"union" FS options which, logically, merged multiple partitions. Just
wondering if anyone knows how *those* dealt with identical filenames and
any other obstacles?

-K

Re: Merging two linux partitions

By Liam Proven at 05/14/2019 - 12:07

On Tue, 14 May 2019 at 16:01, Ken D'Ambrosio < ... at jots dot org> wrote:
PartitionMagic could do it.

If, for example, you merged drives C, D and E, you ended up with a big
drive C which contained subfolders called "\D\" with the full contents
of D: and "\E\" with the full contents of E:

It was then up to you to move stuff around to sort it.

However, the thing is this:

When you move from one drive to another drive, including separate
partitions, the OS must copy the data from source to destination, then
when it's copied, remove the original file... then repeat this for
every file. This is unavoidably slow. It applies even on the same
physical drive, if there are multiple partitions.

But if you move a file from one folder to another folder in the same
partition, on any modern filesystem, the OS can just rename the file
from

/data/my/old/file

... to...

/data/my/new/file

The actual contents of "file" don't move. So it's very, very fast.

So cleaning up the folders left by a PQMagic partition merge was quite
quick. It was the merge that took hours. It copied as much data as
would fit, shrank D: as much as possible by moving the start, enlarged
C: and then copied some more... and repeat. This could be a *very*
lengthy process.

This kind of thing is the reason that logical volume management systems exist:

https://en.wikipedia.org/wiki/Logical_Volume_Manager_(Linux)

LVM is complicated and hard to understand. If the above article makes
little sense, don't blame yourself. For standalone workstations, I
recommend avoiding it.

So, there's LVM, then on top of the LVM space, you have partitions.
Those are formatted with a filesystem, such as ext4, or older
enterprise filesystems from old commercial Unixes, such as JFS (from
IBM's AIX and OS/2), or XFS (from SGI IRIX).

<a href="https://en.wikipedia.org/wiki/XFS" title="https://en.wikipedia.org/wiki/XFS">https://en.wikipedia.org/wiki/XFS</a>

https://en.wikipedia.org/wiki/JFS_(file_system)

Fedora enables LVM by default which is just one reason I avoid Fedora.

Then to make matters worse, there are filesystems which support
"subvolumes" inside a partition, e.g. Btrfs.

<a href="https://en.wikipedia.org/wiki/Btrfs" title="https://en.wikipedia.org/wiki/Btrfs">https://en.wikipedia.org/wiki/Btrfs</a>

Btrfs is the default FS of SUSE Linux.

Then you have subvolumes inside partitions on top of LVM volumes on
top of disks, and personally it all makes my head spin.

*Because* LVM is hard, and its functionality overlaps with
partitioning, there are projects that try to merge them.

For Linux, there was EVMS:

<a href="http://evms.sourceforge.net/" title="http://evms.sourceforge.net/">http://evms.sourceforge.net/</a>

Unfortunately, it did not catch on, so we have LVM instead.

<a href="https://lwn.net/Articles/14816/" title="https://lwn.net/Articles/14816/">https://lwn.net/Articles/14816/</a>

<a href="https://unix.stackexchange.com/questions/22885/is-there-a-more-modern-or-more-popular-version-of-evms2" title="https://unix.stackexchange.com/questions/22885/is-there-a-more-modern-or-more-popular-version-of-evms2">https://unix.stackexchange.com/questions/22885/is-there-a-more-modern-or...</a>

RH does not support Btrfs. However, because it wants some of the
features of Btrfs, RH is now building its own new combined logical
volume manager / partitioner / filesystem, Stratis:

<a href="https://stratis-storage.github.io/" title="https://stratis-storage.github.io/">https://stratis-storage.github.io/</a>

Stratis combines an LVM layer with the XFS filesystem.

I have heard comments that Stratis is in effect re-creating a subset
of the functionality of EVMS.

This is a very typical Linux development path.

The richest filesystem/volume manager from commercial Unix is ZFS,
from Sun (now Oracle) Solaris.

<a href="https://en.wikipedia.org/wiki/ZFS" title="https://en.wikipedia.org/wiki/ZFS">https://en.wikipedia.org/wiki/ZFS</a>

Like JFS and XFS, ZFS is now open source. However, under a licence
that is incompatible with the Linux kernel's GPL licence.

So you _can_ compile a Linux kernel with built-in XFS, but it violates
the licence.

However, Ubuntu has found a way around this, with ZFS being a loadable
module (AIUI) that isn't part of the kernel itself.

(AIUI. IANAL. Clarification welcome.)

Ubuntu Server offers ZFS instead, in place of Btrfs in SUSE or Stratis
in Fedora (or XFS in all of them).

ZFS can replace the LVM _and_ also ext4/XFS/JFS, and therefore Stratis
too, but neither SUSE nor RH will bundle ZFS because of licence
concerns.

Apple _was_ going to bundle ZFS but it too decided the licensing was
too tricky and it has developed its own system, APFS. But then Apple
no longer is trying to compete in the server market.

<a href="https://en.wikipedia.org/wiki/Apple_File_System" title="https://en.wikipedia.org/wiki/Apple_File_System">https://en.wikipedia.org/wiki/Apple_File_System</a>

Yes, it is confusing. Yes, it is a mess. Yes, there are too many standards.

<a href="https://xkcd.com/927/" title="https://xkcd.com/927/">https://xkcd.com/927/</a>

Re: Merging two linux partitions

By Robert Heller at 05/14/2019 - 09:22

1) delete the (new empty) 800GB partition (note down its end). You did not
need to make it.
2) edit the existing linux partition so its end is what the 800GB partition
was. Be careful to leave the start exactly as it is.
3) You now have a 120G file system sitting at the beginning of a 920GB
partition.
4) You may need to reboot to get the partition info updated in the kernel.
5) Use resize2fs to resize the partition

Re: Merging two linux partitions

By oxy via ubuntu-users at 05/14/2019 - 09:48

On Tue, 14 May 2019 09:22:49 -0400 (EDT), Robert Heller wrote:
Btw. even if the OP should need to move the partition, it unlikely will
cause more trouble than the need to reinstall the bootloader. I never
lost data when using gparted or command line to reorganise partitions
or at least I don't remember that this ever happened. I only wanted to
point out that using a copy command is not failure-prone compared to
reorganising partitions. Usually nothing unwanted happens when
using gparted or command line. However, a backup doesn't harm.

Re: Merging two linux partitions

By Robert Heller at 05/14/2019 - 14:44

I *always* set up disks with at most 2 "physical" partitions: one for /boot
and the other a LVM PV. Then I create as many however large or small LVM
Volumes. Growing or shrinking a LVM Volume is a trivial (and generally safe)
task. When transfering to new/larger disk, I just add the new disk's PV, and
move things off the old disk and then remove the old disk's PV. No need to
mess with cloning or even resizing, unless/until I need to. The only really
tricky bit is migrating from one RAID mirror set to another.

Re: Merging two linux partitions

By Ken D'Ambrosio at 05/13/2019 - 22:12

On 2019-05-13 21:51, Phil wrote:
Heh. Glad you asked! Because you're overthinking this, and making it
w-a-y harder than it really is. What you want to do:
* Delete your new, empty partition
* Resize your partition with data on it
Done! Gparted will happily help you with the resize.

-Ken

Re: Merging two linux partitions

By oxy via ubuntu-users at 05/14/2019 - 01:12

On Mon, 13 May 2019 22:12:00 -0400, Ken D'Ambrosio wrote:
Hi,

in theory this is how it should work(, too). IMO it would be better to
do it different in the first place.

FWIW resizing the partition might require to reinstall the bootloader,
a link is provided at the end of this email.

I'm puzzled about what "cloned" is for.

If the old partition was cloned via "dd", the partition table could be
broken.

If I would migrate from a 120 GB SSD to a 1 TB SSD, I would simply
format the new partition/s to the wanted size/s and from an Ubuntu live
DVD I would run

sudo -i
cp -ai /mount_point_of_old_SSD_linux_partition_1/* /mount_point_of_new_SSD_linux_partition_1/

When using a simple copy command (or by restoring from a tar archive backup
or by rsync etc., too), it doesn't matter if the old partition should be
smaller than the new partition.

If the install does use more than one partition, I would repeat it for
each partition.

cp -ai /mount_point_of_old_SSD_linux_partition_n/* /mount_point_of_new_SSD_linux_partition_n/

Note, globbing is tricky, the asterisk ignores hidden items. If there
should be a hidden item inside the parent directory (it doesn't matter
for hidden items inside of sub directories) of a Linux partition, those
items need to be copied by additional steps.

cp -ai /mount_point_of_old_SSD_linux_partition_n/.hidden_item /mount_point_of_new_SSD_linux_partition_n/

After copying, I would install the bootloader. How to do this, depends
on the used bootloader. I'm using syslinux. Since the default for
Ubuntu is Grub 2, here is a link on how to install Grub 2 via a live
media:
<a href="https://help.ubuntu.com/community/Grub2/Installing#via_the_LiveCD_terminal" title="https://help.ubuntu.com/community/Grub2/Installing#via_the_LiveCD_terminal">https://help.ubuntu.com/community/Grub2/Installing#via_the_LiveCD_terminal</a>

Perhaps you need to consider something else, if you should use UEFI.
I've never used it.

Regards,
Ralf

Re: Merging two linux partitions

By Liam Proven at 05/14/2019 - 06:17

On Tue, 14 May 2019 at 07:14, Ralf Mardorf via ubuntu-users

Um. I hesitate to criticise but that is a _really_ failure-prone,
complex and difficult way of doing it, and it doesn't address other
OSes on the disk, either.

Easy way:
* attach new disk as well as old, e.g. externally on USB
* use cloning tool to image old disk to new
* use Gparted to adjust sizes as desired

Done.

Re: Merging two linux partitions

By oxy via ubuntu-users at 05/14/2019 - 07:14

On Tue, 2019-05-14 at 12:17 +0200, Liam Proven wrote:
Apart from the asterisk it's idiot-proof. I decided to post a copy
command using an asterisk, since apart from some unwanted .trash/ folder
root directories very seldom contain hidden files.

However, you could verify after coping:

[rocketmouse@archlinux computer]$ ls -lA new_SSD_partition/ old_SSD_partition/{.,}parent_?
new_SSD_partition/:
total 0

old_SSD_partition/parent_1:
total 0
drwxr-xr-x 2 rocketmouse rocketmouse 40 May 14 12:42 sub_1
drwxr-xr-x 2 rocketmouse rocketmouse 40 May 14 12:42 .sub_2

old_SSD_partition/.parent_2:
total 0
drwxr-xr-x 2 rocketmouse rocketmouse 40 May 14 12:42 sub_1.2
drwxr-xr-x 2 rocketmouse rocketmouse 40 May 14 12:42 .sub_2.2
[rocketmouse@archlinux computer]$ cp -ai old_SSD_partition/* new_SSD_partition/; echo $?
0
[rocketmouse@archlinux computer]$ diff -r old_SSD_partition/ new_SSD_partition/
Only in old_SSD_partition/: .parent_2

The output of "echo $?" confirms that the exit status is "0", IOW no error happened.

The recursive diff command informs about a forgotten .hidden item.

Yes, but the OP only wants to resize the Linux partition.

Gparted will open two dialogs, one that recommends to backup the
install, since resizing a partition could cause data loss and another
that informs that resizing a root partition could break the bootloader.

So why not doing it less risky all in one in the first place by a copy,
rsync, tar archive or something similar? Only the bootloader needs to be
installed, too, but copying from an existing drive to a partition of the
wanted size, is way more secure, than resizing a partition after using a
clone tool.

Re: Merging two linux partitions

By oxy via ubuntu-users at 05/14/2019 - 07:38

location of the unallocated space. Perhaps several partitions needs to
be moved, before the target partition could be resized ;). This could
take very long and the longer it takes to actually copy data, the more
failure-prone it becomes.

Re: Merging two linux partitions

By Phil at 05/14/2019 - 00:38

On 14/5/19 12:12 pm, Ken D'Ambrosio wrote:
Thank you Ken for your reply,

I had attempted resizing the Linux partition, before asking the
question, using the copy of gparted that's on the computer because I
cannot find the gparted boot disk. I don't think that the partition can
be resized while the partition is in use. I'll just have to download and
create another gparted boot disk.

Re: Merging two linux partitions

By Liam Proven at 05/14/2019 - 06:18

On Tue, 14 May 2019 at 06:41, Phil < ... at gmail dot com> wrote:
No, that won't work. It can't resize the disk it is currently running from.

No need. Just use any Ubuntu installation disk.

Re: Merging two linux partitions

By oxy via ubuntu-users at 05/14/2019 - 01:22

On Tue, 14 May 2019 14:38:06 +1000, Phil wrote:
Use an Ubuntu live DVD from the same Ubuntu release of your installed
Ubuntu, since resizing the partition by using gparted could require to
reinstall Grub.