Bug 470675 - Broken NTFS after moving partition
Summary: Broken NTFS after moving partition
Status: REPORTED
Alias: None
Product: partitionmanager
Classification: Applications
Component: general (show other bugs)
Version: 23.04.0
Platform: Arch Linux Linux
: NOR normal
Target Milestone: ---
Assignee: Andrius Štikonas
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2023-06-05 16:36 UTC by Kochi
Modified: 2024-10-08 17:52 UTC (History)
2 users (show)

See Also:
Latest Commit:
Version Fixed In:
Sentry Crash Report:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Kochi 2023-06-05 16:36:39 UTC
SUMMARY
***
NOTE: If you are reporting a crash, please try to attach a backtrace with debug symbols.
See https://community.kde.org/Guidelines_and_HOWTOs/Debugging/How_to_create_useful_crash_reports
***


STEPS TO REPRODUCE
1. Select "Resize/Move"
2. Input space before=0 to move partition left
3. Execute operation

OBSERVED RESULT
Operation reported error (See log below). Got "Reserved fields aren't zero (0, 0, 0, 0, 1, 0)" when checking the partition.

EXPECTED RESULT


SOFTWARE/OS VERSIONS
Windows: 
macOS: 
Linux/KDE Plasma: 
(available in About System)
KDE Plasma Version: 
KDE Frameworks Version: 5.105.0
Qt Version: 5.15.9 (built against 5.15.8)

ADDITIONAL INFORMATION
Logs:

KDE Partition Manager: SMART Status Report

Program version: 	23.04.0
Backend: 	pmsfdiskbackendplugin (1)
KDE Frameworks version: 	5.105.0
Machine: 	Linux 6.2.12-arch1-1 #1 SMP PREEMPT_DYNAMIC Thu, 20 Apr 2023 16:11:55 +0000 x86_64

Move partition ‘/dev/sda4’ to the left by 1.92 TiB
Job: Check file system on partition ‘/dev/sda4’
Command: ntfsresize --no-progress-bar --info --force --verbose /dev/sda4
Check file system on partition ‘/dev/sda4’: Success
Job: Set geometry of partition ‘/dev/sda4’: Start sector: 4,922,705,920, length: 6,442,448,896
Command: sfdisk --force /dev/sda -N 4
Set geometry of partition ‘/dev/sda4’: Start sector: 4,922,705,920, length: 6,442,448,896: Success
Job: Move the file system on partition ‘/dev/sda4’ to sector 4,922,705,920

Copying 314,572 chunks (3,298,533,834,752 bytes) from 4,634,308,509,696 to 2,520,425,431,040, direction: left.
Copying 18 MiB/second, estimated time left: 20:40:00
Copying 18 MiB/second, estimated time left: 18:40:29
Copying 18 MiB/second, estimated time left: 16:23:04
Copying 18 MiB/second, estimated time left: 14:05:59
Copying 18 MiB/second, estimated time left: 11:45:46
Copying 18 MiB/second, estimated time left: 09:23:13
Copying 18 MiB/second, estimated time left: 06:55:28
Copying 18 MiB/second, estimated time left: 04:27:18
Copying 18 MiB/second, estimated time left: 02:01:35
Copying 18 MiB/second, estimated time left: 23:37:35
Copying 18 MiB/second, estimated time left: 21:18:05
Copying 18 MiB/second, estimated time left: 18:58:25
Copying 18 MiB/second, estimated time left: 16:37:58
Copying 18 MiB/second, estimated time left: 14:16:55
Copying 18 MiB/second, estimated time left: 11:55:30
Copying 18 MiB/second, estimated time left: 09:32:28
Copying 18 MiB/second, estimated time left: 07:09:04
Copying 18 MiB/second, estimated time left: 04:45:56
Copying 18 MiB/second, estimated time left: 02:22:58
Copying 18 MiB/second, estimated time left: 00:00:00
Copying remainder of chunk size 7,340,032 from 7,932,835,004,416 to 5,818,951,925,760.
Copying 314,572 chunks (3,298,533,834,752 bytes) finished.
Closing device. This may take a few seconds.

Updating boot sector for NTFS file system on partition ‘/dev/sda4’.

Command:
Command:
Move the file system on partition ‘/dev/sda4’ to sector 4,922,705,920: Success
Job: Check file system on partition ‘/dev/sda4’
Command: ntfsresize --no-progress-bar --info --force --verbose /dev/sda4
Check file system on partition ‘/dev/sda4’: Error
Checking partition ‘/dev/sda4’ after resize/move failed.
Move partition ‘/dev/sda4’ to the left by 1.92 TiB: Error

******

The operation actually moved the data because I've compared the bytes in the unallocated  space (where the orignal data was) with the broken partition after moving.
The part I have most concerned about is what partition manager did in "Updating boot sector for NTFS file system". The partition was compressed NTFS so this step might cause unexpected result I suppose. And because I'm seek for data recovery service's help, such detail should be very helpful.
How is "Updating boot sector for NTFS file system" implemented? I searched partition manager's source code but didn't find anything useful. Could developers make some explanation?
Comment 1 Andrius Štikonas 2023-06-05 22:44:33 UTC
Implementation of update boot sector code is at https://invent.kde.org/system/kpmcore/-/blob/master/src/fs/ntfs.cpp#L180

I.e it just writes it to position 28 of the first and last sectors.
Comment 2 Ishrak 2024-07-17 07:56:32 UTC
Does *partitionmanager* use *parted* under the hood? If so, I recently found that parted would fail to format disk.img but gparted will work. I think I messed up the start of the first partition in parted but *gparted* just did automatically.
Comment 3 Ishrak 2024-07-17 08:00:53 UTC
Does *partitionmanager* use *parted* under the hood? If so, I recently found that parted would fail to format disk.img but gparted will work. I think I messed up the START of the first partition in parted but *gparted* just did automatically.
Comment 4 Andrius Štikonas 2024-07-17 18:23:02 UTC
(In reply to Ishrak from comment #3)
> Does *partitionmanager* use *parted* under the hood? If so, I recently found
> that parted would fail to format disk.img but gparted will work. I think I
> messed up the START of the first partition in parted but *gparted* just did
> automatically.

No, it doesn't use parted, it uses fdisk.
Comment 5 Kochi 2024-07-17 18:39:10 UTC
The noti mails of the new replies remind me of this after a whole year. Luckily I didn't end up with data recovery service because I came across this writeup of The InfoSecurity Challenge contest and the level 3 challenge was exactly this problem, and its solution worked like a charm for me. Thanks the author for the great writings. In conclusion somehow after re-partitioning the byte at `0x20` was `01`, which caused the mounting error.

Ref: https://ctf.zeyu2001.com/2022/tisc-2022/level-3-patient0
Comment 6 Thomas Bertels 2024-10-03 08:32:04 UTC
I can reproduce both this and bug 424045 (moving Btrfs partition) both on Manjaro (with an SSD and a HDD) and in a VM with openSUSE TW.
For some reason, it's harder to reproduce with ext4.

Steps:
- Create a 300 MB partition (Btrfs, FAT32 or NTFS) at the start of the free space, apply
- Move it to the end of the free space, apply

Result (on VM): 
Command: btrfs check /dev/sda4 
Check file system on partition ‘/dev/sda4’: Error
Checking partition ‘/dev/sda4’ after resize/move failed.

The partition is then shown as unknown.
Comment 7 Thomas Bertels 2024-10-03 18:51:35 UTC
I've noticed that gparted resizes the partition when moving it: it grows it to use all the space, then moves the data and finally shrinks the partition to its original size.
Shouldn't KDE Partition Manager do the same ?
Comment 8 Andrius Štikonas 2024-10-07 00:40:04 UTC
(In reply to Thomas Bertels from comment #7)
> I've noticed that gparted resizes the partition when moving it: it grows it
> to use all the space, then moves the data and finally shrinks the partition
> to its original size.
> Shouldn't KDE Partition Manager do the same ?

Yes, I think KDE Partition Manager does exactly the same (see https://invent.kde.org/system/kpmcore/-/blob/master/src/ops/resizeoperation.cpp?ref_type=heads#L69). So I'm a bit confused what exactly causes this.
Comment 9 Thomas Bertels 2024-10-08 16:43:19 UTC
(In reply to Andrius Štikonas from comment #8)
> (In reply to Thomas Bertels from comment #7)
> > I've noticed that gparted resizes the partition when moving it: it grows it
> > to use all the space, then moves the data and finally shrinks the partition
> > to its original size.
> > Shouldn't KDE Partition Manager do the same ?
> 
> Yes, I think KDE Partition Manager does exactly the same (see
> https://invent.kde.org/system/kpmcore/-/blob/master/src/ops/resizeoperation.
> cpp?ref_type=heads#L69). So I'm a bit confused what exactly causes this.

As I understand it, resizeAction() (https://invent.kde.org/system/kpmcore/-/blob/master/src/ops/resizeoperation.cpp?ref_type=heads#L232) finds out based on start and end sector if KDE Partition Manager needs to move, grow, shrink (or a combination) the partition and only one of those operations happen.

The bitwise & operator may make it seem like Shrink and Grow always happen (note that it would be in the wrong order as gparted grows it then shrinks it), but I don't think that's the case based on enum ResizeAction (https://invent.kde.org/system/kpmcore/-/blob/master/src/ops/resizeoperation.h?ref_type=heads#L51).
Comment 10 Andrius Štikonas 2024-10-08 17:52:47 UTC
(In reply to Thomas Bertels from comment #9)
> The bitwise & operator may make it seem like Shrink and Grow always happen
> (note that it would be in the wrong order as gparted grows it then shrinks
> it), but I don't think that's the case based on enum ResizeAction
> (https://invent.kde.org/system/kpmcore/-/blob/master/src/ops/resizeoperation.
> h?ref_type=heads#L51).

Indeed, there is either growth or shrink operation or keep the same size... Never both growth and shrink. There is a non-trivial interaction between growth/shrink and left/right moves but I doubt that it is wrong.

Still we need to understand what is going on. I don't think I ever saw data loss when moving/resizing partition myself. I guess the first thing is I should try to reproduce it somehow.