Bug 447248 - KDE Partition Manager delays for several minutes on Intel RAID/NTFS volume scan
Summary: KDE Partition Manager delays for several minutes on Intel RAID/NTFS volume scan
Status: RESOLVED FIXED
Alias: None
Product: partitionmanager
Classification: Applications
Component: general (show other bugs)
Version: 21.12.0
Platform: Other Linux
: NOR normal
Target Milestone: ---
Assignee: Andrius Štikonas
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2021-12-20 01:44 UTC by Ihor
Modified: 2021-12-25 11:55 UTC (History)
0 users

See Also:
Latest Commit:
Version Fixed In:


Attachments
lsusb log (2.79 KB, text/plain)
2021-12-20 01:44 UTC, Ihor
Details
partitionmanager incorrectly detects partition size, which is lower than used size (132.21 KB, image/png)
2021-12-21 23:33 UTC, Ihor
Details
gparted has no problem detecting partition size correctly (46.91 KB, image/png)
2021-12-21 23:34 UTC, Ihor
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Ihor 2021-12-20 01:44:41 UTC
Created attachment 144694 [details]
lsusb log

SUMMARY
KDE Partition Manager delays for several minutes on USB flash scan

STEPS TO REPRODUCE
1. Take SMI Flash drive 
2. Use balenaEtcher to write latest Manjaro KDE iso
3. Boot Manjaro from this flash drive, try to install Manjaro on local hdd.

OBSERVED RESULT
KDE Partition Manager delays for several minutes on USB flash scan
Manjaro installer (which bundles KDE Partition Manager) hangs forever and often crashes, 
making Manjaro installation a "Quest for the Holy Grail".

EXPECTED RESULT
There should not be such long delay.

SOFTWARE/OS VERSIONS
Operating System: Manjaro Linux
KDE Plasma Version: 5.23.4
KDE Frameworks Version: 5.88.0
Qt Version: 5.15.2
Kernel Version: 5.15.6-2-MANJARO (64-bit)
Graphics Platform: X11
Processors: 12 × Intel® Xeon® CPU X5675 @ 3.07GHz
Memory: 47.0 GiB of RAM
Graphics Processor: NVIDIA GeForce GTX 970/PCIe/SSE2

ADDITIONAL INFORMATION
lsusb log attached
Comment 1 Andrius Štikonas 2021-12-20 10:04:53 UTC
Try running partitoinmanager with

KPMCORE_DEBUG=y partitionmanager

It will print list of commands that it is executing. Unfortunately without timestamps but you should still be able to visually see if one command takes a long time. (I suspect it might be udevadm)
Comment 2 Ihor 2021-12-21 23:29:03 UTC
(In reply to Andrius Štikonas from comment #1)
> Try running partitoinmanager with
> 
> KPMCORE_DEBUG=y partitionmanager
> 
> It will print list of commands that it is executing. Unfortunately without
> timestamps but you should still be able to visually see if one command takes
> a long time. (I suspect it might be udevadm)

Thanks, I got timestamps with this command
KPMCORE_DEBUG=y partitionmanager 1>&1 2>&1 | gnomon --high=10  

That wasn't SMI flash drive responsible, but ntfsresize command that runs every time so long:
 121.7743s   "Command: ntfsresize --info --force --no-progress-bar /dev/md124p1"

md124p is ICH10R "half-software" RAID0 MBR volume based on two WD RE3 drives
md124p1 is NTFS partition of 1.65 TB which was resized some time ago by EaseUS in Windows. 
chkdsk doesn't find errors prior running partitionmanager 
partitionmanager incorrectly detects its total size (total size is lower than used size)
gparted detects it correctly

PS: manjaro was successfully installed via manjaro-architect
Comment 3 Ihor 2021-12-21 23:33:23 UTC
Created attachment 144770 [details]
partitionmanager incorrectly detects partition size, which is lower than used size
Comment 4 Ihor 2021-12-21 23:34:10 UTC
Created attachment 144771 [details]
gparted has no problem detecting partition size correctly
Comment 5 Andrius Štikonas 2021-12-21 23:36:37 UTC
(In reply to Ihor from comment #2)
> gparted detects it correctly
> 
Looks like these days gparted uses ntfsinfo command to read used sectors (I think ntfsinfo command is somewhat newer and wasn't available when ntfs support was added). Gparted seems to run

ntfsinfo --mft --force deviceNode

Can you check if this command works fast? We can then switch to it too.
Comment 6 Andrius Štikonas 2021-12-21 23:38:17 UTC
Also partition size might be wrong because nobody worked on RAID support for partitionmanager. So something might be going wrong there.
Comment 7 Ihor 2021-12-22 01:20:24 UTC
(In reply to Andrius Štikonas from comment #5)
> ntfsinfo --mft --force deviceNode
> 
> Can you check if this command works fast? We can then switch to it too.

It works almost instantly:

-----------------------------------------------------------------------------------------------------------------------------
time sudo ntfsinfo --mft --force /dev/md124p1
Volume Information 
        Name of device: /dev/md124p1
        Device state: 11
        Volume Name: Media
        Volume State: 91
        Volume Flags: 0x0000
        Volume Version: 3.1
        Sector Size: 512
        Cluster Size: 4096
        Index Block Size: 4096
        Volume Size in Clusters: 443812216
MFT Information 
        MFT Record Size: 1024
        MFT Zone Multiplier: 0
        MFT Data Position: 24
        MFT Zone Start: 441274685
        MFT Zone End: 443008326
        MFT Zone Position: 441274685
        Current Position in First Data Zone: 443008326
        Current Position in Second Data Zone: 0
        Allocated clusters 493056 (0,1%)
        LCN of Data Attribute for FILE_MFT: 441274685
        FILE_MFTMirr Size: 4
        LCN of Data Attribute for File_MFTMirr: 88294
        Size of Attribute Definition Table: 2560
        Number of Attached Extent Inodes: 0
FILE_Bitmap Information 
        FILE_Bitmap MFT Record Number: 6
        State of FILE_Bitmap Inode: 80
        Length of Attribute List: 0
        Number of Attached Extent Inodes: 0
FILE_Bitmap Data Attribute Information
        Decompressed Runlist: not done yet
        Base Inode: 6
        Attribute Types: not done yet
        Attribute Name Length: 0
        Attribute State: 3
        Attribute Allocated Size: 55480320
        Attribute Data Size: 55476528
        Attribute Initialized Size: 55476528
        Attribute Compressed Size: 0
        Compression Block Size: 0
        Compression Block Size Bits: 0
        Compression Block Clusters: 0
        Free Clusters: 283662933 (63,9%)
sudo ntfsinfo --mft --force /dev/md124p1  0,06s user 0,04s system 14% cpu 0,713 total
-----------------------------------------------------------------------------------------------------------------------------(In reply to Andrius Štikonas from comment #6)

> Also partition size might be wrong because nobody worked on RAID support for
> partitionmanager. So something might be going wrong there.

Intel RAID (half-software) could be harder to implement than real hardware RAID (hardware controller shows it to system as single volume).
Comment 8 Andrius Štikonas 2021-12-24 00:03:32 UTC
Git commit a5bdd5a4eb44607fed3b0318ee64cc4f15abb122 by Andrius Štikonas.
Committed on 24/12/2021 at 00:03.
Pushed by stikonas into branch 'master'.

Switch to ntfsinfo to read NTFS usage.

M  +19   -9    src/fs/ntfs.cpp
M  +1    -0    src/util/externalcommand_whitelist.h

https://invent.kde.org/system/kpmcore/commit/a5bdd5a4eb44607fed3b0318ee64cc4f15abb122
Comment 9 Andrius Štikonas 2021-12-24 00:04:42 UTC
This should at least fix lag of several minutes.

But I'm not sure what it does to RAID size. Could you test this commit?
Comment 10 Ihor 2021-12-25 04:02:44 UTC
(In reply to Andrius Štikonas from comment #9)
> This should at least fix lag of several minutes.
> 
> But I'm not sure what it does to RAID size. Could you test this commit?

Tested Git commit a5bdd5a4eb44607fed3b0318ee64cc4f15abb122:

Startup now takes about 3-4 seconds! But total size check is still broken.
Comment 11 Andrius Štikonas 2021-12-25 11:52:07 UTC
(In reply to Ihor from comment #10)
> (In reply to Andrius Štikonas from comment #9)
> > This should at least fix lag of several minutes.
> > 
> > But I'm not sure what it does to RAID size. Could you test this commit?
> 
> Tested Git commit a5bdd5a4eb44607fed3b0318ee64cc4f15abb122:
> 
> Startup now takes about 3-4 seconds! But total size check is still broken.

If you want, you can open another bug for RAID size issue, but it's a bit hard to fix it without having access for HW.
Comment 12 Andrius Štikonas 2021-12-25 11:55:08 UTC
(In reply to Andrius Štikonas from comment #11)
> (In reply to Ihor from comment #10)
> > (In reply to Andrius Štikonas from comment #9)
> > > This should at least fix lag of several minutes.
> > > 
> > > But I'm not sure what it does to RAID size. Could you test this commit?
> > 
> > Tested Git commit a5bdd5a4eb44607fed3b0318ee64cc4f15abb122:
> > 
> > Startup now takes about 3-4 seconds! But total size check is still broken.
> 
> If you want, you can open another bug for RAID size issue, but it's a bit
> hard to fix it without having access for HW.

If you open a new bug could you please copy in output of sudo ntfsinfo --mft --force /dev/md124p1