Bug 476688 - Importing pictures on NTFS partition results in FS corruption and data loss
Summary: Importing pictures on NTFS partition results in FS corruption and data loss
Status: RESOLVED NOT A BUG
Alias: None
Product: digikam
Classification: Applications
Component: Database-Media (show other bugs)
Version: 8.1.0
Platform: Fedora RPMs Linux
: NOR normal
Target Milestone: ---
Assignee: Digikam Developers
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2023-11-07 18:11 UTC by David
Modified: 2023-11-07 22:42 UTC (History)
2 users (show)

See Also:
Latest Commit:
Version Fixed In: 8.2.0
Sentry Crash Report:


Attachments
Chkdsk sample log (6.62 KB, text/csv)
2023-11-07 18:11 UTC, David
Details
journalctl --since "1 hour ago" (1.11 MB, text/plain)
2023-11-07 18:12 UTC, David
Details

Note You need to log in before you can comment on or make changes to this bug.
Description David 2023-11-07 18:11:10 UTC
Created attachment 162934 [details]
Chkdsk sample log

SUMMARY
Importing pictures to with Digikam to a NTFS partition, results in corruption of the $I30 index in the destination folder, leading to a silent corruption and data loss. 


STEPS TO REPRODUCE
1. Import pictures to a NTFS partition
2. Check the partition with Windows chkdsk
3. 

OBSERVED RESULT
Chkdsk reports the FS as damaged, requiring a repair run. Some files are either missing or inaccessible prior to repair, both from Linux and Windows, some other are irremediably lost if overwritten.

EXPECTED RESULT
No corruption nor data loss.

SOFTWARE/OS VERSIONS
Operating System: Fedora Linux 38
KDE Plasma Version: 5.27.8
KDE Frameworks Version: 5.110.0
Qt Version: 5.15.11
Kernel Version: 6.5.9-200.fc38.x86_64 (64-bit)
Graphics Platform: X11
Processors: 8 × Intel® Core™ i7-8550U CPU @ 1.80GHz
Memory: 15.5 GiB of RAM
Graphics Processor: Mesa Intel® UHD Graphics 620
Manufacturer: Dell Inc.
Product Name: Inspiron 5570

ADDITIONAL INFORMATION
I am attaching a demo log of a chkdsk run, after I intentionally imported photos with digikam to trigger the corruption. I also briefly tried to match files marked by chkdsk as repaired/damaged against my Fedora journal, finding nothing. 
Unfortunately, I did not have time to understand Digikam's code well enough to perform deeper analysis in a wise way. 

The real problem is, I have noticed this behavior by chance because my backup system started hanging and throwing nonsense errors. I then found out it hung on parts of the NTFS that were corrupted and hence traced the issue back to Digikam. All this means the problem started who knows how many months ago, and I have no more clean copies of my data left to restore from. Same for my family members and some friends, and we all have definitively lost different amounts of data.
Comment 1 David 2023-11-07 18:12:52 UTC
Created attachment 162935 [details]
journalctl --since "1 hour ago"
Comment 2 Maik Qualmann 2023-11-07 19:59:26 UTC
digiKam as an application program cannot cause such an error. Fedora seems to have switched from NTFS3G to NTFS3. There are reports on the internet that indicate data corruption:

https://forum.manjaro.org/t/ntfs3-keeps-corrupting-my-ntfs-partitons/145325

$I30 index error:

https://forum.maboxlinux.org/t/once-more-corrupted-ntfs-directory-since-update-to-5-15-kernel/1521

Maik
Comment 3 David 2023-11-07 22:42:02 UTC
(In reply to Maik Qualmann from comment #2)
> digiKam as an application program cannot cause such an error. 
This is something I thought of, but was unsure about, especially given the fact I was not able to reproduce the issue by manually copying files in Dolphin. Thank you for confirming. 

>Fedora seems to have switched from NTFS3G to NTFS3. There are reports on the internet that indicate data corruption:
>https://forum.manjaro.org/t/ntfs3-keeps-corrupting-my-ntfs-partitons/145325
>$I30 index error:
>https://forum.maboxlinux.org/t/once-more-corrupted-ntfs-directory-since-update-to-5-15-kernel/1521
I was completely unaware of this. Shame on them for using almost the same name, good sorcery..
Can you think of any relevant functions which may trigger the undesired behavior in NTFS3 regarding $I30? I'm trying to gather the pieces, meanwhile I'll open a report against fedora-ntfs3 and reenable ntfs-3g on my machines. 
Thanks again!
David