| Summary: | File management just deleted 1000 files | ||
|---|---|---|---|
| Product: | [Applications] digikam | Reporter: | Anders Lund <anderslund> |
| Component: | Database-Files | Assignee: | Digikam Developers <digikam-bugs-null> |
| Status: | CLOSED UNMAINTAINED | ||
| Severity: | normal | CC: | caulier.gilles, metzpinguin |
| Priority: | NOR | ||
| Version First Reported In: | 6.4.0 | ||
| Target Milestone: | --- | ||
| Platform: | Other | ||
| OS: | Linux | ||
| Latest Commit: | Version Fixed/Implemented In: | 7.5.0 | |
| Sentry Crash Report: | |||
|
Description
Anders Lund
2020-06-19 13:04:38 UTC
Where are located the collections hosting files. In local or remote devices ? Which kind of database did you use ? Here i play with my huge collection in local with sqlite, and i never lost data. I move file/albums with drag and drop intensively without problem. Of course i use 7.0.0-rc whch come with more than 750 bug fixes... Best Gilles Caulier I have uninstalled digikam, as I can not use it any longer. Filemanagement is part of what I need to do with my photos, and digikam is not sufficient or efficient at that. Feel free to ignore this bug - though this have happened several times. Other reasons for not using digikam is that it is notoriosly slow at many tasks, and loads tons of stuff I never use, as do not use the built in editor. Well without relevant info, we cannot help you... File management in digikam have been a problem for me for years and years.
Bugs does not get fixed, and it is not possible to drag folders if the
destination is not visible in the albums pane. There is no facility for moving
multiple folders. And moving folders between devices leads to data loss,
probably because I am an IDIOT, but nevertheless, it has happened several
times. Today, I have (hopefully) been able to recover my files using PhotoRec,
but I do not want to fight digikam any longer, the benefits not being worth the
problems.
A huge issue for me is digikam not being a prober KDE app any longer, since in
KDE file management actually works, and there is a functional file dialog for
example.
I have given up. I'll have to live with using dolphin and gwenview and other
existing tools, to avoid all the grief that digikam causes.
fredag den 19. juni 2020 16.14.22 CEST skrev du:
> https://bugs.kde.org/show_bug.cgi?id=423227
>
> caulier.gilles@gmail.com changed:
>
> What |Removed |Added
> ----------------------------------------------------------------------------
> Resolution|--- |WAITINGFORINFO
> Status|REPORTED |NEEDSINFO
>
> --- Comment #3 from caulier.gilles@gmail.com ---
> Well without relevant info, we cannot help you...
I can understand that it is annoying for you to lose images. But like Gilles, I work with digiKam every day (for years) and have never lost a image. The story with drag & drop and scrolling in the album view is fixed in digiKam-7.0.0. I know the internal file engine in digiKam really well. I can not imagine that digiKam is the sole cause here. We have really built a lot of safeguards that images are not deleted when a file operation has failed. Just a brief technical description: In principle, a move within a drive / partition is just a simple renaming of the source album. If images are lost here, there is an error at system level. We recursively copy all files between different drives / partitions. The first time an error occurs, it is canceled. The source directory is only deleted when all files have been copied without error feedback. In the worst case, you could only have files double. A loss within digiKam cannot be explained here either. Check your drives for errors, I know from daily work experience that there are a lot of hard drives with bad sectors among users. Maik A main cause of grief is the horrible lack of progress information. Today, I
had a stack of progress bars claiming "Moving Album ...", bot none showed any
progress. There was a technical problem with the drive I moved the files to.
Digikam happily deleted my files, leaving me with the issue of recovering them,
getting them renamed and figuring out how to rename darktables xmp files too
(not solved yet) as well as exported jpeg files (not solved yet).
Using Dolphin no file was deleted, when the technical issue with the disk
appeared, the progress information helped me solve the problem with no data
loss.
In earlier events, the lack of proper progress information along with the
insanely slow operation (due to the database queries?) of digikam have been an
issue as well. Without proper progress information, I give up, and meanwhile
digikam appearently just deletes files. I have experienced this at least 3
times.
Maybe digikam works for you because you know the internals well. I don't. I
don't think there were issues when digiKam was a real KDE app.
--
fredag den 19. juni 2020 18.46.18 CEST skrev du:
> https://bugs.kde.org/show_bug.cgi?id=423227
>
> Maik Qualmann <metzpinguin@gmail.com> changed:
>
> What |Removed |Added
> ----------------------------------------------------------------------------
> CC| |metzpinguin@gmail.com
>
> --- Comment #5 from Maik Qualmann <metzpinguin@gmail.com> ---
> I can understand that it is annoying for you to lose images. But like
> Gilles, I work with digiKam every day (for years) and have never lost a
> image.
>
> The story with drag & drop and scrolling in the album view is fixed in
> digiKam-7.0.0.
>
> I know the internal file engine in digiKam really well. I can not imagine
> that digiKam is the sole cause here. We have really built a lot of
> safeguards that images are not deleted when a file operation has failed.
>
> Just a brief technical description:
> In principle, a move within a drive / partition is just a simple renaming of
> the source album. If images are lost here, there is an error at system
> level.
>
> We recursively copy all files between different drives / partitions. The
> first time an error occurs, it is canceled. The source directory is only
> deleted when all files have been copied without error feedback. In the
> worst case, you could only have files double. A loss within digiKam cannot
> be explained here either.
>
> Check your drives for errors, I know from daily work experience that there
> are a lot of hard drives with bad sectors among users.
>
> Maik
Anders, Technically, digiKAm have been a KDE application until 3.0.0 as KIO slave was used to perform serialized database operation. We leaves KIOSlave technology as it's not portable outside Linux. Performance was really bad with large collections. Multithreading was used instead. Step by step we transform digiKam as a pure Qt5 application. It still KDE API used mostly with GUI and main view. digiKam has never shared API with dolphin or other KDE application, excepted for plugin through KIPI. Now this point was also left as we have a pure Qt5 plugin architecture, more simple, and powerful. The problem with album tree-view and drag and drop are probably due to bugs in Qt5 API. Personalty, all work as expected here, i never seen this king of dysfunction here. And i use all the time the D&d features, and it's a pleasure with multi-screens. Note: here you will see the list of KDE API dependencies used by digiKam. As you can see, digiKam always depends of KDE (KF5 components), even if less API calls are used : https://www.digikam.org/api/index.html#externaldeps Gilles Caulier Well, in the old time digikam had a proper file dialog, iirc - now it doesn't.
And again, I don't care. I can't use it, as it is unreliable.
--
lørdag den 20. juni 2020 15.01.54 CEST skrev du:
> https://bugs.kde.org/show_bug.cgi?id=423227
>
> --- Comment #7 from caulier.gilles@gmail.com ---
> Anders,
>
> Technically, digiKAm have been a KDE application until 3.0.0 as KIO slave
> was used to perform serialized database operation.
>
> We leaves KIOSlave technology as it's not portable outside Linux.
> Performance was really bad with large collections. Multithreading was used
> instead.
>
> Step by step we transform digiKam as a pure Qt5 application. It still KDE
> API used mostly with GUI and main view.
>
> digiKam has never shared API with dolphin or other KDE application, excepted
> for plugin through KIPI. Now this point was also left as we have a pure Qt5
> plugin architecture, more simple, and powerful.
>
> The problem with album tree-view and drag and drop are probably due to bugs
> in Qt5 API. Personalty, all work as expected here, i never seen this king
> of dysfunction here. And i use all the time the D&d features, and it's a
> pleasure with multi-screens.
>
> Note: here you will see the list of KDE API dependencies used by digiKam. As
> you can see, digiKam always depends of KDE (KF5 components), even if less
> API calls are used :
>
> https://www.digikam.org/api/index.html#externaldeps
>
>
> Gilles Caulier
lørdag den 20. juni 2020 15.01.54 CEST skrev du:
> The problem with album tree-view and drag and drop are probably due to bugs
> in Qt5 API. Personalty, all work as expected here, i never seen this king
> of dysfunction here. And i use all the time the D&d features, and it's a
> pleasure with multi-screens.
I don't even care if the drag-n-drop thing gets fixed, since I need proper file
management. But not from digikam any longer, as I will not use it any more.
There are a few things I will miss, but not many, and I will stop fearing the
broken file management. I don't care if it is a pleasure with multiscreen,
because most of the time I have only one screen. Good for you, but it does not
solve the brokenness.
lørdag den 20. juni 2020 15.01.54 CEST skrev du:
> digiKam has never shared API with dolphin or other KDE application, excepted
> for plugin through KIPI. Now this point was also left as we have a pure Qt5
> plugin architecture, more simple, and powerful.
The file management in digikam is horrible. Even if it works for you, comparing
to KDE applications, it is a joke. When moving files with kde, files are not
deleted prematurely, and there is proper progress information, as opposed to
using digikam, where there are a spinner, but NO useful information, and files
are deleted when not properly moved. Trying to back out by stopping the action
does not help either - in digikam, in kde it WORKS. digikam is powerful when
it comes to unwanted file deletion, other than that not really as I see it.
Sorry.
You talk about native file dialog ? There is an option in Setup/Misc page to use it. Gilles Caulier You can imagine that i delete also photo on my computer, and a lots of items... And i never lost files, never... How i can fix a problem that i cannot reproduce on my computer ? Also about the progress information dialog with cancel, it's implemented in current 7.0.0. Gilles Caulier Git commit 3eb48089a1ce737ab0371d73566e5c9a5cd1d4a3 by Maik Qualmann. Committed on 20/06/2020 at 21:40. Pushed by mqualmann into branch 'master'. show better progress if we have to copy recursively M +4 -2 core/libs/iojobs/iojob.cpp M +37 -2 core/libs/threadimageio/engine/dfileoperations.cpp M +3 -1 core/libs/threadimageio/engine/dfileoperations.h https://invent.kde.org/graphics/digikam/commit/3eb48089a1ce737ab0371d73566e5c9a5cd1d4a3 That is of course nice, and a step in the right difection, but you still
manage to unlink files that wasn't successfully written to target. My problems
appear when moving files between (usb) disks.
lørdag den 20. juni 2020 23.42.50 CEST skrev du:
> --- Comment #13 from Maik Qualmann <metzpinguin@gmail.com> ---
> Git commit 3eb48089a1ce737ab0371d73566e5c9a5cd1d4a3 by Maik Qualmann.
> Committed on 20/06/2020 at 21:40.
> Pushed by mqualmann into branch 'master'.
>
> show better progress if we have to copy recursively
>
> M +4 -2 core/libs/iojobs/iojob.cpp
> M +37 -2 core/libs/threadimageio/engine/dfileoperations.cpp
> M +3 -1 core/libs/threadimageio/engine/dfileoperations.h
>
> https://invent.kde.org/graphics/digikam/commit/3eb48089a1ce737ab0371d73566e5
> c9a5cd1d4a3
What exactly was the problem with your USB drive? All attempts to reproduce a data loss here fail. Even disconnecting the USB drive during the moving process does not lead to data loss. Maik Same here. I played we some USB keys where i take, move, copy files. If i disconnect the key violently, sure it break the process with an error, but i don't lost files. Also the USB device file system is important here. You muust always use a journalized file system, aka NTFS or Ext4 for ex, never use *Fat* from Microsoft. With this kind of test, you will be sure to lost data. Gilles Caulier søndag den 21. juni 2020 10.34.52 CEST skrev du:
> What exactly was the problem with your USB drive?
When moving files with dolphin, there was a message saying it didn't answer at
some point. I moved the cable/connector a bit, and it continued. The progress
started moving again.
During my attempt to move files using digikam, digikam just appeared to do
nothing, so I pushed the stop button on all the "move album" items, hoping
that at least some files would be saved. I was not able to unmount the disk
(probably due to the usb connection problem). All was lost, appearently
digikam had unlinked ALL the files. I later managed to restore files using
photorec, but it is of course a mess. This is the third time I have this
experience with digikam, so I have decided to try and replace it with
functional softwares.
So, on your computer you have a wrong electric connection with USB device. My tip : change cables, and if it don't work better change the device. This can be a power failure on the +5V delivered to the connected device. I already seen this kind of dysfunction with magnetic external hardrive which need more power more than expected. In my office, when maintenance team backup magnetic hardrive, they always use an external CC and never use USB power. Note : if you use SSD, there is less power required so far and there are 20x faster than HDD. The Dolphin detection of USB devices disconnection is certainly managed by Solid interface. digiKam use this also in background, but not to detect device disconnection. This must never happen, because it's dangerous! Gilles Caulier digiKam AppImage bundle for Linux is just recompiled with last Maik changes : https://files.kde.org/digikam/ Gilles Caulier søndag den 21. juni 2020 11.29.22 CEST skrev du:
> https://bugs.kde.org/show_bug.cgi?id=423227
>
> --- Comment #18 from caulier.gilles@gmail.com ---
> So, on your computer you have a wrong electric connection with USB device.
>
> My tip : change cables, and if it don't work better change the device.
> This can be a power failure on the +5V delivered to the connected device. I
> already seen this kind of dysfunction with magnetic external hardrive which
> need more power more than expected. In my office, when maintenance team
> backup magnetic hardrive, they always use an external CC and never use USB
> power.
What I see is this: With digikam, this leads to data loss, without digikam it
does not. So I don't really care about your tips and attempts to make this
problem not exist. But since I will not be using digikam again anytime soon, I
will not be affected anymore.
Another answer to this is that in earlier events where I lost files, there was NO device issues. Just the very slow speed in combination with digikams poor progress (nonexisting in some of those events in my understanding). And still, even with the process interrupted, it should not mean massive data loss, as it did more than once. Dear Bug Submitter, This bug has been in NEEDSINFO status with no change for at least 15 days. Please provide the requested information as soon as possible and set the bug status as REPORTED. Due to regular bug tracker maintenance, if the bug is still in NEEDSINFO status with no change in 30 days the bug will be closed as RESOLVED > WORKSFORME due to lack of needed information. For more information about our bug triaging procedures please read the wiki located here: https://community.kde.org/Guidelines_and_HOWTOs/Bug_triaging If you have already provided the requested information, please mark the bug as REPORTED so that the KDE team knows that the bug is ready to be confirmed. Thank you for helping us make KDE software even better for everyone! I do not have digikam installed any longer, since I can not trust it. So you
get no information from me. Conveniently for you, you can just remove this
bug, and not fix your broken software.
mandag den 6. juli 2020 06.33.08 CEST skrev du:
> https://bugs.kde.org/show_bug.cgi?id=423227
>
> --- Comment #22 from Bug Janitor Service <bug-janitor@kde.org> ---
> Dear Bug Submitter,
>
> This bug has been in NEEDSINFO status with no change for at least
> 15 days. Please provide the requested information as soon as
> possible and set the bug status as REPORTED. Due to regular bug
> tracker maintenance, if the bug is still in NEEDSINFO status with
> no change in 30 days the bug will be closed as RESOLVED > WORKSFORME
> due to lack of needed information.
>
> For more information about our bug triaging procedures please read the
> wiki located here:
> https://community.kde.org/Guidelines_and_HOWTOs/Bug_triaging
>
> If you have already provided the requested information, please
> mark the bug as REPORTED so that the KDE team knows that the bug is
> ready to be confirmed.
>
> Thank you for helping us make KDE software even better for everyone!
Dear Bug Submitter, This bug has been in NEEDSINFO status with no change for at least 15 days. Please provide the requested information as soon as possible and set the bug status as REPORTED. Due to regular bug tracker maintenance, if the bug is still in NEEDSINFO status with no change in 30 days the bug will be closed as RESOLVED > WORKSFORME due to lack of needed information. For more information about our bug triaging procedures please read the wiki located here: https://community.kde.org/Guidelines_and_HOWTOs/Bug_triaging If you have already provided the requested information, please mark the bug as REPORTED so that the KDE team knows that the bug is ready to be confirmed. Thank you for helping us make KDE software even better for everyone! Read the comment thread. You can safely delete this bug, and I will not be
using this buggy software any longer.
--
tirsdag den 21. juli 2020 06.33.09 CEST skrev du:
> https://bugs.kde.org/show_bug.cgi?id=423227
>
> --- Comment #24 from Bug Janitor Service <bug-janitor@kde.org> ---
> Dear Bug Submitter,
>
> This bug has been in NEEDSINFO status with no change for at least
> 15 days. Please provide the requested information as soon as
> possible and set the bug status as REPORTED. Due to regular bug
> tracker maintenance, if the bug is still in NEEDSINFO status with
> no change in 30 days the bug will be closed as RESOLVED > WORKSFORME
> due to lack of needed information.
>
> For more information about our bug triaging procedures please read the
> wiki located here:
> https://community.kde.org/Guidelines_and_HOWTOs/Bug_triaging
>
> If you have already provided the requested information, please
> mark the bug as REPORTED so that the KDE team knows that the bug is
> ready to be confirmed.
>
> Thank you for helping us make KDE software even better for everyone!
Nothing is deleted. The last comment are done by automated script which scan automatically the entries. About your problem : fix your hardware _FIRST_. All the removal device using USB3 are really very fragile and dysfunction can be easily reproduced. I personalty face with this kind of problem with my son who travel with a removal MSTATA drive in a bag without protection and with cable connected to the device. Of course quickly a connector failure appear and now when the device is connected to the computer it disappear and reappear randomly. digiKam is not designed to fix hardware failure. By chance for your the mechanism implemented in Dolphin is more robust but it's a special case. I test to copy files with Windows 10 or MacOS Catalina with my broken device to see if a similar mechanism exists, and i can said no. If the device diseapear while a copy, file are deleted while the process. Gilles Caulier Stop blaming my hardware. Moving files with functional software, such as
dolphin, does not produce problems. I have experienced this issue with
multiple laptops, and ONLY with digikam. I also filed bugs about it earlier,
which you refused.
Fix digikam if you'd like to, but I am DONE with digikam, as it is not
reliable.
--
tirsdag den 21. juli 2020 10.36.55 CEST skrev du:
> https://bugs.kde.org/show_bug.cgi?id=423227
>
> --- Comment #26 from caulier.gilles@gmail.com ---
> Nothing is deleted. The last comment are done by automated script which scan
> automatically the entries.
>
> About your problem : fix your hardware _FIRST_. All the removal device using
> USB3 are really very fragile and dysfunction can be easily reproduced. I
> personalty face with this kind of problem with my son who travel with a
> removal MSTATA drive in a bag without protection and with cable connected
> to the device. Of course quickly a connector failure appear and now when
> the device is connected to the computer it disappear and reappear randomly.
>
> digiKam is not designed to fix hardware failure. By chance for your the
> mechanism implemented in Dolphin is more robust but it's a special case.
>
> I test to copy files with Windows 10 or MacOS Catalina with my broken device
> to see if a similar mechanism exists, and i can said no. If the device
> diseapear while a copy, file are deleted while the process.
>
> Gilles Caulier
This is idiotic. Digikam deletes my files, dolphin does not, because it have
functional file management. Digikam does not, and unlinks files without ensuring
that they are actually written. But hey, I have decided not to use digikam any
longer, for this specific reason.
torsdag den 30. juli 2020 11.34.57 CEST skrev du:
> https://bugs.kde.org/show_bug.cgi?id=423227
>
> caulier.gilles@gmail.com changed:
>
> What |Removed |Added
> ----------------------------------------------------------------------------
> Status|NEEDSINFO |RESOLVED
> Resolution|WAITINGFORINFO |NOT A BUG
I stopped using digikam due to this madness. It would be solved quite easily
by using KIO, which is safe. Good luck to you in the future.
søndag den 2. januar 2022 14.00.27 CET skrev du:
> https://bugs.kde.org/show_bug.cgi?id=423227
>
> caulier.gilles@gmail.com changed:
>
> What |Removed |Added
> ----------------------------------------------------------------------------
> Status|RESOLVED |CLOSED
> Resolution|NOT A BUG |UNMAINTAINED
|