Bug 472275 - kphotoalbum hangs on new image detection.
Summary: kphotoalbum hangs on new image detection.
Status: RESOLVED WORKSFORME
Alias: None
Product: kphotoalbum
Classification: Applications
Component: general (show other bugs)
Version: 5.11.0
Platform: Arch Linux Linux
: NOR critical
Target Milestone: ---
Assignee: KPhotoAlbum Bugs
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2023-07-15 10:48 UTC by v
Modified: 2023-08-16 03:45 UTC (History)
3 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description v 2023-07-15 10:48:47 UTC
SUMMARY
***
Starting kphotoalbum it will open it's little window to search for new images. This will always hang on the same number indefinitely. 
***


STEPS TO REPRODUCE
1. Open kphotoalbum
2. it will hang on image detection

OBSERVED RESULT
There is nothing I can do here but give up on the application. There is no timeout or error message coming up. When starting from the terminal there is zero output on which pictures or folders it is scanning. I assume there might be a file it is having issues with but kphotoalbum gives you zero tools to see it's status and which file is the culprit so the program just stopped working the day I added some random file it did not like. I removed the index.xml thinking it might be something corrupt in the database but that isn't the cause. Building a new database has the same issue

EXPECTED RESULT
It should probably timeout on an error or show what is causing the issue

SOFTWARE/OS VERSIONS
KDE Plasma Version: 5.27.6
KDE Frameworks Version: 5.108.0
Qt Version: 5.15.10

ADDITIONAL INFORMATION
Comment 1 Andreas Schleth 2023-07-15 14:42:25 UTC
Do not give up so easily :-)

Could you possibly find the file that caused the hang? 
Sort the files by date? 
Or copy them in batches to a new folder (make a new database there) and try to read in KPA after each addition?

With the file in hand, one could start to debug and fix the problem. If you find and attach it here, I could try to reproduce the bug on my system.

Also, there is the option to compile KPA with some more debugging info. 
Before starting cmake enter: 

export CMAKE_BUILD_TYPE=RelWithDebInfo

There is a setting ("file search" or so): search for new images on startup. You might uncheck that. Then you need to trigger the search manually via the maintenance menu. Startup of KPA will be much faster but after adding new images you always need to import them manually.
Comment 2 v 2023-07-15 15:15:57 UTC
I am sorry but compiling isn't something i know how to do and I don't see a good way to find out on what file the program hangs on.
It would be nice if kphotoalbum would just show you on what file it is working on to identify such an issue
Comment 3 Andreas Schleth 2023-07-15 15:36:38 UTC
There is quite a good way to find the right file. It will take some patience but not too much:
1. create a new folder somewhere outside your current image folder
2. copy a few (maybe 10-100) of your latest images there
3. create a new KPA database in the new folder and see if it hangs
  a) it hangs: one of the images is the culprit goto 4.
  b) it does not hang: add a few more images - repeat until you hit a hang (goto 2.)
4. The culprit is one of the last batch. Let's call it the "suspects". Copy the suspects somewhere you find them.
5. Start again with 1. (an empty folder) and put just half of the suspects there. Repeat the process until you are down to one image that makes KPA hang. Have a look at that image with another tool - does it look normal or strange?

This sounds quite tedious but it converges quite fast: for ~1000 images, you would only need 10 passes to find the culprit. 
Welcome to binary search!
Comment 4 v 2023-07-15 15:52:14 UTC
that is a lot of work for a big library like mine and totally unnecessary one if the program would simply offer proper transparency about what it is doing and where it is running into issues. Thank you for the fast response but your suggestion is not a solution to my problem, sorry.
Comment 5 Tobias Leupold 2023-07-15 16:14:57 UTC
Hey, thanks for your bug report. But we can't fix a "It no work on me system", if we can't reproduce the problem.

It would be for sure no problem to add a debugging output showing which exact file is currently processed. But you would have to compile this code to get the output. We can't add some code as a random shot, so that you can wait until our next release to get it.

So either, you do what Andreas suggested (that's the way I would search for the problem, too), or you learn how to manually compile KPA. It's quite easy, you can read this here: https://community.kde.org/KPhotoAlbum/build_instructions – in this case I can send a patch containing the code that will output the currently scanned image.

I don't see another way to fix this if we can't see the problem …
Comment 6 Andreas Schleth 2023-07-15 16:43:38 UTC
Yes, it does not solve the problem immediately, but it would open up the path to fixing the issue. The guys programming KPA put in a lot of effort and are quite good at fixing problems they can reproduce.

We have a twofold problem here: 
a) the program hangs (should be fixed) 
b) you suggest a program improvement (somehow show the current filename)

To my understanding, it would help a lot if we know about a) (why does it hang? can we reproduce it?). The hang might not even be due to a corrupt file or so but due to some other filesystem problems (permissions, strange filename or whatever). I often encounter problems with non-ASCII letters in filenames. One program interprets them as ISO8859 the other as UTF-8, ... So the hang might even occur before the program has any chance to write out the filename. So my first idea would be to have a critical look at the folder names. Image files usually have standardized names from the camera or phone that are pure ASCII.

"... so the program just stopped working the day I added some random file it did not like. ..."  If you have an idea about the date, you only need to look at the files (and folders!) from around that date and not the whole database. Anyway, binary search is very nice to you, even with a large number of files. 10000 instead of 1000 images would only take 3-4 more loops. 
Just imagine how much more (unpaid!) effort goes into improving and fixing the program :-)
Comment 7 Tobias Leupold 2023-07-15 19:09:27 UTC
Displaying which file is currently processed is for sure no bad idea. It's only that we never ran into such a problem during the last two decades, so nobody came up with this idea yet ;-)

So, v@phux.dev, it would be very kind if you would spare some time to track this down. If we can identify some image/file which causes this, which we can use to reproduce the problem, we will of course try to fix this. In spite of that, it's a good idea to be more verbose about what's going on when scanning for new files. We will take that into account.
Comment 8 v 2023-07-17 11:41:16 UTC
I pinpointed the issue down to mp4 video files. I have a video downloaded from TikTok which is causing kphotoalbum to hang. But as I was moving around some folders I also found out that a folder with videos I did with my previous smartphone also causes the same issue. Not every file is effected but like 1 in 10 of them will cause the hang. These videos were in kphotoalbum before and just now came up to cause the issue as moving them caused a rescan on them. So might be the issue is more recent.

File upload limits prevent me to attach any videos here so let me know how to send them to someone and I can supply examples
Comment 9 Tobias Leupold 2023-07-17 12:13:02 UTC
Thanks for your investigation! Maybe you could upload some example videos to Google Drive or such (and delete them again after we or at least one of us obtained the files)? I don't think we have such KDE infrastructure, do we?!
Comment 10 Andreas Schleth 2023-07-17 16:21:46 UTC
(In reply to v from comment #8)
> I pinpointed the issue down to mp4 video files. I have a video downloaded
> from TikTok which is causing kphotoalbum to hang. But as I was moving around
> some folders I also found out that a folder with videos I did with my
> previous smartphone also causes the same issue. Not every file is effected
> but like 1 in 10 of them will cause the hang. These videos were in
> kphotoalbum before and just now came up to cause the issue as moving them
> caused a rescan on them. So might be the issue is more recent.
> 
> File upload limits prevent me to attach any videos here so let me know how
> to send them to someone and I can supply examples

Great work! Have you tried an attachment here? A smallish TikTok might just come below the limit of 4MB.
Comment 11 Bug Janitor Service 2023-08-01 03:44:54 UTC
Dear Bug Submitter,

This bug has been in NEEDSINFO status with no change for at least
15 days. Please provide the requested information as soon as
possible and set the bug status as REPORTED. Due to regular bug
tracker maintenance, if the bug is still in NEEDSINFO status with
no change in 30 days the bug will be closed as RESOLVED > WORKSFORME
due to lack of needed information.

For more information about our bug triaging procedures please read the
wiki located here:
https://community.kde.org/Guidelines_and_HOWTOs/Bug_triaging

If you have already provided the requested information, please
mark the bug as REPORTED so that the KDE team knows that the bug is
ready to be confirmed.

Thank you for helping us make KDE software even better for everyone!
Comment 12 Bug Janitor Service 2023-08-16 03:45:12 UTC
This bug has been in NEEDSINFO status with no change for at least
30 days. The bug is now closed as RESOLVED > WORKSFORME
due to lack of needed information.

For more information about our bug triaging procedures please read the
wiki located here:
https://community.kde.org/Guidelines_and_HOWTOs/Bug_triaging

Thank you for helping us make KDE software even better for everyone!