Bug 463197

Showing items in sub-albums is of course problematic when you select the top level album. In the worst case, 250k items are then loaded into the view, which just takes time. I've already experimented with loading items dynamically and only as many as fit in the view. The downside is you can't even quickly scroll to the bottom. As with some websites, only the next section is then loaded.

Maik

See also this Bug 411099. I see in your video that the search only shows about 3000 items. There should be no freezing. Which database type do you use MySQL or SQLite? Local or server?

Maik

(In reply to Maik Qualmann from comment #1)
> Showing items in sub-albums is of course problematic when you select the top
> level album. In the worst case, 250k items are then loaded into the view,
> which just takes time. 

Hmm, that's how I've always used Lightroom though, & never seen any lag at all - certainly no lockups of the entire UI.

(In reply to Maik Qualmann from comment #1)
> I've already experimented with loading items
> dynamically and only as many as fit in the view. The downside is you can't
> even quickly scroll to the bottom. As with some websites, only the next
> section is then loaded.

Looks like what LR does is it immediately reserves the "spots" for the thumbnails, but if you scroll very quickly, they appear as either grey empty boxes or ultra low-res thumbnails. Then it fills out the actual thumbnails after a moment or two. So essentially it loads on-demand, but you *can* scroll immediately to the very bottom (i.e. you don't have to wait for any next section to load before you can scroll farther). Here's a video: https://jiij.cc/snaps/2022-12-18_09.42.06.mp4

(In reply to Maik Qualmann from comment #2)
> See also this Bug 411099. I see in your video that the search only shows
> about 3000 items. There should be no freezing. Which database type do you
> use MySQL or SQLite? Local or server?

I'm using sqlite, local. If you're referring to the number 3,364 in the section header, that's just that "section" of the results (i.e. that "leaf" in the album tree). It seems to group results by albums, so if you scroll down, there are other groups with other portions of the results.

Git commit 78b3bb6f62633c220d49c4797c7876ed8af7616b by Maik Qualmann.
Committed on 18/12/2022 at 18:59.
Pushed by mqualmann into branch 'master'.

enable batched QListView layout mode

M  +2    -0    core/libs/widgets/itemview/itemviewcategorized.cpp

https://invent.kde.org/graphics/digikam/commit/78b3bb6f62633c220d49c4797c7876ed8af7616b

I use digiKam with a mysql database, 200,000 photos, and the situation is not better... (#411099)

"The idea here is actually that the user in the Iconview is not facing an empty view." (#411099)
I still don't understand why it's a problem if the user sees an empty search window...  ...it does not matter.
Which user expects results before starting a search?
digiKam spends a lot of time unnecessarily re-finding irrelevant elements (previous search, the search result can be even days earlier), while block the user (and often itself) from starting a new search.
This is very sad. :-(
digiKam is a great software, and you developers are great people!
Sorry for my comment, but I think: option to interrupt the search process must be specified, or a search should not be started until the user presses ENTER.

(In reply to Maik Qualmann from comment #5)
> Git commit 78b3bb6f62633c220d49c4797c7876ed8af7616b by Maik Qualmann.
> Committed on 18/12/2022 at 18:59.
> Pushed by mqualmann into branch 'master'.
> 
> enable batched QListView layout mode
> 
> M  +2    -0    core/libs/widgets/itemview/itemviewcategorized.cpp
> 
> https://invent.kde.org/graphics/digikam/commit/
> 78b3bb6f62633c220d49c4797c7876ed8af7616b

Hi Maik,
This patch what's doing it?
Regards
Peter

Possibly a small optimization of the QListView performance with a large number of items.

Maik

It seems like the more generalized solution would be i.e.:

1) Set the total height of the scrollable container to the correct final height;
2) Start *asynchronously* loading/filling out the thumbnails (i.e. not on the same thread as is used for processing user interaction)
3) As the user scrolls, reprioritize loading so that the images near the currently-visible view are loaded first
...Until done.

By pre-setting the correct scroll height right from the bat, the user will be able to drag directly to any vertical offset (i.e. no need to scroll to the bottom & wait for the current batch to load before they can scroll farther).  And by loading not on the thread used for user interaction, it should never lockup the UI, no matter how many there are to load.

Addendum: It seems like the Labels tab isn't really usable either - similar to search, it locks the whole application up as soon as I enter :/

Does the above make sense?

Ie. Shouldn't anything that scales linearly with the number of photos be offloaded to a different thread? Then the main application message loop couldn't get blocked regardless of the # of photos - just like it doesn't in LR, for other O(n) operation in digiKam (ie reading metadata etc).

Hi,

In digiKam, all o(n) operations are multi-threaded, or at least runs in a separated thread. This include the the stream-lined query to send to the database of course and the read metadata from files.

For the rest, especially the GUI operations, X11 under Linux is not re-entrant, and threading cannot be used.

Gilles Caulier

(In reply to caulier.gilles from comment #11)
> For the rest, especially the GUI operations, X11 under Linux is not
> re-entrant, and threading cannot be used.

1) I see. So I'm assuming that in this case, it has already been determined that the operation which locks up the software is purely the UI calls themselves - in other words, there isn't anything else that could be pushed to another thread?

2) For what it's worth, in C#/WinForms, you also cannot directly perform any UI updates from background threads - however, in effect you can accomplish the same with something like the following. Not sure if something similar exists in qt:

// This function runs in a background worker thread
private void backgroundWorker_DoWork(object sender, DoWorkEventArgs e){
            // If you need to update the UI, you cannot do it directly from here - this would not be threadsafe/could crash:
            //uiElement.Text = "whatever"

            // However, you can safely accomplish the same by using a MethodInvoker to dispatch it back to the UI thread:
            MethodInvoker invoker1 = new MethodInvoker(delegate ()
            {
                 // Can manipulate any UI elements here. So effectively the operation was "originated" from the background worker, but the actual update
                 // gets queued to safely occur on the main UI thread
                 uiElement.Text = "whatever";
            });
            this.BeginInvoke(invoker1);
}

3) Even if there's no way to put it on another thread, it really seems like there should be creative solutions that would prevent it from locking up, regardless of the number of images. For example, break up the images into batches, and only add at most "maxN" items to the UI per invocation, before returning control to the message loop so that user interaction can still take place. I'm not directly familiar with how qt's message loop is implemented, but:
* Does that just mean inserting a brief sleep() or similar in the code that adds all the image thumbnails to the UI, so any user interaction messages could be processed before continuing?
* Or perhaps using a custom "SET_UI_IMAGES" message, which gets processed by adding maxN images to the UI, and re-queuing the same message type until all images have been added? (and when the user scrolls to a particular part of the scrollable container, on the next iteration of SET_UI_IMAGES, it would prioritize adding load the corresponding subset of maxN to the UI)

Basically, just some way to hand control back to the main message loop often enough that this long process doesn't have to run all in one go - and thus regardless of the total number of images, it need not block everything else until it's done.

Just saw that 8.0.0 was officially released (congrats). Unfortunately, search still remains unusable (locks up the entire UI for several minutes just attempting to enter that tab). Any comment on my message from 4 months ago? I thought I provided some pretty valid suggestions there.

In any event, even if properly threading/batching it is a bigger project, at the very least it seems like by default it could avoid trying to show every item in the library when entering that tab, so you can at least get into that UI without it completely locking the application. Then if you do a search that returns i.e. 500 results, it only needs to populate 500. Rather than the current situation, where it first populates the entire library before you can even attempt the search you intended to do (and then waiting again while it locks up the UI to populate the result of the search). Cutting out that first step seems like it would at least partially alleviate how this is implemented (just leave the pane empty until an explicit search is performed). As it stands, that pane is really truly not usable.

@Metal450,

This problem still reproducible with the new digiKam 8.2.0 pre-release Windows
installer available at usual place:

https://files.kde.org/digikam/

This new bundle is based on last Qt framework 5.15.11 and KDE framework 5.110.

Thanks in advance

Gilles Caulier

(In reply to caulier.gilles from comment #14)
> @Metal450,
> 
> This problem still reproducible with the new digiKam 8.2.0 pre-release
> Windows
> installer available at usual place:
> 
> https://files.kde.org/digikam/
> 
> This new bundle is based on last Qt framework 5.15.11 and KDE framework
> 5.110.
> 
> Thanks in advance
> 
> Gilles Caulier

Just installed it & gave it a try; the behavior is identical. The UI still locks up & becomes unresponsive the moment you try to use either of those tabs.

Any thoughts on my responses from 10 months ago, & follow-up 6 months ago?

A search interrupt button would be nice.

(In reply to Peter from comment #16)
> A search interrupt button would be nice.

A button wouldn't do anything if the UI thread is already locked up & not processing user inputs.

(In reply to Metal450 from comment #15)
> (In reply to caulier.gilles from comment #14)
> > @Metal450,
> > 
> > This problem still reproducible with the new digiKam 8.2.0 pre-release
> > Windows
> > installer available at usual place:
> > 
> > https://files.kde.org/digikam/
> > 
> > This new bundle is based on last Qt framework 5.15.11 and KDE framework
> > 5.110.
> > 
> > Thanks in advance
> > 
> > Gilles Caulier
> 
> Just installed it & gave it a try; the behavior is identical. The UI still
> locks up & becomes unresponsive the moment you try to use either of those
> tabs.
> 
> Any thoughts on my responses from 10 months ago, & follow-up 6 months ago?

Any thoughts on my responses from 13 months ago, & follow-up 9 months ago?

(In reply to spiesant from comment #18)
> (In reply to Metal450 from comment #15)
> > (In reply to caulier.gilles from comment #14)
> > > @Metal450,
> > > 
> > > This problem still reproducible with the new digiKam 8.2.0 pre-release
> > > Windows
> > > installer available at usual place:
> > > 
> > > https://files.kde.org/digikam/
> > > 
> > > This new bundle is based on last Qt framework 5.15.11 and KDE framework
> > > 5.110.
> > > 
> > > Thanks in advance
> > > 
> > > Gilles Caulier
> > 
> > Just installed it & gave it a try; the behavior is identical. The UI still
> > locks up & becomes unresponsive the moment you try to use either of those
> > tabs.
> > 
> > Any thoughts on my responses from 10 months ago, & follow-up 6 months ago?
> 
> Any thoughts on my responses from 13 months ago, & follow-up 9 months ago?

Is there a reason responses here are just ignored?

@spiesant,

Nothing is ignored. This thread history still available in time. We process report when time permit.

8.3.0 is now released with 250 files closed on bugzilla. Problem still reproducible ?

> 8.3.0 is now released with 250 files closed on bugzilla. Problem still reproducible ?

Yup, no change. Just switching to the "Search" tab freezes the whole application for over a minute.

> Nothing is ignored

Was referring specifically to Comment 12 above - there was good constructive discussion up to that point, & then just went silent for 1.25 years.

I'm having this issue too. Shocked that more users haven't reported it as the lock ups when trying to search are so extreme I believed it had actually crashed. But it turned out that everything just freezed for more than a minute and eventually it comes back. Every time I try to search it's always the same. Never had this issue in Lightroom on the same catalog (300k images). The difference is night and day.

I guess you haven't found any workaround @spiesant?

(In reply to jungelexplorer88 from comment #23)
> I guess you haven't found any workaround @spiesant?

No workaround. I'm still using Lightroom in a Windows virtual machine because of this. Digikam's UI just freezes up way too often to be usable.

I had some specific technical questions/ideas 1.5 years ago, but was never able to get anyone to respond to them: https://bugs.kde.org/show_bug.cgi?id=463197#c12

Tested it again in 8.4.0; still broken. Froze up for >2min on the first attempt to search, had to force-terminate.

In my case, lags accompany literally all actions in the application: scrolling through all lists (album folders, tags, settings), absolutely all clicks (menu, buttons, checkboxes), searching for faces in a single 1280 x 853 px photo took 56 seconds. This is probably related to #501907, but I'm not sure.

digiKam: 8.6.0
KDE Frameworks: 6.10.0
Qt: Using 6.8.1 and built against 6.8.1
Microsoft Windows 10 Professional (x64) Build 19045.5608 (22H2)
Build ABI: x86_64-little_endian-llp64
Kernel: winnt 10.0.19045

Intel Core i7-3770K CPU @ 3.50GHz
Total Memory Size: 32 GBytes
Memory Type: DDR3 SDRAM
Current Timing (tCAS-tRCD-tRP-tRAS): 11-13-13-31

P.S. I understand that the comparison is inappropriate, but, nevertheless, it’s funny for me to wait for a second for the appearance of a drop-down list or any other application response when Cyberpunk 2077 was completed on the same computer with maximum graphics settings and FPS 30+.

This is certainly a videa driver incompatibility with Qt framework used in background by digiKam.

Go to DK/Setup/Misc/System dialog page and turn on  "Use the software OpenGL rendering"

https://docs.digikam.org/en/setup_application/miscs_settings.html#system-settings

Hi,

The 8.7.0 pre-release Windows installer from today have been rebuilt from
scratch with Qt 6.8.3, KDE 6.12, OpenCV 4.11 + CUDA support, Exiv2 0.28.5, ExifTool 13.27, ffmpeg 7, all image codecs updated to last version (jxl, avif, heif, aom, etc.).

Please try with this version to see if your problem still reproducible...

https://files.kde.org/digikam/

Thanks in advance
Best regards

Gilles Caulier

Yes, the same issue persists.

I believe the issue korwin reported is not the same as the one in this thread.

Still haven't received any response on https://bugs.kde.org/show_bug.cgi?id=463197#c12, my comment from 2022.

(In reply to caulier.gilles from comment #27)
> This is certainly a videa driver incompatibility with Qt framework used in background by digiKam.
> Go to DK/Setup/Misc/System dialog page and turn on  "Use the software OpenGL rendering"
> https://docs.digikam.org/en/setup_application/miscs_settings.html#system-settings

digiKam 8.7.0
Qt: Using 6.8.3 and built against 6.8.3
"Use the software OpenGL rendering": off
Nvidia GeForce RTX 3070 Ti with driver 572.83

There are no interface lags. Whatever caused them, the problem is fixed for me.

Wonderful, thanks for the feedback...

Gilles Caulier

Why is this marked as fixed?!

I reported an entirely different issue 2 years ago, it was ignored since then, someone else came in and added a random other/unrelated issue to this thread, you fixed that other unrelated issue, then marked the entire thread as fixed despite my clearly stating that the actual issue in the thread was NOT fixed?

comment 25: "There are no interface lags. Whatever caused them, the problem is fixed for me." 

So for me i expect that it's fixed. simple no ?

(In reply to caulier.gilles from comment #33)
> comment 25: "There are no interface lags. Whatever caused them, the problem
> is fixed for me." 
> 
> So for me i expect that it's fixed. simple no ?

He thread-hijacked this with a completely different issue than the one I originally reported. His issue is fixed, but that was never the issue described in this thread. Just because he thread-hijacked with something else, and that other thing was fixed, doesn't mean the actual topic of the thread was fixed. Simple no?

The issue reported in this thread is that the software appears to be doing operations on the UI thread that scale linearly with the number of photos, so as the collection grows very large, populating views becomes extremely slow to the poitn that it can start to lockup the UI. Per the original video I posted & many subsequent follow-ups.

He came in later and said something about every operation in the UI lagging - i.e. scrolling, clicks, etc. That's clearly not the same thing.

Here's another video showing the exact same behavior as described originally, see "STEPS TO REPRODUCE, 2" in the OP: https://jiij.cc/snaps/2025-04-12_08.59.03.mp4

This will definitely not be fixed just by underlying libraries. It's an implementation issue.

@spiesant,

Before to continue this VERY constructive talk in this room, please read that urgently:

https://kde.org/code-of-conduct/

Notes:
- Do not try to explain to me how to manage an open source project created in 2001.
- About your comment 11, seriously, you think that i have been coding for a few weeks only.

Thank you for your consideration and your contribution, it's always a pleasure...

Gilles Caulier

(In reply to caulier.gilles from comment #36)
> @spiesant,
> 
> Before to continue this VERY constructive talk in this room, please read
> that urgently:
> 
> https://kde.org/code-of-conduct/
> 
> Notes:
> - Do not try to explain to me how to manage an open source project created
> in 2001.
> - About your comment 11, seriously, you think that i have been coding for a
> few weeks only.
> 
> Thank you for your consideration and your contribution, it's always a
> pleasure...
> 
> Gilles Caulier

I read it in full. I apologize if that response was overly abrasive - I will admit that I let my frustration get the best of me and shouldn't have. Not that it excuses my frustrated way of responding, but hopefully there's at least some understanding why this sequence of events could lead to frustration:
* I posted an issue & attempted to engage in constructive discussion. In 2022, that led up to a comment that I thought *might* provide provide context to push it forward. It may or may not have been useful. But as it was ignored with no response that directly addresses the points it made, I don't know.
* For 2.5 years I waited, occasionally pinging in hopes that someone might reply. (Yes, I do realize this is an open-source project and its developers are volunteers with no obligation to reply. I'm only attempting to convey the sequence of events.)
* In 2025, a different user joined the thread and posted an unrelated issue.
* The unrelated issue was *immediately* addressed, and my topic closed.
* I responded that my issue was not addressed.
* You responded again, and it's possible that at this point I misinterpreted your reply. My interpretation of your "simple no?" comment was that you were saying i.e. "duh, it's simple to see that it's fixed." that may have been wrong.

Try to put yourself in my shoes. I've been watching this issue for 2.5 years, someone else jumps in, changes the subject, and the issue is closed. So I overreacted. I apologize.

> About your comment 11, seriously, you think that i have been coding for a few weeks only.

* I assume you're referring to comment 12 (comment 11 was from you). 
* If the suggestions in my comment 12 are incorrect/irrelevant, that's exactly what I've been asking for a response for 2.5 years. Maybe they're stupid and maybe not, but I can't know if there's no meaningful response, can I?
* (This also seems like a sarcastic/abrasive of a comment per the code of conduct, is it not?)

> Do not try to explain to me how to manage an open source project created in 2001.

Honestly I'm not sure what this refers to exactly.

My goal is really quite simple: to try to make progress toward identifying this issue. As I have to keep using Lightroom in a Windows VM, every so often I'm reminded "man I wish I could just use Digikam." So I try it again, see that it has the same issue, come here to check for progress/replies, find that there's none. My intention is not to stir up trouble. It's just to try to get it fixed.