Showing items in sub-albums is of course problematic when you select the top level album. In the worst case, 250k items are then loaded into the view, which just takes time. I've already experimented with loading items dynamically and only as many as fit in the view. The downside is you can't even quickly scroll to the bottom. As with some websites, only the next section is then loaded.

Maik

See also this Bug 411099. I see in your video that the search only shows about 3000 items. There should be no freezing. Which database type do you use MySQL or SQLite? Local or server?

Maik

(In reply to Maik Qualmann from comment #1)
> Showing items in sub-albums is of course problematic when you select the top
> level album. In the worst case, 250k items are then loaded into the view,
> which just takes time. 

Hmm, that's how I've always used Lightroom though, & never seen any lag at all - certainly no lockups of the entire UI.

(In reply to Maik Qualmann from comment #1)
> I've already experimented with loading items
> dynamically and only as many as fit in the view. The downside is you can't
> even quickly scroll to the bottom. As with some websites, only the next
> section is then loaded.

Looks like what LR does is it immediately reserves the "spots" for the thumbnails, but if you scroll very quickly, they appear as either grey empty boxes or ultra low-res thumbnails. Then it fills out the actual thumbnails after a moment or two. So essentially it loads on-demand, but you *can* scroll immediately to the very bottom (i.e. you don't have to wait for any next section to load before you can scroll farther). Here's a video: https://jiij.cc/snaps/2022-12-18_09.42.06.mp4

(In reply to Maik Qualmann from comment #2)
> See also this Bug 411099. I see in your video that the search only shows
> about 3000 items. There should be no freezing. Which database type do you
> use MySQL or SQLite? Local or server?

I'm using sqlite, local. If you're referring to the number 3,364 in the section header, that's just that "section" of the results (i.e. that "leaf" in the album tree). It seems to group results by albums, so if you scroll down, there are other groups with other portions of the results.

Git commit 78b3bb6f62633c220d49c4797c7876ed8af7616b by Maik Qualmann.
Committed on 18/12/2022 at 18:59.
Pushed by mqualmann into branch 'master'.

enable batched QListView layout mode

M  +2    -0    core/libs/widgets/itemview/itemviewcategorized.cpp

https://invent.kde.org/graphics/digikam/commit/78b3bb6f62633c220d49c4797c7876ed8af7616b

I use digiKam with a mysql database, 200,000 photos, and the situation is not better... (#411099)

"The idea here is actually that the user in the Iconview is not facing an empty view." (#411099)
I still don't understand why it's a problem if the user sees an empty search window...  ...it does not matter.
Which user expects results before starting a search?
digiKam spends a lot of time unnecessarily re-finding irrelevant elements (previous search, the search result can be even days earlier), while block the user (and often itself) from starting a new search.
This is very sad. :-(
digiKam is a great software, and you developers are great people!
Sorry for my comment, but I think: option to interrupt the search process must be specified, or a search should not be started until the user presses ENTER.

(In reply to Maik Qualmann from comment #5)
> Git commit 78b3bb6f62633c220d49c4797c7876ed8af7616b by Maik Qualmann.
> Committed on 18/12/2022 at 18:59.
> Pushed by mqualmann into branch 'master'.
> 
> enable batched QListView layout mode
> 
> M  +2    -0    core/libs/widgets/itemview/itemviewcategorized.cpp
> 
> https://invent.kde.org/graphics/digikam/commit/
> 78b3bb6f62633c220d49c4797c7876ed8af7616b

Hi Maik,
This patch what's doing it?
Regards
Peter

Possibly a small optimization of the QListView performance with a large number of items.

Maik

It seems like the more generalized solution would be i.e.:

1) Set the total height of the scrollable container to the correct final height;
2) Start *asynchronously* loading/filling out the thumbnails (i.e. not on the same thread as is used for processing user interaction)
3) As the user scrolls, reprioritize loading so that the images near the currently-visible view are loaded first
...Until done.

By pre-setting the correct scroll height right from the bat, the user will be able to drag directly to any vertical offset (i.e. no need to scroll to the bottom & wait for the current batch to load before they can scroll farther).  And by loading not on the thread used for user interaction, it should never lockup the UI, no matter how many there are to load.

Addendum: It seems like the Labels tab isn't really usable either - similar to search, it locks the whole application up as soon as I enter :/

Does the above make sense?

Ie. Shouldn't anything that scales linearly with the number of photos be offloaded to a different thread? Then the main application message loop couldn't get blocked regardless of the # of photos - just like it doesn't in LR, for other O(n) operation in digiKam (ie reading metadata etc).

Hi,

In digiKam, all o(n) operations are multi-threaded, or at least runs in a separated thread. This include the the stream-lined query to send to the database of course and the read metadata from files.

For the rest, especially the GUI operations, X11 under Linux is not re-entrant, and threading cannot be used.

Gilles Caulier

(In reply to caulier.gilles from comment #11)
> For the rest, especially the GUI operations, X11 under Linux is not
> re-entrant, and threading cannot be used.

1) I see. So I'm assuming that in this case, it has already been determined that the operation which locks up the software is purely the UI calls themselves - in other words, there isn't anything else that could be pushed to another thread?

2) For what it's worth, in C#/WinForms, you also cannot directly perform any UI updates from background threads - however, in effect you can accomplish the same with something like the following. Not sure if something similar exists in qt:

// This function runs in a background worker thread
private void backgroundWorker_DoWork(object sender, DoWorkEventArgs e){
            // If you need to update the UI, you cannot do it directly from here - this would not be threadsafe/could crash:
            //uiElement.Text = "whatever"

            // However, you can safely accomplish the same by using a MethodInvoker to dispatch it back to the UI thread:
            MethodInvoker invoker1 = new MethodInvoker(delegate ()
            {
                 // Can manipulate any UI elements here. So effectively the operation was "originated" from the background worker, but the actual update
                 // gets queued to safely occur on the main UI thread
                 uiElement.Text = "whatever";
            });
            this.BeginInvoke(invoker1);
}

3) Even if there's no way to put it on another thread, it really seems like there should be creative solutions that would prevent it from locking up, regardless of the number of images. For example, break up the images into batches, and only add at most "maxN" items to the UI per invocation, before returning control to the message loop so that user interaction can still take place. I'm not directly familiar with how qt's message loop is implemented, but:
* Does that just mean inserting a brief sleep() or similar in the code that adds all the image thumbnails to the UI, so any user interaction messages could be processed before continuing?
* Or perhaps using a custom "SET_UI_IMAGES" message, which gets processed by adding maxN images to the UI, and re-queuing the same message type until all images have been added? (and when the user scrolls to a particular part of the scrollable container, on the next iteration of SET_UI_IMAGES, it would prioritize adding load the corresponding subset of maxN to the UI)

Basically, just some way to hand control back to the main message loop often enough that this long process doesn't have to run all in one go - and thus regardless of the total number of images, it need not block everything else until it's done.

Just saw that 8.0.0 was officially released (congrats). Unfortunately, search still remains unusable (locks up the entire UI for several minutes just attempting to enter that tab). Any comment on my message from 4 months ago? I thought I provided some pretty valid suggestions there.

In any event, even if properly threading/batching it is a bigger project, at the very least it seems like by default it could avoid trying to show every item in the library when entering that tab, so you can at least get into that UI without it completely locking the application. Then if you do a search that returns i.e. 500 results, it only needs to populate 500. Rather than the current situation, where it first populates the entire library before you can even attempt the search you intended to do (and then waiting again while it locks up the UI to populate the result of the search). Cutting out that first step seems like it would at least partially alleviate how this is implemented (just leave the pane empty until an explicit search is performed). As it stands, that pane is really truly not usable.

@Metal450,

This problem still reproducible with the new digiKam 8.2.0 pre-release Windows
installer available at usual place:

https://files.kde.org/digikam/

This new bundle is based on last Qt framework 5.15.11 and KDE framework 5.110.

Thanks in advance

Gilles Caulier

(In reply to caulier.gilles from comment #14)
> @Metal450,
> 
> This problem still reproducible with the new digiKam 8.2.0 pre-release
> Windows
> installer available at usual place:
> 
> https://files.kde.org/digikam/
> 
> This new bundle is based on last Qt framework 5.15.11 and KDE framework
> 5.110.
> 
> Thanks in advance
> 
> Gilles Caulier

Just installed it & gave it a try; the behavior is identical. The UI still locks up & becomes unresponsive the moment you try to use either of those tabs.

Any thoughts on my responses from 10 months ago, & follow-up 6 months ago?

A search interrupt button would be nice.

(In reply to Peter from comment #16)
> A search interrupt button would be nice.

A button wouldn't do anything if the UI thread is already locked up & not processing user inputs.

(In reply to Metal450 from comment #15)
> (In reply to caulier.gilles from comment #14)
> > @Metal450,
> > 
> > This problem still reproducible with the new digiKam 8.2.0 pre-release
> > Windows
> > installer available at usual place:
> > 
> > https://files.kde.org/digikam/
> > 
> > This new bundle is based on last Qt framework 5.15.11 and KDE framework
> > 5.110.
> > 
> > Thanks in advance
> > 
> > Gilles Caulier
> 
> Just installed it & gave it a try; the behavior is identical. The UI still
> locks up & becomes unresponsive the moment you try to use either of those
> tabs.
> 
> Any thoughts on my responses from 10 months ago, & follow-up 6 months ago?

Any thoughts on my responses from 13 months ago, & follow-up 9 months ago?

(In reply to spiesant from comment #18)
> (In reply to Metal450 from comment #15)
> > (In reply to caulier.gilles from comment #14)
> > > @Metal450,
> > > 
> > > This problem still reproducible with the new digiKam 8.2.0 pre-release
> > > Windows
> > > installer available at usual place:
> > > 
> > > https://files.kde.org/digikam/
> > > 
> > > This new bundle is based on last Qt framework 5.15.11 and KDE framework
> > > 5.110.
> > > 
> > > Thanks in advance
> > > 
> > > Gilles Caulier
> > 
> > Just installed it & gave it a try; the behavior is identical. The UI still
> > locks up & becomes unresponsive the moment you try to use either of those
> > tabs.
> > 
> > Any thoughts on my responses from 10 months ago, & follow-up 6 months ago?
> 
> Any thoughts on my responses from 13 months ago, & follow-up 9 months ago?

Is there a reason responses here are just ignored?

@spiesant,

Nothing is ignored. This thread history still available in time. We process report when time permit.

8.3.0 is now released with 250 files closed on bugzilla. Problem still reproducible ?

> 8.3.0 is now released with 250 files closed on bugzilla. Problem still reproducible ?

Yup, no change. Just switching to the "Search" tab freezes the whole application for over a minute.

> Nothing is ignored

Was referring specifically to Comment 12 above - there was good constructive discussion up to that point, & then just went silent for 1.25 years.