Bug 96388 - show number of images in the album
Summary: show number of images in the album
Status: RESOLVED FIXED
Alias: None
Product: digikam
Classification: Applications
Component: Albums-MainView (show other bugs)
Version: 0.7.0
Platform: unspecified Linux
: NOR wishlist
Target Milestone: ---
Assignee: Digikam Developers
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2005-01-05 20:34 UTC by Allen Winter
Modified: 2012-08-26 14:47 UTC (History)
4 users (show)

See Also:
Latest Commit:
Version Fixed In: 0.9.4


Attachments
Display the total number of images in status-line (4.83 KB, patch)
2007-06-30 17:48 UTC, Arnd Baecker
Details
test patch to display items number in folders (4.30 KB, patch)
2007-11-28 14:26 UTC, caulier.gilles
Details
screenshot of patch in action (302.84 KB, image/png)
2007-11-28 14:30 UTC, caulier.gilles
Details
patch version 2 (12.12 KB, patch)
2007-11-30 14:32 UTC, caulier.gilles
Details
patch version 3 (24.27 KB, patch)
2007-11-30 22:06 UTC, caulier.gilles
Details
patch version 4 (32.89 KB, patch)
2007-12-02 17:04 UTC, caulier.gilles
Details
patch version 5 (58.80 KB, patch)
2007-12-02 23:52 UTC, caulier.gilles
Details
patch version 6 (58.86 KB, patch)
2007-12-03 07:54 UTC, caulier.gilles
Details
patch version 7 (58.84 KB, patch)
2007-12-03 08:22 UTC, caulier.gilles
Details
patch version 8 (57.41 KB, patch)
2007-12-03 16:35 UTC, caulier.gilles
Details
patch version 9 (56.97 KB, patch)
2007-12-03 16:45 UTC, caulier.gilles
Details
patch version 10 (64.52 KB, patch)
2007-12-04 15:21 UTC, caulier.gilles
Details
patch version 11 (72.82 KB, patch)
2007-12-05 09:08 UTC, caulier.gilles
Details
patch version 12 (80.26 KB, patch)
2007-12-05 11:22 UTC, caulier.gilles
Details
patch version 13 (94.24 KB, patch)
2007-12-05 12:24 UTC, caulier.gilles
Details
patch version 14 (134.78 KB, patch)
2007-12-06 09:26 UTC, caulier.gilles
Details
patch version 15 (146.52 KB, patch)
2007-12-06 10:23 UTC, caulier.gilles
Details
patch version 16 (146.52 KB, patch)
2007-12-06 15:45 UTC, caulier.gilles
Details
patch version 17 (146.84 KB, patch)
2007-12-06 21:17 UTC, caulier.gilles
Details
patch version 18 (141.89 KB, patch)
2007-12-07 10:39 UTC, caulier.gilles
Details
patch version 19 (129.91 KB, patch)
2007-12-09 21:11 UTC, caulier.gilles
Details
new screenshot with timeline widget in action (320.17 KB, image/png)
2007-12-09 21:14 UTC, caulier.gilles
Details
patch version 20 (137.99 KB, patch)
2007-12-10 10:28 UTC, caulier.gilles
Details
patch version 21 (150.28 KB, patch)
2007-12-10 15:56 UTC, caulier.gilles
Details
timeline widget in action (176.49 KB, image/png)
2007-12-10 15:58 UTC, caulier.gilles
Details
patch version 22 (149.52 KB, patch)
2007-12-10 20:43 UTC, caulier.gilles
Details
patch version 23 (156.87 KB, patch)
2007-12-12 15:05 UTC, caulier.gilles
Details
timeline widget in action (310.99 KB, image/png)
2007-12-12 15:09 UTC, caulier.gilles
Details
patch version 24 (161.74 KB, patch)
2007-12-13 11:31 UTC, caulier.gilles
Details
timeline widget in action (317.09 KB, image/png)
2007-12-13 11:36 UTC, caulier.gilles
Details
patch version 25 (165.32 KB, patch)
2007-12-13 15:27 UTC, caulier.gilles
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Allen Winter 2005-01-05 20:34:04 UTC
Version:           0.7 (using KDE KDE 3.3.2)
Installed from:    Unlisted Binary Package

I'd like to see the number of images contained in the album, next to the album name, in parens.  Sorta like Kmail does when it shows the number of unread messages in a folder.
Album "parents" could sum up the number of images in each of its children... I suppose.

Thanks for your consideration -Allen
Comment 1 Mikolaj Machowski 2005-01-05 23:28:23 UTC
CC rejected so:

As option, yes. And once again development: album tooltips with comment,
date, number of images, info about sub-albums if exist.

Comment 2 Joern Ahrens 2005-06-11 12:33:57 UTC
*** Bug 105010 has been marked as a duplicate of this bug. ***
Comment 3 Otto Brandstaetter 2005-08-07 15:04:49 UTC
Yes, this would be VERY cool !
maybe a kind of display for the hdd space usage would also be cool !

Thanks in advance !
Comment 4 Otto Brandstaetter 2005-08-08 09:39:40 UTC
Oh, I am not really sure, if this is the right place here, but it would also be nice, if the album preview image would be saved somewhere (so they re-appear after a restart of digicam)
Comment 5 Joern Ahrens 2005-08-08 17:32:55 UTC
Yep, this wasn't the right place here. Please open a new report for every new bug/wish.
But what do you understand under "album preview image"? Do you mean the album thumbnails, which are shown on the left hand side? Actually these icons are stored to disk and (should) reapear after a restart.
Comment 6 Nicolas Brisset 2006-01-13 01:09:43 UTC
I would also love to see image numbers next to the album AND tag names :-) 
Comment 7 Christo 2006-08-10 01:05:56 UTC
good idea !!!
Comment 8 caulier.gilles 2006-10-22 22:14:13 UTC
*** Bug 128667 has been marked as a duplicate of this bug. ***
Comment 9 caulier.gilles 2007-02-08 07:47:42 UTC
*** Bug 141361 has been marked as a duplicate of this bug. ***
Comment 10 Arnd Baecker 2007-06-30 17:48:37 UTC
Created attachment 20998 [details]
Display the total number of images in status-line

Attached is a patch (with a lot of help by Gilles - big thanks!),
which displays the total number of images in the album view in the status-line.
Comment 11 Arnd Baecker 2007-08-30 16:29:18 UTC
Gilles, what about this patch, may I apply it?
(Needs also to be backported to KDE4).
Note, that it does not solve the original wish of this bug,
while still being useful
Comment 12 Arnd Baecker 2007-08-31 20:40:32 UTC
The patch from #10 is applied to svn (revision 706991).

To solve the original issue, the number of images has to 
be determined by a query to the database.
Depending on how time-consuming this is, it might have to be done
in the background and the GUI entries have to be updated once the
required information is available. 
Note that in principle one could also do this for Tags, Searches and Dates
in the left-hand side-bar.
(Of course, the number of images have to be updated when changes occur...)

Comment 13 Faf 2007-10-12 23:11:38 UTC
it would be nice if the "New Album" and "Edit Album" dialogs had an option to make a the album a "sequenced drop target", thus renaming the files in the order that it was dropped {moved or copied into}. for when you create albums/slideshows from other multiple albums, when order and duplicate filenames may be an issue.
Comment 14 caulier.gilles 2007-11-28 14:04:46 UTC
Ok, 

It's time now to study this report...

With a first try using a very small patch on my computer, i can see the number of items on Albums folder view and Tags folder view.

It's just a try to check what we can do easily in KDE3 implementation. 

Gilles
Comment 15 caulier.gilles 2007-11-28 14:26:46 UTC
Created attachment 22232 [details]
test patch to display items number in folders

This is not a final patch. I just display the number of item in:

- Album tree view from left side bar (non recursive mode).
- Tags tree view from left side bar (recursive mode).
- Tags tree view from Caption & Tags on right side bar (recursive mode).
- Tags Filter View from right side bar (recursive mode).

Notes: non-recursive mode with all Tags tree view is possible. It's just a bool
flag to handle.

Arnd, Marcel, i have few question about AlbumDB and database query:

1/ How to support recursive mode with Album tree view ?
2/ How to support Date tree view ?
3/ How to support Search tree view ?

Gilles
Comment 16 caulier.gilles 2007-11-28 14:30:40 UTC
Created attachment 22233 [details]
screenshot of patch in action
Comment 17 caulier.gilles 2007-11-28 16:21:28 UTC
Arnd, Marcel,

I have changed the assignation email witch catch this thread. Look my previous comments on this file and let's me hear your viewpoints...

Gilles

NOTE: i recommend to never change the default assignation email in B.K.O
Comment 18 Arnd Baecker 2007-11-28 17:47:03 UTC
Hi Gilles,

very nice! Some comments:
- the total number of files in an album is displayed. This is different
  from the number of images/movies. This can be confusing
  (One solution for this would be:  https://bugs.kde.org/show_bug.cgi?id=145743 )
- 1/ this is a difficult question, because I would think that
  depending on whether the tree is collapsed or not, different numbers should 
  be obtained:

  [+] album1 (200)   vs. [-] album1 (0)
                             * subalbumA (90)
                             * subalbumB (110)
  So in my opinion this is independent of the recursive view.
  Of course one could always display 200 for album1, maybe this is even better?
- I am not sure, whether a recursive mode for tags makes sense:
  Contrary to the album view (where subalbumA is, by organization, contained
  in album1), subtag1 and tag1 can be ticked independently from each
  other (even though they are organized in a hierarchical way).
- I think that for 1/ 2/ and 3/ corresponding queries like
     QStringList AlbumDB::getItemNamesInAlbum(int albumID)
  have to be added to albumdb.cpp.
- Marcel, are such queries fast enough so that one can easily
  do several of those, or would there be problems on slower machines?

Best, Arnd
Comment 19 caulier.gilles 2007-11-28 18:10:52 UTC
> the total number of files in an album is displayed. This is different
> from the number of images/movies. This can be confusing
> (One solution for this would be:  https://bugs.kde.org/show_bug.cgi?id=145743 ) 

Yes. I can reproduce here. A filtering using mimetype registartion in setup must be done after to query AlbumDB.

Gilles
Comment 20 caulier.gilles 2007-11-28 18:17:39 UTC
> I think that for 1/ 2/ and 3/ corresponding queries like
> QStringList AlbumDB::getItemNamesInAlbum(int albumID)
> have to be added to albumdb.cpp. 

Yes, getItemNamesInAlbum() function is only fine to query physical Albums, not virtuals...

Marcel, can you proposal DB queries for that ?

Gilles

Comment 21 Arnd Baecker 2007-11-28 21:54:36 UTC
To get the number of entries for a given album specified by a kurl,
including its sub-albums, the following should do the job 
(see digikamalbums.cpp)
  QStringList albumvalues;
  urlWithTrailingSlash = kurl.path(1);
  m_sqlDB.execSql(QString("SELECT DISTINCT id, url FROM Albums WHERE
        url='%1' OR url LIKE'\%%2\%';")
  .arg(escapeString(url)).arg(escapeString(urlWithTrailingSlash)),&albumvalues);

For the dates one would have to copy the code from 
void kio_digikamdates::special(const QByteArray& data)

And for searches, void kio_digikamsearch::special(const QByteArray&
data).

The routines involve iterators and some explicit tests...

This sounds like a lot of code duplication.
Maybe it is possible to extract the corresponding parts?
Comment 22 Marcel Wiesweg 2007-11-29 20:59:00 UTC
Physical albums and tags is easy, SQL (for 0.9 dbs):
SELECT COUNT(id) FROM Images WHERE dirid=?;
SELECT COUNT(imageid) FROM ImageTags WHERE tagid=?;

These queries are sufficiently fast, they also can easily be cached in the PAlbum/TAlbum objects.

This can also be done for dates, though potentially a bit slower (we can create an index on creation date for 0.10).

The problem with Searches is that any change in metadata or whatever can affect any search, and that the queries for searches are potentially much slower. So caching is a problem (because you have to listen for everything to keep is up to date), and doing live queries is a problem because it may take longer.
Comment 23 Mikolaj Machowski 2007-11-29 22:19:12 UTC
I disagree with Arnd on 1/ IMO number should show only number of items in current album, without subalbums - always. Exactly like in KMail.

Besides - I suppose whatever way you choose, in few weeks there  will be opened BKO entry to   make it other way  :)
Comment 24 Thomas McGuire 2007-11-29 22:30:14 UTC
>I disagree with Arnd on 1/ IMO number should show only number of items in current album, without subalbums - always. Exactly like in KMail. 
In KMail, when the parent folder is collapsed, the count of the unread mails of the subfolders is added.
Example:

Folder A (2)
|-- Subfolder B (3)
\-- Subfolder C (4)

becomes, after collapsing:

Folder A (2 + 7)
Comment 25 Mikolaj Machowski 2007-11-30 00:28:51 UTC
@KMail

But number of total messages is always the same. Maybe thing with
+ would be best compromise?
Comment 26 caulier.gilles 2007-11-30 14:23:59 UTC
To Marcel, #22:

About count Physical Albums, your solution is not enough. You need to check if file name match the type mime settings, else you will see unwanted files in results.

Also, your solution do not provide a way to check recurssive physical albums items count... But I can feel than make a recursive parsing will take a while and cannot be done easily.

About Tags, this problem do not exist (i think). So your solution if fine for me.

I have not yet studied Date and Search Albums case... 

Gilles
Comment 27 caulier.gilles 2007-11-30 14:32:48 UTC
Created attachment 22256 [details]
patch version 2

This new version supports refresh of Physical Albums folder view when items are
moved/removed/added.

Marcel, please check DIO and AlbumManager class. I'm not sure than code is the
better one... You know better than me these parts...

The setting to display number of items in Physical Albums is hard-forced to
code. 

Tags folders view patch still unchanged. next patch is under progress...

Questions: 

- where we will place the settings to toggle on/off this option ? In Album
Setup page or in View menu ?

- Are you need to separate options to toggle on/off number of items in all tree
views ? one for Physical Albums, one for Date, one for Tags, etc... ?

Gilles
Comment 28 Arnd Baecker 2007-11-30 15:00:15 UTC
I am not sure if a setting would be needed. But if so, I would
suggest to have it in the album setup page and not in the view menu.
This should be a setting which need not be changed that often.

Presumably one option for all types of views is ok.
Comment 29 Marcel Wiesweg 2007-11-30 17:21:21 UTC
If the code works it is ok for me. Note that this will not be ported straightforwardly to 0.10; this is an area with a lot of changes and some things can be done differently there.
Comment 30 caulier.gilles 2007-11-30 22:06:22 UTC
Created attachment 22262 [details]
patch version 3

New version of the patch. Now, all Tags folder views supports fully to display
number of items, like previous patch about Physical Albums.

TODO:

-Date and Search view still not yet implemented...
-There is no yet a setting to toggle on/off this feature...

Marcel, please check if all is fine for you... Thanks in advance

Gilles
Comment 31 caulier.gilles 2007-11-30 22:10:54 UTC
Mik, Marcel, Gerhard,

- Where do you want to see the settings to toggle on/off options : Album Setup page or View main menu ?

- How many options you want to see : only one to toggle all folders view at the same time or one option for each folders view ?

Gilles
Comment 32 Hans Meier 2007-12-01 09:15:20 UTC
I disagree with comment #23 - the parent folder should show all images including the ones contained in subfolders. That way, the root folder would immediately show how many pictures there are in the collection.
Comment 33 Mikolaj Machowski 2007-12-01 18:54:04 UTC
> - Where do you want to see the settings to toggle on/off options : Album
> Setup page or View main menu ?

Album setup page (there is already size of thumbs, no sense to split
options between two places).
> - How many options you want to see : only one to toggle all folders view
> at the same time or one option for each folders view ?

One option.
Comment 34 Gerhard Kulzer 2007-12-02 08:54:51 UTC
Concerning the number of items showing my opinion is this:
- When an entry has no hidden subs it shows its own number (obviously)
- When an entry has subs but the tree is open (-) it shows its own number only
- When an entry has subs and tree is closed (+) it shows the total content of the branch in one number (I think the kmail style : Folder A (2 + 7) is more confusing than bringing information)

Concerning the menu entry: No clear preference
An ideal solution could be to :
- set it in the setup page which is stored in the rc file. This setting defines the behaviour across open/close of application.
- allow a change of display from View menu as temporary toggle. This falls back to settings after restart of application

Gerhard
Comment 35 Arnd Baecker 2007-12-02 09:55:36 UTC
Gerhard: full agreement on all points
(only about the change from View menu as a temporary toggle I am not sure.
Usually I am much in favour of quick configuration changes, but here
I am not so sure if this is really needed in a menu, which is already pretty full:
how often would one really want to change it?)
Comment 36 Mikolaj Machowski 2007-12-02 09:57:49 UTC
> - set it in the setup page which is stored in the rc file. This setting
> defines the behaviour across open/close of application. - allow a change
> of display from View menu as temporary toggle. This falls back to
> settings after restart of application

Definitely no. It will really confuse users.
Comment 37 caulier.gilles 2007-12-02 16:40:16 UTC
To Gerhard #34

>Concerning the number of items showing my opinion is this:
>- When an entry has no hidden subs it shows its own number (obviously)
>- When an entry has subs but the tree is open (-) it shows its own number only
>- When an entry has subs and tree is closed (+) it shows the total content of >the branch in one number (I think the kmail style : Folder A (2 + 7) is more >confusing than bringing information)

Agree with all! Here in my computer Album Gui work like this now. Also, i have implemented reccursive physical album count ! 

Still TODO Tags folder view. New patch in progress...


>Concerning the menu entry: No clear preference
>An ideal solution could be to :
>- set it in the setup page which is stored in the rc file. This setting defines >the behaviour across open/close of application.
>- allow a change of display from View menu as temporary toggle. This falls back >to settings after restart of application

I will provide a common setting for all views in Album setup dialog page. We will wait and see users feedback if a menu entry is require.

Gilles
Comment 38 caulier.gilles 2007-12-02 17:04:01 UTC
Created attachment 22282 [details]
patch version 4

In this version, items count in folder tree views is computed accordingly with
Gerhard tips posted in #34...
Comment 39 Arnd Baecker 2007-12-02 21:23:19 UTC
Very nice!
It seems that the counter for tags is adjusted properly, when
a tag is added to an image. However, when removing
a tag from an image (using the right side-bar), the count does not drop.

Sometimes, it seems when ticking more than one tag (using the right side-bar)
and pressing apply, I recieve a crash.
(It could be that this happens not for the first, but second time one does this 
to an image). 

I did a couple of further tests, like moving images around,
and it seems to work fine.
Comment 40 caulier.gilles 2007-12-02 23:19:02 UTC
Marcel,

I can reproduce the crash like Arnd said when a tag is removed from a picture. It's relevant of QMap container returned MetadataHub::tagIDs(). Using an iterator do not work very well... I don't know why...

Apply the patch version 4 and look the code in imagedescedittab.cpp::line 454. Backtrace is here:

#0  0xb7de58a8 in QMapConstIterator<int, Digikam::MetadataHub::TagStatus>::inc (this=0xbfdd316c) at /usr/lib/qt3//include/qmap.h:287
#1  0xb7de5913 in QMapConstIterator<int, Digikam::MetadataHub::TagStatus>::operator++ (this=0xbfdd316c) at /usr/lib/qt3//include/qmap.h:259
#2  0xb7de442e in Digikam::ImageDescEditTab::slotApplyAllChanges (this=0x8295ca0) at imagedescedittab.cpp:454
#3  0xb7de5219 in Digikam::ImageDescEditTab::qt_invoke (this=0x8295ca0, _id=46, _o=0xbfdd3264) at imagedescedittab.moc:253
#4  0xb6114f5d in QObject::activate_signal () from /usr/lib/qt3/lib/libqt-mt.so.3
#5  0xb6115add in QObject::activate_signal () from /usr/lib/qt3/lib/libqt-mt.so.3
#6  0xb6429bfc in QButton::clicked () from /usr/lib/qt3/lib/libqt-mt.so.3
#7  0xb61a61ed in QButton::mouseReleaseEvent () from /usr/lib/qt3/lib/libqt-mt.so.3
#8  0xb614d564 in QWidget::event () from /usr/lib/qt3/lib/libqt-mt.so.3
#9  0xb60b7cea in QApplication::internalNotify () from /usr/lib/qt3/lib/libqt-mt.so.3
#10 0xb60b8cca in QApplication::notify () from /usr/lib/qt3/lib/libqt-mt.so.3
#11 0xb67cbdba in KApplication::notify () from /usr/lib/libkdecore.so.4
#12 0xb60593e1 in QETWidget::translateMouseEvent () from /usr/lib/qt3/lib/libqt-mt.so.3
#13 0xb6058156 in QApplication::x11ProcessEvent () from /usr/lib/qt3/lib/libqt-mt.so.3
#14 0xb60682ad in QEventLoop::processEvents () from /usr/lib/qt3/lib/libqt-mt.so.3
#15 0xb60cd9e0 in QEventLoop::enterLoop () from /usr/lib/qt3/lib/libqt-mt.so.3
#16 0xb60cd876 in QEventLoop::exec () from /usr/lib/qt3/lib/libqt-mt.so.3
#17 0xb60b790f in QApplication::exec () from /usr/lib/qt3/lib/libqt-mt.so.3
#18 0x0804b131 in main (argc=-1208785332, argv=0xb5633162) at main.cpp:332

Gilles
Comment 41 caulier.gilles 2007-12-02 23:52:52 UTC
Created attachment 22290 [details]
patch version 5

New version of patch including an option from setup dialog to toggle on/off
this new feature.
Comment 42 caulier.gilles 2007-12-03 07:54:44 UTC
Created attachment 22295 [details]
patch version 6

New version of patch witch fix crash when user apply changes in Caption and
Tags sidebar tab.
Comment 43 caulier.gilles 2007-12-03 08:22:16 UTC
Created attachment 22297 [details]
patch version 7

New version of patch witch apply properly all changes in Captions & Tags
Comment 44 caulier.gilles 2007-12-03 16:35:23 UTC
Created attachment 22305 [details]
patch version 8

New version of the patch including Date Folder View support...

Marcel: Code have been simplified. DIO is untouched noww and AlbumbManager
changes are minor. All work fine with all Tags View, but with Physical Album
view, to remove items or albums do not update properlly the items counts in
Albums Folder View and Date Folder View accordinly with this report:

http://bugs.kde.org/show_bug.cgi?id=152961

Please take a look in #152961. There is an important bug here...

Gilles
Comment 45 caulier.gilles 2007-12-03 16:45:32 UTC
Created attachment 22306 [details]
patch version 9

minor update to compile fine...
Comment 46 caulier.gilles 2007-12-04 15:21:17 UTC
Created attachment 22335 [details]
patch version 10

"Et voilà"...

New version of the patch witch add support of Search Folder View count of
items...

Marcel, 

with Search Folder View, i use Kio-slave. It's better to query Database
directly instead to use AlbumDB. Search number of items will be computed in
background without to freezze GUI.

I can do the same for Date Folder View witch can take a while if collection is
huge... What do you think about ?

Arnd, Gerhard, Mik, etc... 

it's time to test this patch indeep, epecially in stress conditions : huge
album collection hierarchy, huge tags collection, huge date range, huge Search
list...

I would to apply this patch after to have released 0.9.3 final. 

Thanks in advance for your helps.

Gilles
Comment 47 Arnd Baecker 2007-12-04 17:35:53 UTC
- non-image files are still counted.
- the counting of tags is not intuitive:
  Assume you have: 
  10 images tagged with D and 3 additional, different ones, 
  are tagged with subD
  This gives:
    - D (10)
      - SubD (3)
  and collapsed
    + D (13)

  So clicking on D one would expect 13 images, but
  this is only correct if recursion into sub-tags is active;
  otherwise only 10 images are shown.

  I think that this can be pretty confusing to the user ...

- I had a weird problem on the first activation
  of the configuration setting that digikam 
  completely hung. It could only be killed,
  but then it did not want to restart
  (hanging while "reading database ...").

  Unfortunately I cannot (and won't dare to) reproduce
  this, because I did this on the machine with my "real" images.
 
Comment 48 Arnd Baecker 2007-12-04 17:57:58 UTC
I forgot the most important thing: this is really a great feature - thanks Gilles!
Comment 49 caulier.gilles 2007-12-05 09:04:58 UTC
Arnd, Gerhard,

Thanks for the reports.

There are severals speed-up improvements to do in the patch, especially to use kio-slave with Album folders view and tags folder view, and to query database after than GUI is initialized. Currently the queries are done during folder list view items construction in memory... This is why digiKam take a while to start-up.

I will fix it...

Gilles
Comment 50 caulier.gilles 2007-12-05 09:08:10 UTC
Created attachment 22348 [details]
patch version 11

In this version :

- Date Folder view use Kio-slave, not AlbumBD directly.
- Search folder view delay items counts after than gui is initialized.

TODO : use kioslave with Album folder view and all other Tags folder views.

Gilles
Comment 51 caulier.gilles 2007-12-05 09:14:23 UTC
Marcel,

Look in patch version 11 how i query database with Dates ioslave to handle folders  statistics. Now, ioslave return a QMap of number of items by month, not just a simple a list of QDate available to identify witch month must be created as DAlbum  in AlbumManager.

AlbumManger implementation is just adapted to use the new QMap as well and continue to work like before.

The idea is to use this QMap later to draw the future TimeLine bar in Date folder view (an histogram of number of items by month). See http://bugs.kde.org/show_bug.cgi?id=146760 for details...

Gilles
Comment 52 caulier.gilles 2007-12-05 11:22:18 UTC
Created attachment 22349 [details]
patch version 12

New version where Tags Folder View use KIO-slave

TODO: use kio-slave with Tags Filter View, Album Folder View and Caption&Tags
Folder View

Gilles
Comment 53 caulier.gilles 2007-12-05 12:24:09 UTC
Created attachment 22351 [details]
patch version 13

New version where Tags Filter View use KIO-slave

TODO: use kio-slave with Album Folder View and Caption&Tags Folder View
Comment 54 caulier.gilles 2007-12-06 09:26:02 UTC
Created attachment 22377 [details]
patch version 14

New version of patch where all implementations have been simplified and cleaned
everywhere... excepted in Search Folder View.

In this version, AlbumManager is god. 2 new instances of Albums and Tags
kio-slave are created to query database about simple and recursive items count
in albums. Kio-salves include now a "folders" mode (like Date kio-slave) to
stats album contents.

Normally, this way will not freezze gui everywhere. 

Of course, i need to use the same way about Search Folder view...

Gilles
Comment 55 caulier.gilles 2007-12-06 10:23:51 UTC
Created attachment 22378 [details]
patch version 15

This last version fix Search Folder View and kio-slave. Now, all is driven by
AlbumManager...

Please let's me hear if this version is fine in all cases, especially with huge
collection of physical albums/tags/searches...

Thanks in advance

Gilles
Comment 56 caulier.gilles 2007-12-06 15:45:43 UTC
Created attachment 22382 [details]
patch version 16

minor fix to compile with current changes from svn
Comment 57 caulier.gilles 2007-12-06 21:17:44 UTC
Created attachment 22388 [details]
patch version 17

Force Album Manager to refresh albums stats when count of items option is
toogle on, else at the first time than you use it, counts will be set to zero
everywhere...

Gilles
Comment 58 Mikolaj Machowski 2007-12-06 22:17:30 UTC
@v16

Two problems:

- after turning it on counter show (0) on all folders, it worked only
  after restart
- update in numbers was very slow - several seconds on 304 items in
  recursively shown albums (after deleting of 5 items)

Apart from that I don't see any issues.
Comment 59 caulier.gilles 2007-12-07 07:16:42 UTC
>after turning it on counter show (0) on all folders, it worked only
> after restart

Patch version 17 fix this problem...

>update in numbers was very slow - several seconds on 304 items in
> recursively shown albums (after deleting of 5 items) 

scan of folder items to count is delegate to digikamalbums kio-slave witch query database. There are 2 query performed : one to count items in simple mode, one to count items in recursive mode.

The first one is simple and fast. The second is more complex, but can be removed.... I can delegate the recursive count of items to the GUI album tree view witch know folder hierarchy. I can count recursive values when a folder branch is not open, following hierarchy.

Gilles
Comment 60 caulier.gilles 2007-12-07 10:39:11 UTC
Created attachment 22391 [details]
patch version 18

New version with an optimized way to scan physical album contents to count
items.

Now digikamalbums kio-slave only parse DB to get simple count of album
contents. Recursive count is delegate to folder tree view GUI.

==> Instead to have 2 complex queries in database witch take a while, there
still only one simple query... It's more fast.

Gilles
Comment 61 Arnd Baecker 2007-12-07 17:04:36 UTC
Speedwise all seems great!
Open issues I found are the first two points of
http://bugs.kde.org/show_bug.cgi?id=96388#c47
Comment 62 Arnd Baecker 2007-12-07 21:08:19 UTC
Correction: onlhy the second point of http://bugs.kde.org/show_bug.cgi?id=96388#c47  is open
(the problem I had with point 1 was due to images with negative rating,
a "feature" which was recently cured ;-).
Comment 63 Mikolaj Machowski 2007-12-08 11:15:53 UTC
@patch 18

Faster but still slowly. I wonder if it would be possibly to give user
some visual feedback about "I am now computing number, don't trust this
number". Eg. directly after removing change font to italic, compute
value, update value and switch to regular font.
Comment 64 caulier.gilles 2007-12-09 21:11:43 UTC
Created attachment 22437 [details]
patch version 19

In this new version:

- Counts of items in Search Folder view is removed. This part take too many
time to run and is not really informative for end-users.
- Speed improvement in Tag Kioslave to handle count of item. Same way is used
here than albums count of items.
- Introduction of Timeline in Date Folder View. This new tool is not yet
complete. Please do not yet report anything about and let's me more time to
complete it...

Marcel, 

There is certainly more speedup improvements to do in digikamalbum and
digikamtags kioslave to handle count of items. I thining about to query only
one time the DB (outside the loop) to get values in QStringList and only parse
the values contents in the loop. It's always for speed to loop QT api than to
query database. 

What do you think about ?

Gilles
Comment 65 caulier.gilles 2007-12-09 21:14:53 UTC
Created attachment 22438 [details]
new screenshot with timeline widget in action
Comment 66 caulier.gilles 2007-12-10 10:28:42 UTC
Created attachment 22442 [details]
patch version 20

In this new version, the digikamalbum kioslave only query one time DB to get
albums statistic. The loop used to parse count of items play only on a QMap and
QStringList: There is no more database acess in loop. 

Like there is only one database query, it's more fast...

Gilles
Comment 67 caulier.gilles 2007-12-10 15:56:09 UTC
Created attachment 22453 [details]
patch version 21

new version of the patch witch include more speed up improvement in digikamtags
kio-slave and several improvment in timeline widget.

Code is not optimum yet. We need to find a simple query to DB outside the loop
witch count items in tags albums.

Marcel, code sound like below :

// --------------------------------------
QMap<int, int> tagsStatMap;
int	       tagID, imageID;
QStringList    values, fileNames;

m_db.execSql(QString("SELECT tagid, imageid FROM ImageTags"), &values);

for ( QStringList::iterator it = values.begin(); it != values.end(); )
{
   tagID = (*it).toInt();
   ++it;
   imageID = (*it).toInt();
   ++it;
   fileNames.clear();
   m_db.execSql(QString("SELECT name FROM Images WHERE id=%1").arg(imageID),
&fileNames);

   if ( matchFilterList( regex, fileNames.first() ) )
   {
      QMap<int, int>::iterator it2 = tagsStatMap.find(tagID);
      if ( it2 == tagsStatMap.end() )
	 tagsStatMap.insert(tagID, 1);
      else
	 tagsStatMap.replace(tagID, it2.data() + 1);
   }
}

QDataStream os(ba, IO_WriteOnly);
os << tagsStatMap;

// --------------------------------------

Notes: 

- The first DB query outside the loop get the complete list of tags/images IDs.

- The second one inside the loop get the filename associed to the image ID to
check mime type in items count.

It would be nice if we can get the list of tagsIDs/imageFilenames directly
outside the loop using only one query. This will limit DB access and speedup
the process...

But it's complex to do because tagsIDs/imageIds and imageIds/imageFilenames
lists are stored in separate DB tables. 

I need help from an SQL guru here (:=)))...

Gilles
Comment 68 caulier.gilles 2007-12-10 15:58:55 UTC
Created attachment 22454 [details]
timeline widget in action
Comment 69 Thomas Thrainer 2007-12-10 19:01:03 UTC
Hello

I have just read your comment, and it looks like you are looking for a join in your code.

Try this:

SELECT ImageTags.tagid, Images.name FROM ImageTags, Images WHERE ImageTags.imageid = Images.id; 

This gives you a list of tagid/name pairs of all images which have tags. If you want the list sorted (by tagid), try

SELECT ImageTags.tagid, Images.name FROM ImageTags, Images WHERE ImageTags.imageid = Images.id ORDER BY ImageTags.tagid; 

To get a list of all tags and the number of associated images, try

SELECT ImageTags.tagid, COUNT(Images.id) FROM ImageTags, Images WHERE ImageTags.imageid = Images.id GROUP BY ImageTags.tagid;

As you have written, that you need to check against the filename, this could be done in this way (if I have understood you right):

SELECT ImageTags.tagid, COUNT(Images.id) FROM ImageTags, Images WHERE ImageTags.imageid = Images.id AND
(Images.name LIKE '%.jpg' OR
Images.name LIKE '%.bmp')
GROUP BY ImageTags.tagid;     

(add OR clauses as you need them for the mime-type, but I am not sure if this approach is more performant than your's with Regex and QMap)

again, to get a sorted list, try:

SELECT ImageTags.tagid, COUNT(Images.id) FROM ImageTags, Images WHERE ImageTags.imageid = Images.id AND
(Images.name LIKE '%.jpg' OR
Images.name LIKE '%.bmp')
GROUP BY ImageTags.tagid
ORDER BY ImageTags.tagid;

I don't know if you also want tagid's with no images in the list. These querries only return tagid's which have at least one image assigned to it, but it would be possible to return a list of all tagid's, even those with no images.

I tried these querries against my own digikam3.db file, they all work for me.

I am not a big SQL guru neither, but maybe I could help.

Thomas
Comment 70 caulier.gilles 2007-12-10 20:25:10 UTC
Thanks Thomas,

SELECT ImageTags.tagid, Images.name FROM ImageTags, Images WHERE ImageTags.imageid = Images.id; 

... work fine for me. I just need to have the currently assigned tags list to images.

Note : i have tried something similar today but without success. I need to re-read my SQL books (:=)))

I will update the patch.

Gilles
Comment 71 caulier.gilles 2007-12-10 20:43:21 UTC
Created attachment 22462 [details]
patch version 22

New version using SQL query from Thomas about Tags counts of items.

Gilles
Comment 72 Mikolaj Machowski 2007-12-11 23:12:10 UTC
Cannot compile latest version of patch:

widget.Tpo -c -o timelinewidget.lo timelinewidget.cpp
In file included from timelinewidget.cpp:41:
timelinewidget.moc:113: error: redefinition of 'const char* 
Digikam::TimeLineWidget::className() const'
timelinewidget.moc:22: error: 'virtual const char* 
Digikam::TimeLineWidget::className() const' previously defined here
timelinewidget.moc:118: error: redefinition of 'QMetaObject* 
Digikam::TimeLineWidget::metaObj'
timelinewidget.moc:27: error: 'QMetaObject* 
Digikam::TimeLineWidget::metaObj' previously defined here
timelinewidget.moc:119: error: redefinition of 'QMetaObjectCleanUp 
cleanUp_Digikam__TimeLineWidget'
timelinewidget.moc:28: error: 'QMetaObjectCleanUp 
cleanUp_Digikam__TimeLineWidget' previously declared here
timelinewidget.moc:141: error: redefinition of 'static QMetaObject* 
Digikam::TimeLineWidget::staticMetaObject()'
timelinewidget.moc:50: error: 'static QMetaObject* 
Digikam::TimeLineWidget::staticMetaObject()' previously defined here

And several hundred lines of such messages. Some file wasn't included in
patch?
Comment 73 Marcel Wiesweg 2007-12-11 23:19:12 UTC
I usually write the JOIN explicitly:

SELECT ImageTags.tagid, Images.name FROM ImageTags LEFT JOIN Images ON ImageTags.imageid=Images.id;

which gives the same result. A real SQL guru could explain the difference.

It is the right way to go here to limit the number of SQL queries to speed things up (as long as the result set is not too large for memory ;-) )
A query always involves much more computing work than accessing structures in memory.

And I have a much better feeling now that you removed search counting. The other three can be handled, but searches get complex (as testing has proved).
I will have time tomorrow to test the patch, finally.
Comment 74 caulier.gilles 2007-12-12 06:24:59 UTC
Thomas,

Do you know the difference between your SQL query :


SELECT ImageTags.tagid, Images.name FROM ImageTags, Images WHERE ImageTags.imageid = Images.id; 

... and Marcel query :

SELECT ImageTags.tagid, Images.name FROM ImageTags LEFT JOIN Images ON ImageTags.imageid=Images.id; 

Thanks in advance...

Gilles
Comment 75 caulier.gilles 2007-12-12 06:29:55 UTC
to Mik #72 ==> sound like a moc file problem. try make clean before to compile 

Gilles
Comment 76 Fabien 2007-12-12 11:02:15 UTC
About #74

If I remember correctly my studies, it's about unmatched records.
With LEFT JOIN, you will also get records which do not match (where one image id exists in the table but not in the other)...
Comment 77 Thomas Thrainer 2007-12-12 11:44:59 UTC
Hello

This query:

SELECT ImageTags.tagid, Images.name FROM ImageTags, Images WHERE ImageTags.imageid = Images.id;

is the implicit way of writing

SELECT ImageTags.tagid, Images.name FROM ImageTags INNER JOIN Images ON ImageTags.imageid=Images.id;

The database should treat both queries in exactly the same way (and optimise both queries to the same disk/database operations).
The second way of writing it is more explicit, and stresses the fact that there is a join involved. But the database (should) work out that the WHERE clause in the first query is equivalent. MySQL, in fact, replaces the JOIN clause internally by WHERE clauses.


In the second query, we used an INNER JOIN. This join is different from the LEFT JOIN, but due to the nature of the table ImageTags they produce the same result in this case.
The LEFT JOIN ensures that all rows from the "left" table (in our case ImageTags) are included in the resulting set. If there were no corresponding row in the Images table, it would fill in null values. But as there are only valid imageid's in ImageTags, the database finds for each row in ImageTags an Image, and the result is the same as for the INNER JOIN (which would never add null-values).

So, this query is (in this case) also equivalent:

SELECT ImageTags.tagid, Images.name FROM ImageTags LEFT JOIN Images ON ImageTags.imageid=Images.id;

There exists also a RIGHT JOIN and a FULL OUTER JOIN. SQLite 3 does not support them, and MySQL does not support the FULL OUTER JOIN. Also, there are some other variants of joins, like NATURAL joins, but I'm not sure which databases support which joins, and where are the exact differences.


So, to conclude, all three queries return the same result, and a database should optimize all three of them to the same low level operations (note that the LEFT JOIN could possibly constrain the database's optimizations, but in this case there is no difference, as there are no further WHERE clauses).

So, I think it comes down to personal taste which of the three queries you use.

Hopefully I could clarify a bit,

Thomas
Comment 78 caulier.gilles 2007-12-12 14:59:53 UTC
Thomas,

Thanks. It's very instructive...

So the argument to choose the right SQL query will the speed. 

What will be the most faster query ?

Gilles
Comment 79 caulier.gilles 2007-12-12 15:05:04 UTC
Created attachment 22502 [details]
patch version 23

In this patch, timeline widget have been improved again to display more
information behind time scale tick.

Still TODO to complete the widget : selection of date histogram bar to dispatch
date on month widget and date folders view...

Gilles
Comment 80 caulier.gilles 2007-12-12 15:09:01 UTC
Created attachment 22503 [details]
timeline widget in action
Comment 81 Thomas Thrainer 2007-12-12 18:00:09 UTC
About #77

IMO, all three queries should have the same performance.

I would probably use

SELECT ImageTags.tagid, Images.name FROM ImageTags INNER JOIN Images ON ImageTags.imageid=Images.id;


Thomas
Comment 82 Mikolaj Machowski 2007-12-12 22:21:20 UTC
@patch 23

It is fast. Slower in recursive albums but acceptable.
Comment 83 caulier.gilles 2007-12-13 11:28:39 UTC
Mik,

Thanks for the report. 

Just to be sure: how many pictures/tags have you in your collection ? 

Gilles
Comment 84 caulier.gilles 2007-12-13 11:31:32 UTC
Created attachment 22521 [details]
patch version 24

New version of patch where simple selection of datebar is implemeted in
timeline widget. Currently, only one bar can be selected. Multiple selection is
planed, but i need to change DAlbum class to support date range (currently one
month is supported).
Comment 85 caulier.gilles 2007-12-13 11:36:35 UTC
Created attachment 22522 [details]
timeline widget in action
Comment 86 Mikolaj Machowski 2007-12-13 11:52:23 UTC
Gilles,

Around 9000 photos and films.

----------------------------------------------------
Mam dla Ciebie ciekaw
Comment 87 caulier.gilles 2007-12-13 15:27:39 UTC
Created attachment 22526 [details]
patch version 25

First version of patch where timeline widget dispatch date selection to album
folder view. Not all is yet fully implemented of course...

Gilles
Comment 88 caulier.gilles 2007-12-30 10:00:34 UTC
SVN commit 754570 by cgilles:

digiKam from KDE3 branch : important changes here : add capability to display count of items in all Folder View.
The number of items contained in virtual or physical albums is now displayed on the right of album name. 
If a tree branch is collapsed, parents album sum-up the number of items from all undisplayed children albums.
Count of items is performed in background by digiKam KIO-Slaves.
A new option from Setup/Album dialog page can toggle on/off this feature.

Screenshot of digiKam in action using this new feature can be seen at this url:

http://bugs.kde.org/attachment.cgi?id=22233&action=view

Folder view witch supports this feature are :

- Physical Albums from left side bar.
- Virtual Date Albums (calendar) from left side bar.
- Virtual Tags Albums from left side bar.
- Virtual Tags Albums from Captions & Tags right side bar tab.
- Virtual Tags Albums Filter from right side bar.

Note: for performance reasons, Search folder view do not support count of items. Search queries on digiKam database
can take a while and slow down digiKam reactivity. See B.K.O 96388 story for details...

BUG: 96388
CCMAIL: digikam-devel@kde.org
TODO: KDE4PORT



 M  +128 -45   digikam/albumfolderview.cpp  
 M  +5 -2      digikam/albumfolderview.h  
 M  +5 -5      digikam/albumlister.cpp  
 M  +1 -1      digikam/albumlister.h  
 M  +207 -44   digikam/albummanager.cpp  
 M  +37 -22    digikam/albummanager.h  
 M  +122 -104  digikam/albumsettings.cpp  
 M  +3 -0      digikam/albumsettings.h  
 M  +164 -59   digikam/datefolderview.cpp  
 M  +46 -3     digikam/datefolderview.h  
 M  +11 -0     digikam/digikamapp.cpp  
 M  +10 -0     digikam/digikamview.cpp  
 M  +1 -0      digikam/digikamview.h  
 M  +3 -3      digikam/monthwidget.cpp  
 M  +222 -99   digikam/tagfilterview.cpp  
 M  +3 -1      digikam/tagfilterview.h  
 M  +125 -45   digikam/tagfolderview.cpp  
 M  +4 -2      digikam/tagfolderview.h  
 M  +147 -113  kioslave/digikamalbums.cpp  
 M  +17 -23    kioslave/digikamdates.cpp  
 M  +129 -95   kioslave/digikamtags.cpp  
 M  +28 -29    libs/imageproperties/imagedescedittab.cpp  
 M  +1 -0      libs/imageproperties/imagedescedittab.h  
 M  +5 -0      libs/imageproperties/imagepropertiessidebardb.cpp  
 M  +1 -0      libs/imageproperties/imagepropertiessidebardb.h  
 M  +107 -10   libs/imageproperties/talbumlistview.cpp  
 M  +20 -5     libs/imageproperties/talbumlistview.h  
 M  +2 -2      project/project.kdevelop  
 M  +6 -0      utilities/imageeditor/editor/imagewindow.cpp  
 M  +1 -0      utilities/imageeditor/editor/imagewindow.h  
 M  +7 -0      utilities/lighttable/lighttablewindow.cpp  
 M  +1 -0      utilities/lighttable/lighttablewindow.h  
 M  +25 -17    utilities/setup/setupgeneral.cpp  


WebSVN link: http://websvn.kde.org/?view=rev&revision=754570
Comment 89 jos poortvliet 2007-12-30 14:56:21 UTC
Nice work, guys ;-)
Comment 90 caulier.gilles 2007-12-30 15:21:56 UTC
And I have created a new blog entry about this subject on digiKam web project site:

http://www.digikam.org/?q=node/283

Gilles
Comment 91 caulier.gilles 2008-01-06 15:48:12 UTC
SVN commit 757930 by cgilles:

complete backport of commit #754570 from KDE3 to KDE4.
B.K.O #96388 is now fully implemented to digiKam for KDE4
CCBUGS: 96388


 M  +11 -0     digikam/digikamapp.cpp  
 M  +10 -0     digikam/digikamview.cpp  
 M  +1 -0      digikam/digikamview.h  
 M  +12 -0     project/project.kdevelop  
 M  +6 -0      utilities/imageeditor/editor/imagewindow.cpp  
 M  +1 -0      utilities/imageeditor/editor/imagewindow.h  
 M  +7 -0      utilities/lighttable/lighttablewindow.cpp  
 M  +1 -0      utilities/lighttable/lighttablewindow.h  


WebSVN link: http://websvn.kde.org/?view=rev&revision=757930