Bug 134594

Halim,

We are concient of the SQlite limitation and there are a lot of report about to use an universal database interface with digiKam.

I'm agree with you than sqlite is a limitation for digiKam, especially to use a remote database (using NFS for example). I would to start DB stuff later than 0.9.0 because it a big job.

Marcel, what do you think about ?

Gilles

In a completely different context, we talked about consolidating the database code spread in albumdb, scanlib, ioslave in one lib.
For this we will need to define database interfaces anyway, so it gives an opportunity to abstract the db interface.

After 0.9.0 of course.

Providing the interface is the one thing, coding and maintaining a db backend the other thing. But at least we could then say "here is the interface, code a plugin" ;-)

Qt4 has extended SQL bindings. Maybe wait for Digikam on Qt4 for that?
I think Amarok is considering using just Qt backend for future database.

There are too many entries in BKO related to digikam db problems. While digiKam is an adavanced app in a lot of aspects, the db stuff is really a fault.

INMO, more advanced db interface and backends are needed by digiKam. We are making a hard work to have the best metadata write and read tool, the best tags system, the best image editor, a color managed app, etc. but, if we have a poor db stuff, well, we have a great hole in the base of the app.

Paco Cruz.

Thanks for your viewpoint Paco. I'm totally agree with you.

Gilles

Seb Ruiz is blogging about Amarok's considerations to move:
http://www.sebruiz.net/162

Docs on Qt4's SQL support can be found here:
http://doc.trolltech.com/4.2/qtsql.html

To me it looks promising.

I'm agree with you Marcel. Qt4 is exactly what we need about to make a new DB interface.

This is want mean that we will delay this task later to have ported digiKam to Qt4/CMake, of course, later 0.9.0 relase.

Other point to talk is about the database structure : if we use new DB backends, we can change something in structure at the same time ?

There are other files in B.K.o witch require database changes, especially to perform search about photograph informations, like aperture for example, or GPS coordinates.

I think we can change database structure at the same time because in all cases, we will need an tool to migrate from sqlite to new DB backend. We just need to take a care about the new structure to be more flexible than the actual.

Question : Marcel, Paco, are you already some experiences about database?

Personaly, i have already managed database used PHP. I know all base of DB management, but i'm not a specialist (:=))).

Gilles

I'm working with databases at work, but in a web environment. With mysql, postgres and oracle from php and java. Some time ago, I wrote some apps for personal use which worked with mysql, using qt db plugins.

*** This bug has been confirmed by popular vote. ***

If anybody want to have a look to the actual qt 3.3 sql support, this is the link:

http://doc.trolltech.com/3.3/sql.html

Would it make sense to at least run this discussion by the developer of other 
software that digikam is likely to be cooperating and coexisting with?

I'm firstly thinking of mythTV / freevo. I assume (haven't done a poll - might 
be worth while) that most computer users prefer to manage/sort their pictures 
on their computer, but to share them with friends and family on the web and 
in the living-room TV. I don't know much about Myth/Freevo developers, but it 
seems clear to me that end-users would greatly benefit from some kind of 
interaction! Same would go for web-photoalbum projects such as for instance 
coppermine.

Realistically, a common database format for myth-photoalbum, freevo, digikam, 
coppermine and others might be difficult to achieve, but at least a well 
documented API so that other primary viewing-apps can benefit from the work 
done through digikam.

Thoughts?

On Tuesday 26 September 2006 13:57, Gilles Caulier wrote:
> Other point to talk is about the database structure : if we use new DB
> backends, we can change something in structure at the same time ?
>
> There are other files in B.K.o witch require database changes, especially
> to perform search about photograph informations, like aperture for example,
> or GPS coordinates.
>
> I think we can change database structure at the same time because in all
> cases, we will need an tool to migrate from sqlite to new DB backend. We
> just need to take a care about the new structure to be more flexible than
> the actual.

I think common database is impossible to achieve - also various apps
have different goals, mythTV is even different medium. Good
documentation is always wonderful thing.

In reply to comment 11:

I have posted a few times to the MythTV dev mailing lists about my intensions to "integrate" Digikam and MythTV but it is not through a direct route. I am currently working on (and discussing on the Kipi ML) at making the Kipi Gallery plugin better. By making the Gallery "Export" as it is now, into more of a "Sync" it will become very easy to publish your photos on your website.

The second part of what I want to do will be to implement support in MythGallery for reading remote Galleries (e.g. those of your friends or your own personal one.)

So, if I use Digikam to manage my photos, then use the Gallery Kipi Plugin to publish them to my website, I can then use MythGallery to view my website's gallery!!!

Like I say it's a little indirect, but it would achieve the desired results! At least for me! (I want it on my website anyway and as it happens I host my website from my internal network so the latency would be minimal, but suitible caching in MythGallery for remote galleries should make it work nicely for properly remote galleries).

Other idea is to look at KexiDB, KDE4's database module. With QtSQL you'll still have a problem with SQLite dependency. KexiDB have maintained and builtin SQLite version since early 2004.

Non-GUI parts of KexiDB are already fully ported to KDE4.

You'll also get PostreSQL, MySQL support OOTB, and integration with querying tools.

Thanks Staniek for this information.

Is KexiDB have been used by ShowImg or Amarok ?

Gilles Caulier

This problem also occurs with SMB/CIFS-shares and is not mentioned in the docs. I do not understand why this is a problem with sqlite since digikam only sees a local mount point and everything else is abstracted over the VFS or am I mistaken?

It's a shame that I cannot use this excellent app in my environment, because I use a separate shared storage for all media stuff, which is (as already mentioned) quite a common usecase I would say.

Any progress on the Qt4-usage? Most people will have Qt3 and Qt4 installed anyway now that more and more apps are ported to be ready for KDE4, so it would be a good solution IMO to migrate digikam too and solve this long standing issue in one row...

>Any progress on the Qt4-usage? Most people will have Qt3 and Qt4 installed anyway >now that more and more apps are ported to be ready for KDE4, so it would be a >good solution IMO to migrate digikam too and solve this long standing issue in >one row... 

Yes... Googd news

Monday, Marcel i sent to me a mail to said than the DB interface is under developement using Qt3 and current implementation of digiKam from svn...

The code is re-factorized and polished. It's not complete and not yet on svn. The code still on Marcel computer's.

Marcel need more time to test indeep. DB stuff is a tiedous job...

He said than the new interface is ready to use another DB backend than sqlite. Of course nothing is yet tested in this way but the future is back.

Marcel is in hollidays actually and not available. I think than he will proposal a patch as soon to have feedback.

Gilles Caulier

This is _excellent_ news - thank you so much for your continued work on this. 
While this sudden burst of joy may be off-topic, I just wanted to thank you 
for not forgetting about this important request.

Thanks again, and happy coding!

On Wednesday 28 March 2007 14:54, Gilles Caulier wrote:
[bugs.kde.org quoted mail]

I really like the idea for mythtv and freevo support, I wish that it takes it a step further and develop an open standard for slide shows so that it's possible to define wipes, music and other features items for all similar applications like mythtv, freevo, kphotobook etc.

Hi there!
Are there any news about this topic?

Oli

With a year of no update on this bug, much has changed. KDE4 has a database interface using Qt SQL, which means in theory, any DB supported by Qt can be used. In practice however we'd need to review the SQL code we have and adapt it to the target system, at least fixing incompatibilities. That is to be done.

This is really a shame. I wish this would get sorted out more quickly. I love using digikam, but tbh with every upgrade I install it gets slower! (Currently 0.9.3 on Debian.)
My photos folder is on a local, reasonably fast HDD, I have a reasonably fast CPU, but the CPU is 100% pegged for ~20 seconds "Reading Database" when starting digikam or when making ANY change to the database.
It was quicker with 0.8.x, with the same database!

I can improve matters by cutting down the number of items in the folders (currently ~50,000 files), but that's not a good thing to have to work around an otherwise excellent application.

As a general comment, I don't really like the central database concept. I'd be happy to see digikam able to distribute metadata about a particular album / photo somewhere in each folder, for example. The centralised database makes it harder than it needs to be to reorganise things..

I hope this database performance issue can be resolved SOON. It's killing me...

On Sat, 17 May 2008, Matt R wrote:

> This is really a shame. I wish this would get sorted out more quickly.


May I point out that digiKam is an open source project, which
in this case in particular means that the development is done in
the free time of the contributors.
Moreover, because it is open source, you can contribute
code to the project!
See http://www.digikam.org/?q=download for information on
how to install current svn.
This would then give the basis for you to debug,
why digikam is slow in your case
(i.e. I don't experience any problems with a similar number of images).

Same for me. I have a SATA 300Gb harddrive dedicated to host all my photos (JPEG+RAW+PNG : around 70000 files). It's very fast to start digiKam (PIV-3.6Ghz-2Gb Ram)

Gilles Caulier

I had assumed this was a general slowness caused by the choice of database backend.. but if other people don't have the same problem with similar numbers of files it must be something in my (debian's) build, so I will look more closely at that.
Thanks.

(And FWIW, I have a Core2Duo 2.1 GHz with 4GB RAM. It's not my hardware making it slow!)

There is one issue about slow behviour, related to the sqlite version, see
http://bugs.kde.org/show_bug.cgi?id=160966, but your problem might be completely
different ... 

Thanks for the hint, I believe I owe you a pint! I downgraded to libsqlite3-0 version 3.5.7-2 and my performance problem has gone away.. :)
Looks like digikam was a victim. Thanks again!

Isn't this duplicate of Bug 127321 - or maybe other way? Here is longer discussion and more voting points.

Yes Mik, work in progress into #127321. I set this one as duplicate...

*** This bug has been marked as a duplicate of bug 127321 ***