Bug 165348 - Problems when trying to import KDE3 data
Summary: Problems when trying to import KDE3 data
Status: RESOLVED FIXED
Alias: None
Product: digikam
Classification: Applications
Component: Database-Migration (show other bugs)
Version: 0.10.0
Platform: Compiled Sources Linux
: NOR crash
Target Milestone: ---
Assignee: Digikam Developers
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2008-06-29 18:14 UTC by Thomas McGuire
Modified: 2017-07-26 06:57 UTC (History)
1 user (show)

See Also:
Latest Commit:
Version Fixed In: 0.10.0


Attachments
My compressed digikam4.db file (597.41 KB, application/octet-stream)
2008-06-30 18:38 UTC, Thomas McGuire
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Thomas McGuire 2008-06-29 18:14:10 UTC
Version:           0.10.0-svn (using Devel)
Installed from:    Compiled sources
OS:                Linux

Importing my old Digikam data does not seems to work, or has too many problems to work correctly.

I didn't want to loose all my tags, so I copied over my old $HOME/share/config/digikamrc and $HOME/share/apps/digikam/ over to my KDE4 home directory.
My picture directory, which also contained the database files, remained the same.

1.
When I then started Digikam, A progress dialog was shown, which contained a listview which showed the folder which was currently processed.
Problem: There was no label or caption saying what digikam actually does at that moment.

2.
During processing, I got a lot of warnings like:

Cannot find Exif key 'Exif.Pentax.LensType' into image using Exiv2  (Error #6: Invalid key `Exif.Pentax.LensType')
digikam(23164): "/media/daten/Bilder/Fotos/Zuhause/2005-07-28/imag0031.jpg"  : JPEG file identified
DateTime => Exif.Photo.DateTimeOriginal =>  QDateTime("Thu Jul 28 06:40:17 2005")

Orientation => Exif.Image.Orientation =>  1

Not sure if these warnings matter

3.
At some point, the dialog froze completely.
The last lines of output were:

digikam(23164): Returning true from updating to 5
digikam(23164): Success updating to v5

I attached GDB to the process and it showed the following backtrace for the frozen Digikam:

#0  0xffffe403 in __kernel_vsyscall ()
#1  0xb7d07566 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
#2  0xb7da8230 in QMutexPrivate::wait () from /media/kdedev/trunk/qt-copy/lib/libQtCore.so.4
#3  0xb7da2892 in QMutex::lock () from /media/kdedev/trunk/qt-copy/lib/libQtCore.so.4
#4  0xb70eb004 in DatabaseAccess (this=0xbfe21feb) at /media/kdedev/trunk/src/KDE/graphics/digikam/libs/database/databaseaccess.cpp:73
#5  0xb70fd81f in Digikam::ImageInfoCache::slotImageChanged (this=0x8432a00, changeset=@0xb1739378) at /media/kdedev/trunk/src/KDE/graphics/digikam/libs/database/imageinfocache.cpp:92
#6  0xb70fe27a in Digikam::ImageInfoCache::qt_metacall (this=0x8432a00, _c=QMetaObject::InvokeMetaMethod, _id=0, _a=0xb1716768) at /media/kdedev/trunk/build/KDE/graphics/digikam/digikam/imageinfocache.moc:68
#7  0xb7eac5a6 in QMetaCallEvent::placeMetaCall () from /media/kdedev/trunk/qt-copy/lib/libQtCore.so.4
#8  0xb7eb094c in QObject::event () from /media/kdedev/trunk/qt-copy/lib/libQtCore.so.4
#9  0xb5860e0d in QApplicationPrivate::notify_helper () from /media/kdedev/trunk/qt-copy/lib/libQtGui.so.4
#10 0xb5861126 in QApplication::notify () from /media/kdedev/trunk/qt-copy/lib/libQtGui.so.4
#11 0xb7988f6b in KApplication::notify (this=0xbfe229cc, receiver=0x8432a00, event=0xb1730c40) at /media/kdedev/trunk/src/KDE/kdelibs/kdeui/kernel/kapplication.cpp:311
#12 0xb7e9d88a in QCoreApplication::notifyInternal () from /media/kdedev/trunk/qt-copy/lib/libQtCore.so.4
#13 0xb7ea13c9 in QCoreApplication::sendEvent () from /media/kdedev/trunk/qt-copy/lib/libQtCore.so.4
#14 0xb7e9dda2 in QCoreApplicationPrivate::sendPostedEvents () from /media/kdedev/trunk/qt-copy/lib/libQtCore.so.4
#15 0xb7e9df53 in QCoreApplication::sendPostedEvents () from /media/kdedev/trunk/qt-copy/lib/libQtCore.so.4
#16 0xb7eceb1e in QCoreApplication::sendPostedEvents () from /media/kdedev/trunk/qt-copy/lib/libQtCore.so.4
#17 0xb7ecdd2f in postEventSourceDispatch () from /media/kdedev/trunk/qt-copy/lib/libQtCore.so.4
#18 0xb50725d6 in g_main_context_dispatch () from /usr/lib/libglib-2.0.so.0
#19 0xb5075972 in ?? () from /usr/lib/libglib-2.0.so.0
#20 0x08384c90 in ?? ()
#21 0x00000000 in ?? ()

I killed Digikam afterwards.

4.
I restarted digikam. It showed the progress dialog again, and then eventually crashed with the following backtrace:

Application: digiKam (digikam), signal SIGSEGV
[Thread debugging using libthread_db enabled]
[New Thread 0xb49e66d0 (LWP 23376)]
[New Thread 0xb298eb90 (LWP 23390)]
[New Thread 0xb218db90 (LWP 23389)]
[New Thread 0xb3401b90 (LWP 23380)]
[KCrash handler]
#6  0x081ebc1c in Digikam::Album::insertChild (this=0x1, child=0xc135f80)
    at /media/kdedev/trunk/src/KDE/graphics/digikam/digikam/album.cpp:101
#7  0x081ebc9f in Digikam::Album::setParent (this=0xc135f80, parent=0x1)
    at /media/kdedev/trunk/src/KDE/graphics/digikam/digikam/album.cpp:67
#8  0x0821fc3e in Digikam::AlbumManager::scanTAlbums (this=0x83ef878)
    at /media/kdedev/trunk/src/KDE/graphics/digikam/digikam/albummanager.cpp:728
#9  0x08222324 in Digikam::AlbumManager::refresh (this=0x83ef878)
    at /media/kdedev/trunk/src/KDE/graphics/digikam/digikam/albummanager.cpp:497
#10 0x08222a07 in Digikam::AlbumManager::startScan (this=0x83ef878)
    at /media/kdedev/trunk/src/KDE/graphics/digikam/digikam/albummanager.cpp:435
#11 0x0825a921 in DigikamApp (this=0x84ed488)
    at /media/kdedev/trunk/src/KDE/graphics/digikam/digikam/digikamapp.cpp:200
#12 0x0828fce2 in main (argc=1, argv=0xbfe2dde4)
    at /media/kdedev/trunk/src/KDE/graphics/digikam/digikam/main.cpp:318
#0  0xffffe410 in __kernel_vsyscall ()



In short: I was unable to import my old tags, and thus I cannot upgrade to the KDE4 version at the moment.
Comment 1 Arnd Baecker 2008-06-29 18:58:27 UTC
On Sun, 29 Jun 2008, Thomas McGuire wrote:

> In short: I was unable to import my old tags,
> and thus I cannot upgrade to the KDE4 version at the moment.


I think it is important to emphasize  that digikam 0.10
is not yet meant to be used in production.

However, testing all the different aspects is extremely important
and such detailed reports are surely very much appreciated!

Best, Arnd
Comment 2 Marcel Wiesweg 2008-06-29 19:53:33 UTC
You'll see that importing my old digikam database works well and is tested. It is quite fast and you'll see some messages with the very first lines on the console.

The warnings under point 2 are irrelevant. They come from somewhere in libkexiv2 or libexiv2. I have a lot of them too.

3 and 4 are relevant and important crashes.
3: Can you reproduce it? Do you have information where the other threads are hanging when the process locks?

4: I need to study the code. In the case that I cannot guess the problem, I would need your digikam4.db file. But more on that later.
Comment 3 caulier.gilles 2008-06-29 23:48:24 UTC
>Cannot find Exif key 'Exif.Pentax.LensType' into image using Exiv2  (Error #6: >Invalid key `Exif.Pentax.LensType') 

Sound like you use an old Exiv2 version on your computer. Look like Pentax Makernote include this tag: 

http://www.exiv2.org/tags-pentax.html

Gilles Caulier
Comment 4 Marcel Wiesweg 2008-06-30 17:38:31 UTC
SVN commit 826370 by mwiesweg:

This should avoid the crash of point 4) of the bugreport.
Still unclear though why the parent of the tag is not found.
I'd need to analyze the digikam4.db file to find that out.
Note: The tag table is not touched during the 0.9 -> 0.10 upgrade process.

CCBUG: 165348


 M  +2 -2      albummanager.cpp  


WebSVN link: http://websvn.kde.org/?view=rev&revision=826370
Comment 5 Thomas McGuire 2008-06-30 18:36:49 UTC
>I think it is important to emphasize  that digikam 0.10 
>is not yet meant to be used in production. 
Yep, I'm aware of that, I backed up my database and my pictures before testing.

> 3: Can you reproduce it? Do you have information where the other threads are hanging when the process locks? 

Yes, I can reproduce it every time when I try to copy my old digikam config over to my KDE4 home and restore the old databases.
There was another thread (sorry, didn't see that before), here are the backtraces for the two threads:

#0  0xffffe410 in __kernel_vsyscall ()
#1  0xb7d86566 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
#2  0xb7e28f34 in QWaitConditionPrivate::wait () from /media/kdedev/trunk/qt-copy/lib/libQtCore.so.4
#3  0xb7e28a33 in QWaitCondition::wait () from /media/kdedev/trunk/qt-copy/lib/libQtCore.so.4
#4  0xb7e23d56 in QSemaphore::acquire () from /media/kdedev/trunk/qt-copy/lib/libQtCore.so.4
#5  0xb7f31310 in blocking_activate () from /media/kdedev/trunk/qt-copy/lib/libQtCore.so.4
#6  0xb7f31561 in QMetaObject::activate () from /media/kdedev/trunk/qt-copy/lib/libQtCore.so.4
#7  0xb7f31ab5 in QMetaObject::activate () from /media/kdedev/trunk/qt-copy/lib/libQtCore.so.4
#8  0xb7165609 in Digikam::CollectionManager::triggerUpdateVolumesList (this=0x83fd9f0) at /media/kdedev/trunk/build/KDE/graphics/digikam/digikam/collectionmanager.moc:95
#9  0xb7165665 in Digikam::CollectionManagerPrivate::listVolumes (this=0x83eff38) at /media/kdedev/trunk/src/KDE/graphics/digikam/libs/database/collectionmanager.cpp:228
#10 0xb7165f81 in Digikam::CollectionManager::updateLocations (this=0x83fd9f0) at /media/kdedev/trunk/src/KDE/graphics/digikam/libs/database/collectionmanager.cpp:899
#11 0xb7167009 in Digikam::CollectionManager::refresh (this=0x83fd9f0) at /media/kdedev/trunk/src/KDE/graphics/digikam/libs/database/collectionmanager.cpp:533
#12 0xb7169f6d in Digikam::DatabaseAccess::checkReadyForUse (observer=0x84315d0) at /media/kdedev/trunk/src/KDE/graphics/digikam/libs/database/databaseaccess.cpp:207
#13 0x0829e0bf in Digikam::ScanController::run (this=0x84315c8) at /media/kdedev/trunk/src/KDE/graphics/digikam/digikam/scancontroller.cpp:304
#14 0xb7e284e9 in QThreadPrivate::start () from /media/kdedev/trunk/qt-copy/lib/libQtCore.so.4
#15 0xb7d82192 in start_thread () from /lib/libpthread.so.0
#16 0xb523502e in clone () from /lib/libc.so.6


#0  0xffffe403 in __kernel_vsyscall ()
#1  0xb7d86566 in pthread_cond_wait@@GLIBC_2.3.2 () from /lib/libpthread.so.0
#2  0xb7e27230 in QMutexPrivate::wait () from /media/kdedev/trunk/qt-copy/lib/libQtCore.so.4
#3  0xb7e21892 in QMutex::lock () from /media/kdedev/trunk/qt-copy/lib/libQtCore.so.4
#4  0xb716a004 in DatabaseAccess (this=0xbfc7a8db) at /media/kdedev/trunk/src/KDE/graphics/digikam/libs/database/databaseaccess.cpp:73
#5  0xb717c81f in Digikam::ImageInfoCache::slotImageChanged (this=0x8432948, changeset=@0x854fb70) at /media/kdedev/trunk/src/KDE/graphics/digikam/libs/database/imageinfocache.cpp:92
#6  0xb717d27a in Digikam::ImageInfoCache::qt_metacall (this=0x8432948, _c=QMetaObject::InvokeMetaMethod, _id=0, _a=0x843bc88) at /media/kdedev/trunk/build/KDE/graphics/digikam/digikam/imageinfocache.moc:68
#7  0xb7f2b5a6 in QMetaCallEvent::placeMetaCall () from /media/kdedev/trunk/qt-copy/lib/libQtCore.so.4
#8  0xb7f2f94c in QObject::event () from /media/kdedev/trunk/qt-copy/lib/libQtCore.so.4
#9  0xb58dfe0d in QApplicationPrivate::notify_helper () from /media/kdedev/trunk/qt-copy/lib/libQtGui.so.4
#10 0xb58e0126 in QApplication::notify () from /media/kdedev/trunk/qt-copy/lib/libQtGui.so.4
#11 0xb7a07f6b in KApplication::notify (this=0xbfc7b2bc, receiver=0x8432948, event=0x8445700) at /media/kdedev/trunk/src/KDE/kdelibs/kdeui/kernel/kapplication.cpp:311
#12 0xb7f1c88a in QCoreApplication::notifyInternal () from /media/kdedev/trunk/qt-copy/lib/libQtCore.so.4
#13 0xb7f203c9 in QCoreApplication::sendEvent () from /media/kdedev/trunk/qt-copy/lib/libQtCore.so.4
#14 0xb7f1cda2 in QCoreApplicationPrivate::sendPostedEvents () from /media/kdedev/trunk/qt-copy/lib/libQtCore.so.4
#15 0xb7f1cf53 in QCoreApplication::sendPostedEvents () from /media/kdedev/trunk/qt-copy/lib/libQtCore.so.4
#16 0xb7f4db1e in QCoreApplication::sendPostedEvents () from /media/kdedev/trunk/qt-copy/lib/libQtCore.so.4
#17 0xb7f4cd2f in postEventSourceDispatch () from /media/kdedev/trunk/qt-copy/lib/libQtCore.so.4
#18 0xb50f15d6 in g_main_context_dispatch () from /usr/lib/libglib-2.0.so.0
#19 0xb50f4972 in ?? () from /usr/lib/libglib-2.0.so.0
#20 0x08384c90 in ?? ()
#21 0x00000000 in ?? ()
Comment 6 Thomas McGuire 2008-06-30 18:38:56 UTC
Created attachment 25738 [details]
My compressed digikam4.db file
Comment 7 Thomas McGuire 2008-06-30 20:15:10 UTC
> SVN commit 826370 by mwiesweg: 
Thanks, that fixed the crash on startup, and the database upgrade seemed to have worked well, all my tags are still there.
Comment 8 caulier.gilles 2008-06-30 22:10:14 UTC
Thomas,

Fine for you to close this report ?

Gilles Caulier
Comment 9 Arnd Baecker 2008-07-01 10:28:01 UTC
Gilles, Marcel: I tried to start digikam 0.10 with my test images
I used with 0.9.x, but it hangs at 64% and with the last messages:
  Returning true from updating to 5
  Success updating to v5.
Looking at a dump of the converted database itself, it seems to be ok,
So maybe it is a different issue ...

I send you the corresponding digikam3.db in private mail.

Best, Arnd
Comment 10 caulier.gilles 2008-07-01 12:21:30 UTC
Arnd,

This problem can be duing to sqlite version embeded in Qt4 as plugin (yes, the famous problem with sqlite <3.5.9 can be seen here)

This is not easy to check. Qt4.3.3 include the wrong sqlite3 version : > 3.5.6 and < 3.5.9. But i think than packager can set the sqlite3 version that they want when Qt4 package is set. Download the Qt4 tarball source code compiled with your distro and check.

The issue to this problem is to make a QT4::SQLite3 plugin dedicated to digiKam and set in core, like it do with KDE3 version. Like this we have control, else it will be the hell.

What do you think about ?

Gilles Caulier
Comment 11 Arnd Baecker 2008-07-01 14:38:53 UTC
I am on a kubuntu hardy installation and at a quick glance
this uses libsqlite3 3.4.2  on which the libqt4-sql depends.
So I think this is not the reason for the problem.
Comment 12 Thomas McGuire 2008-07-01 19:52:31 UTC
> Fine for you to close this report ?
The deadlock in 3) hasn't been fixed yet, right? Or is that caused by the sqlite version mismatched you talked about here?
Comment 13 Marcel Wiesweg 2008-07-01 21:59:43 UTC
SVN commit 826917 by mwiesweg:

Avoid deadlock:

Do not hold a mutex when emitting a blocking queued signal.
In this case, ensure that DatabaseAccess is released, at least temporarily.
Otherwise there will be a deadlock when emitting the synchronous signal to
retrieve Solid information, and the main thread is currently handling a signal
that wants to acquire DatabaseAccess.
Adds a class that unlocks the recursive mutex on creation, and reqacuires on destruction.
This should be used with care as it undermines assumptions that structures
cannot change while the lock is held. But you should not hold a mutex when calling
methods with side effect (we do this here in at least one situtation), so
I happily break assumptions (and there are no such assumptions currently).

Thanks to Thomas McGuire for providing backtraces.

CCBUG:165348


 M  +25 -16    collectionmanager.cpp  
 M  +69 -3     databaseaccess.cpp  
 M  +23 -0     databaseaccess.h  


WebSVN link: http://websvn.kde.org/?view=rev&revision=826917
Comment 14 caulier.gilles 2008-07-01 22:14:05 UTC
Thomas, Arnd,

Please test current implementation patched by Marcel with commit #826917 and give us a feedback. Thanks in advance

Gilles Caulier
Comment 15 Arnd Baecker 2008-07-01 22:39:14 UTC
Great, it also seems to work in my case. 

However,afterwards I can't see any images.
There are some error messages in the konsole like
QObject::killTimer: timers cannot be stopped from another thread
QObject::startTimer: timers cannot be started from another thread
QPainter::begin: Widget painting can only begin as a result of a paintEvent
QPainter::end: Painter not active, aborted

But I am not sure whether these are related 
(I am currently testing via ssh over a pretty slow connection, so
I will have to wait until next week ...). As Gilles cannot reproduce this,
I suggest to ignore it for the moment ... ;-)
Comment 16 Thomas McGuire 2008-07-02 11:19:40 UTC
Thanks, commit 826917 indeed fixed the deadlock.

However, like Arnd in comment 15, I can't see any pictures after the first digikam start.
Everythings seems to work after a restart, though.
Comment 17 caulier.gilles 2008-07-28 14:01:08 UTC
Marcel,

I would release 0.10.0-beta2. It still something to fix here before ?

Gilles
Comment 18 Marcel Wiesweg 2008-07-28 17:29:32 UTC
The "can't see any pictures after restart" problem can be an installation / KDE problem, I dont know. The two main problems here have been fixed.
Comment 19 caulier.gilles 2008-07-30 19:09:54 UTC
Thanks Marcel,

So I close this file now.

For others subjects reported in this thread, please report in another bugzilla entry or look if anybody as already reported the problem.

Thanks in advance

Gilles Caulier
Comment 20 Thomas McGuire 2008-12-06 14:25:12 UTC
> The "can't see any pictures after restart" problem can be an installation / KDE problem, I dont know. The two main problems here have been fixed.

This is still reproduceable with 0.10.0 beta 7. After importing the KDE3 data, one has to restart digikam to see the pictures.