Bug 277163 - digikam valgrind issues during face recognition testing
Summary: digikam valgrind issues during face recognition testing
Status: RESOLVED FIXED
Alias: None
Product: digikam
Classification: Unclassified
Component: Faces-Recognition (show other bugs)
Version: 2.0.0
Platform: Ubuntu Packages Linux
: NOR normal (vote)
Target Milestone: ---
Assignee: Digikam Developers
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-07-05 22:32 UTC by Richard Mortimer
Modified: 2019-12-23 07:16 UTC (History)
1 user (show)

See Also:
Latest Commit:
Version Fixed In: 7.0.0


Attachments
output from run 1 of valgrind/digikam (24.43 KB, application/octet-stream)
2011-07-05 22:32 UTC, Richard Mortimer
Details
output from 2nd run of valgrind/digikam (22.99 KB, text/plain)
2011-07-05 22:33 UTC, Richard Mortimer
Details
repeat valgrind run after upgrading all libraries to natty versions (102.92 KB, text/plain)
2011-07-06 17:22 UTC, Richard Mortimer
Details
valgrind output when face recognition crashed with SIGSEGV (29.03 KB, text/plain)
2011-07-12 22:49 UTC, Richard Mortimer
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Richard Mortimer 2011-07-05 22:32:21 UTC
Created attachment 61632 [details]
output from run 1 of valgrind/digikam

Version:           2.0.0 (using KDE 4.6.4) 
OS:                Linux

Whilst attempting to reproduce a crash during face recognition/detection I ran Digikam 2.0.0.rc under valgrind on Ubuntu 11.04 using the philip5 ppa binaries.

Valgrind reported a number of issues that can be seen in the attached valgrind output. The issues seem to be spread over a number of libraries/areas of code so it maybe that a number of related bugs need raising to track resolution of each.

Unfortunately I did not manage to recreate but 277099 (or any of the similar ones) at this time but hopefully the valgrind output will help to fix some issues.

Reproducible: Always

Steps to Reproduce:
Start digikam using

valgrind --track-origins=yes digikam

Initiate face detection/recognition on a number of albums and whilst this is running go to the People/Unknown list of photos and start assigning tags to the faces identified.

I ran valgrind twice with similar output each time.


Expected Results:  
Valgrind should not have found any memory allocation/usage errors.
Comment 1 Richard Mortimer 2011-07-05 22:33:27 UTC
Created attachment 61633 [details]
output from 2nd run of valgrind/digikam
Comment 2 caulier.gilles 2011-07-06 10:28:50 UTC
Which version of opencv/libface you use. Look into Help/Components Info for details

Gilles Caulier
Comment 3 caulier.gilles 2011-07-06 10:40:43 UTC
It sound like memory trace from Qt, not digiKam & co, especially QImage::loadFromData()

Gilles Caulier
Comment 4 Richard Mortimer 2011-07-06 10:57:04 UTC
LibKface is 2.0.0   (2.0.0+maverick-ppa3kde46)
LibOpenCV is 2.2.0  (2.2.0-natty-ppa4)

Other versions below.

Actually I notice that there is a libkface 2.0.0-natty-ppa5 available. I guess things didn't get fully fixed up when I moved from Ubuntu 10.10 to 11.04. I'm not sure how much of an issue that will be given that it should be the same version but compiled for a different Ubuntu release. I'll switch versions later today when I get a few minutes.


digiKam version 2.0.0-rc
Exiv2 can write to Jp2: Yes
Exiv2 can write to Jpeg: Yes
Exiv2 can write to Pgf: Yes
Exiv2 can write to Png: Yes
Exiv2 can write to Tiff: Yes
Exiv2 supports XMP metadata: Yes
LibCImg: 130
LibClapack: internal library
LibExiv2: 0.21.1
LibJPEG: 62
LibJasper: 1.900.1
LibKDE: 4.6.4 (4.6.4)
LibKExiv2: 2.0.0
LibKMap: 2.0.0
LibKdcraw: 2.0.0
LibLCMS: 118
LibPGF: 6.11.24 - internal library
LibPNG: 1.2.44
LibQt: 4.7.2
LibRaw: 0.13.4
LibTIFF: LIBTIFF, Version 3.9.4 Copyright (c) 1988-1996 Sam Leffler Copyright (c) 1991-1996 Silicon Graphics, Inc.
Marble Widget: 0.11.2 (Stable Release)
Parallelised demosaicing: Yes
Database backend: QMYSQL
Database internal server: No
LibGphoto2: 2.4.11
LibKface: 2.0.0
LibKipi: 1.2.0
LibOpenCV: 2.2.0
Libface: 0.2
Comment 5 Richard Mortimer 2011-07-06 17:22:58 UTC
Created attachment 61648 [details]
repeat valgrind run after upgrading all libraries to natty versions

I searched for all of the libraries that were still maverick (10.10) versions and replaced these with the natty (11.04) versions. All had the same underlying version number but were just the natty builds of the same library.

I reran the valgrind session and set it off detecting and recognising faces. Whilst this was ongoing I tagged a few faces in People/Unknown.

I then aborted the recognition operation and once it had aborted I closed down digikam. At this stage digikam died with a SIGSEGV. Unfortunately a proper stack trace was not produced presumably because valgrind and the bug helpers do not get along (there is a valgrind warning/bug notice in the attached output to that effect).

There are a large number of "Invalid read of size 4" warnings towards the end of the trace. I did not notice if any of these were produced during face recognition/tagging but know that some of them were produced during closedown just prior to the SIGSEGV crash.
Comment 6 Marcel Wiesweg 2011-07-11 16:03:18 UTC
I see lots of issues accessing the loading cache - sure, it has been deleted at application shutdown:

==15989==  Address 0x1074afe8 is 8 bytes inside a block of size 100 free'd
==15989==    at 0x4025907: operator delete(void*) (vg_replace_malloc.c:387)
==15989==    by 0x50991BB: Digikam::LoadingCache::~LoadingCache() (loadingcache.cpp:183)
==15989==    by 0x50994C1: Digikam::LoadingCache::~LoadingCache() (loadingcache.cpp:185)
==15989==    by 0x50962C5: Digikam::LoadingCache::cleanUp() (loadingcache.cpp:162)
==15989==    by 0x509C176: Digikam::LoadingCacheInterface::cleanUp() (loadingcacheinterface.cpp:40)
==15989==    by 0x81914B8: Digikam::DigikamApp::~DigikamApp() (digikamapp.cpp:325)

Did your tests run include shutdown of the application?
Comment 7 Richard Mortimer 2011-07-11 16:46:02 UTC
The valgrind tests do include appliation shutdown yes. In the last run I didn't take a note in the output where I started the shutdown.

The shutdown was definitely after

digikam(15989)/digikam (core) Digikam::CollectionScanner::scanAlbum: Folder does not exist or is not readable:  "/media/shirephoto/digikam/PictureProject/0116/DSC_1958.JPG" 

But I am not sure if that is the last bit of output before I cancelled face recognition/detection.

If I get some time tonight I will re-run and see if I can identify any specific sequencing so that issues during shutdown can be separated from face detection issues.
Comment 8 Richard Mortimer 2011-07-12 22:49:07 UTC
Created attachment 61826 [details]
valgrind output when face recognition crashed with SIGSEGV

I finally got a valgrind session output where digikam crashed with during face recognition. The output that occurred around the crash time looks very interesting.

I see two issues one where timers have been unregistered improperly and the other where unitialise memory is used.

Application asked to unregister timer 0x1e000013 which is not registered in this thread. Fix application.
Application asked to unregister timer 0x28000069 which is not registered in this thread. Fix application.
==1785== Thread 1:
==1785== Use of uninitialised value of size 4
==1785==    at 0x6D8AD00: QVariant::QVariant(QVariant const&) (qvariant.h:360)
==1785==    by 0x5406A3B: Digikam::ImageModel::data(QModelIndex const&, int) const (imagemodel.cpp:1016)
==1785==    by 0x5418B70: Digikam::ImageThumbnailModel::data(QModelIndex const&, int) const (imagethumbnailmodel.cpp:279)
==1785==    by 0x540AD9C: Digikam::ImageFilterModel::subSortLessThan(QModelIndex const&, QModelIndex const&) const (qabstractitemmodel.h:398)
==1785==    by 0x5CD5AB9: KCategorizedSortFilterProxyModel::lessThan(QModelIndex const&, QModelIndex const&) const (kcategorizedsortfilterproxymodel.cpp:121)
==1785==    by 0x6609960: int* QAlgorithmsPrivate::qLowerBoundHelper<int*, int, QSortFilterProxyModelLessThan>(int*, int*, int const&, QSortFilterProxyModelLessThan) (qsortfilterproxymodel.cpp:80)
==1785==    by 0x66099E1: int* qLowerBound<int*, int, QSortFilterProxyModelLessThan>(int*, int*, int const&, QSortFilterProxyModelLessThan) (qalgorithms.h:252)
==1785==    by 0x6609D48: void QAlgorithmsPrivate::qMerge<int*, int const, QSortFilterProxyModelLessThan>(int*, int*, int*, int const&, QSortFilterProxyModelLessThan) (qalgorithms.h:435)
==1785==    by 0x6609CE1: void QAlgorithmsPrivate::qMerge<int*, int const, QSortFilterProxyModelLessThan>(int*, int*, int*, int const&, QSortFilterProxyModelLessThan) (qalgorithms.h:446)
==1785==    by 0x6609CE1: void QAlgorithmsPrivate::qMerge<int*, int const, QSortFilterProxyModelLessThan>(int*, int*, int*, int const&, QSortFilterProxyModelLessThan) (qalgorithms.h:446)
==1785==    by 0x6609C6E: void QAlgorithmsPrivate::qMerge<int*, int const, QSortFilterProxyModelLessThan>(int*, int*, int*, int const&, QSortFilterProxyModelLessThan) (qalgorithms.h:445)
==1785==    by 0x6609C6E: void QAlgorithmsPrivate::qMerge<int*, int const, QSortFilterProxyModelLessThan>(int*, int*, int*, int const&, QSortFilterProxyModelLessThan) (qalgorithms.h:445)
==1785==  Uninitialised value was created by a heap allocation
==1785==    at 0x4026864: malloc (vg_replace_malloc.c:236)
==1785==    by 0x6C6FB8C: qMalloc(unsigned int) (qmalloc.cpp:55)
==1785==    by 0x6C95840: QListData::detach(int) (qlist.cpp:185)
==1785==    by 0x534B0E0: QList<QVariant>::detach_helper(int) (qlist.h:694)
==1785==    by 0x534B1F9: QList<QVariant>::detach_helper() (qlist.h:710)
==1785==    by 0x5406A22: Digikam::ImageModel::data(QModelIndex const&, int) const (qlist.h:126)
==1785==    by 0x5418B70: Digikam::ImageThumbnailModel::data(QModelIndex const&, int) const (imagethumbnailmodel.cpp:279)
==1785==    by 0x540AD70: Digikam::ImageFilterModel::subSortLessThan(QModelIndex const&, QModelIndex const&) const (qabstractitemmodel.h:398)
==1785==    by 0x5CD5AB9: KCategorizedSortFilterProxyModel::lessThan(QModelIndex const&, QModelIndex const&) const (kcategorizedsortfilterproxymodel.cpp:121)
==1785==    by 0x6609E18: void QAlgorithmsPrivate::qMerge<int*, int const, QSortFilterProxyModelLessThan>(int*, int*, int*, int const&, QSortFilterProxyModelLessThan) (qsortfilterproxymodel.cpp:80)
==1785==    by 0x6609F61: void QAlgorithmsPrivate::qStableSortHelper<int*, int, QSortFilterProxyModelLessThan>(int*, int*, int const&, QSortFilterProxyModelLessThan) (qalgorithms.h:459)
==1785==    by 0x6609F06: void QAlgorithmsPrivate::qStableSortHelper<int*, int, QSortFilterProxyModelLessThan>(int*, int*, int const&, QSortFilterProxyModelLessThan) (qalgorithms.h:458)
==1785== 
==1785== Invalid read of size 4
==1785==    at 0x6D8AD00: QVariant::QVariant(QVariant const&) (qvariant.h:360)
==1785==    by 0x5406A3B: Digikam::ImageModel::data(QModelIndex const&, int) const (imagemodel.cpp:1016)
==1785==    by 0x5418B70: Digikam::ImageThumbnailModel::data(QModelIndex const&, int) const (imagethumbnailmodel.cpp:279)
==1785==    by 0x540AD9C: Digikam::ImageFilterModel::subSortLessThan(QModelIndex const&, QModelIndex const&) const (qabstractitemmodel.h:398)
==1785==    by 0x5CD5AB9: KCategorizedSortFilterProxyModel::lessThan(QModelIndex const&, QModelIndex const&) const (kcategorizedsortfilterproxymodel.cpp:121)
==1785==    by 0x6609960: int* QAlgorithmsPrivate::qLowerBoundHelper<int*, int, QSortFilterProxyModelLessThan>(int*, int*, int const&, QSortFilterProxyModelLessThan) (qsortfilterproxymodel.cpp:80)
==1785==    by 0x66099E1: int* qLowerBound<int*, int, QSortFilterProxyModelLessThan>(int*, int*, int const&, QSortFilterProxyModelLessThan) (qalgorithms.h:252)
==1785==    by 0x6609D48: void QAlgorithmsPrivate::qMerge<int*, int const, QSortFilterProxyModelLessThan>(int*, int*, int*, int const&, QSortFilterProxyModelLessThan) (qalgorithms.h:435)
==1785==    by 0x6609CE1: void QAlgorithmsPrivate::qMerge<int*, int const, QSortFilterProxyModelLessThan>(int*, int*, int*, int const&, QSortFilterProxyModelLessThan) (qalgorithms.h:446)
==1785==    by 0x6609CE1: void QAlgorithmsPrivate::qMerge<int*, int const, QSortFilterProxyModelLessThan>(int*, int*, int*, int const&, QSortFilterProxyModelLessThan) (qalgorithms.h:446)
==1785==    by 0x6609C6E: void QAlgorithmsPrivate::qMerge<int*, int const, QSortFilterProxyModelLessThan>(int*, int*, int*, int const&, QSortFilterProxyModelLessThan) (qalgorithms.h:445)
==1785==    by 0x6609C6E: void QAlgorithmsPrivate::qMerge<int*, int const, QSortFilterProxyModelLessThan>(int*, int*, int*, int const&, QSortFilterProxyModelLessThan) (qalgorithms.h:445)
==1785==  Address 0x0 is not stack'd, malloc'd or (recently) free'd
Comment 9 caulier.gilles 2011-12-14 13:53:02 UTC
Richard,

With digiKam 2.4, can you reproduce memory issues with valgrind ?

Gilles Caulier
Comment 10 Richard Mortimer 2011-12-16 16:58:21 UTC
I re-ran briefly with Valgrind yesterday. I didn't get it to crash (but I likely need to try harder because it can take 30 minutes to crash without valgrind slowing things down).

I noted some memory issues. I've nearly finished doing a quick triage of those and most are in libarary code and already have fixes. One is in kdelibs5 and didn't have a fix so I've knocked on up and it seems to fix that. I'll finish the triage off over the weekend and update the report with that.
Comment 11 caulier.gilles 2013-01-11 11:58:46 UTC
This entry still valid using last stable 2.9.0 release ?

Gilles Caulier
Comment 12 caulier.gilles 2013-08-22 10:19:59 UTC
As with 3.3.0, new face recognition have been written, i recommend to check entry #323888 and post in this file new comments about memory leak in Face Management feature.

Gilles Caulier

*** This bug has been marked as a duplicate of bug 323888 ***
Comment 13 caulier.gilles 2019-12-23 07:16:24 UTC
Problem is fixed with new 7.0.0-beta1 through this long story from this bug

https://bugs.kde.org/show_bug.cgi?id=399923

You can test digiKam 7.0.0-beta1 with bundle available here:

https://download.kde.org/unstable/digikam/

Don't hesitate to give us a fresh feedback about his entry.

Thanks in advance

Gilles Caulier