| Summary: | digikam valgrind issues during face recognition testing | ||
|---|---|---|---|
| Product: | [Applications] digikam | Reporter: | Richard Mortimer <richm+kde> |
| Component: | Faces-Recognition | Assignee: | Digikam Developers <digikam-bugs-null> |
| Status: | RESOLVED FIXED | ||
| Severity: | normal | CC: | caulier.gilles |
| Priority: | NOR | ||
| Version: | 2.0.0 | ||
| Target Milestone: | --- | ||
| Platform: | Ubuntu | ||
| OS: | Linux | ||
| Latest Commit: | Version Fixed In: | 7.0.0 | |
| Sentry Crash Report: | |||
| Attachments: |
output from run 1 of valgrind/digikam
output from 2nd run of valgrind/digikam repeat valgrind run after upgrading all libraries to natty versions valgrind output when face recognition crashed with SIGSEGV |
||
Created attachment 61633 [details]
output from 2nd run of valgrind/digikam
Which version of opencv/libface you use. Look into Help/Components Info for details Gilles Caulier It sound like memory trace from Qt, not digiKam & co, especially QImage::loadFromData() Gilles Caulier LibKface is 2.0.0 (2.0.0+maverick-ppa3kde46) LibOpenCV is 2.2.0 (2.2.0-natty-ppa4) Other versions below. Actually I notice that there is a libkface 2.0.0-natty-ppa5 available. I guess things didn't get fully fixed up when I moved from Ubuntu 10.10 to 11.04. I'm not sure how much of an issue that will be given that it should be the same version but compiled for a different Ubuntu release. I'll switch versions later today when I get a few minutes. digiKam version 2.0.0-rc Exiv2 can write to Jp2: Yes Exiv2 can write to Jpeg: Yes Exiv2 can write to Pgf: Yes Exiv2 can write to Png: Yes Exiv2 can write to Tiff: Yes Exiv2 supports XMP metadata: Yes LibCImg: 130 LibClapack: internal library LibExiv2: 0.21.1 LibJPEG: 62 LibJasper: 1.900.1 LibKDE: 4.6.4 (4.6.4) LibKExiv2: 2.0.0 LibKMap: 2.0.0 LibKdcraw: 2.0.0 LibLCMS: 118 LibPGF: 6.11.24 - internal library LibPNG: 1.2.44 LibQt: 4.7.2 LibRaw: 0.13.4 LibTIFF: LIBTIFF, Version 3.9.4 Copyright (c) 1988-1996 Sam Leffler Copyright (c) 1991-1996 Silicon Graphics, Inc. Marble Widget: 0.11.2 (Stable Release) Parallelised demosaicing: Yes Database backend: QMYSQL Database internal server: No LibGphoto2: 2.4.11 LibKface: 2.0.0 LibKipi: 1.2.0 LibOpenCV: 2.2.0 Libface: 0.2 Created attachment 61648 [details]
repeat valgrind run after upgrading all libraries to natty versions
I searched for all of the libraries that were still maverick (10.10) versions and replaced these with the natty (11.04) versions. All had the same underlying version number but were just the natty builds of the same library.
I reran the valgrind session and set it off detecting and recognising faces. Whilst this was ongoing I tagged a few faces in People/Unknown.
I then aborted the recognition operation and once it had aborted I closed down digikam. At this stage digikam died with a SIGSEGV. Unfortunately a proper stack trace was not produced presumably because valgrind and the bug helpers do not get along (there is a valgrind warning/bug notice in the attached output to that effect).
There are a large number of "Invalid read of size 4" warnings towards the end of the trace. I did not notice if any of these were produced during face recognition/tagging but know that some of them were produced during closedown just prior to the SIGSEGV crash.
I see lots of issues accessing the loading cache - sure, it has been deleted at application shutdown: ==15989== Address 0x1074afe8 is 8 bytes inside a block of size 100 free'd ==15989== at 0x4025907: operator delete(void*) (vg_replace_malloc.c:387) ==15989== by 0x50991BB: Digikam::LoadingCache::~LoadingCache() (loadingcache.cpp:183) ==15989== by 0x50994C1: Digikam::LoadingCache::~LoadingCache() (loadingcache.cpp:185) ==15989== by 0x50962C5: Digikam::LoadingCache::cleanUp() (loadingcache.cpp:162) ==15989== by 0x509C176: Digikam::LoadingCacheInterface::cleanUp() (loadingcacheinterface.cpp:40) ==15989== by 0x81914B8: Digikam::DigikamApp::~DigikamApp() (digikamapp.cpp:325) Did your tests run include shutdown of the application? The valgrind tests do include appliation shutdown yes. In the last run I didn't take a note in the output where I started the shutdown. The shutdown was definitely after digikam(15989)/digikam (core) Digikam::CollectionScanner::scanAlbum: Folder does not exist or is not readable: "/media/shirephoto/digikam/PictureProject/0116/DSC_1958.JPG" But I am not sure if that is the last bit of output before I cancelled face recognition/detection. If I get some time tonight I will re-run and see if I can identify any specific sequencing so that issues during shutdown can be separated from face detection issues. Created attachment 61826 [details]
valgrind output when face recognition crashed with SIGSEGV
I finally got a valgrind session output where digikam crashed with during face recognition. The output that occurred around the crash time looks very interesting.
I see two issues one where timers have been unregistered improperly and the other where unitialise memory is used.
Application asked to unregister timer 0x1e000013 which is not registered in this thread. Fix application.
Application asked to unregister timer 0x28000069 which is not registered in this thread. Fix application.
==1785== Thread 1:
==1785== Use of uninitialised value of size 4
==1785== at 0x6D8AD00: QVariant::QVariant(QVariant const&) (qvariant.h:360)
==1785== by 0x5406A3B: Digikam::ImageModel::data(QModelIndex const&, int) const (imagemodel.cpp:1016)
==1785== by 0x5418B70: Digikam::ImageThumbnailModel::data(QModelIndex const&, int) const (imagethumbnailmodel.cpp:279)
==1785== by 0x540AD9C: Digikam::ImageFilterModel::subSortLessThan(QModelIndex const&, QModelIndex const&) const (qabstractitemmodel.h:398)
==1785== by 0x5CD5AB9: KCategorizedSortFilterProxyModel::lessThan(QModelIndex const&, QModelIndex const&) const (kcategorizedsortfilterproxymodel.cpp:121)
==1785== by 0x6609960: int* QAlgorithmsPrivate::qLowerBoundHelper<int*, int, QSortFilterProxyModelLessThan>(int*, int*, int const&, QSortFilterProxyModelLessThan) (qsortfilterproxymodel.cpp:80)
==1785== by 0x66099E1: int* qLowerBound<int*, int, QSortFilterProxyModelLessThan>(int*, int*, int const&, QSortFilterProxyModelLessThan) (qalgorithms.h:252)
==1785== by 0x6609D48: void QAlgorithmsPrivate::qMerge<int*, int const, QSortFilterProxyModelLessThan>(int*, int*, int*, int const&, QSortFilterProxyModelLessThan) (qalgorithms.h:435)
==1785== by 0x6609CE1: void QAlgorithmsPrivate::qMerge<int*, int const, QSortFilterProxyModelLessThan>(int*, int*, int*, int const&, QSortFilterProxyModelLessThan) (qalgorithms.h:446)
==1785== by 0x6609CE1: void QAlgorithmsPrivate::qMerge<int*, int const, QSortFilterProxyModelLessThan>(int*, int*, int*, int const&, QSortFilterProxyModelLessThan) (qalgorithms.h:446)
==1785== by 0x6609C6E: void QAlgorithmsPrivate::qMerge<int*, int const, QSortFilterProxyModelLessThan>(int*, int*, int*, int const&, QSortFilterProxyModelLessThan) (qalgorithms.h:445)
==1785== by 0x6609C6E: void QAlgorithmsPrivate::qMerge<int*, int const, QSortFilterProxyModelLessThan>(int*, int*, int*, int const&, QSortFilterProxyModelLessThan) (qalgorithms.h:445)
==1785== Uninitialised value was created by a heap allocation
==1785== at 0x4026864: malloc (vg_replace_malloc.c:236)
==1785== by 0x6C6FB8C: qMalloc(unsigned int) (qmalloc.cpp:55)
==1785== by 0x6C95840: QListData::detach(int) (qlist.cpp:185)
==1785== by 0x534B0E0: QList<QVariant>::detach_helper(int) (qlist.h:694)
==1785== by 0x534B1F9: QList<QVariant>::detach_helper() (qlist.h:710)
==1785== by 0x5406A22: Digikam::ImageModel::data(QModelIndex const&, int) const (qlist.h:126)
==1785== by 0x5418B70: Digikam::ImageThumbnailModel::data(QModelIndex const&, int) const (imagethumbnailmodel.cpp:279)
==1785== by 0x540AD70: Digikam::ImageFilterModel::subSortLessThan(QModelIndex const&, QModelIndex const&) const (qabstractitemmodel.h:398)
==1785== by 0x5CD5AB9: KCategorizedSortFilterProxyModel::lessThan(QModelIndex const&, QModelIndex const&) const (kcategorizedsortfilterproxymodel.cpp:121)
==1785== by 0x6609E18: void QAlgorithmsPrivate::qMerge<int*, int const, QSortFilterProxyModelLessThan>(int*, int*, int*, int const&, QSortFilterProxyModelLessThan) (qsortfilterproxymodel.cpp:80)
==1785== by 0x6609F61: void QAlgorithmsPrivate::qStableSortHelper<int*, int, QSortFilterProxyModelLessThan>(int*, int*, int const&, QSortFilterProxyModelLessThan) (qalgorithms.h:459)
==1785== by 0x6609F06: void QAlgorithmsPrivate::qStableSortHelper<int*, int, QSortFilterProxyModelLessThan>(int*, int*, int const&, QSortFilterProxyModelLessThan) (qalgorithms.h:458)
==1785==
==1785== Invalid read of size 4
==1785== at 0x6D8AD00: QVariant::QVariant(QVariant const&) (qvariant.h:360)
==1785== by 0x5406A3B: Digikam::ImageModel::data(QModelIndex const&, int) const (imagemodel.cpp:1016)
==1785== by 0x5418B70: Digikam::ImageThumbnailModel::data(QModelIndex const&, int) const (imagethumbnailmodel.cpp:279)
==1785== by 0x540AD9C: Digikam::ImageFilterModel::subSortLessThan(QModelIndex const&, QModelIndex const&) const (qabstractitemmodel.h:398)
==1785== by 0x5CD5AB9: KCategorizedSortFilterProxyModel::lessThan(QModelIndex const&, QModelIndex const&) const (kcategorizedsortfilterproxymodel.cpp:121)
==1785== by 0x6609960: int* QAlgorithmsPrivate::qLowerBoundHelper<int*, int, QSortFilterProxyModelLessThan>(int*, int*, int const&, QSortFilterProxyModelLessThan) (qsortfilterproxymodel.cpp:80)
==1785== by 0x66099E1: int* qLowerBound<int*, int, QSortFilterProxyModelLessThan>(int*, int*, int const&, QSortFilterProxyModelLessThan) (qalgorithms.h:252)
==1785== by 0x6609D48: void QAlgorithmsPrivate::qMerge<int*, int const, QSortFilterProxyModelLessThan>(int*, int*, int*, int const&, QSortFilterProxyModelLessThan) (qalgorithms.h:435)
==1785== by 0x6609CE1: void QAlgorithmsPrivate::qMerge<int*, int const, QSortFilterProxyModelLessThan>(int*, int*, int*, int const&, QSortFilterProxyModelLessThan) (qalgorithms.h:446)
==1785== by 0x6609CE1: void QAlgorithmsPrivate::qMerge<int*, int const, QSortFilterProxyModelLessThan>(int*, int*, int*, int const&, QSortFilterProxyModelLessThan) (qalgorithms.h:446)
==1785== by 0x6609C6E: void QAlgorithmsPrivate::qMerge<int*, int const, QSortFilterProxyModelLessThan>(int*, int*, int*, int const&, QSortFilterProxyModelLessThan) (qalgorithms.h:445)
==1785== by 0x6609C6E: void QAlgorithmsPrivate::qMerge<int*, int const, QSortFilterProxyModelLessThan>(int*, int*, int*, int const&, QSortFilterProxyModelLessThan) (qalgorithms.h:445)
==1785== Address 0x0 is not stack'd, malloc'd or (recently) free'd
Richard, With digiKam 2.4, can you reproduce memory issues with valgrind ? Gilles Caulier I re-ran briefly with Valgrind yesterday. I didn't get it to crash (but I likely need to try harder because it can take 30 minutes to crash without valgrind slowing things down). I noted some memory issues. I've nearly finished doing a quick triage of those and most are in libarary code and already have fixes. One is in kdelibs5 and didn't have a fix so I've knocked on up and it seems to fix that. I'll finish the triage off over the weekend and update the report with that. This entry still valid using last stable 2.9.0 release ? Gilles Caulier As with 3.3.0, new face recognition have been written, i recommend to check entry #323888 and post in this file new comments about memory leak in Face Management feature. Gilles Caulier *** This bug has been marked as a duplicate of bug 323888 *** Problem is fixed with new 7.0.0-beta1 through this long story from this bug https://bugs.kde.org/show_bug.cgi?id=399923 You can test digiKam 7.0.0-beta1 with bundle available here: https://download.kde.org/unstable/digikam/ Don't hesitate to give us a fresh feedback about his entry. Thanks in advance Gilles Caulier |
Created attachment 61632 [details] output from run 1 of valgrind/digikam Version: 2.0.0 (using KDE 4.6.4) OS: Linux Whilst attempting to reproduce a crash during face recognition/detection I ran Digikam 2.0.0.rc under valgrind on Ubuntu 11.04 using the philip5 ppa binaries. Valgrind reported a number of issues that can be seen in the attached valgrind output. The issues seem to be spread over a number of libraries/areas of code so it maybe that a number of related bugs need raising to track resolution of each. Unfortunately I did not manage to recreate but 277099 (or any of the similar ones) at this time but hopefully the valgrind output will help to fix some issues. Reproducible: Always Steps to Reproduce: Start digikam using valgrind --track-origins=yes digikam Initiate face detection/recognition on a number of albums and whilst this is running go to the People/Unknown list of photos and start assigning tags to the faces identified. I ran valgrind twice with similar output each time. Expected Results: Valgrind should not have found any memory allocation/usage errors.