Bug 323253

Summary: When face tag scanning, digikam exhausts all memory on computer.
Product: [Applications] digikam Reporter: Kristian Karl <kristian.hermann.karl>
Component: Faces-EngineAssignee: Digikam Developers <digikam-bugs-null>
Status: CLOSED FIXED    
Severity: normal CC: caulier.gilles, maheshmhegade, rf-kde
Priority: NOR    
Version: 3.4.0   
Target Milestone: ---   
Platform: Compiled Sources   
OS: Linux   
Latest Commit: Version Fixed In: 7.2.0
Sentry Crash Report:

Description Kristian Karl 2013-08-07 06:57:47 UTC
When scanning a small collection of pics (jpgs), digikam will exhaust all memory on computer.

I compiled digikam from latest master and ran it under a debugger. I interrupted the scanning when the memory consumption started to grow, and found this interesting frame stack for one of the threads:
#0 __printf_fp() at /lib64/libc.so.6
#1 vfprintf() at /lib64/libc.so.6
#2 __vsprintf_chk() at /lib64/libc.so.6
#3 __sprintf_chk() at /lib64/libc.so.6
#4 cvWriteRawData() at /lib64/libopencv_core.so.2.4
#5 icvWriteMat(CvFileStorage*, char const*, void const*, CvAttrList)() at /lib64/libopencv_core.so.2.4
#6 cvWrite() at /lib64/libopencv_core.so.2.4
#7 cv::write(cv::FileStorage&, std::string const&, cv::Mat const&)() at /lib64/libopencv_core.so.2.4
#8 operator<< <cv::Mat>() at /usr/include/opencv2/core/operations.hpp:2909
#9 KFaceIface::LBPHFaceModel::setHistograms() at /home/krikar/dev/kde/digikam/extra/libkface/libkface/recognition-opencv-lbph/lbphfacemodel.cpp:139
#10 KFaceIface::TrainingDB::lbphFaceModel() at /home/krikar/dev/kde/digikam/extra/libkface/libkface/database/trainingdb.cpp:303
#11 lbph() at /home/krikar/dev/kde/digikam/extra/libkface/libkface/recognition-opencv-lbph/opencvlbphfacerecognizer.cpp:67
#12 KFaceIface::OpenCVLBPHFaceRecognizer::recognize() at /home/krikar/dev/kde/digikam/extra/libkface/libkface/recognition-opencv-lbph/opencvlbphfacerecognizer.cpp:153
#13 KFaceIface::RecognitionDatabase::recognizeFaces() at /home/krikar/dev/kde/digikam/extra/libkface/libkface/recognitiondatabase.cpp:620
#14 KFaceIface::RecognitionDatabase::recognizeFaces() at /home/krikar/dev/kde/digikam/extra/libkface/libkface/recognitiondatabase.cpp:594
#15 Digikam::RecognitionWorker::process() at /home/krikar/dev/kde/digikam/core/utilities/facemanagement/facepipeline.cpp:594
#16 Digikam::RecognitionWorker::qt_static_metacall() at /home/krikar/dev/kde/digikam/build/core/digikam/facepipeline_p.moc:406
#17 QObject::event(QEvent*)() at /lib64/libQtCore.so.4
#18 QApplicationPrivate::notify_helper(QObject*, QEvent*)() at /lib64/libQtGui.so.4
#19 QApplication::notify(QObject*, QEvent*)() at /lib64/libQtGui.so.4
#20 KApplication::notify(QObject*, QEvent*)() at /lib64/libkdeui.so.5
#21 QCoreApplication::notifyInternal(QObject*, QEvent*)() at /lib64/libQtCore.so.4
#22 QCoreApplicationPrivate::sendPostedEvents(QObject*, int, QThreadData*)() at /lib64/libQtCore.so.4
#23 postEventSourceDispatch(_GSource*, int (*)(void*), void*)() at /lib64/libQtCore.so.4
#24 g_main_context_dispatch() at /lib64/libglib-2.0.so.0
#25 g_main_context_iterate.isra.22() at /lib64/libglib-2.0.so.0
#26 g_main_context_iteration() at /lib64/libglib-2.0.so.0
#27 QEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>)() at /lib64/libQtCore.so.4
#28 QEventLoop::processEvents(QFlags<QEventLoop::ProcessEventsFlag>)() at /lib64/libQtCore.so.4
#29 QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>)() at /lib64/libQtCore.so.4
#30 Digikam::WorkerObjectRunnable::run() at /home/krikar/dev/kde/digikam/core/libs/threads/threadmanager.cpp:196
#31 QThreadPoolThread::run()() at /lib64/libQtCore.so.4
#32 QThreadPrivate::start(void*)() at /lib64/libQtCore.so.4
#33 start_thread() at /lib64/libpthread.so.0
#34 clone() at /lib64/libc.so.6

It gets stuck in: #9 KFaceIface::LBPHFaceModel::setHistograms() at /home/krikar/dev/kde/digikam/extra/libkface/libkface/recognition-opencv-lbph/lbphfacemodel.cpp:139

If I comment out that line, digikam works as expected. But I guess the recognition part does not work properly if histograms are not saved?

I'm on Fedora 19, using testing repos, and the version of Open CV is:
yum info opencv-devel.x86_64
Installed Packages
Name        : opencv-devel
Arch        : x86_64
Version     : 2.4.6.1
Release     : 1.fc19
Size        : 2.6 M
Repo        : installed
From repo   : updates-testing
Summary     : Development files for using the OpenCV library
URL         : http://opencv.org
License     : BSD
Description : This package contains the OpenCV C/C++ library and header files, as well as
            : documentation. It should be installed if you want to develop programs that
            : will use the OpenCV library. You should consider installing opencv-devel-docs
            : package.


Reproducible: Always

Steps to Reproduce:
1. Start digikam using a new database: digikam --database-directory=/home/user/digikam/tmp/
2. Add a small collection of pics. (I used a collection of 20 jpgs)
3. Scan collection for faces.
Actual Results:  
All memory (8GB RAM) on the computer is used by digikam in just a few seconds.

Expected Results:  
Digikam completes the face scanning without any problems.
Comment 1 Kristian Karl 2013-08-07 07:22:17 UTC
Actually, it it's better to comment out line core/utilities/facemanagement/facepipeline.cpp:594 instead.
Commenting out extra/libkface/libkface/recognition-opencv-lbph/lbphfacemodel.cpp:139 crashes digikam when saving face tags.


diff --git a/utilities/facemanagement/facepipeline.cpp b/utilities/facemanagement/facepipeline.cpp
index e9ad434..19d7af7 100644
--- a/utilities/facemanagement/facepipeline.cpp
+++ b/utilities/facemanagement/facepipeline.cpp
@@ -591,7 +591,7 @@ void RecognitionWorker::process(FacePipelineExtendedPackage::Ptr package)
         images = imageRetriever.getThumbnails(package->filePath, package->databaseFaces.toDatabaseFaceList());
     }
 
-    package->recognitionResults  = database.recognizeFaces(images);
+    //package->recognitionResults  = database.recognizeFaces(images);
     package->processFlags       |= FacePipelinePackage::ProcessedByRecognizer;
 
     emit processed(package);
Comment 2 caulier.gilles 2013-08-07 07:25:10 UTC
Strange. Here with OpenCV 2.4.5 under OSX, i pass scanning with any problem (8Gb of RAM).

Same under Linux with 16Gb of RAM (Collection is 250Gb of pictures). It take a while, but i cannot any memory leak or any dysfunction

I suspect a problem with your OpenCV...

Gilles Caulier
Comment 3 caulier.gilles 2013-08-07 07:29:50 UTC
Mahesh,

I would to know your viewpoint here...

Gilles Caulier
Comment 4 Marcel Wiesweg 2013-08-07 19:40:36 UTC
lbphfacemodel.cpp:139 is a possible problem, OpenCV API does not allow an efficient way of loading parameters here, yet the problem should appear only if a very large number of faces is loaded. Not with 20. OpenCV problem?

Commenting out facepipeline.cpp:594  -  Certainly will skipping the recognition solve any problem. That's not at the root of the problem.
Comment 5 Kristian Karl 2013-08-08 10:12:07 UTC
Some more data:
I uncommented kDebug extra/libkface/libkface/database/trainingdb.cpp:297
and it prints 10680 histograms. Is this a realistic number of histograms to store?
:
digikam(9826)/KFACE KFaceIface::TrainingDB::lbphFaceModel: Adding histogram 10676 identity 19 size 65536
digikam(9826)/KFACE KFaceIface::TrainingDB::lbphFaceModel: Adding histogram 10677 identity 19 size 65536
digikam(9826)/KFACE KFaceIface::TrainingDB::lbphFaceModel: Adding histogram 10678 identity 19 size 65536
digikam(9826)/KFACE KFaceIface::TrainingDB::lbphFaceModel: Adding histogram 10679 identity 19 size 65536
digikam(9826)/KFACE KFaceIface::TrainingDB::lbphFaceModel: Adding histogram 10680 identity 19 size 65536
:

The memory gets exhausted, later, when converting the the histograms to [yaml] cv::String, at:
extra/libkface/libkface/recognition-opencv-lbph/lbphfacemodel.cpp:153

I compiled a debug version of latest opencv, and the amount of bytes that gets allocated to the string ends up at some 1691131415 bytes.
I got the value from variable size_t len, at opencv2/core/cvstd.hpp:565

I found that the memory gets exhausted in the while loop in opencv2/core/cvstd.hpp between lines 567 and 571

#0 cv::String::String<std::_Deque_iterator<char, char&, char*> >() at /home/krikar/dev/opencv/modules/core/include/opencv2/core/cvstd.hpp:566
#1 icvClose() at /home/krikar/dev/opencv/modules/core/src/persistence.cpp:545
#2 cv::FileStorage::releaseAndGetString() at /home/krikar/dev/opencv/modules/core/src/persistence.cpp:5173
#3 KFaceIface::LBPHFaceModel::setHistograms() at /home/krikar/dev/kde/digikam/extra/libkface/libkface/recognition-opencv-lbph/lbphfacemodel.cpp:153
#4 KFaceIface::TrainingDB::lbphFaceModel() at /home/krikar/dev/kde/digikam/extra/libkface/libkface/database/trainingdb.cpp:303
#5 lbph() at /home/krikar/dev/kde/digikam/extra/libkface/libkface/recognition-opencv-lbph/opencvlbphfacerecognizer.cpp:67
#6 KFaceIface::OpenCVLBPHFaceRecognizer::recognize() at /home/krikar/dev/kde/digikam/extra/libkface/libkface/recognition-opencv-lbph/opencvlbphfacerecognizer.cpp:153
#7 KFaceIface::RecognitionDatabase::recognizeFaces() at /home/krikar/dev/kde/digikam/extra/libkface/libkface/recognitiondatabase.cpp:620
#8 KFaceIface::RecognitionDatabase::recognizeFaces() at /home/krikar/dev/kde/digikam/extra/libkface/libkface/recognitiondatabase.cpp:594
#9 Digikam::RecognitionWorker::process() at /home/krikar/dev/kde/digikam/core/utilities/facemanagement/facepipeline.cpp:594
#10 Digikam::RecognitionWorker::qt_static_metacall() at /home/krikar/dev/kde/digikam/build/core/digikam/facepipeline_p.moc:406
#11 QObject::event(QEvent*)() at /lib64/libQtCore.so.4
#12 QApplicationPrivate::notify_helper(QObject*, QEvent*)() at /lib64/libQtGui.so.4
#13 QApplication::notify(QObject*, QEvent*)() at /lib64/libQtGui.so.4
#14 KApplication::notify(QObject*, QEvent*)() at /lib64/libkdeui.so.5
#15 QCoreApplication::notifyInternal(QObject*, QEvent*)() at /lib64/libQtCore.so.4
#16 QCoreApplicationPrivate::sendPostedEvents(QObject*, int, QThreadData*)() at /lib64/libQtCore.so.4
#17 postEventSourceDispatch(_GSource*, int (*)(void*), void*)() at /lib64/libQtCore.so.4
#18 g_main_context_dispatch() at /lib64/libglib-2.0.so.0
#19 g_main_context_iterate.isra.22() at /lib64/libglib-2.0.so.0
#20 g_main_context_iteration() at /lib64/libglib-2.0.so.0
Comment 6 Marcel Wiesweg 2013-08-08 19:41:20 UTC
There is usually one histogram per learned (named + confirmed) face. I wouldn't expect that if there's a collection with 20 jpgs as you say - or are there 10000 faces in the rest of your collection?

The building of a yaml file is dirty, but there is absolutely no other way to feed data into OpenCV for the face recognizer.
In the long run, we'll need to add a hard limit on the number of learned faces.
Comment 7 Kristian Karl 2013-08-08 21:48:57 UTC
Very strange. There's only 2 or 3 faces in each of these 20 jpgs.
I started digikam with a new database: "digikam --database-directory=/home/user/digikam/tmp/", I added 1 single collection, the 20 jpgs.

To make sure. Will the command line above start digikam on a clean slate? Or are there still data from other databases still around, like some training db for the faces, or such?
Comment 8 Marcel Wiesweg 2013-08-09 17:27:56 UTC
Yes, the recognition db is a separate file (it could be shared with other apps in the future)
usually at ~/.kde/share/apps/libkface/database/recognition.db
Comment 9 Kristian Karl 2013-08-10 12:24:11 UTC
When I removed my /home/$USER/.kde/share/apps/libkface/database/recognition.db digikam worked as expected.
Maybe the recognition.db (size ~104 MB) was corrupted in some way.

I closed this ticked.
Comment 10 René Fritz 2013-10-05 15:15:55 UTC
Just as a note. I had the same problem. Removing recognition.db solved the problem. Rebuilding the face db was also much faster then.
Comment 11 caulier.gilles 2020-09-12 05:19:12 UTC
Not reproducible with 7.2.0