Bug 274940 - digikam freezes when using "sharpen: refocus" (cpu 100%)
Summary: digikam freezes when using "sharpen: refocus" (cpu 100%)
Status: RESOLVED FIXED
Alias: None
Product: digikam
Classification: Applications
Component: Plugin-Editor-Sharpen (show other bugs)
Version: 1.9.0
Platform: Gentoo Packages Linux
: NOR crash
Target Milestone: ---
Assignee: Digikam Developers
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-06-04 22:34 UTC by Peter Albrecht
Modified: 2020-08-12 12:54 UTC (History)
4 users (show)

See Also:
Latest Commit:
Version Fixed In: 2.0.0


Attachments
gdb output for "thr appl all bt" (10.29 KB, text/plain)
2011-06-11 22:31 UTC, Peter Albrecht
Details
gdb output for "thr appl all bt" as requested in comment #2 (with clapack debug symbols) (11.83 KB, text/plain)
2011-06-12 07:58 UTC, Peter Albrecht
Details
The emerge settings media-gfx/digikam-1.9.0 and sci-libs/clapack-3.2.1-r4 are currently compiled with (4.55 KB, text/plain)
2011-06-18 14:14 UTC, Peter Albrecht
Details
Output of "echo | gcc -dM -E - -march=native" (4.06 KB, text/plain)
2011-06-18 14:29 UTC, Peter Albrecht
Details
emerge --info sci-libs/clapack (6.64 KB, application/octet-stream)
2011-06-18 15:28 UTC, Francesco Riosa
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Peter Albrecht 2011-06-04 22:34:09 UTC
Version:           1.9.0 (using KDE 4.6.2) 
OS:                Linux

Using the sharpening method "Refocus" seems to initiate a closed loop and renders this sharpening method unuseable.

"Simple Sharpening" works flawless.

Reproducible: Always

Steps to Reproduce:
1. start digikam
2. select image and open it in "Image Editor (F4)
3. in menu: "Enhance" -> "Sharpen..."
4. select "method = Refocus"
5. click "Try"

Actual Results:  
- mouse cursor changes to "please wait, I'm working"
- the progressbar in Image Editor keeps stuck at "0%"
- cpu-usage of digikam goes up to 100% and stays there
- nothing else happens for 5 minutes and more (in digikam 1.2.0 refocus finished within 60 seconds)

Expected Results:  
digikam calculates and shows preview of current refocus settings

To test with a clean profile, I have also deleted every "*digikam*"-file in "~/.kde4/". But that did not help.

Enabling every digikam-entry in kdebugdialog shows me the following two lines at STDOUT:

------------ 8< ------------
digikam(4992)/digikam (core) Digikam::EditorToolThreaded::slotEffect: Preview  "Sharpen"  started...
digikam(4992)/digikam (core) Digikam::RefocusFilter::refocusImage: RefocusFilter::Compute matrix...
------------ >8 ------------

After >30 seconds the following lines are added, but I think they are not important:
------------ 8< ------------
digikam(5147)/khtml (caret) DOM::Selection::moveTo: Selection[ Position( 0x0 "null" : 0 ) Position( 0x0 "null" : 0 ) Position( 0x0 "null" : 0 ) Position( 0x0 "null" : 0 ) 1 ] Position( 0x0 "null" : 0 ) Position( 0x0 "null" : 0 )
digikam(5147)/khtml (caret) DOM::Selection::validate: Selection[ Position( 0x0 "null" : 0 ) Position( 0x0 "null" : 0 ) Position( 0x0 "null" : 0 ) Position( 0x0 "null" : 0 ) 1 ] 0
digikam(5147)/khtml (caret) DOM::Selection::validate: [character:baseIsStart] true Position( 0x0 "null" : 0 ) Position( 0x0 "null" : 0 )
digikam(5147)/kio (Slave) KIO::Slave::kill: killing slave pid 5162 ( "digikamdates://" )
[...]
------------ >8 ------------
Comment 1 caulier.gilles 2011-06-07 10:47:15 UTC
Not reproducible here. As refocus tool run in a separated thread, can you run digiKam through gdb and look if a crash appears ?

Gilles Caulier
Comment 2 Marcel Wiesweg 2011-06-11 16:22:39 UTC
If it does not crash but runs at 100% CPU, please:
- find out the process id of digikam (system monitor, or ps -ax | grep digikam)
- run "gdb att <id>"
- post the output of "thr appl all bt"
Comment 3 Peter Albrecht 2011-06-11 22:31:59 UTC
Created attachment 60915 [details]
gdb output for "thr appl all bt"

This file contains the gdb output of "thr appl all bt", attached to the "100% cpu" state of digikams refocus.

According to top (using thread view), the 100% cpu thread is 13667. That must be thread 2 in gdb.txt. Main process id was 13639.
Comment 4 Peter Albrecht 2011-06-11 22:51:27 UTC
Do you need debug output for libclapack, too?
Comment 5 Peter Albrecht 2011-06-12 07:58:51 UTC
Created attachment 60919 [details]
gdb output for "thr appl all bt" as requested in comment #2 (with clapack debug symbols)

This time with clapack debug symbols.

The 100% cpu thread has ID 31101. Process ID was 26148.
Comment 6 Marcel Wiesweg 2011-06-14 22:18:57 UTC
Code is running inside dgesv from clapack. So at first it looks like normal work being done.
I cannot seem to reproduce the problem with a simle test.
Comment 7 Peter Albrecht 2011-06-17 22:39:07 UTC
I did some more debugging tonight:

The infinity loop seems to be at "/var/tmp/portage/sci-libs/clapack-3.2.1-r4/work/clapack-3.2.1-CMAKE/INSTALL/dlamch.c" lines 348 to 355.

The source code is:
------------------------ CODE ------------------------
	lt = 0;
	a = 1.;
	c__ = 1.;


/* +       WHILE( C.EQ.ONE )LOOP */
L30:
	if (c__ == one) {
	    ++lt;
	    a *= lbeta;
	    c__ = dlamc3_(&a, &one);
	    d__1 = -a;
	    c__ = dlamc3_(&c__, &d__1);
	    goto L30;
	}
/* +       END WHILE */
------------------------ /CODE ------------------------
and it is entered with:
   one = 1
   lbeta = 0


The reason might be found in this bug report: 
https://mailman.cae.wisc.edu/pipermail/bug-octave/2009-November/016247.html

An important sentence in one of the answers:
 "the SLAMCH and DLAMCH routines are sensitive to compiler optimizations."

And since I'm using a gentoo linux, I have plenty of options to "kill" DLAMCH with various compiler options. :(

I guess it is a gentoo bug now and should further be handled at bugs.gentoo.org. What do you think?
Comment 8 Francesco Riosa 2011-06-17 23:01:26 UTC
It's not a Gentoo bug either, if you use something else than "-O2 -pipe -fomit-frame-pointer".
In particular using flags which affect math precision on all packages is simply plain wrong, packages which can benefit from these generally set them by themself.

you can still open a bug on gentoo bugzilla asking for them to filter-flags providing also a `emerge --info sci-libs/clapack`

thank you for investigating this issue

http://en.gentoo-wiki.com/wiki/Safe_Cflags (unofficial)

(In reply to comment #7)
> I did some more debugging tonight:
> 
> The infinity loop seems to be at
> "/var/tmp/portage/sci-libs/clapack-3.2.1-r4/work/clapack-3.2.1-CMAKE/INSTALL/dlamch.c"
> lines 348 to 355.
> 
> The source code is:
> ------------------------ CODE ------------------------
>     lt = 0;
>     a = 1.;
>     c__ = 1.;
> 
> 
> /* +       WHILE( C.EQ.ONE )LOOP */
> L30:
>     if (c__ == one) {
>         ++lt;
>         a *= lbeta;
>         c__ = dlamc3_(&a, &one);
>         d__1 = -a;
>         c__ = dlamc3_(&c__, &d__1);
>         goto L30;
>     }
> /* +       END WHILE */
> ------------------------ /CODE ------------------------
> and it is entered with:
>    one = 1
>    lbeta = 0
> 
> 
> The reason might be found in this bug report: 
> https://mailman.cae.wisc.edu/pipermail/bug-octave/2009-November/016247.html
> 
> An important sentence in one of the answers:
>  "the SLAMCH and DLAMCH routines are sensitive to compiler optimizations."
> 
> And since I'm using a gentoo linux, I have plenty of options to "kill" DLAMCH
> with various compiler options. :(
> 
> I guess it is a gentoo bug now and should further be handled at
> bugs.gentoo.org. What do you think?
Comment 9 Andreas K. Huettel 2011-06-18 02:17:38 UTC
(In reply to comment #8)
> It's not a Gentoo bug either, if you use something else than "-O2 -pipe
> -fomit-frame-pointer".
> In particular using flags which affect math precision on all packages is simply
> plain wrong, packages which can benefit from these generally set them by
> themself.
> 
> you can still open a bug on gentoo bugzilla asking for them to filter-flags
> providing also a `emerge --info sci-libs/clapack`

Gentoo clapack maintainer here :). You can also just tell us your CFLAGS here, I'm watching digikam-devel on the kde bugzilla anyway.
Comment 10 caulier.gilles 2011-06-18 09:40:52 UTC
Note : in OPenCV library, there is a 3rdparty clapack code embeded. As they use Cmake too, perhaps some compilation tunes are done to prevent a similar problem.

Gilles Caulier
Comment 11 Peter Albrecht 2011-06-18 14:12:50 UTC
# emerge --info: (see also the following attachment)
---------------- 8< ----------------
[...]
CFLAGS="-O2 -march=core2 -msse4.1 -pipe"
[...]
CXXFLAGS="-O2 -march=core2 -msse4.1 -pipe"
[...]
---------------- >8 ----------------

If debugging, I add "-ggdb" to CFLAGS and CXXFLAGS and add "nostrip" to FEATURES. The Problem occurs in "debug-" and "non-debug-mode".


# cat /proc/cpuinfo
---------------- >8 ----------------
[...]
vendor_id       : GenuineIntel
cpu family      : 6
model           : 23
model name      : Intel(R) Core(TM)2 Duo CPU     E8400  @ 3.00GHz
[..]
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx lm constant_tsc arch_perfmon pebs bts aperfmperf pni dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm sse4_1 xsave lahf_lm dts tpr_shadow vnmi flexpriority
[...]
---------------- 8< ----------------

According to this Wiki-Article (http://en.gentoo-wiki.com/wiki/Safe_Cflags/Intel#Core_2_Duo.2FQuad.2C_Xeon_51xx.2F53xx.2F54xx.2F3360.2C_Pentium_Dual-Core_T23xx.2B.2FExxxx.2C_Celeron_Dual-Core) those use flags should be ok.
Comment 12 Peter Albrecht 2011-06-18 14:14:59 UTC
Created attachment 61114 [details]
The emerge settings media-gfx/digikam-1.9.0 and sci-libs/clapack-3.2.1-r4 are currently compiled with

Output of "emerge --info".
Comment 13 caulier.gilles 2011-06-18 14:21:52 UTC
Look into OpenCV 3rdparty. CLapack is here and source code is compiled with this options :

https://opencvlibrary.svn.sourceforge.net/svnroot/opencvlibrary/trunk/opencv/3rdparty/lapack/CMakeLists.txt

... look at lines:

    set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -Wno-parentheses")
    set_source_files_properties(dlamch.c slamch.c PROPERTIES COMPILE_FLAGS "-ffloat-store -O0")

perhaps i need to turn on these options with GCC...

Gilles Caulier
Comment 14 Peter Albrecht 2011-06-18 14:29:14 UTC
Created attachment 61115 [details]
Output of "echo | gcc -dM -E - -march=native"

According to http://en.gentoo-wiki.com/wiki/Safe_Cflags#-march.3Dnative this would be the settings, gcc would pic automatically.
Comment 15 Francesco Riosa 2011-06-18 15:28:19 UTC
Created attachment 61117 [details]
emerge --info sci-libs/clapack

(In reply to comment #11)
> # emerge --info: (see also the following attachment)
> ---------------- 8< ----------------
> [...]
> CFLAGS="-O2 -march=core2 -msse4.1 -pipe"
> [...]
> CXXFLAGS="-O2 -march=core2 -msse4.1 -pipe"
> [...]
> ---------------- >8 ----------------
> 
> If debugging, I add "-ggdb" to CFLAGS and CXXFLAGS and add "nostrip" to
> FEATURES. The Problem occurs in "debug-" and "non-debug-mode".
> 
> 
> # cat /proc/cpuinfo
> ---------------- >8 ----------------
> [...]
> vendor_id       : GenuineIntel
> cpu family      : 6
> model           : 23
> model name      : Intel(R) Core(TM)2 Duo CPU     E8400  @ 3.00GHz
> [..]
> flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov
> pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx lm constant_tsc
> arch_perfmon pebs bts aperfmperf pni dtes64 monitor ds_cpl vmx smx est tm2
> ssse3 cx16 xtpr pdcm sse4_1 xsave lahf_lm dts tpr_shadow vnmi flexpriority
> [...]
> ---------------- 8< ----------------
> 
> According to this Wiki-Article
> (http://en.gentoo-wiki.com/wiki/Safe_Cflags/Intel#Core_2_Duo.2FQuad.2C_Xeon_51xx.2F53xx.2F54xx.2F3360.2C_Pentium_Dual-Core_T23xx.2B.2FExxxx.2C_Celeron_Dual-Core)
> those use flags should be ok.

Same CPU here, but using a 64bit environment, recompiled sci-libs/clapack with C*FLAGS="-O2 -march=core2 -msse4.1 -pipe -ggdb", gcc-4.5.2 and the loop is not reproducible.
Same does apply using gcc-4.6.0 and CFLAGS as in attached emerge --info.

Speaking of digikam-2.0 from git because I've problem recompiling digikam-1.9 alone with gcc-4.5.2 (need to rebuild kexif and possibly other)

may be this problem only related to 32 bit builds?
Comment 16 Peter Albrecht 2011-06-18 22:02:31 UTC
(In reply to comment #13)
> Look into OpenCV 3rdparty. CLapack is here and source code is compiled with
> this options :
> 
> https://opencvlibrary.svn.sourceforge.net/svnroot/opencvlibrary/trunk/opencv/3rdparty/lapack/CMakeLists.txt
> 
> ... look at lines:
> 
>     set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -Wno-parentheses")
>     set_source_files_properties(dlamch.c slamch.c PROPERTIES COMPILE_FLAGS
> "-ffloat-store -O0")

As a test, I set "-O2" in C*FLAGS to "-O0" in /etc/make.conf and recompiled "sci-libs/clapack-3.2.1-r4".
=> the infinite loop seems to be eleminated and digikam's Refocus plugin (using clapack) works flawless.

So I think the following line might fix this bug:
set_source_files_properties(dlamch.c slamch.c PROPERTIES COMPILE_FLAGS "-O0")
Comment 17 Peter Albrecht 2011-06-18 22:10:24 UTC
(In reply to comment #15)
> Same CPU here, but using a 64bit environment, recompiled sci-libs/clapack with
> C*FLAGS="-O2 -march=core2 -msse4.1 -pipe -ggdb", gcc-4.5.2 and the loop is not
> reproducible.
> Same does apply using gcc-4.6.0 and CFLAGS as in attached emerge --info.
> 
> Speaking of digikam-2.0 from git because I've problem recompiling digikam-1.9
> alone with gcc-4.5.2 (need to rebuild kexif and possibly other)
> 
> may be this problem only related to 32 bit builds?

I'm using gcc-4.4.5, which also _might_ be the critical difference. But in reference to comment #8, the 32 bit system seems more likely.
Comment 18 Andreas K. Huettel 2011-06-19 22:38:39 UTC
(In reply to comment #16)
> set_source_files_properties(dlamch.c slamch.c PROPERTIES COMPILE_FLAGS "-O0")

This has been patched in sci-libs/clapack-3.2.1-r6 - keyword that and you should be fine. I'll raise the dependencies as soon as it becomes stable.
From the Gentoo side this bug can be closed.
Comment 19 Peter Albrecht 2011-06-20 19:10:55 UTC
I keyworded sci-libs/clapack-3.2.1-r6 and emerged it, but the infinite loop is still there.

 # ebuild /usr/portage/sci-libs/clapack/clapack-3.2.1-r6.ebuild configure
 # cd /var/tmp/portage/sci-libs/clapack-3.2.1-r6
 # grep -nHrD skip "set_source_files_properties" .
gives me:
> ./work/clapack-3.2.1-CMAKE/SRC/CMakeLists.txt:359:   set_source_files_properties(dlamch.c slamch.c PROPERTIES COMPILE_FLAGS "-ffloat-store -O0")

But is this the correct directory/CMakeList.txt? Because
 # find -iname "dlamch*"
gives me:
> ./work/clapack-3.2.1-CMAKE/INSTALL/dlamch.c

=> "set_source_files_properties" is in SRC-Directory, but the specified file is in INSTALL-Directory
Comment 20 Peter Albrecht 2011-06-20 21:18:52 UTC
I tried:

# ebuild /usr/portage/sci-libs/clapack/clapack-3.2.1-r6.ebuild configure

then changed the line to:
> set_source_files_properties(../INSTALL/dlamch.c ../INSTALL/slamch.c PROPERTIES COMPILE_FLAGS "-ffloat-store -O0")

# ebuild /usr/portage/sci-libs/clapack/clapack-3.2.1-r6.ebuild merge

But that did not help. (I did not reemerge digikam, do I have to?)
Comment 21 Andreas K. Huettel 2011-06-20 21:28:34 UTC
My fault, I should do this stuff when I'm more awake. Try the same before the configure phase (i.e. after prepare)... :) You dont have to reemerge digikam. Will test later tonight.

(In reply to comment #20)
> I tried:
> 
> # ebuild /usr/portage/sci-libs/clapack/clapack-3.2.1-r6.ebuild configure
> 
> then changed the line to:
> > set_source_files_properties(../INSTALL/dlamch.c ../INSTALL/slamch.c PROPERTIES COMPILE_FLAGS "-ffloat-store -O0")
> 
> # ebuild /usr/portage/sci-libs/clapack/clapack-3.2.1-r6.ebuild merge
> 
> But that did not help. (I did not reemerge digikam, do I have to?)
Comment 22 Andreas K. Huettel 2011-06-21 01:02:27 UTC
OK, I patched the patch. :) This is now the correct patch:

diff -ruN clapack-3.2.1-CMAKE.orig/SRC/CMakeLists.txt clapack-3.2.1-CMAKE/SRC/CMakeLists.txt
--- clapack-3.2.1-CMAKE.orig/SRC/CMakeLists.txt 2011-06-21 00:29:16.275749375 +0200
+++ clapack-3.2.1-CMAKE/SRC/CMakeLists.txt      2011-06-21 00:31:06.133749448 +0200
@@ -355,6 +355,9 @@
    zla_heamv.c zla_hercond_c.c zla_hercond_x.c zla_herpvgrw.c          
    zla_lin_berr.c zlarscl2.c zlascl2.c zla_wwaddw.c)
 
+if(CMAKE_COMPILER_IS_GNUCC)
+   set_source_files_properties(../INSTALL/dlamch.c ../INSTALL/slamch.c PROPERTIES COMPILE_FLAGS "-ffloat-store -O0")
+endif()
 
 if( USEXBLAS)
   set(ALLXOBJ ${SXLASRC} ${DXLASRC} ${CXLASRC} ${ZXLASRC} ${ALLXAUX})


(note GNUCC instead of GNUCXX!). I checked and the flags are correctly applied during compilation. Committed, you should get it in one/two hours with a sync. Would be great to know if it works... ;)
Comment 23 Peter Albrecht 2011-06-21 20:04:20 UTC
Bug fixed! Hurray! Thanks at all!

Steps to reproduce:
  1. emerge --sync
  2. echo "=sci-libs/clapack-3.2.1-r6" >>/etc/portage/package.keywords
  3. emerge -1 sci-libs/clapack
 => Digikam Refocus-Plugins works again! :)
Comment 24 caulier.gilles 2011-07-06 10:53:26 UTC
*** Bug 242479 has been marked as a duplicate of this bug. ***