Bug 274940

Not reproducible here. As refocus tool run in a separated thread, can you run digiKam through gdb and look if a crash appears ?

Gilles Caulier

If it does not crash but runs at 100% CPU, please:
- find out the process id of digikam (system monitor, or ps -ax | grep digikam)
- run "gdb att <id>"
- post the output of "thr appl all bt"

Created attachment 60915 [details]
gdb output for "thr appl all bt"

This file contains the gdb output of "thr appl all bt", attached to the "100% cpu" state of digikams refocus.

According to top (using thread view), the 100% cpu thread is 13667. That must be thread 2 in gdb.txt. Main process id was 13639.

Do you need debug output for libclapack, too?

Created attachment 60919 [details]
gdb output for "thr appl all bt" as requested in comment #2 (with clapack debug symbols)

This time with clapack debug symbols.

The 100% cpu thread has ID 31101. Process ID was 26148.

Code is running inside dgesv from clapack. So at first it looks like normal work being done.
I cannot seem to reproduce the problem with a simle test.

I did some more debugging tonight:

The infinity loop seems to be at "/var/tmp/portage/sci-libs/clapack-3.2.1-r4/work/clapack-3.2.1-CMAKE/INSTALL/dlamch.c" lines 348 to 355.

The source code is:
------------------------ CODE ------------------------
	lt = 0;
	a = 1.;
	c__ = 1.;


/* +       WHILE( C.EQ.ONE )LOOP */
L30:
	if (c__ == one) {
	    ++lt;
	    a *= lbeta;
	    c__ = dlamc3_(&a, &one);
	    d__1 = -a;
	    c__ = dlamc3_(&c__, &d__1);
	    goto L30;
	}
/* +       END WHILE */
------------------------ /CODE ------------------------
and it is entered with:
   one = 1
   lbeta = 0


The reason might be found in this bug report: 
https://mailman.cae.wisc.edu/pipermail/bug-octave/2009-November/016247.html

An important sentence in one of the answers:
 "the SLAMCH and DLAMCH routines are sensitive to compiler optimizations."

And since I'm using a gentoo linux, I have plenty of options to "kill" DLAMCH with various compiler options. :(

I guess it is a gentoo bug now and should further be handled at bugs.gentoo.org. What do you think?

It's not a Gentoo bug either, if you use something else than "-O2 -pipe -fomit-frame-pointer".
In particular using flags which affect math precision on all packages is simply plain wrong, packages which can benefit from these generally set them by themself.

you can still open a bug on gentoo bugzilla asking for them to filter-flags providing also a `emerge --info sci-libs/clapack`

thank you for investigating this issue

http://en.gentoo-wiki.com/wiki/Safe_Cflags (unofficial)

(In reply to comment #7)
> I did some more debugging tonight:
> 
> The infinity loop seems to be at
> "/var/tmp/portage/sci-libs/clapack-3.2.1-r4/work/clapack-3.2.1-CMAKE/INSTALL/dlamch.c"
> lines 348 to 355.
> 
> The source code is:
> ------------------------ CODE ------------------------
>     lt = 0;
>     a = 1.;
>     c__ = 1.;
> 
> 
> /* +       WHILE( C.EQ.ONE )LOOP */
> L30:
>     if (c__ == one) {
>         ++lt;
>         a *= lbeta;
>         c__ = dlamc3_(&a, &one);
>         d__1 = -a;
>         c__ = dlamc3_(&c__, &d__1);
>         goto L30;
>     }
> /* +       END WHILE */
> ------------------------ /CODE ------------------------
> and it is entered with:
>    one = 1
>    lbeta = 0
> 
> 
> The reason might be found in this bug report: 
> https://mailman.cae.wisc.edu/pipermail/bug-octave/2009-November/016247.html
> 
> An important sentence in one of the answers:
>  "the SLAMCH and DLAMCH routines are sensitive to compiler optimizations."
> 
> And since I'm using a gentoo linux, I have plenty of options to "kill" DLAMCH
> with various compiler options. :(
> 
> I guess it is a gentoo bug now and should further be handled at
> bugs.gentoo.org. What do you think?

(In reply to comment #8)
> It's not a Gentoo bug either, if you use something else than "-O2 -pipe
> -fomit-frame-pointer".
> In particular using flags which affect math precision on all packages is simply
> plain wrong, packages which can benefit from these generally set them by
> themself.
> 
> you can still open a bug on gentoo bugzilla asking for them to filter-flags
> providing also a `emerge --info sci-libs/clapack`

Gentoo clapack maintainer here :). You can also just tell us your CFLAGS here, I'm watching digikam-devel on the kde bugzilla anyway.

Note : in OPenCV library, there is a 3rdparty clapack code embeded. As they use Cmake too, perhaps some compilation tunes are done to prevent a similar problem.

Gilles Caulier

# emerge --info: (see also the following attachment)
---------------- 8< ----------------
[...]
CFLAGS="-O2 -march=core2 -msse4.1 -pipe"
[...]
CXXFLAGS="-O2 -march=core2 -msse4.1 -pipe"
[...]
---------------- >8 ----------------

If debugging, I add "-ggdb" to CFLAGS and CXXFLAGS and add "nostrip" to FEATURES. The Problem occurs in "debug-" and "non-debug-mode".


# cat /proc/cpuinfo
---------------- >8 ----------------
[...]
vendor_id       : GenuineIntel
cpu family      : 6
model           : 23
model name      : Intel(R) Core(TM)2 Duo CPU     E8400  @ 3.00GHz
[..]
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx lm constant_tsc arch_perfmon pebs bts aperfmperf pni dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm sse4_1 xsave lahf_lm dts tpr_shadow vnmi flexpriority
[...]
---------------- 8< ----------------

According to this Wiki-Article (http://en.gentoo-wiki.com/wiki/Safe_Cflags/Intel#Core_2_Duo.2FQuad.2C_Xeon_51xx.2F53xx.2F54xx.2F3360.2C_Pentium_Dual-Core_T23xx.2B.2FExxxx.2C_Celeron_Dual-Core) those use flags should be ok.

Created attachment 61114 [details]
The emerge settings media-gfx/digikam-1.9.0 and sci-libs/clapack-3.2.1-r4 are currently compiled with

Output of "emerge --info".

Look into OpenCV 3rdparty. CLapack is here and source code is compiled with this options :

https://opencvlibrary.svn.sourceforge.net/svnroot/opencvlibrary/trunk/opencv/3rdparty/lapack/CMakeLists.txt

... look at lines:

    set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -Wno-parentheses")
    set_source_files_properties(dlamch.c slamch.c PROPERTIES COMPILE_FLAGS "-ffloat-store -O0")

perhaps i need to turn on these options with GCC...

Gilles Caulier

Created attachment 61115 [details]
Output of "echo | gcc -dM -E - -march=native"

According to http://en.gentoo-wiki.com/wiki/Safe_Cflags#-march.3Dnative this would be the settings, gcc would pic automatically.

Created attachment 61117 [details]
emerge --info sci-libs/clapack

(In reply to comment #11)
> # emerge --info: (see also the following attachment)
> ---------------- 8< ----------------
> [...]
> CFLAGS="-O2 -march=core2 -msse4.1 -pipe"
> [...]
> CXXFLAGS="-O2 -march=core2 -msse4.1 -pipe"
> [...]
> ---------------- >8 ----------------
> 
> If debugging, I add "-ggdb" to CFLAGS and CXXFLAGS and add "nostrip" to
> FEATURES. The Problem occurs in "debug-" and "non-debug-mode".
> 
> 
> # cat /proc/cpuinfo
> ---------------- >8 ----------------
> [...]
> vendor_id       : GenuineIntel
> cpu family      : 6
> model           : 23
> model name      : Intel(R) Core(TM)2 Duo CPU     E8400  @ 3.00GHz
> [..]
> flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov
> pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx lm constant_tsc
> arch_perfmon pebs bts aperfmperf pni dtes64 monitor ds_cpl vmx smx est tm2
> ssse3 cx16 xtpr pdcm sse4_1 xsave lahf_lm dts tpr_shadow vnmi flexpriority
> [...]
> ---------------- 8< ----------------
> 
> According to this Wiki-Article
> (http://en.gentoo-wiki.com/wiki/Safe_Cflags/Intel#Core_2_Duo.2FQuad.2C_Xeon_51xx.2F53xx.2F54xx.2F3360.2C_Pentium_Dual-Core_T23xx.2B.2FExxxx.2C_Celeron_Dual-Core)
> those use flags should be ok.

Same CPU here, but using a 64bit environment, recompiled sci-libs/clapack with C*FLAGS="-O2 -march=core2 -msse4.1 -pipe -ggdb", gcc-4.5.2 and the loop is not reproducible.
Same does apply using gcc-4.6.0 and CFLAGS as in attached emerge --info.

Speaking of digikam-2.0 from git because I've problem recompiling digikam-1.9 alone with gcc-4.5.2 (need to rebuild kexif and possibly other)

may be this problem only related to 32 bit builds?

(In reply to comment #13)
> Look into OpenCV 3rdparty. CLapack is here and source code is compiled with
> this options :
> 
> https://opencvlibrary.svn.sourceforge.net/svnroot/opencvlibrary/trunk/opencv/3rdparty/lapack/CMakeLists.txt
> 
> ... look at lines:
> 
>     set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -Wno-parentheses")
>     set_source_files_properties(dlamch.c slamch.c PROPERTIES COMPILE_FLAGS
> "-ffloat-store -O0")

As a test, I set "-O2" in C*FLAGS to "-O0" in /etc/make.conf and recompiled "sci-libs/clapack-3.2.1-r4".
=> the infinite loop seems to be eleminated and digikam's Refocus plugin (using clapack) works flawless.

So I think the following line might fix this bug:
set_source_files_properties(dlamch.c slamch.c PROPERTIES COMPILE_FLAGS "-O0")

(In reply to comment #15)
> Same CPU here, but using a 64bit environment, recompiled sci-libs/clapack with
> C*FLAGS="-O2 -march=core2 -msse4.1 -pipe -ggdb", gcc-4.5.2 and the loop is not
> reproducible.
> Same does apply using gcc-4.6.0 and CFLAGS as in attached emerge --info.
> 
> Speaking of digikam-2.0 from git because I've problem recompiling digikam-1.9
> alone with gcc-4.5.2 (need to rebuild kexif and possibly other)
> 
> may be this problem only related to 32 bit builds?

I'm using gcc-4.4.5, which also _might_ be the critical difference. But in reference to comment #8, the 32 bit system seems more likely.

(In reply to comment #16)
> set_source_files_properties(dlamch.c slamch.c PROPERTIES COMPILE_FLAGS "-O0")

This has been patched in sci-libs/clapack-3.2.1-r6 - keyword that and you should be fine. I'll raise the dependencies as soon as it becomes stable.
From the Gentoo side this bug can be closed.

I keyworded sci-libs/clapack-3.2.1-r6 and emerged it, but the infinite loop is still there.

 # ebuild /usr/portage/sci-libs/clapack/clapack-3.2.1-r6.ebuild configure
 # cd /var/tmp/portage/sci-libs/clapack-3.2.1-r6
 # grep -nHrD skip "set_source_files_properties" .
gives me:
> ./work/clapack-3.2.1-CMAKE/SRC/CMakeLists.txt:359:   set_source_files_properties(dlamch.c slamch.c PROPERTIES COMPILE_FLAGS "-ffloat-store -O0")

But is this the correct directory/CMakeList.txt? Because
 # find -iname "dlamch*"
gives me:
> ./work/clapack-3.2.1-CMAKE/INSTALL/dlamch.c

=> "set_source_files_properties" is in SRC-Directory, but the specified file is in INSTALL-Directory

I tried:

# ebuild /usr/portage/sci-libs/clapack/clapack-3.2.1-r6.ebuild configure

then changed the line to:
> set_source_files_properties(../INSTALL/dlamch.c ../INSTALL/slamch.c PROPERTIES COMPILE_FLAGS "-ffloat-store -O0")

# ebuild /usr/portage/sci-libs/clapack/clapack-3.2.1-r6.ebuild merge

But that did not help. (I did not reemerge digikam, do I have to?)

My fault, I should do this stuff when I'm more awake. Try the same before the configure phase (i.e. after prepare)... :) You dont have to reemerge digikam. Will test later tonight.

(In reply to comment #20)
> I tried:
> 
> # ebuild /usr/portage/sci-libs/clapack/clapack-3.2.1-r6.ebuild configure
> 
> then changed the line to:
> > set_source_files_properties(../INSTALL/dlamch.c ../INSTALL/slamch.c PROPERTIES COMPILE_FLAGS "-ffloat-store -O0")
> 
> # ebuild /usr/portage/sci-libs/clapack/clapack-3.2.1-r6.ebuild merge
> 
> But that did not help. (I did not reemerge digikam, do I have to?)

OK, I patched the patch. :) This is now the correct patch:

diff -ruN clapack-3.2.1-CMAKE.orig/SRC/CMakeLists.txt clapack-3.2.1-CMAKE/SRC/CMakeLists.txt
--- clapack-3.2.1-CMAKE.orig/SRC/CMakeLists.txt 2011-06-21 00:29:16.275749375 +0200
+++ clapack-3.2.1-CMAKE/SRC/CMakeLists.txt      2011-06-21 00:31:06.133749448 +0200
@@ -355,6 +355,9 @@
    zla_heamv.c zla_hercond_c.c zla_hercond_x.c zla_herpvgrw.c          
    zla_lin_berr.c zlarscl2.c zlascl2.c zla_wwaddw.c)
 
+if(CMAKE_COMPILER_IS_GNUCC)
+   set_source_files_properties(../INSTALL/dlamch.c ../INSTALL/slamch.c PROPERTIES COMPILE_FLAGS "-ffloat-store -O0")
+endif()
 
 if( USEXBLAS)
   set(ALLXOBJ ${SXLASRC} ${DXLASRC} ${CXLASRC} ${ZXLASRC} ${ALLXAUX})


(note GNUCC instead of GNUCXX!). I checked and the flags are correctly applied during compilation. Committed, you should get it in one/two hours with a sync. Would be great to know if it works... ;)

Bug fixed! Hurray! Thanks at all!

Steps to reproduce:
  1. emerge --sync
  2. echo "=sci-libs/clapack-3.2.1-r6" >>/etc/portage/package.keywords
  3. emerge -1 sci-libs/clapack
 => Digikam Refocus-Plugins works again! :)

*** Bug 242479 has been marked as a duplicate of this bug. ***