Bug 403269

Summary: Stacked image and Panorama tools cannot find required binaries
Product: [Applications] digikam Reporter: meku <kde.bugs>
Component: Plugin-Generic-ExpoBlendingAssignee: Digikam Developers <digikam-bugs-null>
Status: RESOLVED FIXED    
Severity: normal CC: caulier.gilles, metzpinguin, rozzo
Priority: NOR    
Version: 6.0.0   
Target Milestone: ---   
Platform: Appimage   
OS: Linux   
Latest Commit: Version Fixed In: 6.4.0
Sentry Crash Report:

Description meku 2019-01-16 12:52:59 UTC
SUMMARY
Stacked images and Panorama tools cannot be used because hugin and enfuse binaries are not found by Digikam.

The binaries are in the path and manually selecting their location with 'Find' button does not accept them.

Installed packages:
hugin-tools v2018.0.0
enfuse v4.2-4

Digikam version: digikam-6.0.0-beta3-20181228T114626-x86-64.appimage

SOFTWARE/OS VERSIONS
KDE Plasma Version: 5.14.5
KDE Frameworks Version: 5.52.0
Qt Version: 5.11.1
Comment 1 Maik Qualmann 2019-01-16 18:23:48 UTC
Here under openSUSE Tumbleweed all tools (same versions hugin-2018 and enfuse 4.2) are also automatically found in the AppImage. In which file path are the hugin tools installed?

Maik
Comment 2 Maik Qualmann 2019-01-16 18:35:46 UTC
Can you post the output from the console, for example, if you choose autooptimiser manually. As an an example:

digikam.general: Testing  "/usr/bin/autooptimiser" ...
digikam.general: "/usr/bin/autooptimiser"  help header line: 
 "autooptimiser version 2018.0.0.be8da0221960"
digikam.general: Found  "/usr/bin/autooptimiser"  version:  "2018.0.0"
digikam.general: All Binaries Found :  true

Maik
Comment 3 meku 2019-01-17 09:22:49 UTC
Hugin tools appear to be in /usr/bin

Console selecting autooptimiser:

Digikam::DBinaryIface::checkDir: Testing  "/usr/bin/autooptimiser" ...
Digikam::adjustedEnvironmentForAppImage: Adjusting environment variables for AppImage bundle
Digikam::DBinaryIface::parseHeader: "autooptimiser"  help header line: 
 "ERROR: ld.so: object '/usr/lib64/libudev.so' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored."
Digikam::DBinarySearch::slotAreBinariesFound: All Binaries Found :  false
Comment 4 caulier.gilles 2019-01-17 09:34:44 UTC
Digikam::adjustedEnvironmentForAppImage: Adjusting environment variables for AppImage bundle
Digikam::DBinaryIface::parseHeader: "autooptimiser"  help header line: 
 "ERROR: ld.so: object '/usr/lib64/libudev.so' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored."

This error come from only the AppImage. This is due to binary compatibility with low level libraries which cannot be embeded in the bundle.

One solution to this problem is to build the bundle under a more recent Linux system. Remember that currently we use CentOS 6.10, which still maintained...

Another solution is to preload shared library in the AppImage run script as we already do. Take a care, it will puzzle the AppImage stating process and can have a side effect at run time. Look the already preload set here :

https://cgit.kde.org/digikam.git/tree/project/bundles/appimage/data/AppRun#n11

As you can see the famous libudev is already in the list of pre-ploaded items... and typically, the .so file cannot be found on the target system...

Gilles Caulier
Comment 5 meku 2019-01-17 10:26:53 UTC
On this ubuntu system the libraries are not at the paths that Digikam has hardcoded:
libfreetype.so -> /usr/lib/x86_64-linux-gnu/libfreetype.so.6
libudev.so -> /lib/x86_64-linux-gnu/libudev.so.1

If I manually create symlinks to /usr/lib64 then the Stacking/Panorama tools work.
Comment 6 caulier.gilles 2019-01-19 09:39:29 UTC
Maik,

How to make a list of shared libs to preload with right path without to make a distro puzzle ?

Perhaps we need a bash function to found the path of specific lib to pre-load to run it.

Gilles
Comment 7 caulier.gilles 2019-01-19 09:49:54 UTC
There is one solution here, but it's not perfect as it return all symlinks pointing to shared lib :

https://stackoverflow.com/questions/13148608/how-can-i-find-the-full-file-path-given-a-library-name-like-libfoo-so-1

ex under Centos6.10 :

[gilles@localhost data]$ echo "$(ldconfig -p | grep libudev.so | tr ' ' '\n' | grep /)"
/lib64/libudev.so.0
/usr/lib64/libudev.so
[gilles@localhost data]$ ll /lib64/libudev.so.0
lrwxrwxrwx. 1 root root 16 29 oct.  11:09 /lib64/libudev.so.0 -> libudev.so.0.5.1
[gilles@localhost data]$ echo "$(ldconfig -p | grep libfreetype.so | tr ' ' '\n' | grep /)"
/usr/lib64/libfreetype.so.6
/usr/lib64/libfreetype.so

Gilles
Comment 8 caulier.gilles 2019-01-19 13:38:59 UTC
Ok the solution is something like this :

echo "$(ldconfig -p | grep libfreetype.so | tr ' ' '\n' | grep / | grep ".*\.so$" )"
Comment 9 caulier.gilles 2019-01-19 15:28:53 UTC
meku,

I upload a new Linux AppImage bundle 64bits with a patch for LD_PRELOAD to be more universal.

Please try with this version here :


https://files.kde.org/digikam/

Gilles Caulier
Comment 10 caulier.gilles 2019-01-19 15:47:38 UTC
Hum, in fact my way to use hard coded library path is the best, as ldconfig can be run as root only in some Linux... So i need to revisit my patch
Comment 11 Maik Qualmann 2019-01-19 16:10:34 UTC
Yes, the same here under openSUSE, ldconfig needs root rights.

Maik
Comment 12 caulier.gilles 2019-01-19 17:54:38 UTC
Git commit ee630892eab59649fbc6276d53b51dbedff5b9ef by Gilles Caulier.
Committed on 19/01/2019 at 17:53.
Pushed by cgilles into branch 'master'.

add rules to check shared lib paths

M  +34   -8    project/bundles/appimage/data/AppRun

https://commits.kde.org/digikam/ee630892eab59649fbc6276d53b51dbedff5b9ef
Comment 13 meku 2019-01-20 03:23:34 UTC
The above commit fails to preload of the shared libs because the lib filenames are not matched. As previously noted, ubuntu paths are:
/usr/lib/x86_64-linux-gnu/libfreetype.so.6
/lib/x86_64-linux-gnu/libudev.so.1

Yet this build DOES work for stacking and panorama tools. Is it because $LD_PRELOAD is left unset in this new build?

Output from console at startup:

-- digiKam AppImage Bundle                                                                                                                  
-- Use 'help' as CLI argument to know all available options for digiKam application                                       
-- Preloading shared libs:
Comment 14 caulier.gilles 2019-01-20 05:30:51 UTC
Git commit a50d64cb0243dbceaa39a50817bd62962c7a9682 by Gilles Caulier.
Committed on 20/01/2019 at 05:29.
Pushed by cgilles into branch 'development/dplugins'.

forget to remove this debug line

M  +1    -1    project/bundles/appimage/data/AppRun

https://commits.kde.org/digikam/a50d64cb0243dbceaa39a50817bd62962c7a9682
Comment 15 Maik Qualmann 2019-01-20 07:24:14 UTC
This would also work for normal users. I just do not know if ldconfig is located under /sbin/ in all Linux distributions.

echo "$(/sbin/ldconfig -p | grep libfreetype.so | tr ' ' '\n' | grep / | grep ".*\.so$" )"

Maik
Comment 16 caulier.gilles 2019-01-20 09:06:08 UTC
You are right Maik, if i pass the full path to ldconfig, with a standard user it work.

I can revisit the patch then...

Gilles
Comment 17 caulier.gilles 2019-03-20 15:16:10 UTC
After 3 weeks of work, i finally completed the compilation of AppImage using Qt
5.11.3 + QWebkit 5.212.

New 6.1.0 pre-release AppImage bundle can be found here (64 bits only for the
moment) :

https://files.kde.org/digikam/

Please check if this bugzilla entry still valid.

Thanks in advance

Gilles Caulier
Comment 18 meku 2019-04-26 06:22:44 UTC
Stacked image tool is not working in DK 6.1 or DK 6.2 appimages.

The interface now finds the binaries but shows a waiting page with "Pre-processing is under progress, please wait." Looking at running processes there doesn't appear to be any image processing going on.

I can confirm it works as expected in final DK 6.0 release.
Comment 19 caulier.gilles 2019-04-26 06:50:27 UTC
Between 6.0.0 and 6.1.0, nothing has changed in this plugins about processing.

I suspect a change in binary program used in background, especially about CLI interface.

Which versions do you use with mandatory programs for this plugin ?

Gilles Caulier
Comment 20 meku 2019-04-26 07:26:47 UTC
The mandatory programs do not appear to have changed since my first post.

Installed are:
hugin-tools v2018.0.0
enfuse v4.2-4
Comment 21 Maik Qualmann 2019-04-26 10:45:11 UTC
Git commit d6eb516fb99da981499b6fdc3f816c391d5f7962 by Maik Qualmann.
Committed on 26/04/2019 at 10:44.
Pushed by mqualmann into branch 'master'.

fix signal / slots from the ExpoBlendingThread

M  +4    -4    core/dplugins/generic/tools/expoblending/blendingdlg/expoblendingdlg.cpp
M  +4    -4    core/dplugins/generic/tools/expoblending/wizard/expoblendingitemspage.cpp
M  +6    -6    core/dplugins/generic/tools/expoblending/wizard/expoblendingpreprocesspage.cpp

https://invent.kde.org/kde/digikam/commit/d6eb516fb99da981499b6fdc3f816c391d5f7962
Comment 22 meku 2019-04-27 09:37:39 UTC
Appears to be working again in digikam-6.2.0-git-20190426T172018-qtwebkit-x86-64.appimage

Note that running the appimage with 'debug' then the stacked tool cannot find the binaries, though this appears to have been an issue in all versions - including DK 6.0.0
Comment 23 caulier.gilles 2019-04-27 09:38:47 UTC
Which one is not found exactly ?

Gilles Caulier
Comment 24 caulier.gilles 2019-04-27 09:39:19 UTC
And it's the same for Panorama plugin ?

Gilles Caulier
Comment 25 meku 2019-04-27 12:46:06 UTC
In debug: stacked does not find any required binaries and panorama only finds 'cpfind' and 'make'.

Without debug: all the required binaries are successfully detected.
Comment 26 Maik Qualmann 2019-09-13 09:46:16 UTC
*** Bug 411882 has been marked as a duplicate of this bug. ***
Comment 27 Maik Qualmann 2019-09-13 16:24:04 UTC
OK, I close the bug now. Without debug mode, all programs are now found. If necessary, reopen the bug.

Maik