Bug 344537 - Explicit module load location breaks different path installs
Summary: Explicit module load location breaks different path installs
Status: RESOLVED WORKSFORME
Alias: None
Product: trojita
Classification: Applications
Component: Core (show other bugs)
Version: git
Platform: Compiled Sources Linux
: VHI wishlist
Target Milestone: 0.6
Assignee: Trojita default assignee
URL:
Keywords: triaged
: 332579 (view as bug list)
Depends on:
Blocks:
 
Reported: 2015-02-24 17:40 UTC by Helio Chissini de Castro
Modified: 2018-10-27 02:36 UTC (History)
2 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Helio Chissini de Castro 2015-02-24 17:40:08 UTC
This is for version 0.5

[helio@xxxxxx]$ /opt/kf5/bin/trojita
/opt/kf5/bin/trojita: error while loading shared libraries: libtrojita_plugins.so: cannot open shared object file: No such file or directory

It searchs on system patchs only
The plugin is installed under /opr/kf5/lib64


Reproducible: Always

Steps to Reproduce:
Install in any directory that is not system regular
Comment 1 Pali Rohár 2015-02-24 17:47:31 UTC
libtrojita_plugins.so is not plugin but rather shared runtime library which is needed same as libc.so or libQtCore.so. If you install libtrojita_plugins.so to some non standard path, you need to also configure correctly ldconfig, /etc/ld.so.conf or LD_LIBRARY_PATH. Also if you install libc.so to some non standard path then trojita will not start too. So I'm closing this report as INVALID.
Comment 2 Helio Chissini de Castro 2015-02-24 17:57:39 UTC
Nope, is not invalid at all

The fact this is a .so, invalidates on distributions to proper ship it.
It should be a proper library, with a proper soname or linked static.
This not enter in regular library conformity, should be reviewed as properly on code.

I'm reopening as wish list, very high priority for 0.6 release.
On this current release, i will need to do some changes to generate the distro packaging i aiming.

Regards.
Comment 3 Pali Rohár 2015-02-24 18:03:21 UTC
(In reply to Helio Chissini de Castro from comment #2)
> This not enter in regular library conformity

Why?
Comment 4 Helio Chissini de Castro 2015-02-24 18:20:27 UTC
Shared library conventions is well defined in Linux aside some real exceptions.
This is very old reference.
http://tldp.org/HOWTO/Program-Library-HOWTO/shared-libraries.html

The fact that a single .so in available, you can't rely on a ABI breakage on subsequent versions.

Two possible scenarios

- If you assume that "Only trojita uses it", so why the reason of be a library ? Why is not linked in the software ?  This will make a proper upgrade in future release, as many distros use to split libraries and main binaries.

- If you assume that this library will be used for other programs, means then that you will need a versioning scheme to keep the compatibility, otherwise, as soon a new ABI breakage happens on the library, every other program depending on that will break. And proper headers, development library, so, the soname + so link.

[]'s
Comment 5 Pali Rohár 2015-02-24 19:06:02 UTC
Library libtrojita_plugins.so is used for all trojita executables and plugins. It does not have stable ABI and is it not for external (out of trojita tree) applications.

> Why the reason of be a library ?

It is linked into all trojita plugins and executables.

> Why is not linked in the software ?

Without hacks or linux specific linker and compiler flags we cannot statically link it into main trojita executable. Trojita is not linux-only application, so we cannot use linux specific flags & configuration.

But if other trojita developers agree I can write patches which allow optional cmake option which cause that libtrojita_plugins.so will be statically linked into trojita binary, but only on linux with gcc and gnu linker. Previosly when I designed API for plugins this support for cmake option was rejected. But now other devs can reconsider it...

Problem is that Trojita is C++ application (now C++11) and libtrojita_plugins.so is also C++ library. Not C! And there are problems with C++ runtime and plugin libraries which cannot be easily fixed due to C++ language. Also your link does not mention how to solve such problems.

If you told me what force you to consider that libtrojita_plugins.so is not regular library (like libQtCore.so or libc.so) I would try to fix it.
Comment 6 Helio Chissini de Castro 2015-02-24 21:15:36 UTC
First of all, we're not talking about linux specifics, but how Linux
distros and most of projects under Linux do a proper library soname
naming.

About libQt5Core.so, or libc.so, here's what is in a linux install environment:

lrwxrwxrwx. 1 root root      19 Dez 10 17:55 /usr/lib64/libQt5Core.so
-> libQt5Core.so.5.4.0
lrwxrwxrwx. 1 root root      19 Dez 10 17:55
/usr/lib64/libQt5Core.so.5 -> libQt5Core.so.5.4.0
lrwxrwxrwx. 1 root root      19 Dez 10 17:55
/usr/lib64/libQt5Core.so.5.4 -> libQt5Core.so.5.4.0
-rwxr-xr-x. 1 root root 5086376 Dez 10 18:02 /usr/lib64/libQt5Core.so.5.4.0

So the .so private libs is a link to the main sonamed version


On Tue, Feb 24, 2015 at 4:06 PM, Pali Rohár <pali.rohar@gmail.com> wrote:
> https://bugs.kde.org/show_bug.cgi?id=344537
>
> --- Comment #5 from Pali Rohár <pali.rohar@gmail.com> ---
> Library libtrojita_plugins.so is used for all trojita executables and plugins.
> It does not have stable ABI and is it not for external (out of trojita tree)
> applications.
>
>> Why the reason of be a library ?
>
> It is linked into all trojita plugins and executables.
>
>> Why is not linked in the software ?
>
> Without hacks or linux specific linker and compiler flags we cannot statically
> link it into main trojita executable. Trojita is not linux-only application, so
> we cannot use linux specific flags & configuration.
>
> But if other trojita developers agree I can write patches which allow optional
> cmake option which cause that libtrojita_plugins.so will be statically linked
> into trojita binary, but only on linux with gcc and gnu linker. Previosly when
> I designed API for plugins this support for cmake option was rejected. But now
> other devs can reconsider it...
>
> Problem is that Trojita is C++ application (now C++11) and
> libtrojita_plugins.so is also C++ library. Not C! And there are problems with
> C++ runtime and plugin libraries which cannot be easily fixed due to C++
> language. Also your link does not mention how to solve such problems.
>
> If you told me what force you to consider that libtrojita_plugins.so is not
> regular library (like libQtCore.so or libc.so) I would try to fix it.
>
> --
> You are receiving this mail because:
> You reported the bug.
Comment 7 Thomas Lübking 2015-02-24 21:36:37 UTC
I think you're missing the point: this is no public API nor currently intended to ever become.
It's merely a technical requirement (as Pali has explained) - the shared object is part of one trojita source compilation.

Also, I do really not see how the question whether this lib needs to be versioned (cannot harm, but is not required either) is any related to the reported bug:
You cannot simply put a shared lib "somewhere" and hope the linker will "magically" resolve the path.

Not having INSTALL_PREFIX/lib$(LIB_SUFFIX) in LD_LIBRARY_PATH is like begging for trouble - this does by far not affect trojita alone and is not related to the library name either.

I assume the problem boiled up because you set RUNPATH (shadowing RPATH) rather than LD_LIBRARY_PATH?
Comment 8 Pali Rohár 2015-02-24 22:34:11 UTC
Maybe I should point that trojita can be compiled & installed only by cmake.

And if you install trojita executables and libraries via make install, then trojita should work correctly and also correctly handle library directory. And it should work also if you specify non standard CMAKE_INSTALL_PREFIX.

But if copy libraries and executables from trojita build tree yourself (or move installed libraries/executables), then this is not supported and never will be. Those problems you should fix by yourself.

Cmake handle ELF RPATH and other properties, so non standard directories should also work.

I do not see any reason to version libtrojita_plugins.so library. That library has no defined API/ABI and we are not going to do that.
Comment 9 Jan Kundrát 2015-02-24 23:26:02 UTC
Hi Helio,
the reason for requiring a shared library is described at [1]. At the same time, this is purely a technical requirement, and the library is abolutely not ABI stable -- hence a lack of soname versioning. We have no plans to settle on a stable ABI. If there's a better way to indicate this rather than not introducing ABI versioning, please let us know -- I'll be happy to adopt a reasonable standard.

With that out of mind, I wonder what you're actually trying to do. What is the sequence of commands that you're running? What is the end result you expect to get? Does some KDE software actually work that way?

[1] http://thread.gmane.org/gmane.comp.lib.qt.user/8057/focus=8065
Comment 10 Helio Chissini de Castro 2015-02-25 12:39:26 UTC
Hello Jan

First of all, sorry to create this discussion, not intentional, but
i'm maintaining LXQt in Fedora right now, and i was planning to add
Trojita package officially before Fedora 22 release.

The reason behind i was complaining about the private lib is because
the very restrict guidelines we have, and this is we, not you, but
reasoning you can see here, on the Downstream .so part
http://fedoraproject.org/wiki/Packaging:Guidelines#Shared_Libraries

The bug was open on the start because my confusion if this was a
plugin, development library, and as usual, my test environment uses
different than system directories to avoid polute the regular dirs
where only rpm is installed.

This is where i install my personal KDE development tree, and most of
the libraries there are using runpath. To compile trojita i used same
cmake command line i use for every Frameworks 5, so this caused the
alert.

Thinking in a near future, there's a plan on RedHat to sandboxing
applications, including qt one, an this couls lead to again create the
soname mechanism.

Again, sorry for all noise.

On Tue, Feb 24, 2015 at 8:26 PM, Jan Kundrát <jkt@kde.org> wrote:
> https://bugs.kde.org/show_bug.cgi?id=344537
>
> Jan Kundrát <jkt@kde.org> changed:
>
>            What    |Removed                     |Added
> ----------------------------------------------------------------------------
>          Resolution|---                         |WAITINGFORINFO
>              Status|REOPENED                    |NEEDSINFO
>
> --- Comment #9 from Jan Kundrát <jkt@kde.org> ---
> Hi Helio,
> the reason for requiring a shared library is described at [1]. At the same
> time, this is purely a technical requirement, and the library is abolutely not
> ABI stable -- hence a lack of soname versioning. We have no plans to settle on
> a stable ABI. If there's a better way to indicate this rather than not
> introducing ABI versioning, please let us know -- I'll be happy to adopt a
> reasonable standard.
>
> With that out of mind, I wonder what you're actually trying to do. What is the
> sequence of commands that you're running? What is the end result you expect to
> get? Does some KDE software actually work that way?
>
> [1] http://thread.gmane.org/gmane.comp.lib.qt.user/8057/focus=8065
>
> --
> You are receiving this mail because:
> You reported the bug.
Comment 11 Jan Kundrát 2015-02-25 13:23:24 UTC
> First of all, sorry to create this discussion, not intentional, but
> i'm maintaining LXQt in Fedora right now, and i was planning to add
> Trojita package officially before Fedora 22 release.

Cool -- there's already https://bugzilla.redhat.com/show_bug.cgi?id=1080411 
.

> The reason behind i was complaining about the private lib is because
> the very restrict guidelines we have, and this is we, not you, but
> reasoning you can see here, on the Downstream .so part
> http://fedoraproject.org/wiki/Packaging:Guidelines#Shared_Libraries

Fair enough; I can see the point of this. So, what is the Fedora's 
suggestion for shipping shared libraries with no soname versioning and no 
ABI guarantees? Should we use something like e.g. $LIBDIR/trojita/ along 
with RPATH?

Anyway, I'll be happy to do reasonable things, and pleasing packagers is a 
good thing in general.
Comment 12 Helio Chissini de Castro 2015-02-25 13:44:51 UTC
In my side i don't think i will have the option to disrupt that much
your work, as it is a library, not a plugin.

In general, what i would do in the case of current status of the
package would be exactly this:
https://paste.kde.org/ptp2tvz5k

This is acceptable for our guidelines and i will be able to check ABI
compliance, and not affects the Windows building at all, as Pali
strongly notice me.

[]'s




On Wed, Feb 25, 2015 at 10:23 AM, Jan Kundrát <jkt@kde.org> wrote:
> https://bugs.kde.org/show_bug.cgi?id=344537
>
> --- Comment #11 from Jan Kundrát <jkt@kde.org> ---
>> First of all, sorry to create this discussion, not intentional, but
>> i'm maintaining LXQt in Fedora right now, and i was planning to add
>> Trojita package officially before Fedora 22 release.
>
> Cool -- there's already https://bugzilla.redhat.com/show_bug.cgi?id=1080411
> .
>
>> The reason behind i was complaining about the private lib is because
>> the very restrict guidelines we have, and this is we, not you, but
>> reasoning you can see here, on the Downstream .so part
>> http://fedoraproject.org/wiki/Packaging:Guidelines#Shared_Libraries
>
> Fair enough; I can see the point of this. So, what is the Fedora's
> suggestion for shipping shared libraries with no soname versioning and no
> ABI guarantees? Should we use something like e.g. $LIBDIR/trojita/ along
> with RPATH?
>
> Anyway, I'll be happy to do reasonable things, and pleasing packagers is a
> good thing in general.
>
> --
> You are receiving this mail because:
> You reported the bug.
Comment 13 Pali Rohár 2015-02-25 14:40:04 UTC
So, if we set library SONAME to trojita version, it is ok for you? We already have cmake code which parse trojita version string and output four 16bit numbers, so I think we can reuse it.
Comment 14 Jan Kundrát 2015-02-25 14:47:13 UTC
Right now we do not provide any guarantees about the API/ABI of this library, not even between a hypotethical 0.6.2 and 0.6.3 releases -- it's really subject to arbitrary changes at any given time. AFAIU tieing the soname version with Trojita's version would change this, so I am not thrilled by this approach. Do I understand it right that e.g. libfoobar.so.5.1.2 and libfoobar.so.5.1.3 are supposed to be ABI-compatible?
Comment 15 Helio Chissini de Castro 2015-02-25 14:59:59 UTC
In this case would be 0.5.0 and 0.5.1 and 0.5.2 you mean. Is not usual
to have patch minor.

Is usually expected the the Major versions can be ABI compatible,
since the .so.0 will be seeing by applications.
But considering that this library is now used only by trojita itself,
is safe for now.

At the moment that you have more than one application that uses this
library and aren't released at same time, then will be an issue.

Btw, i looked on the bugzilla entry you pasted. To avoid sidetrack, i
will help the developer to move ahead his proposal.


On Wed, Feb 25, 2015 at 11:47 AM, Jan Kundrát <jkt@kde.org> wrote:
> https://bugs.kde.org/show_bug.cgi?id=344537
>
> --- Comment #14 from Jan Kundrát <jkt@kde.org> ---
> Right now we do not provide any guarantees about the API/ABI of this library,
> not even between a hypotethical 0.6.2 and 0.6.3 releases -- it's really subject
> to arbitrary changes at any given time. AFAIU tieing the soname version with
> Trojita's version would change this, so I am not thrilled by this approach. Do
> I understand it right that e.g. libfoobar.so.5.1.2 and libfoobar.so.5.1.3 are
> supposed to be ABI-compatible?
>
> --
> You are receiving this mail because:
> You reported the bug.
Comment 16 Pali Rohár 2015-02-25 15:12:47 UTC
Now I think we have discussion about non-existent problem (or some cmake problem).

$ mkdir build && cd build
$ cmake -DCMAKE_INSTALL_PREFIX=/tmp/trojita -DCMAKE_INSTALL_LIBDIR=/tmp/trojita/lib -DCMAKE_INSTALL_RPATH=/tmp/trojita/lib -DWITH_TESTS=OFF ..
$ make
$ make install

$ readelf -d /tmp/trojita/bin/trojita | grep trojita
 0x0000000000000001 (NEEDED)             Shared library: [libtrojita_plugins.so]
 0x000000000000000f (RPATH)              Library rpath: [/tmp/trojita/lib]

$ ldd /tmp/trojita/bin/trojita | grep trojita
        libtrojita_plugins.so => /tmp/trojita/lib/libtrojita_plugins.so (0x00007fd86171b000)

I used official 0.5.0 tarball and installed everything into /tmp/trojita. And starting /tmp/trojita/bin/trojita worked fine.

@Helio Chissini de Castro: Can you check if you set correctly all cmake settings at compile time? I think it must work also with path /opr/kf5/lib64. If not there is bug in cmake or our CMakeLists.txt, but not in trojita itself.
Comment 17 Pali Rohár 2015-02-25 16:11:48 UTC
And here is patch which add new cmake option to support external plugins without need for libtrojita_plugins.so library (that library can be static linked to executable on supported platforms & toolchains):

https://gerrit.vesnicky.cesnet.cz/r/#/c/402/
Comment 18 Jan Kundrát 2016-01-25 21:34:29 UTC
*** Bug 332579 has been marked as a duplicate of this bug. ***
Comment 19 Andrew Crouthamel 2018-09-25 21:47:54 UTC
Dear Bug Submitter,

This bug has been in NEEDSINFO status with no change for at least 15 days. Please provide the requested information as soon as possible and set the bug status as REPORTED. Due to regular bug tracker maintenance, if the bug is still in NEEDSINFO status with no change in 30 days, the bug will be closed as RESOLVED > WORKSFORME due to lack of needed information.

For more information about our bug triaging procedures please read the wiki located here: https://community.kde.org/Guidelines_and_HOWTOs/Bug_triaging

If you have already provided the requested information, please set the bug status as REPORTED so that the KDE team knows that the bug is ready to be confirmed.

Thank you for helping us make KDE software even better for everyone!
Comment 20 Andrew Crouthamel 2018-10-27 02:36:40 UTC
Dear Bug Submitter,

This bug has been in NEEDSINFO status with no change for at least 30 days. The bug is now closed as RESOLVED > WORKSFORME due to lack of needed information.

For more information about our bug triaging procedures please read the wiki located here: https://community.kde.org/Guidelines_and_HOWTOs/Bug_triaging

Thank you for helping us make KDE software even better for everyone!