Bug 402428 - Try to install non-KDE build dependencies automatically
Summary: Try to install non-KDE build dependencies automatically
Status: RESOLVED MOVED
Alias: None
Product: kdesrc-build
Classification: Developer tools
Component: project metadata (show other bugs)
Version: Git
Platform: Other Linux
: NOR normal
Target Milestone: ---
Assignee: Michael Pyne
URL: https://invent.kde.org/kde/kdesrc-bui...
Keywords:
Depends on:
Blocks:
 
Reported: 2018-12-21 15:42 UTC by Nate Graham
Modified: 2020-04-26 14:42 UTC (History)
2 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Nate Graham 2018-12-21 15:42:54 UTC
Now that we're strongly recommending the use of kdesrc-build in our official documentation, one thing that's coming up more and more is problems getting non-KDE dependencies installed.

It might be nice if kdesrc-build could do a best-effort attempt to install needed non-KDE dependencies using the distro's package manager. It would basically do the equivalent of `dnf builddep` (Fedora), `zypper si -d` (openSUSE) or `apt build-dep` (Ubuntu), but omitting the KDE dependencies if possible. If not, that's not a huge deal since kdesrc-build will just compile them anyway.
Comment 1 Nate Graham 2018-12-21 15:50:27 UTC
And it should do this for all of the KDE dependencies too, so that all of them will be able to compile successfully.
Comment 2 Christoph Feck 2018-12-21 16:25:17 UTC
The problem is that the name of the packages to install is not unified across distributions. How should kdesrc-build know if the package is named 'libqca5-dev', 'libqca-devel', 'qca5-devel', 'libqca-qt5-devel', etc.?
Comment 3 Nate Graham 2018-12-21 17:18:38 UTC
Indeed. Probably we would need to store a mapping of KDE software name to distro package name, and it would need to be maintained as distros irritatingly change their package names.

However I suspect that such a mapping would be less prone to becoming wrong over time compared to the current approach of maintaining lists of all distro package dependency names at https://community.kde.org/Guidelines_and_HOWTOs/Build_from_source/Install_the_dependencies.
Comment 4 Nate Graham 2018-12-24 02:16:35 UTC
After attempting to actually use our developer documentation that recommends kdesrc-build while pretending to not know anything that I know, the issue of fetching non-KDE dependencies kept coming up again and again. kdesrc-build works fantastically well as long as you have all the non-KDE dependencies installed. If you don't, it's an exercise in extreme frustration reading log files one by one trying to parse CMake errors and map them to package names that you then go out and manually install. I think it must be a minor miracle when a new contributor has the patience to work through this.

I would be happy to contribute to fixing this bug by compiling the mapping of KDE-project-to-distro-package-name, and I can help with the code too since I know Perl.

Conceptually I think we need a mapping that allows for each piece of software to have one or more distro packages associated with it. For example, KIO:

frameworks/kio
##############
Arch: kio
KDE Neon: kio
Ubuntu: libkf5kiocore5 libkf5kiofilewidgets5 libkf5kiogui5 libkf5kiontlm5 libkf5kiowidgets5

frameworks/kconfigwidgets
Arch: kconfigwidgets
KDE Neon: kconfigwidgets
Ubuntu: libkf5configwidgets5

[etc]

So when you do `kdesrc-build-kio --include-dependencies` it would go out and fetch the dependencies for kio and all of its KDE dependencies using the distro's package manager, e.g. sudo apt build-dep kio kconfigwidgets [etc]

We might also need a way of adding specificity regarding the major version of the KDE software, plus a way to have the package names change over time with different distro version releases.
Comment 5 Michael Pyne 2018-12-25 01:18:28 UTC
Part of this is already starting, though it's focused on getting kdesrc-build and critical dependencies installed common to most of KDE software (e.g. cmake, gcc, X11 or Wayland but not necessarily packages specific only to Krita or KDevelop).

See modules/ksb/OSSupport.pm (to figure out which distro we're on) and modules/ksb/FirstRun.pm (to encode which distro releases go to which packages to install). 

In particular OSSupport.pm already tries to provide a concept of distro versions, we could add metadata there to return the proper per-distro packager cmdline invocation. OSSupport.pm uses the systemd os-release standard which allows scripts to interrogate to increasingly more-generic versions of distros (e.g. Neon -> Ubuntu -> Debian or Manjaro -> Arch) so we hopefully would only need to support a few.

For per-app dependencies we could use the existing kde-build-metadata repository to hold the data mapping (rather than putting it in kdesrc-build itself). Would need to be JSON or YAML (we already have to support both). YAML is probably easier since you can provide comments :P

Conceptually we could then have a pseudo-module be added to the build list before the user's module list (but after kde-build-metadata), with a special Updater/BuildSystem setup to use the distro's own package manager to install dependencies for all modules in the build list.

I'm absolutely willing to push this feature forward but I could use help. :) Remember we're trialing Gitlab-based workflow for kdesrc-build on https://invent.kde.org/ so if you haven't already hopped over I'd love to get you integrated into some issues/merge requests.
Comment 6 Michael Pyne 2018-12-25 01:21:30 UTC
See https://invent.kde.org/kde/kdesrc-build/issues/9 for the KDE Invent issue I've added for this. I would like to track progress for this feature there.
Comment 7 Nate Graham 2018-12-26 00:28:37 UTC
Gotcha, let's do that.
Comment 8 Andrey 2020-04-26 13:51:19 UTC
(In reply to Nate Graham from comment #3)
> However I suspect that such a mapping would be less prone to becoming wrong
> over time compared to the current approach of maintaining lists of all
> distro package dependency names at
> https://community.kde.org/Guidelines_and_HOWTOs/Build_from_source/
> Install_the_dependencies.

Suffering from it meantime. BTW, the instruction above is outdated for Fedora.
Comment 9 Nate Graham 2020-04-26 14:42:51 UTC
It's a wiki, you can fix it now that you've figured out the true list of dependencies, right? :)

BTW This issue has been moved to https://invent.kde.org/kde/kdesrc-build/issues/9