Bug 350446 - Port MacOS PKG to Arm64 Apple Silicon.
Summary: Port MacOS PKG to Arm64 Apple Silicon.
Status: RESOLVED FIXED
Alias: None
Product: digikam
Classification: Applications
Component: Bundle-MacOS (show other bugs)
Version: 8.2.0
Platform: macOS (DMG) macOS
: NOR wishlist
Target Milestone: ---
Assignee: Digikam Developers
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2015-07-21 09:47 UTC by kertase
Modified: 2024-09-14 21:27 UTC (History)
5 users (show)

See Also:
Latest Commit:
Version Fixed In: 8.5.0
Sentry Crash Report:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description kertase 2015-07-21 09:47:38 UTC
May it be possible to integrate the api? https://www.deviantart.com/developers/
Comment 1 Maik Qualmann 2017-08-23 18:05:19 UTC

*** This bug has been marked as a duplicate of bug 306362 ***
Comment 2 caulier.gilles 2023-10-19 08:55:36 UTC
Macports has a Qt 6.4.3 package:

https://ports.macports.org/port/qt6/details/

but we needs Qt 6.5.3 LTS to support Qt6::Multimedia based on ffmpeg. There is a blocking report :

https://trac.macports.org/ticket/66975

A PR is now commited to support Qt 6.5.0 : 

https://github.com/macports/macports-ports/pull/18186

Wait and see...

GilleS Caulier
Comment 3 caulier.gilles 2024-01-04 12:04:29 UTC
Maik,

I build a new Macbook pro M1 (Apple Silicon). CPU is Arm64. I will provide a PKG installer for this platform, as Apple has abandoned Intel CPU since 2021. The current PKG Intel will still alive until my older Macbook continue to work (currently it has a swollen battery).

The other problem is the port to Qt6. Macports still frozen to 6.4 and there is no move on the right way since a while. 

https://ports.macports.org/port/qt6/details/

Alternative is to use homebrew which has Qt 6.6.1 with Arm64 support:

https://formulae.brew.sh/formula/qt

Note : VCPKG from M$ do not support yet Apple Silicon officially.

Gilles
Comment 4 caulier.gilles 2024-01-16 09:40:27 UTC
Git commit 0385d543c59a79472440aaf2a5d9e6fe147f82b0 by Gilles Caulier.
Committed on 16/01/2024 at 10:40.
Pushed by cgilles into branch 'master'.

Start to add HomeBrew build scripts for MacOS Silicon + Qt6

A  +380  -0    project/bundles/homebrew/01-build-macports.sh
A  +118  -0    project/bundles/homebrew/02-build-extralibs.sh
A  +224  -0    project/bundles/homebrew/03-build-digikam.sh
A  +705  -0    project/bundles/homebrew/04-build-installer.sh [INFRASTRUCTURE] [INFRASTRUCTURE] [INFRASTRUCTURE] [INFRASTRUCTURE]
A  +213  -0    project/bundles/homebrew/README [INFRASTRUCTURE]
A  +8    -0    project/bundles/homebrew/TODO
A  +260  -0    project/bundles/homebrew/common.sh
A  +139  -0    project/bundles/homebrew/config.sh [INFRASTRUCTURE] [INFRASTRUCTURE] [INFRASTRUCTURE] [INFRASTRUCTURE] [INFRASTRUCTURE]
A  +4    -0    project/bundles/homebrew/data/qt.conf
A  +12   -0    project/bundles/homebrew/fixbundledatapath.sh
A  +15   -0    project/bundles/homebrew/installer/ABOUT.txt
A  +340  -0    project/bundles/homebrew/installer/GPL.txt
A  +1077 -0    project/bundles/homebrew/installer/digikam.pkgproj
A  +-    --    project/bundles/homebrew/installer/logo.png
A  +56   -0    project/bundles/homebrew/installer/releasenotes.html
A  +92   -0    project/bundles/homebrew/makeall.sh [INFRASTRUCTURE] [INFRASTRUCTURE]
A  +186  -0    project/bundles/homebrew/rll.py
A  +41   -0    project/bundles/homebrew/tools/cleanupKF5framework.sh
A  +34   -0    project/bundles/homebrew/tools/console.sh
A  +77   -0    project/bundles/homebrew/tools/installextralibsdep.sh
A  +82   -0    project/bundles/homebrew/tools/installmacportsdeps.sh
A  +39   -0    project/bundles/homebrew/tools/macports-uninstall.sh
A  +29   -0    project/bundles/homebrew/tools/macports-update.sh
A  +78   -0    project/bundles/homebrew/update.sh [INFRASTRUCTURE] [INFRASTRUCTURE]

https://invent.kde.org/graphics/digikam/-/commit/0385d543c59a79472440aaf2a5d9e6fe147f82b0
Comment 5 caulier.gilles 2024-01-16 16:46:38 UTC
Hi Maik,

Some fresh news about the HomeBrew+Apple Sillicon+Qt6 port for MacOS.

My new MacBook Pro M1 is really a rock compared to the Intel 2015 version. The CPU rarely use the fan even if compilation are intensive. Apple have really well works with Arm architecture.

HomeBrew is very powerfull, better than MAcports as package manager. All low levels dependencies are there (this is not the case with Macports). All packages are precompiled and stored in a cache from Internet. Instead of 4 hours to compile whole Qt6 & co, it take 2 minutes 

So i complete stages 1 and 2. Now i will compile digiKam and make the PKG installer.

Gilles
Comment 6 caulier.gilles 2024-01-16 16:47:52 UTC
Git commit 5202e3a655c39aea972e8bdaaeaf8158f8f79615 by Gilles Caulier.
Committed on 16/01/2024 at 17:47.
Pushed by cgilles into branch 'master'.

stage 1 and 2 are now complete with HomeBrew.

M  +8    -2    project/bundles/homebrew/02-build-extralibs.sh

https://invent.kde.org/graphics/digikam/-/commit/5202e3a655c39aea972e8bdaaeaf8158f8f79615
Comment 7 caulier.gilles 2024-01-17 07:13:55 UTC
Git commit 73ef0147abb613face432dd161cd908489e8ebf1 by Gilles Caulier.
Committed on 17/01/2024 at 08:12.
Pushed by cgilles into branch 'master'.

HomeBrew Apple Silicon Qt6: stage 3 pass completly

M  +0    -2    project/bundles/homebrew/03-build-digikam.sh

https://invent.kde.org/graphics/digikam/-/commit/73ef0147abb613face432dd161cd908489e8ebf1
Comment 8 griffiths_andy 2024-03-25 15:37:30 UTC
I was reading through the documentation for the Homebrew build, along with `01-build-homebrew.sh`, and noticed the specification of the custom Homebrew instance being placed in `/opt/homebrew`. 

Does this create a custom build somewhere else inside here (iirc the Macports build does the same, but a while since I tried it?), or will it conflict with an existing general Homebrew installation at `/opt/homebrew` for Apple Silicon machines?

I'd like to give this a go, but don't want to jeopardise my existing Homebrew install.

I also saw a comment (maybe in the mailing list) re a problem still outstanding at the packaging stage. Is this still an issue?

Thanks..
Comment 9 caulier.gilles 2024-03-25 16:50:41 UTC
Yes, packaging is still and issue, especially to re-sign all binaries after to relocate it (typically from /opt/homebrew to a relative path).

But there are more problems with homebrew that i don't seen with macports (the current packaging framework : the customization of ports with variants. This do not exists in homebrew without to patch one by one all ports. Homebrew is a puzzle from me.

This is why i prefer so far Macports than Homebrew. The only blocking point to build digiKam for Arm64 with Macport is the Qt6 version available : 6.4.x instead 6.6.x with Homebrew....

Gilles Caulier
Comment 10 Michael Miller 2024-07-27 22:38:13 UTC
It's been 4 months since the last comment.  Are there any updates on an Apple ARM port of Digikam? 

Is there a branch I can look at to help?
Comment 11 caulier.gilles 2024-07-27 22:43:55 UTC
Hi, 

There is no branch. Packaging script are in master to this directory :

https://invent.kde.org/graphics/digikam/-/tree/master/project/bundles/homebrew?ref_type=heads

Best

Gilles Caulier
Comment 12 Michael Miller 2024-08-21 20:25:49 UTC
I'm working on a MacOS ARM build of digiKam using Homebrew instead of Macports.  I'll post updates here as my work progresses. 

Current Status: I have an almost clean build using v8.4.0 tag.  There's still several issues I'm debugging to get a usable binary, but I believe most (of not all) of the issues are due to resource or library pathing.  I'm using the most recently released version to avoid introducing unstable code.  Once I have a good arm64 package of v8.4.0, I'll grab the latest from Main and slowly introduce my changes to keep the commits small.
Comment 13 caulier.gilles 2024-08-22 05:22:56 UTC
Hi,

I tried a few month ago to build the PKG installer of digiKam using homebrew instead macports on my M1 computer. It was a failure due to multiple reasons.

First, homebrew do not have a port variant option as macports. This is a very important feature that main pakages manager as macports or vcpkg supports. For digiKam this is important for Qt6 installation which do not support the mysql/mariadb qtsqlplugin by default and the qtmultimedia backend compiled with ffmpeg.

Second problem is the target directories structure generated by homebrew which is completelydifferent that macports or vcpkg and need extra works to rearrange files in the PKG to work as expected. Typically, we attempt to found a unix like treeview to hst all files by categories (or something like that. This point is really a pain and a big failure at run time when the PKG is completed.

Compiling the wole code with homebrew trough the scripts is not a problem (even if variant is not supported by homebrew). The packaging is a complex deal, but i must admit that i discover homebrew for the first time.

Best

Gilles Caulier
Comment 14 Michael Miller 2024-08-22 14:35:23 UTC
I agree.  The directory structure for Homebrew is a challenge when packaging the libraries.  I think I might have found a solution, though.  I'll be able to test it in a day or two.

Quick update: I am able to pin a specific variant of a Homebrew package, so I'm building with Qt 6.5.3_1.  Are there any other packages the need a specific version?

Overall, progress is slow, but it's progress.
Comment 15 Michael Miller 2024-08-22 14:36:13 UTC
Gilles,
Do you mind if I assign this issue to myself?
Comment 16 caulier.gilles 2024-08-22 15:30:14 UTC
yes sure, all contrib are welcome (:=)))
Comment 17 caulier.gilles 2024-08-22 15:38:08 UTC
Why Qt 6.5.3, where homebrew provides 6.7.0 ?

For the variant, look the Macports scripts :

FFMPEG needs non free and others codecs : https://invent.kde.org/graphics/digikam/-/blob/master/project/bundles/macports/01-build-macports.sh?ref_type=heads#L326

Qt needs the mysql plugin : https://invent.kde.org/graphics/digikam/-/blob/master/project/bundles/macports/01-build-macports.sh?ref_type=heads#L342

QtMultimedia needs also to be compiled with the ffmpeg support.

We need last libjpeg-turbo, not the legacy libjpeg, else decoding recent DNG files will fail. There is a report for that in bugzilla. 

Gilles
Comment 18 Thomas Friedrichsmeier 2024-08-22 16:23:55 UTC
Sorry to be adding noise to this ticket (feel free to ignore me). RKWard developer, there, wondering why you aren't using craft for the package building, which may have its issues, but integrates nicely with the KDE CD? We do have an ARM machine, too, there is even an option for notarization (which I'll still have to figure out for RKWard), and at least the Qt side of the dependency chain will generally be in an ok state, even on our not-so-strong platforms. Any special requirements that would make this a no-go for you?
Comment 19 Michael Miller 2024-08-22 17:06:15 UTC
I think the biggest blocker to using Craft package manager was I wasn't aware it existed.  :) I'll take a look at it tonight.

Thank you for the suggestion. 

Cheers,
Mike
Comment 20 Maik Qualmann 2024-08-22 18:54:09 UTC
I don't know if it has changed yet, but Craft didn't have MySQL and Heif support.

Maik
Comment 21 caulier.gilles 2024-08-22 21:22:28 UTC
For Craft, i tried few years ago to use it without success :

Mysql/mariadb support in Qt not supported. I ask to add it but it was immediately refused : this require changes with server included and many dependencies.

HEIF support not implemented with X264 and x265 codecs. About patents, this cannot be included safety.

ffmpeg do not include non free / gpl3 codecs support.

The puzzle: plenty of applications uses craft build with different needs / config / usage. This is not compatible with the digiKam requirements. We needs a stable env to bundle, not soething broken days after days because plenty of developer touch the configurations without to double check all the usage.

The CI/CD from KDE uses gitlab, whoch is limited to one hour of compilation. digiKam and 3rd party plugins + build packaging explode this limit. digiKam is 1.5 M od C++ code. If we touch something in core API, all will be recompiled and compilation cannot be completed. 

For all this problems, i focused to Macports and this take age for me to maintain and finalize packaging. Here i have 2 macbook pro : one intel and one Arm. I written ann bash scripts to package all digiKam needs, as Mysql support, codecs, relocatable install, etc.

Of course installer is not notarized, but we don't care... It's always possible to install digiKam as 3r party application outside the store.

Note: the same problematic exists with the Windows build. We don't use Craft but VCPKG from Microsoft and it do very well the job. Here 2 Windows 10 VMs run on my Linux host computer : one for Qt5, one for Qt6. 

Voilà, so for me Craft is not a safe solution for digiKam...

Gilles Caulier
Comment 22 Michael Miller 2024-08-24 00:18:28 UTC
Thank you Gilles.  I've stopped exploring Craft, and I'm continuing with Homebrew.

Update: I've given up trying to pin to a specific package variant with Homebrew, and I'm now using the latest of each package.  Three issues I've found so far.
1. Some Homebrew bottles have the dynamic library paths hard coded to /opt/homebrew instead of @rpath.  If you try to use install_name_tool on them, the code signature becomes invalid.  You need to run the codesign tool to update the signature, but since you didn't compile the binary yourself, codesign fails.  One way to fix this is to just build all the packages yourself using "brew --build-from-source install XXXXX".  This however takes a very long time.  An alternate solution is to install Homebrew to a custom directory.  By doing this, Homebrew will automatically build from source only the packages (bottles) that are hard-coded to /opt/homebrew, so it is faster.  This also has the added benefit of not "polluting" an existing Homebrew installation.
2. Not all of the needed libraries are being packaged by 04-build-installer.sh.  This is slow work going through and finding the libraries that are missing.
3. I think not all the updated binaries are being re-signed by codesign.  Again, it's slow work going through them.
Comment 23 caulier.gilles 2024-08-24 06:30:07 UTC
Hi Michael,

> 1. Some Homebrew bottles have the dynamic library paths hard coded to /opt/homebrew instead of @rpath.  If you try to use install_name_tool
> on them, the code signature becomes invalid. 

yes, i already see this information while investigating with HomeBrew.

> You need to run the codesign tool to update the signature, 

yes, the script do it here : 

https://invent.kde.org/graphics/digikam/-/blob/master/project/bundles/homebrew/04-build-installer.sh?ref_type=heads#L514

> but since you didn't compile the binary 
> yourself, codesign fails.

because binaries are not compiled on the computer, but taken from a homebrew cache on the web ?

>  One way to fix this is to just build all the packages yourself using "brew --build-from-source install XXXXX".  This however 
> takes a very long time.

Well i know that already with Macports. On my MacbookPro 2015, this take around 8 hours, especially about clang/gcc/qtwebengine build. 

> An alternate solution is to install Homebrew to a custom directory.  By doing this, Homebrew will automatically build from 
> source only the packages (bottles) that are hard-coded to /opt/homebrew, so it is faster. 

Interesting tune to know...

> This also has the added benefit of not "polluting" an
> existing Homebrew installation.

Right. The install path must be /opt/digiKam._arch_ here :

https://invent.kde.org/graphics/digikam/-/blob/master/project/bundles/homebrew/config.sh?ref_type=heads#L63

as i do with Macports already :

https://invent.kde.org/graphics/digikam/-/blob/master/project/bundles/macports/config.sh?ref_type=heads#L65

> 2. Not all of the needed libraries are being packaged by 04-build-installer.sh.  This is slow work going through and finding the libraries that are missing.

Why not all libs ?

> 3. I think not all the updated binaries are being re-signed by codesign.  Again, it's slow work going through them.

Same Q ?

Gilles
Comment 24 Michael Miller 2024-08-24 12:20:31 UTC
Hi Gilles,
Here's a list of libs I've found so far.  These are either missing or unable to be found by @rpath/[optional dir/]<libname>:
1. QtDBus.framework/Versions/A/QtDBus:  Not found.  Should be in ./opt/qt/lib/
2. libopencv_ml.410.dylib: Incorrect @rpath/libopencv_core.410.dylib.  Should be @rpath/opt/opencv/lib/libopencv_core.410.dylib
3. libgcc_s.1.1.dylib: Not found.  Should be in ./opt/gcc/lib/gcc/current/
4. libjxl_cms.0.10.dylib: Wrong location.  Should be ./opt/jpeg-xl/lib
5. QtQuickWidgets.framework/Versions/A/QtQuickWidgets: At this point I went to bed.

I also noticed we need a better app Info.plist.  Additional items:
1. architecture in LSExecutableArchitectures
2. update the LSEnvironment with HOMEBREW_PREFIX, and updated PATH
3. LSMinimumSystemVersion should match config.sh->OSX_MIN_TARGET

I have a static Info.plist I'm using for now.  I'll add it to my list of "to do" for the arm64 build.
Comment 25 Michael Miller 2024-08-28 21:24:09 UTC
I have a (mostly) working build using Homebrew.  I've just been struggling with creating a good bundle.  The 2 items I had to comment out was the dark-icons and gmic.  I'll revisit them once I figure out how to bundle.  Specifically, it's Qt.  I've tried multiple approaches to getting it to work, but still no luck.
Comment 26 caulier.gilles 2024-08-29 16:39:04 UTC
>I have a (mostly) working build using Homebrew.  I've just been struggling with creating a good bundle.  The 2 items I had to comment out was the >dark-icons 

The KDE frameworks components ?

>and gmic.  I'll revisit them once I figure out how to bundle. 

GMic-Qt and 2 others 3rd-party plugins uses the same rules to compile and install. If you face specific problems with Gmic-qt, code is hosted is my own github repository (it's not the original one that i patch myself for digiKam). So i can adjust some points if necessary.

> Specifically, it's Qt.  I've tried multiple approaches to getting it to work, but still no luck.

Any traces of the errors ?

Gilles
Comment 27 Michael Miller 2024-08-30 02:17:41 UTC
Good to know about gmic.  I'll post error logs of the gmic build tomorrow.   

The biggest problem I'm running into is Qt not finding the plugins.  If I run digikam without relocating the libraries (still hard-coded to /opt/homebrew), everything runs fine.  Once I rebase the library paths, it can't find the plugins anymore.  I've tried using the qt.conf file to point to the plugins, and putting the environment variables in the LSEnvironment section of the Info.plist, but neither works.  If I run digikam from the command line, and I have QT_PLUGIN_PATH and QT_QPA_PLATFORM_PLUGIN_PATH set in the environment it works fine as well.  I am using a different program to relocate the binaries and run install_make_tool.  I have a Python script I wrote a while ago to allow me to make a relocatable install of MariaDB.  I modified RelocatableBinaries in common.sh to use my tool.  It's a bit slower, but it is giving me the results I need. 

Can you think of any reason why it's not picking up the settings in qt.conf?  Per the Qt documentation, I've put qt.conf in Contents/Resources.  I've also tried the bundle root, and Contents/MacOS so it's beside the main binary.  It's just not picking up the settings.  I think my next step is to build Qt from source so I can debug into it to see why it's not using the settings in qt.conf.

As for the dark-icons, the cmake file is completely different, so the patch doesn't work.  I've looked at it, and I think I can figure it out with a little time.

if I have time, I'll clone digikam into my own repo so you can see my changes.

One final note, total build time is less than an hour in my UTM arm64 dev VM from 01-04 if Homebrew is installed in /opt/homebrew.  So that's a nice bonus.
Comment 28 Michael Miller 2024-08-30 23:46:21 UTC
A forked the project into my own repo.  I changed about 15 or 16 files, so I thought this might be easier for you to take a look.  I'm still having path issues with Qt.  Maybe you will have better luck.

https://invent.kde.org/michmill/digi-kam-mac-os-arm-64-michmill
Comment 29 caulier.gilles 2024-08-31 06:46:55 UTC
Hi Michael,

Ok i will take a look to your git fork asap.

> total build time is less than an hour in my UTM arm64 dev VM from 01-04 if Homebrew is installed in /opt/homebrew

It's normal as you use directly the pre-compiled version of the packages. But in fact we cannot do it as the Qt6Mysql plugin is not provided in precompiled version (at least i seen in my previous tests 2 months ago). So whole Qt6 must be recompiled and it's take age.

About the gmic-qt plugin, it's possible that binary file is not installed at the right place and must be moved by the script before packaging.
The main difference between digiKam plugins and 3rdparty plugins is the KDE framework dependency which is dropped for simplification. In the KDE framework the ECM script is used in digiKam to identify the install path of plugins. In the 3rd-party, a simplified CMake base rule is used instead, and i already seen few differences. Code relevant is here :

In digiKam : https://invent.kde.org/graphics/digikam/-/blob/master/core/cmake/macros/MacroDPlugins.cmake?ref_type=heads#L93

Note the ${KDE_INSTALL_FULL_PLUGINDIR} variable taken from ECM.

In GMic-Qt : https://github.com/cgilles/gmic-qt/blob/master/src/editor/EditorPluginRules.cmake#L46

Note the ${QT_PLUGINS_DIR} variable taken from CMake.

Gilles
Comment 30 Michael Miller 2024-09-01 19:04:10 UTC
Hi Gilles,

> It's normal as you use directly the pre-compiled version of the packages. But in fact we cannot do it as the Qt6Mysql plugin is not provided in precompiled version (at least i seen in my previous tests 2 months ago). So whole Qt6 must be recompiled and it's take age.

Doesn't including qt-mysql or my-mariadb fix this?  Look at the module log, I can see the qt-mariadb plugin being loaded.

>Note the ${KDE_INSTALL_FULL_PLUGINDIR} variable taken from ECM.
> Note the ${QT_PLUGINS_DIR} variable taken from CMake.

I'm looking for a way for us to modify these paths at run-time.  According to the Qt documentation I should be able to do this for Qt with the qt.conf file.  Do you know if there is a similar file for KF?
Comment 31 caulier.gilles 2024-09-02 08:20:43 UTC
> Doesn't including qt-mysql or my-mariadb fix this?  Look at the module log, I can see the qt-mariadb plugin being loaded.

Well in this case it's perfect...

>Do you know if there is a similar file for KF?

I think the KDE framework ECM will use the Qt variable as well as it's and extension of. Look here :

https://invent.kde.org/frameworks/extra-cmake-modules/-/blob/master/kde-modules/KDEInstallDirs6.cmake?ref_type=heads#L16

Gilles
Comment 32 Michael Miller 2024-09-08 14:19:29 UTC
Hi Gilles,
I've been continuing to work on a Homebrew based arm64 build of digiKam.  I'm still struggling with setting the runtime paths to libraries, plugins, translations, and other correctly.  I've looked at all the makefiles and the scripts, but I just can't figure out which flags are used at compile time, and which flags are embedded in the code to be used at runtime.  The only solution I've found so far is by using several environment variables to redirect Qt and other modules at runtime.  

If you have time, can you take a look at my private repo to see what I'm doing wrong? Use the "fix_bundle_01" branch, and look at README-MM in ./project/bundles/homebrew for additional details.

https://invent.kde.org/michmill/digi-kam-mac-os-arm-64-michmill.git
Comment 33 caulier.gilles 2024-09-08 14:47:32 UTC
With cmake, to check which flags are used at compile time, just use "export VERBOSE=1". This will generate a very verbose output on the console while compiling code and show all flags passed to compiler.

yes i will look on your repo when time permit. I must complete an important patches before in core implementation.

Gilles
Comment 34 Michael Miller 2024-09-08 17:01:26 UTC
Hi Gilles,
I got most of the Qt paths working in the past hour.  It's now picking up the values in the ./Resources/qt.conf file, so that's fixed.  I'm working on QtWebEngine and ImageMagick paths now. 

The latest commit to the fox_bundle_01 branch has my changes.
Comment 35 griffiths_andy 2024-09-08 17:36:45 UTC
I've been subscribed to this bug for a while now, and I just thought I'd leave some thanks for the effort you're putting into this! I always had trouble trying to do a Macports build, primarily because I prefer to use Brew and I don't want to mix things up, so I will be looking forward to you getting this completed.

Thank You!
Andy
Comment 36 caulier.gilles 2024-09-08 18:30:43 UTC
For the ImageMagick path, look the AppImage startup scrip :

https://invent.kde.org/graphics/digikam/-/blob/master/project/bundles/appimage/data/AppRun?ref_type=heads#L101

Gilles
Comment 37 Michael Miller 2024-09-09 02:39:04 UTC
Final update for today. App package and package installed DK starts. DK is finding most of the libraries.  Running the app from the installer works. ImageMagick works, as do most of the tools.  It's almost usable as an installed app.

Current issues:
* Still can't figure out how to load the kxmlgui5/digikam/*.rc files from within the app bundle.  Any help would be greatly appreciated.
* Even when placing the kxmlgui5/digikam/*.rc files in ~/Library/kxmlgui5/digikam, the UI is still not correct.  Icons are missing and the UI proportions are off.  Again, any help would be appreciated.
* Can't start exiftool.  I can run it from the command line.  DK can find it but not start it. 
* No available spell-check back-ends.  It appears sonnet can't find the language files. 
* Need to update MariaDB bundling steps.

I feel like the remaining issues are just simple pathing problems.  Hopefully you can find some time to look, Gilles.  Thank you for all your time.
Comment 38 caulier.gilles 2024-09-09 05:32:48 UTC
For the spellcheck backend, i can see the same behavior.

In fact the backend is here, but not the dictionaries data files. It's a packaging problem with VCPKG which do not provide the files. PErhaps there is an issue with the dat afile license or something like that, i don't yet investigated this point.

The Macport with Qt 6.7.2 building is under progress. I found the solution with the Perl 5 dependency cycle, which is in fact a git install deps to drop with Macports. If the Macports install is complete, we will be able to compare the PKG from both package managers.

Gilles
Comment 39 caulier.gilles 2024-09-09 05:39:46 UTC
>* Still can't figure out how to load the kxmlgui5/digikam/*.rc files from within the app bundle.  Any help would be greatly appreciated.
>* Even when placing the kxmlgui5/digikam/*.rc files in ~/Library/kxmlgui5/digikam, the UI is still not correct.  Icons are missing and the UI >proportions are off.  Again, any help would be appreciated.

Install the current digiKam Qt5 X86 PKG and look like the rc files are installed here in the bundle :

# pwd
/Applications/digiKam.org/digikam.app/Contents/Resources/kxmlgui5/digikam
# ls -al
total 64
drwxr-xr-x  7 root  wheel   224 May  7 22:43 .
drwxr-xr-x  4 root  wheel   128 Jan 10  2024 ..
-rw-r--r--  1 root  wheel  6307 May 21  2023 digikamui5.rc
-rw-r--r--  1 root  wheel  5359 May 21  2023 imageeditorui5.rc
-rw-r--r--  1 root  wheel  3867 May 21  2023 importui5.rc
-rw-r--r--  1 root  wheel  4247 May 21  2023 lighttablewindowui5.rc
-rw-r--r--  1 root  wheel  2812 May 21  2023 queuemgrwindowui5.rc

Gilles
Comment 40 caulier.gilles 2024-09-09 05:42:32 UTC
>* Can't start exiftool.  I can run it from the command line.  DK can find it but not start it. 

In the x86 bundle, ExifTool is installed here :

# pwd
/Applications/digiKam.org/digikam.app/Contents/bin
# ls -al
total 320
drwxr-xr-x   8 root  wheel     256 May  7 22:43 .
drwxr-xr-x  10 root  wheel     320 May  7 22:44 ..
lrwxr-xr-x   1 root  wheel      22 May  7 22:43 Image-ExifTool -> ./Image-ExifTool-12.84
drwxr-xr-x  17 root  wheel     544 May  7 22:43 Image-ExifTool-12.84
lrwxr-xr-x   1 root  wheel      25 May  7 22:43 exiftool -> ./Image-ExifTool/exiftool
-rwxr-xr-x   1 root  wheel  346440 May  7 18:45 ffmpeg
-rwxr-xr-x   1 root  wheel   69096 May  7 18:45 kbuildsycoca5
-rwxr-xr-x   1 root  wheel  116648 May  7 18:45 solid-hardware5

Gilles
Comment 41 caulier.gilles 2024-09-09 05:45:39 UTC
Don't forget one important point : the data install path for Qt is patched while compiling KDE framework, digiKam core, and plugins. It's a long long long story about a Qt5 bug under MacOS. For Qt6, i don't know if the problem have been fixed. There is a script to patch source codes before compiling :

https://invent.kde.org/graphics/digikam/-/blob/master/project/bundles/macports/fixbundledatapath.sh?ref_type=heads

Gilles
Comment 42 Michael Miller 2024-09-09 13:29:42 UTC
Hi Gilles,

>Install the current digiKam Qt5 X86 PKG and look like the rc files are installed here in the bundle :
>/Applications/digiKam.org/digikam.app/Contents/Resources/kxmlgui5/digikam
Yes, I have the files in the same place in my bundle, but digiKam isn't looking for them there.  I used Inspector to look at all the file fstats and file opens.  DK isn't trying to look for the .rc files in ./Resources/xmlgui5/digikam.

>Don't forget one important point : the data install path for Qt is patched while compiling KDE framework, digiKam core, and plugins. It's a long long long story about a Qt5 bug under MacOS. For Qt6, i don't know if the problem have been fixed. There is a script to patch source codes before compiling :
YES!  This is it.  fixbundledatapath.sh is being copied and added to the CONFIGURE_COMMAND in the ext_kf5/CMakeLists.txt, but it's NOT in ext_kf6/CMakeLists.txt.

>In the x86 bundle, ExifTool is installed here :
>/Applications/digiKam.org/digikam.app/Contents/bin
ExifTool is in the same place in my bundle.  Again, DK isn't looking there.  I have the bin directory defined in my qt.com as Binaries=bin.  Using Inspector, I can see the DK is just using the PATH, and not looking in the Qt defined bin directory.
Comment 43 Michael Miller 2024-09-09 13:49:21 UTC
I meant to say to the PATCH_COMMAND, not CONFIGURE_COMMAND.
Comment 44 Michael Miller 2024-09-09 14:35:29 UTC
I created a new branch from digikam master to update the ext_kf6 CMakeFile.txt so it includes fixbundledatapath.sh, but I'm unable to push my new branch because I'm not a project member.  Any chance you can add me?
Comment 45 caulier.gilles 2024-09-09 15:10:12 UTC
Michael,

Usually, the forked project contributions must be merged back to the master using merge request mechanism from gitlab. Create a merge request form your project and the workflow will appears to the digiKam project for review and integration...

If you want to contribute to digiKam on a more long time, yes you can be add to the developers team from the master project, but we have no admin right privilege to do it. To add a new member, a KDE core admin request must be send by the phabricator.kde.org. Let's me hear if i need to process in this way.

Best 

Gilles
Comment 46 Michael Miller 2024-09-10 02:32:26 UTC
Hi Gilles,
I saw your comment over on bug https://bugs.kde.org/show_bug.cgi?id=491857.  I wanted to let you know about a week ago I spent about 1.5 days on the Orbit2 bug and could not figure it out, so I went back to using Homebrew.  My thought is I'm probably 90% there using Homebrew, and I can't even get through script 01-build-macports.  Plus, Homebrew is so much faster to set up.

As for the issues I'm working on, thank you for the info about fixbudnledatapath.sh.  Once I updated the ext_kf6 CMakeFile.txt to call the patch, all the kxmlgui5 files are loading from the bundle.  Thank you again.

I'm in progress of fixing sonnet/hunspell dictionary issues ([kf.sonnet.core] No language dictionaries for the language: en_US).  Homebrew doesn't provide dictionaries with Sonnet or Hunspell, so I'm downloading them separately.  At the moment I'm using https://github.com/LibreOffice/dictionaries.  Is there a list of languages I should install?

I could use some help with loading icons.  DXmlGuiWindow::setupIconThere finds both Breeze and Breeze-dark icon .rcc files, it loads the .rcc files, but the icons aren't displayed in the UI.  Breeze and Breeze-Dark are not options in Settings->Miscellaneous->Appearance tab. Are other files needed more than the .rcc files?  Could this be an issue with Qt6/KF6?

Cheers,
Mike
Comment 47 caulier.gilles 2024-09-10 02:52:41 UTC
The problem with Orbit 2 is the autoconf detection of target OSX version... As usual the autoconf dinosaure come form another age and it's really  time to drop definitively this kind of framework to compile open source programs.

Deps : qtwebengine -> qtpositionning -> gconf -> orbit2. I dropped gconf deps in positioning deps of webengine, and in fact it compile now. SO wait and see:

--->  Cleaning qt6-mysql-plugin
--->  Scanning binaries for linking errors
--->  No broken files found.
--->  No broken ports found.
Portfile for qt6-qtwebengine changed since last build; discarding previous state.
--->  Computing dependencies for qt6-qtwebengine
--->  Dependencies to be installed: qt6-qtpositioning
--->  Fetching distfiles for qt6-qtpositioning
--->  Attempting to fetch qtpositioning-everywhere-src-6.7.2.tar.xz from https://distfiles.macports.org/qt6
--->  Verifying checksums for qt6-qtpositioning
--->  Extracting qt6-qtpositioning
--->  Configuring qt6-qtpositioning
--->  Building qt6-qtpositioning
--->  Staging qt6-qtpositioning into destroot
--->  Installing qt6-qtpositioning @6.7.2_0
--->  Activating qt6-qtpositioning @6.7.2_0
--->  Cleaning qt6-qtpositioning
--->  Fetching distfiles for qt6-qtwebengine
--->  Attempting to fetch qtwebengine-everywhere-src-6.7.2.tar.xz from https://distfiles.macports.org/qt6
--->  Verifying checksums for qt6-qtwebengine
--->  Extracting qt6-qtwebengine
--->  Configuring qt6-qtwebengine
--->  Building qt6-qtwebengine
...

In fact, we don't care about gnome/orbit2 deps in Qt under MacOS...
Comment 48 caulier.gilles 2024-09-10 02:56:07 UTC
About spellcheck dictionaries, well i think this must be downloaded in user demand from client application. LibreOffice for ex do not bundle dics in installer. You need extra download for that. We need certainly the same in digiKam. So i recommend to delay this point for the moment.

Gilles
Comment 49 caulier.gilles 2024-09-10 03:04:23 UTC
About the Breeze icon with KF6 : yes there is a problem, already reported with the Qt6 AppImage. Look this report :

https://bugs.kde.org/show_bug.cgi?id=487799#c13
Comment 50 Michael Miller 2024-09-10 03:41:55 UTC
Hi Gilles,
Your suggestions sound good. I’ll wait until later to work on the spell checker and the icons.  Since I have a working build, I’ll start doing pull requests of my changes small pieces at a time to get them into master. I’ll start with ext_kf6 tomorrow. Let me know how quickly you want the changes. I don’t want to overload the team with a bunch of merge requests at once, nor is it a good idea to submit one massive change.

Do the tests work on Mac? I have working Mac ARM64, Mac x86-64, and Kubuntu x86-64 dev environments, so I’d like to test my changes as many ways as possible before I do the PR.
Comment 51 caulier.gilles 2024-09-10 05:51:48 UTC
Hi Michael,

The Arm64 macOS version need only to be tested on M* based computer. Do not try to build an universal version, this will bloat the bundles for nothing as a x86_64 version already exists. It's better to provide only for one architecture by bundle to prevent side effects. Also the Intel version work fine with Rosetta2 emulator on Arm, so this is a safe alternative.

Else, as developer, i mostly work 70% of time under Linux and i test under Windows 10 VM with Qt5 and Qt6 version. Note that the days are numbers about the Qt5 version.

Under Linux i use Kubuntu 22.04 and 24.04. The last one is the host computer, the first one a VM for regression tests.

Linux AppImage are compiled under xubuntu 20.04 for Qt5 and xubuntu 22.04 for Qt6. Both run in dedicated VMs.

The Macos Intel is compiled with Macports under my MacBook Pro 2015. Another Macbook Pro 2021 M1 is my lead computer to code and test everywhere. I use a UTM VM arm64 running Ubuntu 24.04 migrated to Kubuntu 24.04. It work perfectly (Arm CPU rock really so far compared to Intel).

When you talk about "tests", you want mean the unit-tests suite from core/tests/*. It must be, but i don't check this point at run time since a while.

About the MR, please use atomic patch by feature, not too huge, but also not too small (human compatible workflow (:-))). Note that in all cases the MR will be sync with master code before to be merged, as all the days, the i18n are pushed and updated in master repo. So if a review take age, a sync is mandatory.

Gilles
Comment 52 Michael Miller 2024-09-10 10:32:39 UTC
Hi Gilles,
>Do not try to build an universal version, this will bloat the bundles for nothing as a x86_64 version already exists. 

No, my plan is to do at least 3 different builds Mac ARM64, Mac x86-64, Linux x86-64 (AppImage) and minimally test each one to make sure I didn't break anything.

My build machines:
* Mac ARM64 - Sonoma (M1)
* Mac x86-64 Sonoma (Core I9) (I don't have an older Mac with an old version of the OS)
* Linux x86-64 - Kubuntu 24.02 (VM on Core I9 Mac)

Since almost all my changes are to the 01-04 build Bah files, CMake and patch files I think this should be sufficient to ensure I didn't break anything.  I don't think I had to change any source files other than core/app/main/main.cpp.
Comment 53 Michael Miller 2024-09-10 20:58:02 UTC
Hi Gilles,
I submitted the first MR.  It's for calling fixbundledatapath.sh as part of patching ext_kf6.  Please take a look and accept when you can, it should help you with Macports when you get to this point.

I appears other contributors are submitting commits after I submitted my MR, so while the MR was all green when I submitted it, it's now red because it's at least 1 commit behind.
Comment 54 caulier.gilles 2024-09-11 12:44:25 UTC
Michael,

As now my Macports Qt6 based is fully installed, i can compile digiKam with units tests. I fixed all broken build in core and tests implementation. So typically, the unit-tests can be processed under MacOS if necessary. I hope that run time will be fine with Apple computers, if application is not yet installed on the system....

Anyway, the Linux unit tests workflow with the CI will be enough in all cases. We need at least to append a new job using Qt6. The gitlab config in the digiKam repo must be patched and a new KDE admin request using phabricator must be open to branch the digiKam project with the Qt6 based VM.

Gilles
Comment 55 caulier.gilles 2024-09-14 21:27:41 UTC
Next digiKam 8.5.0 macOS bundles will compiled in native for Apple Silicon (arm64 architecture).
It will use Qt 6.7.2 and KDE framework 6.5.0.

Screenshot: https://i.imgur.com/BAuRI4i.png