Bug 430949

Summary: digiKam-7.2.0-beta2 (macOS (DMG)) breaks existing local MySQL/MariaDB instances on upgrade
Product: [Applications] digikam Reporter: Hackintosh HD <hackintoshhd>
Component: Database-MysqlAssignee: Digikam Developers <digikam-bugs-null>
Status: RESOLVED FIXED    
Severity: major CC: carolinacorreaorozco, caulier.gilles, metzpinguin, rjvbertin
Priority: NOR    
Version: 7.2.0   
Target Milestone: ---   
Platform: macOS (DMG)   
OS: macOS   
Latest Commit: Version Fixed In: 7.2.0
Sentry Crash Report:

Description Hackintosh HD 2020-12-29 17:44:40 UTC
SUMMARY
On my MacBook Pro (17" MacBookPro5,2, macOS 10.14.6 with Security Upddate 2020-007) the upgrade from digiKam 7.2.0-beta2-20201219T112949 to digiKam 7.2.0 (official beta2 released today) has broken the MySQL / MariaDB backend.

After analyzing the issue I suspect this is caused by the installer moving the MariaDB binaries from /opt/digikam/lib/mariadb/bin/ to some place inside the digikam.app in digiKam-7.2.0-beta2 while obviously NOT upgrading the MySQL / MariaDB path variables in /Users/$user/Library/preferences/digikamrc.

STEPS TO REPRODUCE
1. Create a digiKam 7.1 installation using the local MySQL / MariaDB database variant. Launch digiKam and make sure it works. 
2. Upgrade to the just release digiKam 7.2.0-beta2.
3. After the digiKam 7.2.0-beta2 is finished, launch digiKam.

OBSERVED RESULT
digiKam can't launch the local MySQL daemon and the whole launch fails as a consequence.


EXPECTED RESULT
digiKam should launch normally.

SOFTWARE/OS VERSIONS
Windows: 
macOS: 
Linux/KDE Plasma: 
(available in About System)
KDE Plasma Version: 
KDE Frameworks Version: 
Qt Version: 

ADDITIONAL INFORMATION
Comment 1 Hackintosh HD 2020-12-29 18:29:33 UTC
Changed to Component "Databse-Mysql".
Comment 2 Hackintosh HD 2020-12-29 19:09:10 UTC
To elaborate this further: As far as I can see, versions of digiKam prior to 7.2.0-beta2 have installed MySQL / MariaDB binaries to /opt/digikam/lib/mariadb and have referenced the corresponding paths at least in the following variables in /User/$user/Library/Preferences/digikamrc:

1. MonitorProfileFile[$e]
2. WorkProfileFile[$e]
3. Internal Database Server Mysql Init Command
4. Internal Database Server Mysql Server Command

To my understanding, the installer of digiKam 7.2.0-beta2 deletes /opt/digikam … but does not change /User/$user/Library/Preferences/digikamrc accordingly, which would be necessary to preserve digiKam's ability to launch the local MariaDB server.
Comment 3 caulier.gilles 2020-12-30 09:25:39 UTC
Hi,

Yes, the new relocatable PKG for MAcOS remove older place to install digiKam : /opt/digikam. No need to use 800Mb for nothing.

Now bundle is installed in MacOS full compliant place : /Applications.
It can be also installed in other drive if necessary, if space missing.

Your problem with mysql/mariadb cli tool settings is not only relevant of MacOS version. If you uninstall Mysql/Mariadb from a Linux Computer, and you try to restart digiKam, your dysfunction will appear.

So the ultimate solution is to always check at startup if Mysql/Mariadb CLI tool are present on host computer. If not, the database settings page must be show before to continue.

Also path to found Mysql/Mariadb under MacOS must be adjusted in source code to follow new install place.

Gilles Caulier
Comment 4 caulier.gilles 2020-12-30 09:45:56 UTC
Git commit ded1be247c6bc516d456bf3028d8578ac18b4793 by Gilles Caulier.
Committed on 30/12/2020 at 09:45.
Pushed by cgilles into branch 'master'.

Fix binary paths in new relocatable macOS bundle

M  +1    -1    core/dplugins/generic/tools/expoblending/wizard/expoblendingintropage.cpp
M  +1    -1    core/dplugins/generic/tools/panorama/wizard/panointropage.cpp
M  +1    -1    core/dplugins/generic/tools/printcreator/wizard/advprintintropage.cpp
M  +1    -1    core/dplugins/generic/tools/sendbymail/wizard/mailintropage.cpp
M  +1    -3    core/libs/database/utils/widgets/dbsettingswidget.cpp
M  +2    -2    core/libs/dimg/filters/icc/iccprofile.cpp
M  +1    -1    core/utilities/setup/setupicc.cpp

https://invent.kde.org/graphics/digikam/commit/ded1be247c6bc516d456bf3028d8578ac18b4793
Comment 5 caulier.gilles 2020-12-30 09:50:13 UTC
Git commit 175d2adaed5818942be77a7ec296ee48a1a7898c by Gilles Caulier.
Committed on 30/12/2020 at 09:49.
Pushed by cgilles into branch 'master'.

Add standard Hugin bundle path to CLI tools installed under macOS

M  +4    -0    core/dplugins/generic/tools/expoblending/wizard/expoblendingintropage.cpp
M  +4    -0    core/dplugins/generic/tools/panorama/wizard/panointropage.cpp

https://invent.kde.org/graphics/digikam/commit/175d2adaed5818942be77a7ec296ee48a1a7898c
Comment 6 Maik Qualmann 2020-12-30 20:05:03 UTC
*** Bug 430985 has been marked as a duplicate of this bug. ***
Comment 7 Maik Qualmann 2020-12-30 21:43:41 UTC
Git commit ea71dc4e6aad2d2a7d04c81964864dedf11d9b7d by Maik Qualmann.
Committed on 30/12/2020 at 21:42.
Pushed by mqualmann into branch 'master'.

if the MySQL binary tools are not found, show the setup dialog

M  +48   -0    core/libs/album/manager/albummanager_database.cpp
M  +1    -0    core/libs/album/manager/albummanager_p.h

https://invent.kde.org/graphics/digikam/commit/ea71dc4e6aad2d2a7d04c81964864dedf11d9b7d
Comment 8 Hackintosh HD 2020-12-30 23:08:32 UTC
(In reply to caulier.gilles from comment #3)
> Hi,
> 
> Yes, the new relocatable PKG for MAcOS remove older place to install digiKam
> : /opt/digikam. No need to use 800Mb for nothing.
> 
> Now bundle is installed in MacOS full compliant place : /Applications.
> It can be also installed in other drive if necessary, if space missing.
> 
> Your problem with mysql/mariadb cli tool settings is not only relevant of
> MacOS version. If you uninstall Mysql/Mariadb from a Linux Computer, and you
> try to restart digiKam, your dysfunction will appear.
> 
> So the ultimate solution is to always check at startup if Mysql/Mariadb CLI
> tool are present on host computer. If not, the database settings page must
> be show before to continue.
> 
> Also path to found Mysql/Mariadb under MacOS must be adjusted in source code
> to follow new install place.
> 
> Gilles Caulier

Thank you, Gilles, for fixing the problem that fast, it's highly appreciated.

I'll definitely test the next 7.2.0-rc build from https://files.kde.org/digikam/, once it incorporates your commits and will report back (unfortunately, today's digiKam-7.2.0-rc-20201230T073859-MacOS-x86-64.pkg didn't include your commits yet as it was built a few minutes before them).

Not to be misunderstood, I think it's a really good idea to move the binaries formerly installed to /opt/digikam to the inside of digiKam's macOS app bundle and thus achieving compliance with the regular macOS app structure. That may help to avoid the casual users' "Where did that /opt-leftover-folder come from? - Nevermind, let's delete it!"-type of accidents.
Comment 9 caulier.gilles 2020-12-30 23:24:57 UTC
New digiKam 7.2.0-rc PKG will be built tomorrow morning. It will include last fixes...

Gilles Caulier
Comment 10 caulier.gilles 2020-12-31 10:13:22 UTC
New PKG bundle is now online at usual place and ready to test last changes.

Happy new year

Gilles Caulier
Comment 11 Hackintosh HD 2020-12-31 10:45:00 UTC
Tested this morning's https://files.kde.org/digikam/digiKam-7.2.0-rc-20201231T095642-MacOS-x86-64.pkg:

OBSERVATIONS

1. As expected, digiKam displays a dialog after launch that the MySQL binaries were not found and launches the corresponding setup dialog.

2. After confirming the two found binaries in their respective new locations, digiKam brings up the following error message (personal data anonymized + I'm sorry my macOS user is currently set to German and I don't know the exact English/French localization, my own French translation would be "Erreur lors du démarrage du serveur interne. Détails: Le processus de la base de données a été inopinément interrompu lors de l'etablissement de connexion."):

Fehler beim Start des internen Servers.
Details: Der Datenbank-Prozess ist während der Verbindungsherstellung unerwartet beendet worden.
Programm: /Applications/digiKam.org/digikam.app/Contents/opt/digikam.app/Contents/lib/mariadb/bin/mysqld
Argumente: --defaults-file=/Users/$MYUSERNAME/Library/Application Support/digikam/digikam/mysql.conf, --datadir=/Users/$MYUSERNAME/Nextcloud/$CLOUD.DOMAIN.TLD/$SOMESUBFOLDERWITHABLANK/digiKam/.mysql.digikam/db_data, --socket=/Users/$MYUSERNAME/Library/Application Support/digikam/digikam/db_misc/mysql.socket
Prozess-Fehler: Unknown error


ADDITIONAL INFO

In ~/Library/Preferences/digikamrc, digiKam's installer has apparently left over two variables, MonitorProfileFile[$e] and WorkProfileFile[$e], that are still referring to /opt/… :

~/Library/Preferences $ grep -i '/opt' digikamrc
MonitorProfileFile[$e]=/opt/digikam/Applications/KF5/digikam.app/Contents/Resources/digikam/profiles/srgb-d65.icm
WorkProfileFile[$e]=/opt/digikam/Applications/KF5/digikam.app/Contents/Resources/digikam/profiles/srgb-d65.icm
Internal Database Server Mysql Init Command=/Applications/digiKam.org/digikam.app/Contents/opt/digikam.app/Contents/lib/mariadb/bin/mysql_install_db
Internal Database Server Mysql Server Command=/Applications/digiKam.org/digikam.app/Contents/opt/digikam.app/Contents/lib/mariadb/bin/mysqld
Comment 12 Hackintosh HD 2020-12-31 10:57:21 UTC
ADDITIONAL COMMENTS

Please feel free to let me know if I can provide further logging / debugging info.

And just an idea, but if possible, wouldn't it be helpful to reference the MariaDB binaries by their relative instead of their absolute paths? - Since the official 7.2.0-beta2, digiKam's installer creates the subfolder /Applications/digiKam.org. With the current MariaDB binaries references as absolute paths, a MySQL / MariaDB-backended digiKam instance will already break if the user only decides to move the two app bundles out of the digiKam.org subfolder into the main Application folder again. Which IMHO is not a too-far-out-idea as it is simply the former application structure and the macOS user base may have created aliases to digiKam.app (e.g. by simply putting digiKam into the macOS dock), which now are no longer working after 7.2.0-beta2's execution.
Comment 13 caulier.gilles 2020-12-31 10:59:07 UTC
Too weird. Mariadb cli tools are script with hard-coded paths...

bash-3.2$ /Applications/digiKam.org/digikam.app/Contents/opt/digikam.app/Contents/lib/mariadb/bin/mysql_install_db

FATAL ERROR: Could not find /opt/digikam/lib/mariadb/bin/my_print_defaults

If you compiled from source, you need to either run 'make install' to
copy the software into the correct location ready for operation.
If you don't want to do a full install, you can use the --srcdir
option to only install the mysql database and privilege tables.

If you are using a binary release, you must either be at the top
level of the extracted archive, or pass the --basedir option
pointing to that location.

The latest information about mysql_install_db is available at
https://mariadb.com/kb/en/installing-system-tables-mysql_install_db
bash-3.2$ cat /Applications/digiKam.org/digikam.app/Contents/opt/digikam.app/Contents/lib/mariadb/bin/mysql_install_db
#!/bin/sh
# Copyright (c) 2000, 2013, Oracle and/or its affiliates.
# Copyright (c) 2009, 2013, Monty Program Ab
# 
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; version 2 of the License.
# 
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
# 
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA  02110-1335  USA

# This scripts creates the MariaDB Server system tables
#
# All unrecognized arguments to this script are passed to mysqld.

basedir=""
builddir=""
ldata="/opt/digikam/var/db/mariadb"
langdir=""
srcdir=""

args=""
defaults=""
defaults_group_suffix=""
mysqld_opt=""
user=""
...
Comment 14 caulier.gilles 2020-12-31 11:02:14 UTC
For the second one, which is a binary, it refer to a configuration file where all is hard coded...

Solution: when bundle is computed, patch all script and config files from mariadb to change /opt/digikam/ to /Applications/digiKam.org/

Gilles Caulier
Comment 15 Maik Qualmann 2020-12-31 11:56:58 UTC
I think the cause is the wrong directory too

/Users/$MYUSERNAME/Library/Application Support/digikam/digikam/mysql.conf

The double digikam/ is certainly wrong and is related to the QStandardPaths and is easy to fix. What is the actual path like?

Maik
Comment 16 Hackintosh HD 2020-12-31 13:22:49 UTC
(In reply to Maik Qualmann from comment #15)
> I think the cause is the wrong directory too
> 
> /Users/$MYUSERNAME/Library/Application Support/digikam/digikam/mysql.conf
> 
> The double digikam/ is certainly wrong and is related to the QStandardPaths
> and is easy to fix. What is the actual path like?
> 
> Maik

If I remember correctly, my digiKam installation was originally created by version 6.3.0 or 6.4.0. The double "digikam" folder structure was probably created by the versions' installers to separate the database files from the face recognition and the QtWebEngine files. The content of this first, subordinated "digikam" folder currently is:

~/Library/Application Support/digikam $ find .
.
./digikam
./digikam/db_misc
./digikam/file_db_data
./digikam/mysql.conf
./facesengine
./facesengine/deploy.prototxt
./facesengine/openface_nn4.small2.v1.t7
./facesengine/res10_300x300_ssd_iter_140000_fp16.caffemodel
./facesengine/shapepredictor.dat
./facesengine/yolov3-face.cfg
./facesengine/yolov3-wider_16000.weights
./QtWebEngine
./QtWebEngine/Default
./QtWebEngine/Default/blob_storage
./QtWebEngine/Default/blob_storage/56acb600-af1b-4f90-8a7e-4529428f820e
./QtWebEngine/Default/GPUCache
./QtWebEngine/Default/GPUCache/data_0
./QtWebEngine/Default/GPUCache/data_1
./QtWebEngine/Default/GPUCache/data_2
./QtWebEngine/Default/GPUCache/data_3
./QtWebEngine/Default/GPUCache/index
./QtWebEngine/Default/Network Persistent State
./QtWebEngine/Default/Platform Notifications
./QtWebEngine/Default/Platform Notifications/000003.log
./QtWebEngine/Default/Platform Notifications/CURRENT
./QtWebEngine/Default/Platform Notifications/LOCK
./QtWebEngine/Default/Platform Notifications/LOG
./QtWebEngine/Default/Platform Notifications/LOG.old
./QtWebEngine/Default/Platform Notifications/MANIFEST-000001
./QtWebEngine/Default/Session Storage
./QtWebEngine/Default/Session Storage/000003.log
./QtWebEngine/Default/Session Storage/CURRENT
./QtWebEngine/Default/Session Storage/LOCK
./QtWebEngine/Default/Session Storage/LOG
./QtWebEngine/Default/Session Storage/LOG.old
./QtWebEngine/Default/Session Storage/MANIFEST-000001
./QtWebEngine/Default/user_prefs.json
./QtWebEngine/Default/Visited Links
Comment 17 caulier.gilles 2020-12-31 13:49:31 UTC
Git commit 74e135b352ddba35f5419b45c036185cad5d457c by Gilles Caulier.
Committed on 31/12/2020 at 13:48.
Pushed by cgilles into branch 'master'.

clean up unecessary test/bench files from Mariadb

M  +3    -0    project/bundles/macports/04-build-installer.sh

https://invent.kde.org/graphics/digikam/commit/74e135b352ddba35f5419b45c036185cad5d457c
Comment 18 Maik Qualmann 2020-12-31 14:17:00 UTC
Reply to comment 16:

./digikam
./digikam/db_misc
./digikam/file_db_data
./digikam/mysql.conf

These files were created when the database was first created. This double "digikam/" subdirectory is basically wrong back then. Because QStandardPaths::GenericDataLocation under MacOS already returns the app name in the path, unlike Linux and Windows.

Maik
Comment 19 Carolina 2020-12-31 14:25:42 UTC
In order to launch Digikam, I just delete this:

/User/$user/Library/Preferences/digikamrc and digikam.plist and maybe digikam_systemrc, sorry not be sure, I just was traying.

As I said I was erase MySQL so program launch normally and fully fuctionaly
Comment 20 Maik Qualmann 2020-12-31 14:29:14 UTC
The paths in comment 11 match, the unknown error must be internal to MariaDB. Perhaps we shouldn't put MariaDB in the digiKam bundle like we do with all other platforms. The only problem is that there are very few MariaDB versions for MacOS in PKG format. MariaDB currently only offers a slightly older 10.2.x version for MacOS.

Maik
Comment 21 caulier.gilles 2020-12-31 14:38:33 UTC
Maik,

Mariadb is compiled with Macports and included in the PKG bundle. I finalize a patch for the PKG creation bash script to fix all text files from mariadb included in the bundles (script and config files).


Gilles
Comment 22 caulier.gilles 2020-12-31 14:43:52 UTC
Notes:

- there is no official PKG or DMG installer for Mariadb under macOS. Only Windows and Linux are proposed. So we need to include Mariadb compiled with Macports in the bundle, no other choice.

- Very nice: Oracle only propose for free a Mysql version for 30 days trial. Welcome to US company, seriously ! How oracle detroy an open source project !!!


Gilles
Comment 23 caulier.gilles 2020-12-31 14:48:46 UTC
Voilà, my patch much work as expected:


---------- Patch config and script files in bundle

--- Patching /Users/gilles/Devel/7.x/project/bundles/macports//Applications/digiKam.org/digikam.app/Contents/opt/digikam.app/Contents/etc/mariadb/my.cnf...
--- Patching /Users/gilles/Devel/7.x/project/bundles/macports//Applications/digiKam.org/digikam.app/Contents/opt/digikam.app/Contents/etc/mariadb/macports-default.cnf...
--- Patching /Users/gilles/Devel/7.x/project/bundles/macports//Applications/digiKam.org/digikam.app/Contents/opt/digikam.app/Contents/lib/mariadb/bin/mysql_install_db...
--- Patching /Users/gilles/Devel/7.x/project/bundles/macports//Applications/digiKam.org/digikam.app/Contents/opt/digikam.app/Contents/lib/mariadb/bin/mysql_config...
--- Patching /Users/gilles/Devel/7.x/project/bundles/macports//Applications/digiKam.org/digikam.app/Contents/opt/digikam.app/Contents/lib/mariadb/bin/mysql_secure_installation...
--- Patching /Users/gilles/Devel/7.x/project/bundles/macports//Applications/digiKam.org/digikam.app/Contents/opt/digikam.app/Contents/lib/mariadb/bin/msql2mysql...
--- Patching /Users/gilles/Devel/7.x/project/bundles/macports//Applications/digiKam.org/digikam.app/Contents/opt/digikam.app/Contents/lib/mariadb/bin/mysqld_safe...
--- Patching /Users/gilles/Devel/7.x/project/bundles/macports//Applications/digiKam.org/digikam.app/Contents/opt/digikam.app/Contents/share/mariadb/support-files/my-huge.cnf...
--- Patching /Users/gilles/Devel/7.x/project/bundles/macports//Applications/digiKam.org/digikam.app/Contents/opt/digikam.app/Contents/share/mariadb/support-files/my-innodb-heavy-4G.cnf...
--- Patching /Users/gilles/Devel/7.x/project/bundles/macports//Applications/digiKam.org/digikam.app/Contents/opt/digikam.app/Contents/share/mariadb/support-files/my-medium.cnf...
--- Patching /Users/gilles/Devel/7.x/project/bundles/macports//Applications/digiKam.org/digikam.app/Contents/opt/digikam.app/Contents/share/mariadb/support-files/mysql-log-rotate...
--- Patching /Users/gilles/Devel/7.x/project/bundles/macports//Applications/digiKam.org/digikam.app/Contents/opt/digikam.app/Contents/share/mariadb/support-files/macports/my.cnf...
--- Patching /Users/gilles/Devel/7.x/project/bundles/macports//Applications/digiKam.org/digikam.app/Contents/opt/digikam.app/Contents/share/mariadb/support-files/my-large.cnf...
--- Patching /Users/gilles/Devel/7.x/project/bundles/macports//Applications/digiKam.org/digikam.app/Contents/opt/digikam.app/Contents/share/mariadb/support-files/my-small.cnf...
--- Patching /Users/gilles/Devel/7.x/project/bundles/macports//Applications/digiKam.org/digikam.app/Contents/opt/digikam.app/Contents/share/mariadb/support-files/mysql.server...
---------- Create MacOS package for digiKam 7.2.0-rc
Target PKG file : /Users/gilles/Devel/7.x/project/bundles/macports/bundle/digiKam-7.2.0-rc-20201231T154529-MacOS-x86-64.pkg
Building Project (15:45:29) at path: /Users/gilles/Devel/7.x/project/bundles/macports/installer/digikam.pkgproj 
------------------------------------------------------------------------------ 
...

Gilles
Comment 24 caulier.gilles 2020-12-31 14:50:59 UTC
Git commit 8087748a32e09b86bbdbe9a75e9b9beefeaf9ab0 by Gilles Caulier.
Committed on 31/12/2020 at 14:49.
Pushed by cgilles into branch 'master'.

Missing to copy etc/mariadb config files in the bundle.
Patch all Mariadb scripts and config files with relocation prefix

M  +37   -0    project/bundles/macports/04-build-installer.sh
M  +2    -2    project/bundles/macports/config.sh

https://invent.kde.org/graphics/digikam/commit/8087748a32e09b86bbdbe9a75e9b9beefeaf9ab0
Comment 25 caulier.gilles 2020-12-31 15:09:59 UTC
Maik,

My Mariab patch to relocate script and config files work as expected. I can run digiKam with Mariadb under macOS.

I see a strange effect if i switch from a work sqlite db to internal mariadb (from setup dialog, not using migration tool). when i press Ok from setup dialog, mariadb database is initialized immediatly as expected, the collection appears in album tree-view, and scan for new items start. But at the same time, the Collection setup dialog page appears on the front. This is a non sense because at least one collection is set already (taken from sqlite settings).

Can you reproduce ? Here i can and not only on macOS, as Linux give the same result...

Gilles
Comment 26 Maik Qualmann 2020-12-31 15:41:23 UTC
Hi Gilles,

Yes, this behavior is also known under Linux. It is probably triggered because at first there is no root collection in the database. I'll check it out in the next few days.

Maik
Comment 27 Hackintosh HD 2020-12-31 15:57:51 UTC
(In reply to Maik Qualmann from comment #20)

Thank you, Maik, I'll try to collect relevant MariaDB debug info then.

I'm afraid I also should check how the integrated MariaDB binaries of digiKam 7.2.0-beta2 were compiled considering the fact I'm running digiKam on a MacBook Pro with an older Core2Duo CPU, which nowadays can be problematic with some projects (example: https://github.com/Homebrew/homebrew-core/issues/50411).
Comment 28 caulier.gilles 2020-12-31 17:00:05 UTC
Remember that whole Macports package are compiled under BigSur 11.1 and last Xcode. Only the binary compatibility backward to macOS 10.13.

MariaDB 5.5.68 (not Mysql) is compiled with Macports. There are few different version avaailble if necessary:

https://ports.macports.org/?search=mariadb&search_by=name

So let's me hear if we need to tune MariaDB version.

Note: i just updated again the PKG at usual place.

Gilles Caulier
Comment 29 Hackintosh HD 2020-12-31 17:54:21 UTC
(In reply to caulier.gilles from comment #28)

Merci beaucoup, Gilles, I've downloaded and installed digiKam-7.2.0-rc-20201231T170945-MacOS-x86-64.pkg now. Unfortunately, the error dialog still appears and the problem remains the same.

I've then tried to launch mysqld in verbose mode manually using the parameters from the error message:

~ $ /Applications/digiKam.org/digikam.app/Contents/opt/digikam.app/Contents/lib/mariadb/bin/mysqld --defaults-file="/Users/$MYUSERNAME/Library/Application Support/digikam/digikam/mysql.conf" --datadir="/Users/$MYUSERNAME/Nextcloud/cloud.domain.tld/$SOMEFOLDERNAMEWITHABLANK/digiKam/.mysql.digikam/db_data" --socket="/Users/$MYUSERNAME/Library/Application Support/digikam/digikam/db_misc/mysql.socket" --verbose
201231 18:44:53 [Warning] options --log-slow-admin-statements, --log-queries-not-using-indexes and --log-slow-slave-statements have no effect if --log_slow_queries is not set
201231 18:44:53 [Note] /Applications/digiKam.org/digikam.app/Contents/opt/digikam.app/Contents/lib/mariadb/bin/mysqld (mysqld 5.5.68-MariaDB) starting as process 7617 ...
201231 18:44:53 [ERROR] Can't find messagefile '/opt/digikam.org/share/mariadb/errmsg.sys'
201231 18:44:53 [ERROR] Aborting

201231 18:44:53 [Note]

To me, it looks like something goes wrong, but MariaDB remains unable to report it accurately because it doesn't find its message ressources (which still has a reference to /opt…)?
Comment 30 Hackintosh HD 2020-12-31 18:41:49 UTC
(In reply to Hackintosh HD from comment #29)

Problem partly solved by a workaround: I did a

~ $ sudo mkdir -p /opt/digikam.org/share/mariadb/ && sudo cp -p /Applications/digiKam.org/digikam.app/Contents/opt/digikam.app/Contents/share/mariadb/english/errmsg.sys /opt/digikam.org/share/mariadb/

to get around digiKam 7.2.0-beta2's obviously missing / not found localized message resources as indicated by the error message in my comment #29 above. In other words, afterwards the English errmsg.sys is the only component located under /opt/digikam.org/… again, while all other MariaDB components still reside within the app bundle.

To my surprise, this already solves the problem as you can see from the following console output:

~ $ /Applications/digiKam.org/digikam.app/Contents/opt/digikam.app/Contents/lib/mariadb/bin/mysqld --defaults-file="/Users/$MYUSERNAME/Library/Application Support/digikam/digikam/mysql.conf" --datadir="/Users/$MYUSERNAME/Nextcloud/cloud.domain.tld/$SOMEFOLDERNAMEWITHABLANK/digiKam/.mysql.digikam/db_data" --socket="/Users/$MYUSERNAME/Library/Application Support/digikam/digikam/db_misc/mysql.socket" --verbose
201231 19:30:15 [Warning] options --log-slow-admin-statements, --log-queries-not-using-indexes and --log-slow-slave-statements have no effect if --log_slow_queries is not set
201231 19:30:15 [Note] /Applications/digiKam.org/digikam.app/Contents/opt/digikam.app/Contents/lib/mariadb/bin/mysqld (mysqld 5.5.68-MariaDB) starting as process 7787 ...
201231 19:30:15 [Warning] An old style --language or -lc-message-dir value with language specific part detected: /opt/digikam.org/share/mariadb/
201231 19:30:15 [Warning] Use --lc-messages-dir without language specific part instead.
201231 19:30:15 [Warning] Setting lower_case_table_names=2 because file system for /Users/$MYUSERNAME/Nextcloud/cloud.domain.tld/$SOMEFOLDERNAMEWITHABLANK/digiKam/.mysql.digikam/db_data/ is case insensitive

So as far as I understand, there must be one final reference to /opt/digikam.org/share/mariadb/errmess.sys left that still needs to be updated.
Comment 31 caulier.gilles 2020-12-31 23:43:25 UTC
In fact, problem with mysqld can resolved simply by passing one argument to command line. look below :



bash-3.2$ ./mysqld --verbose --help
210101  0:34:32 [ERROR] mysqld: Can't find file: './mysql/plugin.frm' (errno: 20)
210101  0:34:32 [Warning] Could not open mysql.plugin table. Some options may be missing from the help text
./mysqld  Ver 5.5.68-MariaDB for osx10.13 on x86_64 (Source distribution)
Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Starts the MariaDB database server.

Usage: ./mysqld [OPTIONS]

...

  -b, --basedir=name  Path to installation directory. All paths are usually
                      resolved relative to this

...

aria-log-dir-path                                 /opt/digikam.org/var/db/mariadb/
basedir                                           /opt/digikam.org/                 <===== THIS ONE
character-sets-dir                                /opt/digikam.org/share/mariadb/charsets/
datadir                                           /opt/digikam.org/var/db/mariadb/
pid-file                                          /opt/digikam.org/var/db/mariadb/MBP-de-gilles.pid
plugin-dir                                        /opt/digikam.org/lib/mariadb/plugin/
socket                                            /opt/digikam.org/var/run/mariadb/mysqld.sock

Here, we need to pass -b /Applications/digiKam.org/digikam.app/Contents/opt/digikam.app/Contents/ to mysqld under macOS, and all must be fine...

Gilles
Comment 32 caulier.gilles 2021-01-01 09:58:39 UTC
Git commit a0c50d2763bac53fe0da3f289e28ba11179faeac by Gilles Caulier.
Committed on 01/01/2021 at 09:55.
Pushed by cgilles into branch 'master'.

Internal Mysql database server: append "--basedir=" argument to mysql_install_db and mysqld cli tools under macOS to respect relocatable bundle path.

M  +27   -2    core/libs/database/server/databaseserver.cpp
M  +3    -3    core/libs/database/server/databaseserver.h

https://invent.kde.org/graphics/digikam/commit/a0c50d2763bac53fe0da3f289e28ba11179faeac
Comment 33 Hackintosh HD 2021-01-01 12:31:53 UTC
Encore une fois merci, Gilles. Si la solution est si simple, tant mieux! :)

(In reply to caulier.gilles from comment #31)

> Here, we need to pass -b
> /Applications/digiKam.org/digikam.app/Contents/opt/digikam.app/Contents/ to
> mysqld under macOS, and all must be fine...

Yes, one should just keep in mind that any attempt from a macOS user to move digikam.app and showfoto.app from the new /Applications/digiKam.org back to /Applications will break this entire construction again due to the absolute notation of the path ('just saying, because I know quite a lot of Mac users who have the habit to eliminate subfolders from their Application folder).
Comment 34 Hackintosh HD 2021-01-01 13:31:27 UTC
Another scenario that would break the app is an installation to /Users/$USERNAME/Applications, which according to Apple's guidelines is an entirely valid install location for macOS applications.

Wouldn't it be possible to generate $BASEDIR dynamically on launch before passing it over to MariaDB?
Comment 35 caulier.gilles 2021-01-01 14:28:43 UTC
Actually, the PKG installer do not permit to install digiKam in a specific directory. Only the drive target can be used and the directory will be /Applications/digiKam.org/

But you are right, if we open PKg installer to custom target directory feature, we will take a care to pass the current executable directory as basedir.

Gilles Caulier
Comment 36 Hackintosh HD 2021-01-01 15:14:10 UTC
(In reply to caulier.gilles from comment #35)

Oops, you're right, sorry. I did not consider that I was going through a PKG installer and not just dragging the two app bundles from a DMG to my /Application folder.

Anyway, once there is a new macOS build in https://files.kde.org/digikam, I'll test it and can hopefully confirm that it also fixes my installation. Thank you again for fixing the problem so fast.
Comment 37 Hackintosh HD 2021-01-01 17:43:42 UTC
Confirmation: https://files.kde.org/digikam/digiKam-7.2.0-rc-20210101T154336-MacOS-x86-64.pkg indeed fixes the problem.

REMAINING PROBLEM

The PKG installer still did *not* update MonitorProfileFile[$e] and WorkProfileFile[$e] in my /Users/$USERNAME/Library/Preferences/digikamrc, which remain set to the now incorrect /opt/digikam/… structure:

~/Library/Preferences $ grep -i '=/opt/' digikamrc
MonitorProfileFile[$e]=/opt/digikam/Applications/KF5/digikam.app/Contents/Resources/digikam/profiles/srgb-d65.icm
WorkProfileFile[$e]=/opt/digikam/Applications/KF5/digikam.app/Contents/Resources/digikam/profiles/srgb-d65.icm

The file that should be referenced by these entries now seems to be located here:

/Applications/digiKam.org/digikam.app/Contents/opt/digikam.app/Contents/Resources/digikam/profiles/srgb-d65.icm
Comment 38 Maik Qualmann 2021-01-01 17:50:07 UTC
The problem with the monitor profiles is known and has already been reported for the Linux AppImage. The path changes with the AppImage even after every start.

Maik
Comment 39 Maik Qualmann 2021-01-01 18:30:28 UTC
Git commit 97394d1ed35d09482606b1be3e24658f03d591f1 by Maik Qualmann.
Committed on 01/01/2021 at 18:28.
Pushed by mqualmann into branch 'master'.

fix color profile paths
Related: bug 415617
FIXED-IN: 7.2.0

M  +2    -1    NEWS
M  +41   -1    core/libs/dimg/filters/icc/iccsettingscontainer.cpp

https://invent.kde.org/graphics/digikam/commit/97394d1ed35d09482606b1be3e24658f03d591f1
Comment 40 caulier.gilles 2021-01-01 18:33:22 UTC
I will rebuild the PKG installer this evening to check if all is fixed now from this file.

Gilles Caulier
Comment 41 Maik Qualmann 2021-01-01 21:37:50 UTC
Git commit df9fd6011206f44cdf0576bae1ad17feda9bb63b by Maik Qualmann.
Committed on 01/01/2021 at 21:36.
Pushed by mqualmann into branch 'master'.

better fix for the color profile path, write entry too

M  +27   -41   core/libs/dimg/filters/icc/iccsettingscontainer.cpp
M  +4    -0    core/libs/dimg/filters/icc/iccsettingscontainer.h

https://invent.kde.org/graphics/digikam/commit/df9fd6011206f44cdf0576bae1ad17feda9bb63b
Comment 42 caulier.gilles 2021-01-02 09:41:56 UTC
Voilà new PKG is available for download and test with ICC path corrections from digiKam config. 

Gilles Caulier
Comment 43 Hackintosh HD 2021-01-02 11:35:56 UTC
(In reply to caulier.gilles from comment #42)

Confirmation: digiKam-7.2.0-rc-20210101T154336-MacOS-x86-64.pkg with Maik's patch from comment #41 now also fixes MonitorProfileFile[$e] and WorkProfileFile[$e] of my installation's digikamrc.

Before executing the PKG installer:

~/Library/Preferences $ grep -i '=/opt/' digikamrc
MonitorProfileFile[$e]=/opt/digikam/Applications/KF5/digikam.app/Contents/Resources/digikam/profiles/srgb-d65.icm
WorkProfileFile[$e]=/opt/digikam/Applications/KF5/digikam.app/Contents/Resources/digikam/profiles/srgb-d65.icm


After PKG installer execution:

~/Library/Preferences $ grep -i '=opt/' digikamrc

~/Library/Preferences $ grep -i '/opt/' digikamrc
LastSpecifiedAssignProfile[$e]=/Applications/digiKam.org/digikam.app/Contents/opt/digikam.app/Contents/Resources/digikam/profiles/
MonitorProfileFile[$e]=/Applications/digiKam.org/digikam.app/Contents/opt/digikam.app/Contents/Resources/digikam/profiles/srgb-d65.icm
WorkProfileFile[$e]=/Applications/digiKam.org/digikam.app/Contents/opt/digikam.app/Contents/Resources/digikam/profiles/srgb-d65.icm
Internal Database Server Mysql Init Command=/Applications/digiKam.org/digikam.app/Contents/opt/digikam.app/Contents/lib/mariadb/bin/mysql_install_db
Internal Database Server Mysql Server Command=/Applications/digiKam.org/digikam.app/Contents/opt/digikam.app/Contents/lib/mariadb/bin/mysqld

~/Library/Preferences $


Once again thanks to both of you for fixing the problem that fast.
Comment 44 Hackintosh HD 2021-01-02 11:38:54 UTC
(In reply to Hackintosh HD from comment #43)

> Confirmation: digiKam-7.2.0-rc-20210101T154336-MacOS-x86-64.pkg with Maik's
> patch from comment #41 now also fixes MonitorProfileFile[$e] and
> WorkProfileFile[$e] of my installation's digikamrc.

Sorry, wrong reference due to wrong browser tab - the PKG installer file is of course "digiKam-7.2.0-rc-20210102T101252-MacOS-x86-64.pkg" of today, January 2, 2021.
Comment 45 caulier.gilles 2021-01-02 13:10:17 UTC
Thanks for your feedback. I can close this file now and continue the macOS history to new files from bugzilla.

Gilles Caulier
Comment 46 caulier.gilles 2021-01-02 14:31:41 UTC
The story will continue to bug #431003 where i will simplify the internal bundle architecture. We have legacy applets to remove. See my comment #3

https://bugs.kde.org/show_bug.cgi?id=431003#c3

Gilles Caulier
Comment 47 Hackintosh HD 2021-01-03 09:48:19 UTC
(In reply to caulier.gilles from comment #46)
> The story will continue to bug #431003 where i will simplify the internal
> bundle architecture. We have legacy applets to remove. See my comment #3
> 
> https://bugs.kde.org/show_bug.cgi?id=431003#c3
> 
> Gilles Caulier

I've just tested this morning's macOS build https://files.kde.org/digikam/digiKam-7.2.0-rc-20210103T072125-MacOS-x86-64.pkg and surprisingly it brings up the following GUI warning again on launch, although that problem had already been fixed with digiKam-7.2.0-rc-20210102T101252-MacOS-x86-64.pkg:

" The MySQL binary tools are not found, please set the correct location in the next dialog. "
Comment 48 Hackintosh HD 2021-01-03 09:51:28 UTC
Addendum: After confirming the location of the MariaDB binaries, however, everything seems to be okay (digiKam launches, database is populated with expected RAW images).
Comment 49 caulier.gilles 2021-01-03 09:56:08 UTC
The beavior is normal, as the internal Unix hierarchy as changed in the macOS bundle. I simplifed a lots the structure, so the Mysql binaries place going wrong, and confirming the place fix it at startup.

So it's normal (i see the same happening on my computer).

Please confirm if bug #431003 is fixed or not. For me yes it is...

Gilles Caulier
Comment 50 Hackintosh HD 2021-01-03 10:43:29 UTC
I just wondered if that GUI dialog was intentional as my digiKam instance already was under the /Applications/digiKam.org/digikam.app… -structure before executing the PKG installer. But if it is intented, then of course the bug is fixed.
Comment 51 Hackintosh HD 2021-01-03 10:52:36 UTC
(In reply to caulier.gilles from comment #49)

> Please confirm if bug #431003 is fixed or not. For me yes it is...

Sorry, I'm a bit confused which bug is meant - this one here (#430949) or indeed the one with the two app icons at launch (#431003)? - If it's the second one, there are no new PKG installers at https://files.kde.org/digikam yet that I could test.
Comment 52 caulier.gilles 2021-01-03 11:31:51 UTC
yes, the behavior is normal. when i talk about Unix hierarchy, i want mean the internal Unix structure.

Look the diff of the README:

https://invent.kde.org/graphics/digikam/-/commit/7b15b8b5b2fe95b5774e2598553370d57fe6d86c

Internal paths to access to Mysql binaries has changed and simplified. This is why the dialog appear.

Gilles Caulier
Comment 53 RJVB 2021-04-27 16:52:14 UTC
FWIW, the fix for this assumes (hardcodes) a fixed install location (/Applications/digiKam.org/digikam.app) and breaks my MacPorts build, and indeed any other build that aims to be as "Unixy" as possible (i.e. not making a self-sufficient relocatable app bundle but one that uses libraries and helpers from shared location such as /opt/local).

The former issue can be fixed with QCoreApplication::applicationFilePath() and QDir::cdUp() (twice), the latter can only be fixed by patching.

Might I suggest that instead of adding the mysql --basedir argument `#ifdef Q_OS_MACOS`, you define a token for your own dist builds and check for that here? Fixing this particular regression (for me) was easy but how many remain hidden?