Bug 489197 - Faces problem
Summary: Faces problem
Status: RESOLVED FIXED
Alias: None
Product: digikam
Classification: Applications
Component: Database-Faces (show other bugs)
Version: 8.3.0
Platform: Microsoft Windows Microsoft Windows
: NOR crash
Target Milestone: ---
Assignee: Digikam Developers
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2024-06-25 18:42 UTC by Alexander
Modified: 2024-10-09 21:46 UTC (History)
2 users (show)

See Also:
Latest Commit:
Version Fixed In: 8.5.0
Sentry Crash Report:


Attachments
DebugView log (23.58 KB, text/plain)
2024-07-11 07:31 UTC, Alexander
Details
Another one log (179.89 KB, text/plain)
2024-07-11 19:06 UTC, Alexander
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Alexander 2024-06-25 18:42:38 UTC
***
If you're not sure this is actually a bug, instead post about it at https://discuss.kde.org

If you're reporting a crash, attach a backtrace with debug symbols; see https://community.kde.org/Guidelines_and_HOWTOs/Debugging/How_to_create_useful_crash_reports
***

SUMMARY
After migrating from sqlite to Mysql, almost all people in the People panel disappeared - about 10 people left. It is not possible to create new people by signing names under faces - this attempt is simply ignored. Unlabeled people are also not displayed.

STEPS TO REPRODUCE
1. Use sqlite to index ~200 000 photos and import faces from them.
2. Tire of crashes because of huge DB of thumbnails and migrate to local MariaDB
3. Enjoy lost face tags and impossibility to create any new person

OBSERVED RESULT
New people cannot be created, the list of people is down to 9 people.

EXPECTED RESULT
Enjoy fully navigating through your photo collection.

SOFTWARE/OS VERSIONS
Windows: 10
Comment 1 Maik Qualmann 2024-06-25 22:28:13 UTC
Without a DebugView log we won't be able to help you here. Create a DebugView log of the process of creating a new face.
Description here:

https://www.digikam.org/contribute/

By the way, it is also not normal for a SQLite DB with 200,000 images to crash frequently.

Maik
Comment 2 Maik Qualmann 2024-06-26 05:57:18 UTC
Alternatively, you could provide the digikam4.db file (compress as ZIP) so that we can investigate the problem.

Maik
Comment 3 Bug Janitor Service 2024-07-11 03:47:12 UTC
Dear Bug Submitter,

This bug has been in NEEDSINFO status with no change for at least
15 days. Please provide the requested information as soon as
possible and set the bug status as REPORTED. Due to regular bug
tracker maintenance, if the bug is still in NEEDSINFO status with
no change in 30 days the bug will be closed as RESOLVED > WORKSFORME
due to lack of needed information.

For more information about our bug triaging procedures please read the
wiki located here:
https://community.kde.org/Guidelines_and_HOWTOs/Bug_triaging

If you have already provided the requested information, please
mark the bug as REPORTED so that the KDE team knows that the bug is
ready to be confirmed.

Thank you for helping us make KDE software even better for everyone!
Comment 4 Alexander 2024-07-11 07:31:47 UTC
Created attachment 171559 [details]
DebugView log

This is the log file that was created while I was trying to create a new name in digiKam. In the photo itself, it looks like some name is assigned, but it seems to be empty. In the People section, the new people never showed up.
Comment 5 Alexander 2024-07-11 07:33:08 UTC
Comment on attachment 171559 [details]
DebugView log

This is the log file that was created while I was trying to create a new name in digiKam. In the photo itself, it looks like some name is assigned, but it seems to be empty. In the People section, the new people never showed up.
Comment 6 Alexander 2024-07-11 07:33:21 UTC
This is the log file that was created while I was trying to create a new name in digiKam. In the photo itself, it looks like some name is assigned, but it seems to be empty. In the People section, the new people never showed up.
Comment 7 Alexander 2024-07-11 07:34:20 UTC
Damn, this interface is so unintuitive!
Comment 8 Alexander 2024-07-11 07:38:05 UTC
(In reply to Maik Qualmann from comment #2)
> Alternatively, you could provide the digikam4.db file (compress as ZIP) so
> that we can investigate the problem.
> 
> Maik

Hi Maik. Sorry for the long reply - I was recreating the face database from scratch and it took a very long time, but it didn't help. DigiKam is now running using MySQL, so sending the old digikam4.db probably doesn't make sense. I can enter some command in the database console and send you the answer if that would help diagnostics.
Comment 9 Alexander 2024-07-11 08:05:54 UTC
I found a pattern! People are not created if I write their names in Cyrillic. But they do in Latin.
Comment 10 Maik Qualmann 2024-07-11 16:34:16 UTC
The log shows almost only messages from other programs, the few digiKam messages are warnings that defective JPG files are being read to create thumbnails.
Please activate internal debugging in the digiKam settings under Miscellaneous->System (restart digiKam afterwards). This will make digiKam more "talkative" in the log. And try creating a face tag with Cyrillic letters again, maybe we'll see messages from MySQL.

Maik
Comment 11 Alexander 2024-07-11 19:06:09 UTC
Created attachment 171582 [details]
Another one log
Comment 12 Alexander 2024-07-11 19:06:50 UTC
This new log should have included such an event.
Comment 13 Maik Qualmann 2024-07-11 20:03:11 UTC
I don't see any error messages in your log. I created the face tag "Антипин Сергей" in my MySQL/MariaDB database. No problems here. Is your MySQL server local or on a NAS? I suspect you have incorrect global server settings, so that the MySQL server does not use a UTF-8 code page.

Maik
Comment 14 Alexander 2024-07-11 20:51:02 UTC
(In reply to Maik Qualmann from comment #13)
> I don't see any error messages in your log. I created the face tag "Антипин
> Сергей" in my MySQL/MariaDB database. No problems here.
I don't see any errors here either. Apparently it was possible to create such a name, but... He is not in the list of people in the People section. And the person himself has an empty name under his face when viewing a photo with him. Maybe if any of the Cyrillic letters occur in a name, it can't show up in the People section?

> Is your MySQL server local or on a NAS? I suspect you have incorrect global server settings, so
> that the MySQL server does not use a UTF-8 code page.
It is my local MariaDB server.
MariaDB [digikam]> SHOW VARIABLES LIKE 'character_set%';
+--------------------------+-----------------------------------------------+
| Variable_name            | Value                                         |
+--------------------------+-----------------------------------------------+
| character_set_client     | utf8mb4                                       |
| character_set_collations |                                               |
| character_set_connection | utf8mb4                                       |
| character_set_database   | utf8mb4                                       |
| character_set_filesystem | binary                                        |
| character_set_results    | utf8mb4                                       |
| character_set_server     | utf8mb4                                       |
| character_set_system     | utf8mb3                                       |
| character_sets_dir       | C:\Program Files\MariaDB 11.4\share\charsets\ |
+--------------------------+-----------------------------------------------+
9 rows in set (0.001 sec)
Comment 15 Maik Qualmann 2024-07-12 05:50:57 UTC
Do you see the Cyrillic name directly in the database?

SELECT name FROM Tags;

Maik
Comment 16 Alexander 2024-07-12 10:38:30 UTC
(In reply to Maik Qualmann from comment #15)
> Do you see the Cyrillic name directly in the database?
> SELECT name FROM Tags;
Yes, it is there
Comment 17 Maik Qualmann 2024-07-12 10:54:32 UTC
I don't really have any other ideas at the moment. You are using Windows, it is possible to set a custom font in the digiKam settings for the tree view and others. Have you perhaps set a font that does not support Cyrillic letters?

Maik
Comment 18 Alexander 2024-07-12 11:02:28 UTC
(In reply to Maik Qualmann from comment #17)
> I don't really have any other ideas at the moment. You are using Windows, it
> is possible to set a custom font in the digiKam settings for the tree view
> and others. Have you perhaps set a font that does not support Cyrillic
> letters?

I had the system font selected. Tried changing it to one that exactly supports Cyrillic, but it didn't help - the list of people is still about 20 people instead of about 500. Looks like people are being filtered out at some stage. Also, the list of people who don't have names assigned doesn't show up.
Comment 19 Maik Qualmann 2024-07-12 14:43:29 UTC
There is another reason why so many tags are missing - something went wrong when converting from SQLite to MySQL. The error was probably already in the SQLite DB (missing parent tag or something similar). We could only investigate and possibly fix this once we have the SQLite DB.

Maik
Comment 20 Alexander 2024-07-12 16:17:16 UTC
(In reply to Maik Qualmann from comment #19)
> There is another reason why so many tags are missing - something went wrong
> when converting from SQLite to MySQL. The error was probably already in the
> SQLite DB (missing parent tag or something similar). We could only
> investigate and possibly fix this once we have the SQLite DB.
Unfortunately, I can't share the digikam4.db file because it contains real names of people whose interconnection I'm not authorized to make public.
I put a screenshot here https://keephere.ru/get/KM0iqriBZFR/o/photo.jpg, where MariaDB database is on the left and SQLite is on the right. The first values are from that tables. I can see the difference that in MariaDB there is no record with pid -1
Comment 21 Alexander 2024-07-12 16:36:09 UTC
(In reply to Alexander from comment #20)
> I can see the difference that in MariaDB there is no record with pid -1
Brrrr, the opposite. SQLite has no record with pid -1
Comment 22 Maik Qualmann 2024-07-12 17:25:44 UTC
It is normal that the SQLite database does not have an entry with -1, this is only needed in MySQL. That is not the error.

Maik
Comment 23 Alexander 2024-07-14 19:55:07 UTC
I tried migrating back from MariaDB to SQLite. The resulting digikam4.db file is 64 megabytes, although the MariaDB database is 8 gigabytes. It didn't migrate the thumbnails and probably something else. I set to re-import information from images into SQLite database.
Comment 24 Alexander 2024-07-16 21:28:43 UTC
Well, after reverse migration from MySQL (MariaDB) to SQLite the problem was solved. Yes, the thumbnails were not migrated and had to be re-created. Also, I had to re-import all the information about faces that was in all photo collections. New people can be successfully created. Apparently, there is some problem with digikam working with Cyrillic alphabet using MySQL.
Comment 25 caulier.gilles 2024-10-08 05:33:47 UTC
Hi Maik,

What's the status of this file ? From the last comment, it sounds a problem with Cyrillic characters and Mysql database. Do we need to reassign this file in the right bugzilla place. 

Gilles
Comment 26 Alexander 2024-10-09 20:32:51 UTC
Hi, I upgraded to the latest version of digikam and tried migrating from SQLite to MariaDB again. As far as I can tell, the problem I described here is gone. The People section shows Cyrillic names, everything is fine :) Thanks to those who fixed it!
Comment 27 caulier.gilles 2024-10-09 21:46:29 UTC
Thanks for the feedback. I close this file now.

Gilles Caulier