Bug 375346

Summary: Many tags disappeared from database
Product: [Applications] digikam Reporter: Andrius <aegoreev>
Component: Database-MysqlAssignee: Digikam Developers <digikam-bugs-null>
Status: RESOLVED FIXED    
Severity: critical CC: caulier.gilles, kde.bugs, mattia.verga, metzpinguin, skydivergm
Priority: NOR    
Version: 5.7.0   
Target Milestone: ---   
Platform: Microsoft Windows   
OS: Microsoft Windows   
Latest Commit: Version Fixed In: 5.8.0
Sentry Crash Report:

Description Andrius 2017-01-20 17:35:44 UTC
I just noticed that many tags disappeared from the database. They still present in images metadata but are not shown in tags tree (both left and right panels) nor under the thumbnail.

I am  syncing from images metadata back to the database right now and I see they are showing back up but the bug is pretty critical.

Sorry, I cannot tell where and how the tags disappeared because I might did not notice that.
Comment 1 Andrius 2017-01-20 17:50:54 UTC
I am already seeing that some tags coming back properly e.g. /Activities/ snowboarding but some of them showing up in the root not in the group e.g. /snowboarding instead of /Activities/ snowboarding
Comment 2 caulier.gilles 2017-01-20 17:54:38 UTC
I never seen this dysfunction before (can you imagine)...

Which kind of database type do you use ?

Gilles Caulier
Comment 3 Andrius 2017-01-20 17:57:25 UTC
MySQL MariaDB downloaded using the link digiKam provides
Comment 4 caulier.gilles 2017-01-20 18:01:35 UTC
So A mysql server, not Mysql internal. Right ?

Can you try with a fresh database using sqlite instead, to see if problem is reproducible ?

Gilles Caulier
Comment 5 Andrius 2017-02-22 18:47:28 UTC
(In reply to caulier.gilles from comment #4)
> So A mysql server, not Mysql internal. Right ?
> 
> Can you try with a fresh database using sqlite instead, to see if problem is
> reproducible ?
> 
> Gilles Caulier

Everything seems to work fine after switching to sqlite. I will keep you posted if it pops up again though.
Comment 6 caulier.gilles 2017-04-16 20:21:06 UTC
new 5.6.0 pre-release as bundle is available here :

https://drive.google.com/drive/folders/0BzeiVr-byqt5Y0tIRWVWelRJenM

Please check if this problem still reproducible with these versions.

Thanks in advance

Gilles Caulier
Comment 7 Andrius 2017-05-30 18:41:21 UTC
(In reply to caulier.gilles from comment #6)
> new 5.6.0 pre-release as bundle is available here :
> 
> https://drive.google.com/drive/folders/0BzeiVr-byqt5Y0tIRWVWelRJenM
> 
> Please check if this problem still reproducible with these versions.
> 
> Thanks in advance
> 
> Gilles Caulier

It never happened again since I switched back to SQLite. You probably were correct, it was a mySQL related issue.
Comment 8 caulier.gilles 2017-06-22 21:43:12 UTC
digiKam 5.6.0 is now released and available as bundle for Linux, MacOS and Windows.

https://www.digikam.org/news/2017-06-21-5.6.0-release-announcement/

Can you check if problem still exists with this version ?

Thanks in advance

Gilles Caulier
Comment 9 caulier.gilles 2017-07-23 18:27:50 UTC
New digiKam 5.7.0 are built with current implementation as pre-release bundles:

https://drive.google.com/drive/folders/0BzeiVr-byqt5Y0tIRWVWelRJenM

Problem still reproducible ?
Comment 10 Andrius 2017-07-24 17:35:35 UTC
(In reply to caulier.gilles from comment #9)
> New digiKam 5.7.0 are built with current implementation as pre-release
> bundles:
> 
> https://drive.google.com/drive/folders/0BzeiVr-byqt5Y0tIRWVWelRJenM
> 
> Problem still reproducible ?

Not sure I want to check this. I switched back to sqlite and never seen the issue happen again
Comment 11 Mattia 2017-09-09 07:32:16 UTC
I've hit this problem yesterday, after database migration from SQLite to MySQL external.
The migration went fine, but after changing tags to some images and moving tags from one parent to another I saw Digikam doing something strange (tags in tags manager were not updated correctly). Closed and restarted Digikam, all tags disappeared.

Probably related to bug 383326? There's also bug 383563 which seems a duplicate of this one.

I'm switched back to SQLite, will retry as soon 5.7 will be available in Fedora repositories.
Comment 12 Maik Qualmann 2017-11-10 21:05:28 UTC
Git commit 74ea34d9fa0247a91c341ee99e6f001caa8020c3 by Maik Qualmann.
Committed on 10/11/2017 at 18:35.
Pushed by mqualmann into branch 'master'.

fix updating lft/rgt columns in the Tags table with MySQL
Related: bug 386606
FIXED-IN: 5.8.0

M  +0    -11   app/dragdrop/tagdragdrop.cpp
M  +26   -10   data/database/dbconfig.xml.cmake.in
M  +10   -12   libs/database/coredb/coredb.cpp
M  +0    -7    libs/database/coredb/coredb.h

https://commits.kde.org/digikam/74ea34d9fa0247a91c341ee99e6f001caa8020c3
Comment 13 meku 2017-11-20 11:21:39 UTC
This bug occurred to me in digikam-5.8.0-20171119094605-x86-64.appimage

I removed an unused Tag tree from the Tag Manager and digikam crashed.  After re-opening digikam some other unrelated Tag trees were completely missing from the tags list.
Comment 14 Maik Qualmann 2017-11-20 11:23:48 UTC
SQLite or MySQL?

Maik
Comment 15 meku 2017-11-20 11:32:04 UTC
(In reply to Maik Qualmann from comment #14)
> SQLite or MySQL?
> 
> Maik

external MySQL 5.7
Comment 16 Maik Qualmann 2017-11-20 19:21:46 UTC
Git commit 5bd6f2fddfe1ab585a67cb3c54a81b4d6c079693 by Maik Qualmann.
Committed on 20/11/2017 at 19:20.
Pushed by mqualmann into branch 'master'.

fix deleting tag or tag tree for MySQL

M  +5    -6    data/database/dbconfig.xml.cmake.in

https://commits.kde.org/digikam/5bd6f2fddfe1ab585a67cb3c54a81b4d6c079693
Comment 17 meku 2017-11-21 10:49:58 UTC
I inspected the Tags table before and after the crash and the cause of the loss of tags may be related to corrupt lft/rgt fields (related Bug #383326)
Comment 18 SkyDiver 2017-12-15 13:00:27 UTC
Hi, I posted an issue 2 months ago and them missed Igor's request that I'll post here. So I'm posting now in case it would help (as it seems from the comments that the issue is not yet fixed).

--

After upgrading to 5.7 the Tags/People trees are empty.
(*) Since I haven't used the app in a while, I can't tell if this is a
direct result of the upgrade or not.

I rolled back to 5.6 and the behavior persisted so I re-installed 5.7.

I already lost all tagging once in v5.6 and had to re-tag everything. This
time I first took a look at the database and noticed that all the data is
there, well organized in the various tables.

So I dug deeper into the website and found how to get a dump of the trace
messages (DbgView.exe as I'm running on Windows), and then found the
following suspicious lines:

[18488] digikam.general: Failed to find parent tag for tag
"_Digikam_Internal_Tags_"  with pid  -1
[18488] digikam.general: Failed to find parent tag for tag  "People"  with
pid  -1

Took a look at the tags table, and assuming pid = parent id and assuming
only _Digikam_root_tag_ = 0, I modified the above error tags' pid to 0
(_Digikam_root_tag_).

This fixed my issue and tags got back to normal.
Comment 19 SkyDiver 2017-12-15 13:02:14 UTC
Oops.. sorry, I meant to write "Andrey's request".
Comment 20 caulier.gilles 2017-12-15 14:11:28 UTC
Please test with digital 5.8.0 pre release app image bundle instead native package
Gilles caulier
Comment 21 caulier.gilles 2017-12-15 14:12:46 UTC
5.8.0 bundles are here
https://files.kde.org/digikam/
Comment 22 Maik Qualmann 2017-12-15 15:43:49 UTC
This issue is definitely fixed in digiKam-5.8.0. Another thing is a possibly broken tag table. With digiKam-5.8.0 an export to SQLite and re-import to MySQL is possible to fix the table again. I close this bug again.

Maik