Bug 422015 - Canon THM files (and likely others) are ignored by Sidecar management unless named very specifically
Summary: Canon THM files (and likely others) are ignored by Sidecar management unless ...
Status: RESOLVED FIXED
Alias: None
Product: digikam
Classification: Applications
Component: Metadata-Sidecar (show other bugs)
Version: 7.0.0
Platform: Compiled Sources Linux
: NOR normal
Target Milestone: ---
Assignee: Digikam Developers
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2020-05-24 14:49 UTC by griffiths_andy
Modified: 2020-05-25 23:51 UTC (History)
2 users (show)

See Also:
Latest Commit:
Version Fixed In: 7.0.0


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description griffiths_andy 2020-05-24 14:49:10 UTC
SUMMARY

Tested in 7.0.0-rc3, compiled as Arch AUR 'digikam-git' http://commits.kde.org/digikam/2a389849c15fb0fd5257c823e58e84822c34842e

I specified .THM files as a sidecar in Configure/Metadata/Sidecars. When I moved the main photo between albums I find that the THM file does not travel like the xmp or pp3 files do (checked on command line).

As the original extensions are upper case I specified it as upper case, but find on-rechecking the configure dialog has made it lower case. So I thought maybe it ignored them thru case sensitivity.
So I renamed the extension of my test file from .THM to .thm (matching the sidecar list), but I find that it is still ignored.

I noticed earlier that switching on commercial compatibility changes the filename of new sidecars to e.g. CRW_0930.CRW.xmp so I have tested that as below with the THM also, and found that Digikam recognises it and moves it, but in this case compatibility was still switched off. This makes me question what compatibility is doing?

I've seen https://bugs.kde.org/show_bug.cgi?id=193232 

STEPS TO REPRODUCE
1. Add THM as sidecar extension. Commercial compatibility = OFF. Note that the dialog has changed THM to thm.

Entered 'pp3 THM'. Changed to 'pp3 thm'

2. Import an album, 'Test THM 1', of Canon CRW xmp and THM. File modification dates are irrelevant for the CRW and THM, the original photos were taken in 2005.

'./Test THM 1':
total 5348
-rw-r--r-- 1 andyg andyg 5456666 Aug 13  2015 CRW_0930.CRW
-rw-r--r-- 1 andyg andyg   11406 Jan 14  2005 CRW_0930.THM
-rw-r--r-- 1 andyg andyg    2255 May 23 14:42 CRW_0930.xmp

3. Move picture to a new album 'Test THM 2'

4. Verify on command line and in file browser that the THM *does not* accompany CRW and xmp

'./Test THM 1':
total 12
-rw-r--r-- 1 andyg andyg 11406 Jan 14  2005 CRW_0930.THM

'./Test THM 2':
total 5336
-rw-r--r-- 1 andyg andyg 5456666 Aug 13  2015 CRW_0930.CRW
-rw-r--r-- 1 andyg andyg    2255 May 23 14:42 CRW_0930.xmp

5. Move picture back to 'Test THM 1'

'./Test THM 1':
total 5348
-rw-r--r-- 1 andyg andyg 5456666 Aug 13  2015 CRW_0930.CRW
-rw-r--r-- 1 andyg andyg   11406 Jan 14  2005 CRW_0930.THM
-rw-r--r-- 1 andyg andyg    2255 May 23 14:42 CRW_0930.xmp

'./Test THM 2':
total 0

6. Rename the THM file to thm to match whats in sidecar extension list.

'./Test THM 1':
total 5348
-rw-r--r-- 1 andyg andyg 5456666 Aug 13  2015 CRW_0930.CRW
-rw-r--r-- 1 andyg andyg   11406 Jan 14  2005 CRW_0930.thm
-rw-r--r-- 1 andyg andyg    2255 May 23 14:42 CRW_0930.xmp

'./Test THM 2':
total 0

6. Move picture to album 'Test THM 2'. Verify on command line and in file browser that the thm *does not* accompany CRW and xmp

'./Test THM 1':
total 12
-rw-r--r-- 1 andyg andyg 11406 Jan 14  2005 CRW_0930.thm

'./Test THM 2':
total 5336
-rw-r--r-- 1 andyg andyg 5456666 Aug 13  2015 CRW_0930.CRW
-rw-r--r-- 1 andyg andyg    2255 May 23 14:42 CRW_0930.xmp

7. Move picture back to 'Test THM 1'

'./Test THM 1':
total 5348
-rw-r--r-- 1 andyg andyg 5456666 Aug 13  2015 CRW_0930.CRW
-rw-r--r-- 1 andyg andyg   11406 Jan 14  2005 CRW_0930.thm
-rw-r--r-- 1 andyg andyg    2255 May 23 14:42 CRW_0930.xmp

'./Test THM 2':
total 0

8. Rename the thm to CRW_0930.CRW.thm to match something I'd seen re commercial compatibility naming. Although in this case compatibility is still OFF.

'./Test THM 1':
total 5348
-rw-r--r-- 1 andyg andyg 5456666 Aug 13  2015 CRW_0930.CRW
-rw-r--r-- 1 andyg andyg   11406 Jan 14  2005 CRW_0930.CRW.thm
-rw-r--r-- 1 andyg andyg    2255 May 23 14:42 CRW_0930.xmp

'./Test THM 2':
total 0

9. Move picture to album 'Test THM 2'. Verify on command line that this time the thm *has* accompanied CRW and xmp

'./Test THM 1':
total 0

'./Test THM 2':
total 5348
-rw-r--r-- 1 andyg andyg 5456666 Aug 13  2015 CRW_0930.CRW
-rw-r--r-- 1 andyg andyg   11406 Jan 14  2005 CRW_0930.CRW.thm
-rw-r--r-- 1 andyg andyg    2255 May 23 14:42 CRW_0930.xmp

10. Now to test, does the case of thm matter? Change thm to THM

'./Test THM 1':
total 0

'./Test THM 2':
total 5348
-rw-r--r-- 1 andyg andyg 5456666 Aug 13  2015 CRW_0930.CRW
-rw-r--r-- 1 andyg andyg   11406 Jan 14  2005 CRW_0930.CRW.THM
-rw-r--r-- 1 andyg andyg    2255 May 23 14:42 CRW_0930.xmp

11. Move picture back to Test THM 1. THM has not moved back, so case matters

'./Test THM 1':
total 5336
-rw-r--r-- 1 andyg andyg 5456666 Aug 13  2015 CRW_0930.CRW
-rw-r--r-- 1 andyg andyg    2255 May 23 14:42 CRW_0930.xmp

'./Test THM 2':
total 12
-rw-r--r-- 1 andyg andyg 11406 Jan 14  2005 CRW_0930.CRW.THM

12. One last test to see if xmp file moves are case-insensitive. Rename THM to thm, xmp to XMP. XMP metadata is no longer visible in Digikam (first clue to how this ends..).

'./Test THM 1':
total 5348
-rw-r--r-- 1 andyg andyg 5456666 Aug 13  2015 CRW_0930.CRW
-rw-r--r-- 1 andyg andyg   11406 Jan 14  2005 CRW_0930.CRW.thm
-rw-r--r-- 1 andyg andyg    2255 May 23 14:42 CRW_0930.XMP

'./Test THM 2':
total 0

13. Move picture to Test THM 2. XMP has not moved.

'./Test THM 1':
total 4
-rw-r--r-- 1 andyg andyg 2255 May 23 14:42 CRW_0930.XMP

'./Test THM 2':
total 5344
-rw-r--r-- 1 andyg andyg 5456666 Aug 13  2015 CRW_0930.CRW
-rw-r--r-- 1 andyg andyg   11406 Jan 14  2005 CRW_0930.CRW.thm

So, no, XMP files aren't case-insensitive either.

OBSERVED RESULT
THM files do not accompany main file when moved, even though they're specified in sidecar list. This is for 2 reasons:

1. When entering THM in sidecar list, Digikam changes case to thm. The Sidecar list is case-sensitive. This ignores all my THM files.

2. Regardless of commercial compatibility setting, only THM files named as e.g. CRW_NNNN.CRW.thm are recognised, and moved. This appears not to be that case with existing xmp files.

EXPECTED RESULT
1. THM files should accompany main file when moved, as they are specified in sidecar list. Sidecar list should not translate upper to lower case, as that is an assumption, and does not match the users data. As an aside, I entered 'pp3 thm THM' as my sidecar list, and when checked it, THM was removed giving 'pp3 thm'.

2. Sidecar files should be recognised in all cases, as there should be no assumptions on the filenaming of the user's data, which could come from many sources.

3. The commercial software compatibility naming should not apply to non xmp files. They originate in the chain well before the DAM. I should not need to rename files away from the camera standard.

SOFTWARE/OS VERSIONS
Windows: 
macOS: 
Linux/KDE Plasma: Arch Linux
(available in About System)
KDE Plasma Version: 5.18.5
KDE Frameworks Version: 5.70.0
Qt Version: 5.14.2

ADDITIONAL INFORMATION

BTW thanks for a great bit of software! Getting more work done here that I ever did in other offerings.

Andy..
Comment 1 Maik Qualmann 2020-05-25 06:33:51 UTC
Git commit cc707f5b13f95d194fe9158cd9935eeb9a790d47 by Maik Qualmann.
Committed on 25/05/2020 at 06:08.
Pushed by mqualmann into branch 'master'.

first step save user sidcare extensions as case sensitive

M  +1    -1    core/utilities/setup/metadata/setupmetadata.cpp

https://invent.kde.org/graphics/digikam/commit/cc707f5b13f95d194fe9158cd9935eeb9a790d47
Comment 2 Maik Qualmann 2020-05-25 18:54:56 UTC
Git commit d7628c034fa19b49f5aaacfc0735ae6fb58e2a81 by Maik Qualmann.
Committed on 25/05/2020 at 18:52.
Pushed by mqualmann into branch 'master'.

better handling of sidecar files and fix renaming
FIXED-IN: 7.0.0

M  +2    -1    NEWS
M  +16   -3    core/libs/database/utils/ifaces/dio.cpp
M  +41   -10   core/libs/database/utils/ifaces/diofinders.cpp
M  +1    -0    core/libs/database/utils/ifaces/diofinders.h

https://invent.kde.org/graphics/digikam/commit/d7628c034fa19b49f5aaacfc0735ae6fb58e2a81
Comment 3 Maik Qualmann 2020-05-25 19:02:39 UTC
The ".xmp" file extension is still only supported in lower case. For performance reasons, especially for users of network drives, we cannot check for all possible variants.

The following variants are now supported when moving / copying or renaming sidecars:

BASENAME.EXT.xmp
BASENAME.xmp
BASENAME.EXT.UserSuffix
BASENAME.UserSuffix

Maik
Comment 4 griffiths_andy 2020-05-25 19:29:48 UTC
Hi Maik,
Thanks for your very quick response! I'll give it a test.

re. xmp, their names are fine, wasn't expecting you t do anything different with them.

Thanks

Andy
Comment 5 griffiths_andy 2020-05-25 23:51:23 UTC
Maik,
Thanks again. Just tested your changes and all is fine.