Bug 384577 - Rename of video files stalls and must click abort due to XMP sidecar
Summary: Rename of video files stalls and must click abort due to XMP sidecar
Status: RESOLVED FIXED
Alias: None
Product: digikam
Classification: Applications
Component: Metadata-Sidecar (show other bugs)
Version: 5.6.0
Platform: macOS (DMG) macOS
: NOR normal
Target Milestone: ---
Assignee: Digikam Developers
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2017-09-11 00:30 UTC by Geoff King
Modified: 2017-12-29 17:32 UTC (History)
2 users (show)

See Also:
Latest Commit:
Version Fixed In: 5.8.0
Sentry Crash Report:


Attachments
shows dialog (63.81 KB, image/png)
2017-09-11 00:30 UTC, Geoff King
Details
dialog box (40.79 KB, image/png)
2017-09-11 00:30 UTC, Geoff King
Details
screenshot of error (226.95 KB, image/png)
2017-12-28 21:40 UTC, Geoff King
Details
xmp file (2.26 KB, application/octet-stream)
2017-12-28 21:43 UTC, Geoff King
Details
logfile from terminal using lldb (10.73 KB, text/plain)
2017-12-28 23:42 UTC, Geoff King
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Geoff King 2017-09-11 00:30:03 UTC
Created attachment 107789 [details]
shows dialog

I have an issue where renaming of video files stalls and must be aborted.  One or some of the video files does get renamed, but the remaining do not. This only affects the video files, pictures are renamed correctly. 
For example:
-select movie files
-F2 to invoke rename dialog.(see screenshot)
-results in one of the movie files being renamed and a progress dialog stuck until click abort (screenshot)
-the file in question does get renamed, but the remaining files do not. 

I believe I submitted a bug report same or similar to this in the past, but cannot find it now.
Comment 1 Geoff King 2017-09-11 00:30:34 UTC
Created attachment 107790 [details]
dialog box
Comment 2 Geoff King 2017-09-11 00:37:49 UTC
I can add that this does happen maybe 80% of the time for me today. 
It does not need to be multiple files,  doing a F2 rename on a single file also can trigger it. 
These are MOV files from a Canon 70D, that view fine within digicam.
Comment 3 Maik Qualmann 2017-09-12 10:27:55 UTC
If you used a single file and the dialog box is waiting. Do you see a thumbnail in the dialog box and the path to the file?

Maik
Comment 4 Geoff King 2017-09-12 23:39:33 UTC
(In reply to Maik Qualmann from comment #3)
> If you used a single file and the dialog box is waiting. Do you see a
> thumbnail in the dialog box and the path to the file?

Yes
Comment 5 caulier.gilles 2017-09-13 06:29:18 UTC
Another point : Do you use video file metadata in the rename settings ?

The video metadata do not work with current Exiv2. Just to be sure...
Comment 6 Geoff King 2017-09-13 16:39:02 UTC
(In reply to caulier.gilles from comment #5)
> Another point : Do you use video file metadata in the rename settings ?
> 
> The video metadata do not work with current Exiv2. Just to be sure...

I'm only using the date and filename.  Such as [date:"yyyyMMdd"]_[file]
See the first attachment.
Comment 7 caulier.gilles 2017-09-13 16:49:51 UTC
video date taken is problematic here using Exiv2. This can give errors.

Please try to seen if problem occurs without date handling. Just to identify if problem is relevant of date.
Comment 8 Maik Qualmann 2017-09-13 18:13:51 UTC
Git commit 71210daa7b640acc1e48172966c5e1edbd28b23c by Maik Qualmann.
Committed on 13/09/2017 at 18:12.
Pushed by mqualmann into branch 'master'.

try to fix the renaming problem

M  +1    -0    utilities/advancedrename/advancedrenameprocessdialog.cpp

https://commits.kde.org/digikam/71210daa7b640acc1e48172966c5e1edbd28b23c
Comment 9 Maik Qualmann 2017-09-14 19:09:19 UTC
Git commit 7ea843fc86e000ddf5fb2b45172bc6b36714850c by Maik Qualmann.
Committed on 14/09/2017 at 19:08.
Pushed by mqualmann into branch 'master'.

change signal/slots handling for renaming and show error in the dialog

M  +4    -20   libs/database/utils/dio.cpp
M  +0    -1    libs/database/utils/dio.h
M  +3    -0    libs/iojobs/iojob.cpp
M  +1    -0    libs/iojobs/iojob.h
M  +2    -16   libs/iojobs/iojobsthread.cpp
M  +3    -10   libs/iojobs/iojobsthread.h
M  +9    -6    utilities/advancedrename/advancedrenameprocessdialog.cpp

https://commits.kde.org/digikam/7ea843fc86e000ddf5fb2b45172bc6b36714850c
Comment 10 Geoff King 2017-09-15 01:04:26 UTC
(In reply to caulier.gilles from comment #7)
> video date taken is problematic here using Exiv2. This can give errors.
> 
> Please try to seen if problem occurs without date handling. Just to identify
> if problem is relevant of date.

It does occur without date.  Even occurs with something like "ZZ-[file]" as an example.
Comment 11 Maik Qualmann 2017-10-17 18:35:51 UTC
Git commit 89e61652264777a38578c5bf9b944a4c82173294 by Maik Qualmann.
Committed on 17/10/2017 at 18:32.
Pushed by mqualmann into branch 'master'.

add backward QHash to the ImageInfo cache to resolve changed names
Related: bug 385592, bug 382312

M  +6    -0    libs/database/item/imageinfocache.cpp
M  +1    -0    libs/database/item/imageinfocache.h
M  +2    -2    libs/database/utils/dio.cpp

https://commits.kde.org/digikam/89e61652264777a38578c5bf9b944a4c82173294
Comment 12 Geoff King 2017-10-23 00:49:25 UTC
New observation with 5.8.0-pre - It appears to be aborting on associated xmp files.  I had several test files that had associated xmp files. I deleted the xmp from the file manager and all the images renamed okay.  These xmp's were associated with MOV files.  

In my settings, I have "Write to XMP sidecar for read-only image only" set so I presume that this is how they got there.
Comment 13 Geoff King 2017-12-27 22:41:34 UTC
To elaborate on my earlier observation.  Issues with renaming of video files was resolved for me by deleting the associated XMP sidecar file.   I just tried a test where it failed with the xmp present, then deleted the xmps using file manager, and tried again, and they all renamed fine.  This was with a variety of video formats and a simple rename like "[file]-test"
Comment 14 Maik Qualmann 2017-12-28 19:24:43 UTC
You can still reproduce the problem with the pre-release version on OS X? I can not reproduce it here on Linux with sidecar files. Does the window indicate that the rename failed or does the progress bar simply stop?

Maik
Comment 15 Geoff King 2017-12-28 21:39:50 UTC
Hello - Yes on Mac.  digiKam-5.8.0-20171227T110215-MacOS-x86-64
Still happens with xmp files.  
I have attached screenshot.  Also attached is the xmp file in the error dialog. 

It does appear to rename the xmp file as expected, but aborts with the error.  
The progress bar stops.  
For example:
The File named: 2017-07-13_IMG_2263-aquabats-test-test.MOV.xmp
I ran the rename on all files in the directory with "[file]-test2"
The renaming process aborts with the attached screenshot. 
I go find the file in finder and it has been renamed:  2017-07-13_IMG_2263-aquabats-test-test-test2.MOV.xmp
Comment 16 Geoff King 2017-12-28 21:40:53 UTC
Created attachment 109562 [details]
screenshot of error
Comment 17 Geoff King 2017-12-28 21:43:07 UTC
Created attachment 109563 [details]
xmp file
Comment 18 Maik Qualmann 2017-12-28 22:00:56 UTC
Ok, that's interesting. The renaming of the sidecar file failed. But why? Read-only? Missing user rights? Can you start digiKam from the console? Here should be an error message.

Maik
Comment 19 Maik Qualmann 2017-12-28 22:11:19 UTC
Very strange, the file was renamed and still must have been reported by QFile an error. The log from the console would be fine.

Maik
Comment 20 Geoff King 2017-12-28 22:35:21 UTC
Sorry.  I don't see anything in the terminal when running like this:
open -a /opt/digikam/Applications/digiKam/digikam.app

I don't know much about mac programs.  
This is all I could find in console that happens at the same time. 

usernoted
com.apple.message.domain: com.apple.nc.usage.app.notification.delivered
com.apple.message.signature: digikam
com.apple.message.summarize: YES
SenderMachUUID: D46D49C7-70A6-37F5-8FF1-BCA5F565388E
Comment 21 caulier.gilles 2017-12-28 22:40:14 UTC
geoff,

you run the deskop link to start DK. this is not the real executable.

Look well on this page, i give the path in Mac computer to the real programme called by the link :

https://www.digikam.org/contribute/

Gilles Caulier
Comment 22 Geoff King 2017-12-28 23:42:36 UTC
Created attachment 109567 [details]
logfile from terminal using lldb

New logfile using lldb for the renaming of 3 files.
Comment 23 Maik Qualmann 2017-12-29 07:07:31 UTC
OK, thanks for the log. It recognizes the sidecar file twice and also twice tries to rename it, which of course results in a file exists error. I will continue to watch it tonight.

Maik
Comment 24 Maik Qualmann 2017-12-29 10:13:24 UTC
I could imagine a possible cause. Have you possibly added "xmp" as a sidecar extension in the metadata settings for sidecars?

Maik
Comment 25 Maik Qualmann 2017-12-29 11:45:02 UTC
Git commit ac0a2c912fabc0feff88cc4486cc96be2b8cb037 by Maik Qualmann.
Committed on 29/12/2017 at 11:43.
Pushed by mqualmann into branch 'master'.

fix possible double sidecars files

M  +10   -3    libs/database/utils/dio.cpp

https://commits.kde.org/digikam/ac0a2c912fabc0feff88cc4486cc96be2b8cb037
Comment 26 Geoff King 2017-12-29 14:55:43 UTC
(In reply to Maik Qualmann from comment #24)
> I could imagine a possible cause. Have you possibly added "xmp" as a sidecar
> extension in the metadata settings for sidecars?
> 
> Maik

This is what was in my sidecar list: aae xmp lrv
I may have added those, but don't remember.
I removed xmp from the list and the issue is now resolved (files rename with no error messages). 

I think it would be nice to check to make sure that this double renaming can't happen. 

Thanks.
Comment 27 Maik Qualmann 2017-12-29 17:14:17 UTC
Thanks for the feedback. Double sidecars are now filtered out.

Maik
Comment 28 Maik Qualmann 2017-12-29 17:32:22 UTC
Git commit 2a0221952925d0a93ac872aefa7875413ddac0f9 by Maik Qualmann.
Committed on 29/12/2017 at 17:30.
Pushed by mqualmann into branch 'master'.

clean user sidecar extensions in the GUI when apply the settings

M  +2    -0    utilities/setup/metadata/setupmetadata.cpp

https://commits.kde.org/digikam/2a0221952925d0a93ac872aefa7875413ddac0f9