Bug 437813 - exiftool.exe is preventing the installation if digiKam is terminate inexplicably
Summary: exiftool.exe is preventing the installation if digiKam is terminate inexplicably
Status: RESOLVED FIXED
Alias: None
Product: digikam
Classification: Applications
Component: Metadata-ExifTool (show other bugs)
Version: 7.3.0
Platform: Microsoft Windows Microsoft Windows
: NOR normal
Target Milestone: ---
Assignee: Digikam Developers
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2021-05-29 05:21 UTC by HJ
Modified: 2023-11-17 09:42 UTC (History)
3 users (show)

See Also:
Latest Commit:
Version Fixed In: 8.2.0
Sentry Crash Report:


Attachments
Successful installation log (66.87 KB, text/plain)
2021-06-06 19:32 UTC, HJ
Details
Process monitor -> Process Tree after abnormal closing of digiKam (140.47 KB, image/png)
2021-06-08 09:49 UTC, HJ
Details

Note You need to log in before you can comment on or make changes to this bug.
Description HJ 2021-05-29 05:21:14 UTC
SUMMARY
When updating digiKam, the active window of the old version is closed and the installation of the new version becomes active, but the old version is in the Task Manager as background processes: Console Window Host; exiftool.exe (32-bit) 2 times.

STEPS TO REPRODUCE
1. When you press the digikam dialog, the installation of unsintall.exe (32-bit) will start working.
2. Remove digiKam 7.3.0 - Remove
Remove completed - Close
3. DigiKam 7.3.0. installation - Next> Next> I agree
Next> Install

OBSERVED RESULT
Error Message - DigiKam 7.3.0. installation
Error opening file when writing to C: \ Program Files \ digiKam \ exiftool.exe

EXPECTED RESULT
It is necessary to close the old digiKam process before installation

SOFTWARE/OS VERSIONS
Windows: 
macOS: 
Linux/KDE Plasma: 
(available in About System)
KDE Plasma Version: 
KDE Frameworks Version: 
Qt Version: 

ADDITIONAL INFORMATION

The reason - the old version is open in the Task Manager as background processes: Console Window Host; exiftool.exe (32-bit) 2 times.
Comment 1 caulier.gilles 2021-05-29 05:29:35 UTC
Must be fixed with bug #437750 using last daily bundle build from today.
Comment 2 caulier.gilles 2021-05-29 06:19:10 UTC
Confirmed. Problem is fixed with bug #437750. With the new installer form today, all exiftool instances are killed at end of digiKam session, and Windows installer can process until the end without problem.

Gilles Caulier
Comment 3 HJ 2021-06-02 03:15:06 UTC
Yesterday and today, when installing the new version, I still got the error Error opening file when writing to C: \ Program Files \ digiKam \ exiftool.exe.

Error caused by non-closed
digikam.exe (3)
- Console Window Host
- exiftool.exe (32-bit)
- exiftool.exe (32-bit)
After closing them manually, it worked normally.


Build date: 01.06.2021 19:09 (target: RelWithDebInfo)

Hannes
Comment 4 Maik Qualmann 2021-06-02 05:55:35 UTC
Did the version you already installed contain the fix? The installed version should have been from 05/30 or 05/31.

I tested it yesterday with the current version and all exiftool instances are now closed. 

Maik
Comment 5 HJ 2021-06-02 06:04:44 UTC
When I installed the new version yesterday morning, the same error occurred. And today after yesterday's version again. The error does not occur if the exiftool has not yet started. Today I used digiKam until exiftool started in Task Manager and then started installing the new version.

Hannes
Comment 6 caulier.gilles 2021-06-02 06:08:26 UTC
I test also today with the new daily release, with a version previously installed one day ago, and i cannot reproduce the problem here under Windows 7.

I don't yet checked with Windows 10, but i think Maik use this Windows version.

Gilles Caulier
Comment 7 caulier.gilles 2021-06-02 06:11:40 UTC
Just to be sure, to test in clean environment, can you restart Windows and try again ?

Gilles Caulier
Comment 8 caulier.gilles 2021-06-02 06:36:26 UTC
Maik,

I just tried again on Windows7, after a rebboot, and now the bug is reproducible.

Sometime, the digiKam process is not killed when i press Install from the Online Version Checker Tool. The GUI disappear, but process still in task manager (with exiftool instances of course). 

Another time, digiKam process is down but not the exiftool instances.

And finally, trying again, all is fine.

Conclusion: At the most on time all is fine (i tried 10 times), but sometime no. There is nothing special running in digiKam. I use Sqlite under Windows, and i just start digiKam icon view, open metadata view with ExifTool, and run Onlin Version Checker as well. It a simple use case context (aka no LT, Editor, or BQM)

Gilles
Comment 9 Maik Qualmann 2021-06-02 19:41:03 UTC
Git commit 51e1d00c0ad9784b8febe3ad356b28db0dae6ebf by Maik Qualmann.
Committed on 02/06/2021 at 19:39.
Pushed by mqualmann into branch 'master'.

optimize exiftool terminate

M  +16   -13   core/libs/metadataengine/exiftool/exiftoolprocess.cpp
M  +0    -2    core/libs/metadataengine/exiftool/exiftoolprocess.h
M  +1    -1    core/libs/metadataengine/exiftool/exiftoolprocess_p.cpp

https://invent.kde.org/graphics/digikam/commit/51e1d00c0ad9784b8febe3ad356b28db0dae6ebf
Comment 10 HJ 2021-06-04 21:13:29 UTC
Check for New Version update will not be able to close the Exiftool (32) .exe (2 units) processes that are left open at some point. digiKam installation is prevented and it is necessary to manually close the Exiftool (32) .exe process.
Build date: 04.06.2021 19:09
Comment 11 Maik Qualmann 2021-06-05 08:34:44 UTC
And what date was the already installed version?

Maik
Comment 12 caulier.gilles 2021-06-05 09:11:33 UTC
Maik, 

The problem still reproducible, but only from Check For New Release tool.

Typically, you run digiKam, and your close it, exiftool instances are closed.

If you use Check For New Release, you download installer, and you run Installer, digiKam is closed with qApp->quit(), but exiftool instances still in memory.

https://invent.kde.org/graphics/digikam/-/blob/master/core/libs/onlineversion/onlineversiondlg.cpp#L647

Perhaps under Windows, qApp->quit() is not the right way.

Tested with Windows 7 and installed digiKam build from 03/06/2021 19:10 

Gilles
Comment 13 caulier.gilles 2021-06-05 12:16:47 UTC
Git commit 2fd2a9a8914f416729cb311a6bba803b6f345ac7 by Gilles Caulier.
Committed on 05/06/2021 at 12:15.
Pushed by cgilles into branch 'master'.

call closeAllWindows() slot instead quit() with qApp.

M  +1    -1    core/libs/onlineversion/onlineversiondlg.cpp

https://invent.kde.org/graphics/digikam/commit/2fd2a9a8914f416729cb311a6bba803b6f345ac7
Comment 14 caulier.gilles 2021-06-05 13:41:48 UTC
Git commit c06a6677b53616d271d021ce0cdc239c224dded9 by Gilles Caulier.
Committed on 05/06/2021 at 13:38.
Pushed by cgilles into branch 'master'.

Search for Quit menu role action from main window to trigger after starting installer.
If action is not found, call generic slot instead
Report in the console which method is used.

M  +38   -1    core/libs/onlineversion/onlineversiondlg.cpp

https://invent.kde.org/graphics/digikam/commit/c06a6677b53616d271d021ce0cdc239c224dded9
Comment 15 caulier.gilles 2021-06-06 15:55:45 UTC
Hannesj,

The Windows version to install from today do the job here under Windows 7:

See the debug trace captured from DebugView when digiKam is closed just after downloading the new installer and start it:

[2920] digikam.general: Download is complete:  "C:\\Users\\gilles\\Downloads\\digiKam-7.3.0-20210605T210939-Win64.exe"
[2920] digikam.general: Run installer: "C:\\Users\\gilles\\Downloads\\digiKam-7.3.0-20210605T210939-Win64.exe"
[2920] digikam.general: Generic call to close main windows
[2920] digikam.geoiface: ----
[2920] digikam.metaengine: ExifToolProcess::terminate(): send ExifTool shutdown command...
[2920] digikam.metaengine: ExifTool process finished 0 QProcess::NormalExit
[2920] digikam.general: Cancel Main Thread
[2920] digikam.general: Cancel Main Thread
[2920] QWaitCondition: Destroyed while threads are still waiting

The list of ExifTool.exe process disapear properly:

https://i.imgur.com/O3JD62s.png

To hack this problem, i changed the compilation time-stamp of publishing metadata of the bundle. So for today the build date and the publishing date are different. The consequence, in digiKam, when you run Check For New Version tool, it will always found a new daily version available. This will true for taday and only today.

Install the daily version, run it, and run Version Checker. Look if when digiKAm session is closed, ExifTool instance disappear.

Best

Gilles Caulier
Comment 16 HJ 2021-06-06 19:10:37 UTC
Under normal circumstances, the installation will run smoothly.
However, exiftool.exe files that remain due to abnormal digiKam termination (not responding) do not close during installation. And there are error paths caused by the openness of exiftool.exe



[35884] digikam.general: Run installer: "C: \\ Users \\ hanne \\ Downloads \\ digiKam-7.3.0-20210605T210939-Win64.exe"
[35884] digikam.general: Generic call to close main windows
[35884] digikam.geoiface: ----
[35884] digikam.metaengine: ExifToolProcess :: terminate (): send ExifTool shutdown command ...
[35884] digikam.metaengine: ExifTool process finished 0 QProcess :: NormalExit
[35884] digikam.general: Cancel Main Thread
[35884] digikam.general: Cancel Main Thread
[35884] QWaitCondition: Destroyed while threads are still waiting
[35884] QThreadStorage: Thread 0x8806b20 exited after QThreadStorage 10 destroyed
[35884] QThreadStorage: Thread 0x8806b20 exited after QThreadStorage 7 destroyed
[35884] QWaitCondition: Destroyed while threads are still waiting
[35884] QEventDispatcherWin32 :: wakeUp: Failed to post a message (Invalid window handle.)

Hannes
Comment 17 HJ 2021-06-06 19:32:00 UTC
Created attachment 139047 [details]
Successful installation log

Successful installation log added from download to running a new version of the program.

Hannes
Comment 18 caulier.gilles 2021-06-06 20:29:00 UTC
I'm happy to see that for normal closing conditions, including the install stage from Check For New Version tool, the shutdown behavior for ExifTool instances work fine.

What do you mean by abnormal termination ? Do you process any special digiKam processing in background, as queue manager, faces detection, export to web service ?  

Gilles Caulier
Comment 19 HJ 2021-06-06 22:10:31 UTC
One reason exiftool.exe does not close is the unexpected self-closing of digiKam. During this time, a background process like Update fingerprints has been running, and at the same time I've been reviewing and deleting Similarity Duplicates pairs.
Another reason is digiKam "Not responding" and Table view up to 90,000 entries. A table image with thumbnails will appear and give the impression that the row can be selected, but the message Not Responding will appear on top of some attempts. Time drags on and Ctrl + Q doesn't work, then I choose X to close. Exiftool.exe files remain running.
Ma olen teadlik, et Table view ei ole optimeeritud kuid vahetevahel tuleb seda siiski kasutada.

Normally, X closes all digiKam files.


Hannes
Comment 20 HJ 2021-06-08 04:44:55 UTC
Sorry, I'm translating the previous comment line from Estonian here:

I'm aware that Table view isn't optimized, but it still needs to be used occasionally.

But in general, this issue can be closed as resolved.

Hannes
Comment 21 caulier.gilles 2021-06-08 05:32:35 UTC
Sure, in normal termination ExifTool process are closed properly. This include Check For New Version Tool.

But in abnormal termination, it doesn't work, even if Exiftool process are children than digiKam process. The Q is why ?

Gilles Caulier
Comment 22 caulier.gilles 2021-06-08 05:38:46 UTC
Under Linux, Exiftool appear well as a children process from digiKam.

If i kill digiKam with CRTL+C from the console, ExifTool process disappear properly.

Gilles Caulier
Comment 23 caulier.gilles 2021-06-08 05:45:50 UTC
Under MacOS, Exiftool do not appear as a children process from digiKam directly. It's "perl5.30" process.

If i kill digiKam with CRTL+C from the console, perl5.30 process disappear properly.

Gilles Caulier
Comment 24 caulier.gilles 2021-06-08 06:04:15 UTC
Under Windows, ProcesMonitor application show well exiftool.exe as children than digikam.exe. Windows Tasks Manager doesn't.

If i terminate digikam.exe from Windows Tasks Manager, exiftool.exe do not close properly. ProcessMonitor application, always show the digikam.exe + exiftool.exe process tree, even if digikam.exe has disappear than Windows Tasks Manager.

Gilles Caulier
Comment 25 caulier.gilles 2021-06-08 06:11:28 UTC
Note : Uder Windows, i double check if exiftool.exe process are closed properly when digiKam is closed normally, and yes, it work.

Gilles Caulier
Comment 26 HJ 2021-06-08 09:49:29 UTC
Created attachment 139101 [details]
Process monitor -> Process Tree after abnormal closing of digiKam

Process monitor -> Process Tree after abnormal closing of digiKam - the old process is visible after opening a new digiKam.

The W10 task manager also now shows two digiKam processes, one of which is inactive.
Comment 27 caulier.gilles 2021-06-09 21:32:51 UTC
Maik, 

As explained in this forum:

https://social.msdn.microsoft.com/Forums/vstudio/en-US/7d285149-3f60-4c0f-acbd-aa5d32aa138c/kill-child-process-when-parent-process-is-killed-forcefully?forum=vcgeneral


As we don't have a control in exiftool.exe binary source code, we need to create an intermediary process 'ETLaucher' between digikam an exiftool. ETLauncher will trigger internally a watchdog to check if parent process is dead (digiKam). If yes it will shutdown exiftool instance properly.

Gilles
Comment 28 caulier.gilles 2022-10-11 05:51:34 UTC
Maik,

With your large ExifTool wrapper rewriting task, did you take a care about this entry ?

Typically, to prevent this kind of dysfunction, the same mechanism used with Mysql internal server can be used, I explained also the details in comment #27.

Gilles
Comment 29 Maik Qualmann 2022-10-11 06:07:56 UTC
No, there is no solution for this at the moment, but I'm still looking at it. If digiKam should currently crash, the ExifTool task will remain.

Maik
Comment 30 caulier.gilles 2023-03-14 21:14:12 UTC
*** Bug 467333 has been marked as a duplicate of this bug. ***
Comment 31 caulier.gilles 2023-05-03 08:35:28 UTC
Maik,

Yes there is a solution. Look how local mysql executables are managed. There is no similar issue with it.

Gilles
Comment 32 Maik Qualmann 2023-05-03 10:43:02 UTC
No, the same problem with MySQL, also the local MySQL server does not stop when it crashes. There is just no problem with the installation because MySQL is not part of the installer on Windows.

For ExifTool I have already tested bending the crash handler. This works quite well under Linux. If digiKam crashes, Exiftool can still be closed normally in this way. I haven't tested it on Windows, but after research it should work. As a side effect for a CTRL+C in the terminal but not to end digiKam immediately, first a second CTRL+C.

The other way would be an external program with a watchdog over DBus? checks if digiKam is still alive.

But there is another possibility, a developer in the ExifTool Forum asked for it, that a special sequence is sent every few seconds/minutes, if the main application does not send it, ExifTool then terminates itself. But Phil didn't want to add it at the moment, maybe the wish could be pushed again.

Maik
Comment 33 caulier.gilles 2023-11-16 04:09:25 UTC
Git commit 6dad6403945a91b950eb28239654aa51d25ac6f5 by Gilles Caulier.
Committed on 16/11/2023 at 05:04.
Pushed by cgilles into branch 'master'.

NSIS VCPKG: kill all background exiftool.exe process running in backgroud before to copy file in the system
Related: bug 467333

M  +4    -0    project/bundles/vcpkg/installer/digikam.nsi

https://invent.kde.org/graphics/digikam/-/commit/6dad6403945a91b950eb28239654aa51d25ac6f5
Comment 34 caulier.gilles 2023-11-16 04:09:33 UTC
Git commit fbe53f85b92395f438214e0223792471e06c3c4c by Gilles Caulier.
Committed on 16/11/2023 at 05:08.
Pushed by cgilles into branch 'master'.

NSIS MXE: kill all background exiftool.exe process running in backgroud before to copy file in the system
Related: bug 467333

M  +4    -0    project/bundles/mxe/installer/digikam.nsi

https://invent.kde.org/graphics/digikam/-/commit/fbe53f85b92395f438214e0223792471e06c3c4c
Comment 35 caulier.gilles 2023-11-17 09:42:39 UTC
Git commit 34c75c33ffb215daf70d4c336a8cb75ccc61dd69 by Gilles Caulier.
Committed on 17/11/2023 at 10:38.
Pushed by cgilles into branch 'master'.

kill exiftool.exe also at uninstall stage
The goal works as expected. We can close files now.
Related: bug 467333
FIXED-IN: 8.2.0

M  +4    -0    project/bundles/mxe/installer/digikam.nsi
M  +4    -0    project/bundles/vcpkg/installer/digikam.nsi

https://invent.kde.org/graphics/digikam/-/commit/34c75c33ffb215daf70d4c336a8cb75ccc61dd69