| Summary: | When tagging, "Writing metadata to files" messages never clear from progress queue | ||
|---|---|---|---|
| Product: | [Applications] digikam | Reporter: | Jason Harris <jharris> |
| Component: | ProgressManager-Batch | Assignee: | Digikam Developers <digikam-bugs-null> |
| Status: | RESOLVED FIXED | ||
| Severity: | normal | CC: | andi.clemens, bugs, caulier.gilles, clapautius, cyh47, domlyons, fredericg_99, gerhardk, guillom.pom, jeffnik, jon33040, npomarede, Robert.M.Davies, superandrzej, varun.herale, veaceslav.munteanu90 |
| Priority: | NOR | ||
| Version: | 2.6.0 | ||
| Target Milestone: | --- | ||
| Platform: | Compiled Sources | ||
| OS: | Linux | ||
| Latest Commit: | Version Fixed In: | 2.6.0 | |
| Sentry Crash Report: | |||
| Attachments: |
screenshot of the issue
Debug messages patch 1 Results generated when running with patch 1 applied. Log from digikam with patch applied Log from digikam with patch applied - without raw images fileworkeriface.moc for qt 4.8.1 |
||
|
Description
Jason Harris
2012-03-03 15:49:06 UTC
Update and try again. Some patch have been applied recently by Marcel about... Gilles Caulier I have just updated and rebuilt. The porblem is still present. Can you or someone else confirm that tagging images causes persistent jobs in the process queue, using the latest code? (In reply to comment #1) > Update and try again. Some patch have been applied recently by Marcel > about... > > Gilles Caulier Not reproducible here. I tag 56 files (JPEG and PNG) with one tag. It take 10 s around. Progress Manager close process as expected... Which type of file you tag ? Which Metadata settings you use from digiKam setup panel ? Which type of computer your use (one or multicore cpu) ? Gilles Caulier I tagged between 1 and a few dozen JPEG images at a time, and occassionally some AVI files too. In all cases, the process jobs don't go away until I close Digikam. Not sure what metadata settings you are interested in. There's one section labeled "Write this information to the metadata". Prfeviously, I had none of those items checked. I just tried checking them all, and I am still seeing persistent process jobs after tagging after making that change. My uname -a: Linux localhost 2.6.35-gentoo-r5 #3 SMP Wed Sep 29 22:15:48 PDT 2010 x86_64 AMD Athlon(tm) Dual Core Processor 4050e AuthenticAMD GNU/Linux (In reply to comment #3) > Not reproducible here. I tag 56 files (JPEG and PNG) with one tag. It take > 10 s around. Progress Manager close process as expected... > > Which type of file you tag ? Which Metadata settings you use from digiKam > setup panel ? Which type of computer your use (one or multicore cpu) ? > > Gilles Caulier I can confirm this bug with the latest git (just pulled few seconds ago) Also, it can't rotate images using option from my right click, only progress manager appears and it stops on 0%. I can't reproduce problem with writing metadata anymore (but i remember having the same problem few days ago) But I still can't rotate images (progress Manager 0%) Veaceslav, Which type of file your try to rotate exactly ? If JPEG, which libjpeg you use ? Look in Help/components Info for details. Also, which CPU type you use ? How many processors exactly ? Gilles Caulier Info from components: digiKam version 2.6.0-beta3 Exiv2 can write to Jp2: Yes Exiv2 can write to Jpeg: Yes Exiv2 can write to Pgf: Yes Exiv2 can write to Png: Yes Exiv2 can write to Tiff: Yes Exiv2 supports XMP metadata: Yes LibCImg: 130 LibClapack: internal library LibExiv2: 0.22 LibJPEG: 80 LibJasper: 1.900.1 LibKDE: 4.8.00 (4.8.0 LibKExiv2: 2.1.1 LibKGeoMap: 2.0.0 LibKdcraw: 2.0.1 LibLCMS: 119 LibPGF: 6.11.42 - internal library LibPNG: 1.5.9 LibQt: 4.8.0 LibRaw: 0.14.4 LibTIFF: LIBTIFF, Version 4.0.0 Copyright (c) 1988-1996 Sam Leffler Copyright (c) 1991-1996 Silicon Graphics, Inc. Marble Widget: 0.12.97 (0.13 Release Candidate 2) Parallelized demosaicing: Yes Database backend: QSQLITE LibGphoto2: 2.4.11 LibKface: 2.0.0 LibKipi: 1.3.0 LibOpenCV: 2.3.1 Libface: 0.2 And I have an Intel core i5 2410m 2 cores 4 threads I try to rotate both jpeg and png jpeg: Bit deph: 8 bpp Color mode: YcbCr png: Bit deph 8bpp Color mode: RGB If you process PNG files only, the rotation failed too ? Note that i use also libjpeg 8.0 on my computer, and all work fine to rotate JPEG. No problem too with PNG... Gilles Caulier Yes, it can't rotate even png files only. Created attachment 70184 [details] screenshot of the issue I confirm the issue as originally reported in comment #0 : > Whenever I tag one or more images, the action adds a job to the progress > queue with the text "Writing metadata to files". As far as I can tell, > these jobs never get cleared from the progress queue. when editing metadata (rating, tags, title or description) of PNG files with the latest version compiled from git master (see attached screenshot). Metadata are saved to the database but never written to the files. If you close digikam and "Cancel" the "Finishing tasks" window, then reopen digikam again, it's possible to sync the metadata from the database by writing them to the files using the menu command, but writing them automatically as soon as they're entered doesn't seem to be working right now. Additional information: digikam compiled from git master on opensuse 12.1 64 bits Is someone here on latest git who can reliably reproduce the issue and would be willing to test some patches for debug output? I cannot reproduce it, but it seems to be an issue that several people see. We need remote debugging. I can't test right now (my building has been condemned because of a fire) but I'm happy to test patches when I can access the computer and photo collection where the issue happens (probably in a few days). Marcel, I have the same problem with metadata writing AND rotating, has appeared somewhere in February. I am willing to test your patches. My system: uname -a Linux 3.4.0-030400rc4-generic #201204211835 SMP Sat Apr 21 22:36:13 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux digiKam version 2.6.0-rc Exiv2 can write to Jp2: Yes Exiv2 can write to Jpeg: Yes Exiv2 can write to Pgf: Yes Exiv2 can write to Png: Yes Exiv2 can write to Tiff: Yes Exiv2 supports XMP metadata: Yes LibCImg: 130 LibClapack: internal library LibExiv2: 0.22 LibJPEG: 80 LibJasper: 1.900.1 LibKDE: 4.8.2 (4.8.2) LibKExiv2: 2.1.0 LibKGeoMap: 2.0.0 LibKdcraw: 2.0.1 LibLCMS: 119 LibLensFun: external shared library LibLqr: internal library LibPGF: 6.11.24 - external shared library LibPNG: 1.2.46 LibQt: 4.8.1 LibRaw: 0.14.4 LibTIFF: LIBTIFF, Version 3.9.5 Copyright (c) 1988-1996 Sam Leffler Copyright (c) 1991-1996 Silicon Graphics, Inc. Marble Widget: 0.13.0 (stable release) Parallelized demosaicing: Yes Database backend: QMYSQL Database internal server: No LibGphoto2: 2.4.13 LibKface: 2.0.0 LibKipi: 1.3.0 LibOpenCV: 2.3.1 Libface: 0.2 I also have the problem when assigning a star rating to images. I could probably help debug too. I also see this problem, both with 2.6.0-beta 3 and a checkout from git done yesterday. I'm prepared to try patches although I've never done any C++ programming so you would have to deliver something which can be applied automatically. Created attachment 70895 [details]
Debug messages patch 1
This is a first patch to narrow down the location of the problem. It should give out a number of debug messages on the console.
Is the problem always reproducible? Try to reduce - if you can reproduce with one image in one operation, debug output will be easier to parse than with 100 images in ten operations.
Thanks!
Created attachment 70897 [details]
Results generated when running with patch 1 applied.
The attached file is the complete output from running digikam with patch 1 applied, from starting digikam to exiting. Once digikam had started, I picked one picture, applied & removed a tag in order to trigger a metadata re-write, pressed apply and after a few seconds, the "Writing metadata to files" popup appeared and did not go away.
Oh, please add a tag by drag-and-drop or context menu, the patch was meant for that approach. On the console, a line "Adding/removing ... tags" should appear, then some more output. Still reproducible, or only with the right sidebar? What's you Qt version? Created attachment 70929 [details]
Log from digikam with patch applied
Log from digikam with patch applied. First a tag is applied and removed by right-clicking on a thumbnail. Second a tag is applied by dragging a thumbnail over to the tags list on the RHS of the screen. Each time an instance of the "Writing metadata to files" pop-up was created and did not vanish. Lastly digikam was exited and blocked due to pending writes.
Here's the information from "help" / "components information". digiKam version 2.6.0-rc Exiv2 can write to Jp2: Yes Exiv2 can write to Jpeg: Yes Exiv2 can write to Pgf: Yes Exiv2 can write to Png: Yes Exiv2 can write to Tiff: Yes Exiv2 supports XMP metadata: Yes LibCImg: 130 LibClapack: internal library LibExiv2: 0.22 LibJPEG: 80 LibJasper: 1.900.1 LibKDE: 4.8.3 (4.8.3) LibKExiv2: 2.3.0 LibKGeoMap: 2.0.0 LibKdcraw: 2.1.0 LibLCMS: 119 LibLensFun: 0.2.5-1 - internal library LibLqr: internal library LibPGF: 6.11.42 - internal library LibPNG: 1.2.46 LibQt: 4.8.1 LibRaw: 0.14.6 LibTIFF: LIBTIFF, Version 3.9.5 Copyright (c) 1988-1996 Sam Leffler Copyright (c) 1991-1996 Silicon Graphics, Inc. Marble Widget: 0.13.3 (stable release) Parallelized demosaicing: Yes Database backend: QSQLITE LibGphoto2: 2.4.13 LibKface: 2.0.0 LibKipi: 1.6.0 LibOpenCV: 2.3.1 Libface: 0.2 Created attachment 70934 [details]
Log from digikam with patch applied - without raw images
The previous log file included a lot of noise due to the image I was using being a raw one. This 3rd log file uses TIFF/PNG.
Thanks a lot. To all who have confirmed this problem: Can we see a correlation with Qt version? Qt 4.7 -> no problem Qt 4.8 -> problem? The test sample is very small, so it's hard to say with certainty. I would seem to strengthen the correlation though, as I am using Qt4.8.1 My desktop machine is Qt 4.7.4 (see below) and does not show this problem. Still a very small sample but we have to start somewhere. Jon digiKam version 2.6.0-beta3 Exiv2 can write to Jp2: Yes Exiv2 can write to Jpeg: Yes Exiv2 can write to Pgf: Yes Exiv2 can write to Png: Yes Exiv2 can write to Tiff: Yes Exiv2 supports XMP metadata: Yes LibCImg: 130 LibClapack: internal library LibExiv2: 0.22 LibJPEG: 62 LibJasper: 1.900.1 LibKDE: 4.7.4 (4.7.4) LibKExiv2: 2.3.0 LibKGeoMap: 2.0.0 LibKdcraw: 2.1.0 LibLCMS: 119 LibLensFun: external shared library LibLqr: internal library LibPGF: 6.11.42 - internal library LibPNG: 1.2.46 LibQt: 4.7.4 LibRaw: 0.14.5 LibTIFF: LIBTIFF, Version 3.9.5 Copyright (c) 1988-1996 Sam Leffler Copyright (c) 1991-1996 Silicon Graphics, Inc. Marble Widget: 0.12.2 (stable release) Parallelised demosaicing: Yes Database backend: QSQLITE LibGphoto2: 2.4.13 LibKface: 2.0.0 LibKipi: 1.6.0 LibOpenCV: 2.1.0 Libface: 0.2 Probably should have provided complete component info: digiKam version 2.6.0-beta3 Exiv2 can write to Jp2: Yes Exiv2 can write to Jpeg: Yes Exiv2 can write to Pgf: Yes Exiv2 can write to Png: Yes Exiv2 can write to Tiff: Yes Exiv2 supports XMP metadata: Yes LibCImg: 130 LibClapack: internal library LibExiv2: 0.21.1 LibJPEG: 62 LibJasper: 1.900.1 LibKDE: 4.8.2 (4.8.2) LibKExiv2: 2.1.0 LibKGeoMap: 2.0.0 LibKdcraw: 2.0.1 LibLCMS: 119 LibPGF: 6.11.42 - internal library LibPNG: 1.2.46 LibQt: 4.8.1 LibRaw: 0.14.4 LibTIFF: LIBTIFF, Version 3.9.5 Copyright (c) 1988-1996 Sam Leffler Copyright (c) 1991-1996 Silicon Graphics, Inc. Marble Widget: 0.13.0 (stable release) Parallelised demosaicing: Yes Database backend: QSQLITE LibGphoto2: 2.4.11 LibKface: 2.0.0 LibKipi: 1.3.0 LibOpenCV: 2.1.0 Libface: 0.2 I can see this problem too - and my Qt version is 4.8.1. It looks like nothing is being written into the metadata at all. I see this bug too, for quite a long time now. I posted a bug report here : https://bugs.kde.org/show_bug.cgi?id=295340 Actually, as I feared, the report described 2 bugs, one of them was fixed, but the other one remains. Still, the backtrace included may be of some help... My Qt is 4.8.1, and the only specific setting for me is that my database is stored on mysql. I have this bug on (at least) 2 computers, that share the same storage (one of them locally, the other one accesses it through NFS) and the same database. Digikam is built from source, so I may apply patches if necessary. About the Qt version, here is my yum.log Nov 09 23:03:49 Updated: 1:qt-4.8.0-0.23.rc1.fc16.x86_64 Dec 23 22:32:13 Updated: 1:qt-4.8.0-0.29.rc1.fc16.x86_64 Jan 05 19:00:21 Updated: 1:qt-4.8.0-5.fc16.x86_64 Jan 23 11:25:50 Updated: 1:qt-4.8.0-7.fc16.x86_64 Apr 21 14:04:14 Updated: 1:qt-4.8.1-5.fc16.x86_64 As you can see, I'm running qt-4.8 since november 2011, and the bug is far more recent (I would say february or march), so the relation will not be immediate. PS : I don't know who has the rights to do it, but it may be acceptable to set this bug to "confirmed" I can confirm this issue. I'm using kubuntu 12.04, kde 4.8.3 (tested on kde 4.8.2 as well), qt 4.8.1, arch - 32bit amd 2 cores, digikam 2.6.0-rc compiled from sources. I've just tested this issue on a different computer with qt 4.7.4 (openSuse, kde 4.7.2, digikam 2.6.0-rc) and the issue does not reproduce. The metadata is written to files and everything works as expected. I too am having the issue.
I don't know if proving results from"top" is of help but here you are.
I am using Fedora 1x86_64.
Following is the result from typing "top" in a console.
This was after asking Digikam to close. The small window with progess bar was warning about unfinished task.
If one chooses "cancel" Digikam does close as well as the small progress window.
Upon restarting Digikam, the tags have been added to the photos and works well it seems.
top - 13:23:39 up 6:54, 3 users, load average: 2.56, 2.59, 2.68
Tasks: 178 total, 2 running, 176 sleeping, 0 stopped, 0 zombie
Cpu(s): 47.0%us, 19.8%sy, 0.0%ni, 22.7%id, 10.1%wa, 0.0%hi, 0.5%si, 0.0%st
Mem: 4050492k total, 3933556k used, 116936k free, 9392k buffers
Swap: 7186424k total, 2426216k used, 4760208k free, 424124k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
1736 cyh47 20 0 6287m 2.6g 23m S 52.9 68.4 152:14.10 digikam
9251 cyh47 20 0 630m 6992 5440 R 26.4 0.2 0:00.16 kio_digikamtags
46 root 20 0 0 0 0 S 1.7 0.0 2:00.74 kswapd0
1220 cyh47 20 0 724m 16m 1256 S 1.7 0.4 0:34.87 mysqld
2451 root 20 0 0 0 0 S 1.7 0.0 2:57.66 btrfs-endio-3
9250 root 20 0 15252 1168 824 R 1.7 0.0 0:00.05 top
1 root 20 0 61392 1328 808 S 0.0 0.0 0:03.10 systemd
2 root 20 0 0 0 0 S 0.0 0.0 0:00.01 kthreadd
3 root 20 0 0 0 0 S 0.0 0.0 0:09.77 ksoftirqd/0
6 root RT 0 0 0 0 S 0.0 0.0 0:00.00 migration/0
7 root RT 0 0 0 0 S 0.0 0.0 0:00.79 watchdog/0
8 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 cpuset
9 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 khelper
10 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kdevtmpfs
11 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 netns
12 root 20 0 0 0 0 S 0.0 0.0 0:00.25 sync_supers
13 root 20 0 0 0 0 S 0.0 0.0 0:00.00 bdi-default
14 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 kintegrityd
15 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 kblockd
16 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 ata_sff
17 root 20 0 0 0 0 S 0.0 0.0 0:00.12 khubd
18 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 md
47 root 25 5 0 0 0 S 0.0 0.0 0:00.00 ksmd
Can someone with Qt 4.8 attach the file core/digikam/fileworkeriface.moc from the build dir? Just to rule out some possible causes. I believe for a proper debugging I will need to upgrade to Qt4.8 which is not done in ten minutes. Therefore I guess for 2.6 final, we need a workaround which will only disable the feature that load is distributed over all CPU cores for the relevant tasks (rotation, file metadata editing). Btw: Those of you who can confirm this issue have also a problem when rotating multiple pictures at once? Created attachment 71077 [details]
fileworkeriface.moc for qt 4.8.1
I attached the fileworkeriface.moc file (qt 4.8.1).
As for rotating images, I tried to rotate a raw image (.nef) but the progress bar didn't move from 0%.
I've just done an experiment with moc files. I used the moc file generated on computer with qt 4.7.4 to build digikam with qt 4.8.1 (after disabling the #ifdef protection). And everything seems to work fine. :) Both operations (writing metadata and rotating images) finish with success. It's probably not the best idea but might give you some clues about the problem. I still can't rotate images, with latest git, clean install. Progress stuck at 0%. I get the same problem. I'm cleaning up my tags, so I create new tag hierarchies and then drag-and-drop whole albums on the new tag and click "Assign tag" from the popup. Sometimes this happens with no progress bar, other times I get the "Writing metadata to files" progress bar. It stays at 0%. When I try to close digiKam, a new window pops up, "Finishing tasks" and it also stays up perpetually. I have to click "Cancel". Upon restarting digiKam usually my changes have been applied, but yesterday I came across one main tag /subject/* whose files I tried re-tagging with other tags (e.g. from drag photos from the /subject/flora/fern tag and drop them on the /plants/fern tag), the perpetual progress bar would appear, I would have to "cancel" it to close digiKam, upon restarting the new tags were gone, and the old deletes ones were back. So after restarting, all changes I made to these files from /subject/* (deleting the old tags, settings new ones) would not get applied - time lost. I ended up deleting that whole main tag /subject. I tested just now - created a new tag "foo", dropped seven JPEG photos from tag "mushrooms" on to tag "foo", the perpetual 0% progress bar appeared, I clicked X to close digiKam, clicked "cancel", restarted it, and the new tag "foo" was set. Screenshot here: http://i.imgur.com/6YnUM.png digiKam version 2.6.0-rc Exiv2 can write to Jp2: Yes Exiv2 can write to Jpeg: Yes Exiv2 can write to Pgf: Yes Exiv2 can write to Png: Yes Exiv2 can write to Tiff: Yes Exiv2 supports XMP metadata: Yes LibCImg: 130 LibClapack: external shared library LibExiv2: 0.21.1 LibJPEG: 80 LibJasper: 1.900.1 LibKDE: 4.8.1 (4.8.1) LibKExiv2: 2.1.0 LibKGeoMap: 2.0.0 LibKdcraw: 2.0.1 LibLCMS: 119 LibLensFun: external shared library LibLqr: internal library LibPGF: 6.11.32 - external shared library LibPNG: 1.5.10 LibQt: 4.8.1 LibRaw: 0.14.4 LibTIFF: LIBTIFF, Version 4.0.1 Copyright (c) 1988-1996 Sam Leffler Copyright (c) 1991-1996 Silicon Graphics, Inc. Marble Widget: 0.12.97 (0.13 Release Candidate 2) Parallelized demosaicing: Yes Database backend: QSQLITE LibKface: 2.0.0 LibKipi: 1.3.0 LibOpenCV: 2.3.0 Libface: 0.2 to comment #32 : Hello, I have the same problem under Mandriva cooker, using qt 4.8.1 and digikam 2.6.0.rc2 I tried to disable all cores/cpus at boot using kernel option maxcpus=1, but in that case the problem is still present, the "writing metadata to file" message is still present on screen and never goes away. Are you sure the problem is related to the number of cpu in QT 4.8 ? Nicolas i7 820QM (mobile quad-core) Git commit e254d1b8e53fa513f1433b1c73adb4859874f2f1 by Marcel Wiesweg. Committed on 19/05/2012 at 12:37. Pushed by mwiesweg into branch 'master'. Try to make slot interception work with Qt4.8 4.8's moc makes use of the extraData field of QMetaObject to pass the static_qt_metacall pointer. I know this is a hack (but binary compatibility vows are on our side) M +18 -3 libs/threads/parallelworkers.cpp M +19 -7 libs/threads/parallelworkers.h http://commits.kde.org/digikam/e254d1b8e53fa513f1433b1c73adb4859874f2f1 Can someone try the just committed fix? Only users of Qt4.8 are affected by the bug. I just did a new compile and both rotation and metadata update work. Thanks a lot Marcel! On May 19, 2012 1:01 PM, "Marcel Wiesweg" <marcel.wiesweg@gmx.de> wrote: > https://bugs.kde.org/show_bug.cgi?id=295263 > > --- Comment #39 from Marcel Wiesweg <marcel.wiesweg@gmx.de> --- > Git commit e254d1b8e53fa513f1433b1c73adb4859874f2f1 by Marcel Wiesweg. > Committed on 19/05/2012 at 12:37. > Pushed by mwiesweg into branch 'master'. > > Try to make slot interception work with Qt4.8 > > 4.8's moc makes use of the extraData field of QMetaObject to pass the > static_qt_metacall pointer. > I know this is a hack (but binary compatibility vows are on our side) > > M +18 -3 libs/threads/parallelworkers.cpp > M +19 -7 libs/threads/parallelworkers.h > > http://commits.kde.org/digikam/e254d1b8e53fa513f1433b1c73adb4859874f2f1 > > -- > You are receiving this mail because: > You are on the CC list for the bug. > Fix confirmed. Thank you! Rotate images works! Thanks! Mandriva just released a new version with the patch in comment #39 and this also fixed the issue for me. Thanks *** Bug 295182 has been marked as a duplicate of this bug. *** |
