Summary: | digiKam freezes on Import | ||
---|---|---|---|
Product: | [Applications] digikam | Reporter: | Bernd Buschinski <b.buschinski> |
Component: | Import-Gphoto2 | Assignee: | Digikam Developers <digikam-bugs-null> |
Status: | RESOLVED FIXED | ||
Severity: | crash | CC: | caulier.gilles, kredba |
Priority: | NOR | ||
Version: | 4.2.0 | ||
Target Milestone: | --- | ||
Platform: | Other | ||
OS: | Linux | ||
Latest Commit: | http://commits.kde.org/digikam/54b51f06b9d413e663a07b64d9bbe2eff69d9713 | Version Fixed In: | 4.4.0 |
Sentry Crash Report: | |||
Attachments: |
digikam with kdebug output
Last Actions in History View no Files in, or foldern in Log patch debug counters |
Description
Bernd Buschinski
2014-08-28 17:33:56 UTC
It crash in Harfbuzz not in digiKam : http://www.freedesktop.org/wiki/Software/HarfBuzz/ I suspect a problem in computer configuration. Gilles Caulier Please try to get a better backtrace in GDB following information here : https://www.digikam.org/contrib Gilles Caulier It does NOT crash, this is the bt if I run it in gdb and press ctrl+c Can you turn on debug space using kdebugdialog as explained here : https://www.digikam.org/contrib ... and run digiKam from a console. Report console trace here. Q: do you have any video files on your camera ? Gilles Caulier Created attachment 88524 [details]
digikam with kdebug output
And no, no videos on this device ok, so I went ahead and commented libs/widgets/mainview/dhistoryview.cpp:166 out and it works. No more freezes, I can see and download the images. You have commented this line ? https://projects.kde.org/projects/extragear/graphics/digikam/repository/revisions/master/entry/libs/widgets/mainview/dhistoryview.cpp#L166 Why this will block GUI ??? Typicially DHistoryView is a simple QTreeWidget used to list event from Import Tool... I'm lost Gilles Caulier Call of DHistoryView::addEntry() in Import tool is done here : https://projects.kde.org/projects/extragear/graphics/digikam/repository/revisions/master/entry/utilities/importui/main/importui.cpp#L2557 Nothing special here. It a slot which receive event to log on history. How many message can you see registered in history view from Import GUI before to see digiKam frozen ? Gilles Caulier hm.. I can not find the "history view", how can I access it? is it called somethingelse? where should it be? But yes, just this one line https://projects.kde.org/projects/extragear/graphics/digikam/repository/revisions/master/entry/libs/widgets/mainview/dhistoryview.cpp#L166 History view is not show be default in Import Tool. To turn on, got to View Show History menu... Gilles Caulier Created attachment 88609 [details]
Last Actions in History View
Last Actions in History View, after this digikam is stuck forever
ok, so I added a kDebug() void DHistoryView::addedEntry(const QString& msg, EntryType type, const QVariant& metadata) { DHistoryViewItem* const item = new DHistoryViewItem(this, msg, type, metadata); kDebug() << msg << "," << metadata; setCurrentItem(item); } and it does "not" freeze, it outputs digikam(8810)/digikam (core) Digikam::DHistoryView::addedEntry: "The files in /store_00010001/.qmf/mail/1405495633.981.N6SOU-parts/ have been listed." , QVariant(QStringList, ("", "") ) digikam(8810)/digikam (core) Digikam::DHistoryView::addedEntry: "Auflisten der Ordner in /store_00010001/.qmf/mail/1405495633.981.N6SOU-parts/ ..." , QVariant(QStringList, ("", "") ) digikam(8810)/digikam (core) Digikam::DHistoryView::addedEntry: "Die Ordner in /store_00010001/.qmf/mail/1405495633.981.N6SOU-parts/ wurden aufgelistet." , QVariant(QStringList, ("", "") ) digikam(8810)/digikam (core) Digikam::DHistoryView::addedEntry: "Listing files in /store_00010001/.qmf/mail/1405495633.981.e56mS-parts/..." , QVariant(QStringList, ("", "") ) digikam(8810)/digikam (core) Digikam::DHistoryView::addedEntry: "The files in /store_00010001/.qmf/mail/1405495633.981.e56mS-parts/ have been listed." , QVariant(QStringList, ("", "") ) digikam(8810)/digikam (core) Digikam::DHistoryView::addedEntry: "Auflisten der Ordner in /store_00010001/.qmf/mail/1405495633.981.e56mS-parts/ ..." , QVariant(QStringList, ("", "") ) and such forever Empty metadata(folder & file)? Looks fishy To resume, without kDebug, it do not crash. Right? This is a non sense. kdebug will just slowdown history entry registration... ... or there is a race condition here. But i cannot see where exactly. A signal is emitted from controller with message content. main import window re-route signal to a slot to create new entry in history view. Sound like the right way to do... Gilles Ok, let me clarify this, with or without kDebug the GUI freezes. I just got the impression that the freeze happens in some Qt Code, but your digikam code is still running and it adds millions of items to the history. (As the kDebug output keeps coming for at least like ~5+min, didn't wait longer) NOTE: I am talking about 126 visible images in digikam here, no clue why it scans *ALL* folders anyway. Maybe thats the main question? But yes, there still might be a race condition here, I vague remember some Qt Bug that caused the horizontal scrollbar (even if not directly visible) to cause endless resizes, but not sure here, I will test/search that as soon as I can. And it never crashes, it only freezes, (or maybe takes more than 10min, didn't wait thaaat long) If bombarding a listview is not safe for Qt, well, just comment "setCurrentItem(item);" to let Qt the time to integrate new items. This solve this issue ? Gilles Caulier Yep this will solve the issue, but then you no longer have the latest item selected&visible. So I tried something, like setting the headers to fixed width, but it didn't help. But then I used scrollToItem(item), instead of setCurrentItem(item) and it was "only" frozen for like ~45sec (NOTE: debugbuild) And just to be sure I tried setCurrentItem(item); again, but I gave up after 8min. scrollToItem does not select the latest Item, but at least it is visible. I tried other stuff as well, but it didn't really help. to sum up: setCurrentItem: 8+min and still frozen (guessing 15min) //current code scrollToItem: ~45sec none: just some very minor freezes like 1-2sec I messured the "frozen"-time by looking at the animated digikam logo. So its up to you, what you think is best? how important is it to see&select the latest item in the history? (given that the history is not shown by default) In case you have more ideas, I will test them as soon as I can :) I think a simple a way to fix it is to replace kDebug() statement by kapp->processEvents(). Does it work for you ? I fact, i suspect that Qt dispatch queued events when debug statement is printed to the console. But the ultimate solution here is to limit HistoryView to be bombarded of messages. Typically, at ImportUI start-up, controller get all thumbnails from camera. This can be very fast, especially with UMS media. I think to log all thumbnail events in HistoryView is not very important for end users. We can limit to to start and end of all thumbs process. Downloading items can take more time. So it's a different situation, and to log download states is more important for end users. What do you think about ? Gilles Caulier Git commit 7285bf0badecd05cc8897c8f516a9e62f03cc1d5 by Gilles Caulier. Committed on 08/09/2014 at 21:19. Pushed by cgilles into branch 'master'. dispatch Qt event to not block gui if history recieve a lot of message to register M +4 -2 libs/widgets/mainview/dhistoryview.cpp M +1 -1 libs/widgets/mainview/dhistoryview.h http://commits.kde.org/digikam/7285bf0badecd05cc8897c8f516a9e62f03cc1d5 Bernd, With my last commit, i add a processEvent() in DHistoryView. This enough to solve your problem ? Gilles Caulier So I updated to git master and tested it. Well, it does not completly "freeze", as the gui still repaints and I can even scroll in the import dialog, (NOTE: the scroll is very laggy and extremly slow) BUT it does not solve the issue, I waited for 5min and it still was busy looking in all subdirs and no image thumbnails was displayed. Btw I also tested a very old camera with way more images, but "no" subdirs, it was waaaay faster, like all image thumbnails where displayed almost instantly. So the subdirs(lookups)&history really are the problem. I would generally vote "yes" to the "drop thumbnail History" idea, but I maybe sometimes later, another person will find a bug were the import(lookup) gets stuck in a specific directory, wouldn't that make the debugging harder for him? But this would be the only real use-case I can think of. BTW: Only the "Listing files" parts in the History causes the problems (as I have millions of them), I only have 126 Images, so the "getting thumbnail" part is just a few quick Lines. Thinking about it... Is the "Listing files in ..." part done async in another (multiple) thread(s)? So can it be that try to add 2 history entries at the exact same time? (just guessing) Bernd, Just for testing, to not recieve thumbnail notification to log in history, just comment this line : https://projects.kde.org/projects/extragear/graphics/digikam/repository/revisions/master/entry/utilities/importui/backend/cameracontroller.cpp#L550 The "Listing Files" stage is managed here from controller : https://projects.kde.org/projects/extragear/graphics/digikam/repository/revisions/master/entry/utilities/importui/backend/cameracontroller.cpp#L511 It call d->camera->getItemsInfoList(). This method time processing depend of : - Camera type : UMS or gphoto. - the arguments passed to get extra metadata. This can be modified in digiKam Camera settings from "Behavior" section. The loop to list file from camera interfaces are here : https://projects.kde.org/projects/extragear/graphics/digikam/repository/revisions/master/entry/utilities/importui/backend/gpcamera.cpp#L736 https://projects.kde.org/projects/extragear/graphics/digikam/repository/revisions/master/entry/utilities/importui/backend/umscamera.cpp#L181 Gilles Caulier Created attachment 88634 [details] no Files in, or foldern in Log patch so for testing I commented https://projects.kde.org/projects/extragear/graphics/digikam/repository/revisions/master/entry/utilities/importui/backend/cameracontroller.cpp#L550 but it did not help, waited 5min then I commented the "Listing files in" log but it still did not help, waited 5min again. then I commented "Listing folders in" log and weeeeeeeeh! I have never seen the import dialog that fast :) NOTE: there is a "Failed to list files in ..." so the missing begin/end "files in" log is not a big problem IMHO If you have commented these lines : case (CameraCommand::cam_listfolders): { QString folder = cmd->map["folder"].toString(); - sendLogMsg(i18n("Listing folders in %1...", folder)); + //sendLogMsg(i18n("Listing folders in %1...", folder)); d->camera->getFolders(folder); - sendLogMsg(i18n("The folders in %1 have been listed.", folder)); + //sendLogMsg(i18n("The folders in %1 have been listed.", folder)); break; } To fix the problem, well, i'm lost... There is nothing special here. Only 2 lines are add to history. There is no bombarding events between controler and GUI... Gilles Caulier Created attachment 88635 [details]
debug counters
digikam(23823)/digikam (core) Digikam::CameraController::executeCommand: Files: 1318
digikam(23823)/digikam (core) Digikam::CameraController::executeCommand: Folders: 1318
so everything x2
(as we have sendLogMsg(i18n("Listing files in %1...", folder)); and sendLogMsg(i18n("The files in %1 have been listed.", folder));)
those things add (1318 Files + 1318 Folders) * 2 = 5272 LogLines
Ah, now i understand. You have plenty of dirs on your media. now, i know what's i need to fix. Gilles Caulier *** Bug 338949 has been marked as a duplicate of this bug. *** While trying the Import again (some days old git master), with open history, I noticed that sometimes an "old" item gets selected, not the latest one, and then it freezes. So there is very likely some kind of race condition here. Git commit 54b51f06b9d413e663a07b64d9bbe2eff69d9713 by Gilles Caulier. Committed on 25/09/2014 at 12:32. Pushed by cgilles into branch 'master'. Only report error or warnings in Import console history. No need to bloat history content with unnecessary contents which can freeze GUI FIXED-IN: 4.4.0 M +2 -1 NEWS M +49 -51 utilities/importui/backend/cameracontroller.cpp M +63 -61 utilities/importui/backend/gpcamera.cpp M +31 -30 utilities/importui/backend/umscamera.cpp http://commits.kde.org/digikam/54b51f06b9d413e663a07b64d9bbe2eff69d9713 |