Version: 2.1.1 (using KDE 4.7.1) OS: Linux I have a collection of 3833 pictures. The root folder contains one image and many, many (1630) subfolders, which may be as many as 8 levels deep. When 'Include Album Sub-Tree' is disabled, the main panel displays the only image in the folder. When 'Include Album Sub-Tree' is enabled, nothing is displayed in the main panel at all, even though it should show all the photos. The photos are files downloaded using the 'DownloadHelper' plugin for Firefox. When I select any (all) subfolders of the collection, their contents are displaying correctly - there is no single photo that cannot be displayed, only when they're all displayed together does nothing show up. Reproducible: Always Steps to Reproduce: Select collection, enable 'Include Album Sub-Tree'. Actual Results: Nothing is displayed in main view. Expected Results: 3833 Photos should be shown. OS: Linux (x86_64) release 2.6.38-11-generic Compiler: gcc
Forgot to add terminal output: // start program rob@SCKubuntu:~$ digikam QSqlDatabasePrivate::removeDatabase: connection 'ConnectionTest' is still in use, all queries will cease to work. digikam(4517)/digikam (core) Digikam::ImageInfoJob::slotResult: Failed to list url: "" digikam(4517)/digikam (core) Digikam::ImageInfoJob::slotResult: Failed to list url: "" //reproduce error **no terminal output here** //close digikam QSqlDatabasePrivate::removeDatabase: connection 'thumbnailDatabase-76961744' is still in use, all queries will cease to work. QSqlDatabasePrivate::removeDatabase: connection 'digikamDatabase-42162912' is still in use, all queries will cease to work. QSqlDatabasePrivate::removeDatabase: connection 'thumbnailDatabase-42162912' is still in use, all queries will cease to work. QSqlDatabasePrivate::removeDatabase: connection 'digikamDatabase-20190400' is still in use, all queries will cease to work. QSqlDatabasePrivate::removeDatabase: connection 'thumbnailDatabase-46776240' is still in use, all queries will cease to work. QSqlDatabasePrivate::removeDatabase: connection 'digikamDatabase-46776240' is still in use, all queries will cease to work. QSqlDatabasePrivate::removeDatabase: connection 'digikamDatabase-76961744' is still in use, all queries will cease to work. rob@SCKubuntu:~$
Not reproducible here. I use it all days with current implementation from git master. QSqlDatabasePrivate::removeDatabase: connection 'thumbnailDatabase-76961744' is still in use, all queries will cease to work. ^^^^ this is not an error message, just a warning from Qt database plugin. Gilles Caulier
I would not be to much surprised if the sorting process hangs or goes in some sort of loop, using this much subfolders 8 levels deep. There are definitly sorting problems here as well. Pictures can show up randomly on different places with no alterations to sort options. Rinus Op 27-09-11 17:41, Gilles Caulier schreef: > https://bugs.kde.org/show_bug.cgi?id=282907 > > > > > > --- Comment #2 from Gilles Caulier<caulier gilles gmail com> 2011-09-27 15:41:03 --- > Not reproducible here. I use it all days with current implementation from git > master. > > QSqlDatabasePrivate::removeDatabase: connection 'thumbnailDatabase-76961744' is > still in use, all queries will cease to work. > > ^^^^ this is not an error message, just a warning from Qt database plugin. > > Gilles Caulier >
To be honest, I don't expect this to be reproducible by anyone who doesn't have quite such a complex folder structure... ...Which I suppose makes tracking it down my job. Does anyone know if there's any debugging output generated by whatever might be causing this that I could look at to try to find the cause?
If you are compiling from current git, I could give some patches which introduce debug output
I am not currently compiling from current git... Hopefully that's all about to change though. I've downloaded the git repo, and I've spent the last 45 mins trying to make it build, but I'm still gathering dependencies... I have other plans now so compiling from source will have to wait, perhaps 'til next week, but if you want to put debug code into the repo, then I can pull down the debug version once you've done it...
OK! I'm sorry it's taken so long, but finally I am compiling from latest git! I followed the instructions at http://www.digikam.org/drupal/download/GIT Either you can put in some debugging code yourself, or, if you can point me in the direction of the likely source of the problem (did you mention it might be the sorting routine?), I'll happily start investigating it myself.
Created attachment 65011 [details] First patch to locate the problem This is a first patch to decide if the problem lies on the ioslave or the model side. Apply with patch -p1 < patch-282907-1.diff in digikam/core/ Please collect the console output while trying to list the failed album (album, not tag or search)
OK, I believe I have successfully applied the patch. Here is the terminal output: rob@SCKubuntu:~$ digikam # press enter QSqlDatabasePrivate::removeDatabase: connection 'ConnectionTest' is still in use, all queries will cease to work. # Collection scanner appears here, scans, then disappears. digikam(11411)/digikam (core) Digikam::ImageInfoJob::slotResult: Failed to list url: "" digikam(11411)/digikam (core) Digikam::ImageInfoJob::slotResult: Failed to list url: "" # GUI appears here, no terminal output while loading the faulty folder (or any other) # I also tried loading other folders and then the faulty one again, and turning 'Include Album Sub-Tree off and on again', still no output. QSqlDatabasePrivate::removeDatabase: connection 'digikamDatabase-140305176791872' is still in use, all queries will cease to work. QSqlDatabasePrivate::removeDatabase: connection 'thumbnailDatabase-97517984' is still in use, all queries will cease to work. QSqlDatabasePrivate::removeDatabase: connection 'digikamDatabase-140305176784480' is still in use, all queries will cease to work. QSqlDatabasePrivate::removeDatabase: connection 'thumbnailDatabase-140305176784480' is still in use, all queries will cease to work. QSqlDatabasePrivate::removeDatabase: connection 'digikamDatabase-41402752' is still in use, all queries will cease to work. QSqlDatabasePrivate::removeDatabase: connection 'digikamDatabase-97517984' is still in use, all queries will cease to work. # quit rob@SCKubuntu:~$
If you dont get any output, you either did not apply the patch (can be checked easily with "git diff" to list local changes), or you did not recompile (after applying the patch to an already compiled digikam, running "make" will only recompile the imagealbummodel.cpp file), or, which is most probable, you are running some system-wide installed digikam instead of the locally changed. For the latter, I recommend to run it directly from the binary directory: Assuming that ${DIGIKAM_BUILD} is the toplevel build directory of your digikam-software-compilation, (one line) LD_LIBRARY_PATH=${DIGIKAM_BUILD}/lib:${LD_LIBRARY_PATH} ${DIGIKAM_BUILD}/core/digikam/digikam
Hi, When I got little or no new output on the console, I thought I hadn't patched it properly, so I checked the imagealbummodel.cpp file and the patch had been applied. I deleted the build directory and rebuilt into an empty folder. I run the program using the digikam command in the terminal. I also get this: rob@SCKubuntu:~$ which digikam /usr/bin/digikam Is that where it should be, or is that a different, system-wide version of digikam?
I have now DEFINITELY run the local version (${DIGIKAM_BUILD}/core/digikam/digikam) where: ${DIGIKAM_BUILD} bash: /home/rob/projects/digiKam/digikam-sc/build: Is a directory I am pretty sure it was built with the patch, and there is still no new output. If you cannot think of any other reason why there would be no output, I can wipe the whole source and build directories, reclone from git, repatch, recompile and rerun. Unless it's possible I've not compiled it in debug mode...? Though that seems unlikely. I've been compiling this this: # mkdir build # cd build # cmake -DCMAKE_BUILD_TYPE=debugfull -DCMAKE_INSTALL_PREFIX=`kde4-config --prefix` .. # make # sudo make install
Ok, looks good. I forgot about kdebugdialog: Please run this tool and check that code 50003 is enabled.
Created attachment 65468 [details] Digikam output 10/11/11 Comments are prefixed with # Only debugging from digiKam core is enabled.
I see that 3370 photos are listed for Fat Photos. Apparently some are shown, as the thumbnail creator starts to created thumbnails for Zillertal flowers. Do you see any photos here? Then you switch to the fat/Downloads collection, and digikam crashes. What's happening here? does it usually crash? Any backtrace?
Yes, I see images in Fat Photos. In fact, I see images in every single subfolder of my 'images' collection, and every other folder in digikam, just not when I try to view the whole 'images' collection at once. Only the version I built seems to crash, my main version doesn't. The backtrace is marked as 'probably not useful' It doesn't automatically find any debug packages. The packages containing debug information for the following application and libraries are missing: /home/rob/projects/digiKam/digikam-sc/build/core/digikam/digikam /usr/lib/libhighgui.so.2.1 /usr/lib/libopencv_highgui.so.2.3 Are there steps I can take to make it more useful? Application: digiKam (digikam), signal: Segmentation fault [Current thread is 1 (Thread 0x7f6137589a20 (LWP 26138))] Thread 4 (Thread 0x7f611595f700 (LWP 26140)): #0 0x00007f6130b4a773 in __GI___poll (fds=<optimized out>, nfds=<optimized out>, timeout=<optimized out>) at ../sysdeps/unix/sysv/linux/poll.c:87 #1 0x00007f612a06bf68 in ?? () from /lib/x86_64-linux-gnu/libglib-2.0.so.0 #2 0x00007f612a06c429 in g_main_context_iteration () from /lib/x86_64-linux-gnu/libglib-2.0.so.0 #3 0x00007f61319c5f3e in QEventDispatcherGlib::processEvents (this=0x16365b0, flags=<optimized out>) at kernel/qeventdispatcher_glib.cpp:424 #4 0x00007f6131999cf2 in QEventLoop::processEvents (this=<optimized out>, flags=...) at kernel/qeventloop.cpp:149 #5 0x00007f6131999ef7 in QEventLoop::exec (this=0x7f611595eb30, flags=...) at kernel/qeventloop.cpp:201 #6 0x00007f61318b127f in QThread::exec (this=<optimized out>) at thread/qthread.cpp:498 #7 0x00007f613197ccbf in QInotifyFileSystemWatcherEngine::run (this=0x154bb00) at io/qfilesystemwatcher_inotify.cpp:248 #8 0x00007f61318b3d05 in QThreadPrivate::start (arg=0x154bb00) at thread/qthread_unix.cpp:331 #9 0x00007f612c5e0efc in start_thread (arg=0x7f611595f700) at pthread_create.c:304 #10 0x00007f6130b5689d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112 #11 0x0000000000000000 in ?? () Thread 3 (Thread 0x7f611515e700 (LWP 26186)): #0 pthread_cond_wait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_wait.S:162 #1 0x00007f61318b41ab in wait (time=18446744073709551615, this=0x1bd81e0) at thread/qwaitcondition_unix.cpp:88 #2 QWaitCondition::wait (this=<optimized out>, mutex=0x1bd7d68, time=18446744073709551615) at thread/qwaitcondition_unix.cpp:160 #3 0x00007f6134d7812b in Digikam::ParkingThread::run (this=0x1bd7d50) at /home/rob/projects/digiKam/digikam-sc/core/libs/threads/threadmanager.cpp:119 #4 0x00007f61318b3d05 in QThreadPrivate::start (arg=0x1bd7d50) at thread/qthread_unix.cpp:331 #5 0x00007f612c5e0efc in start_thread (arg=0x7f611515e700) at pthread_create.c:304 #6 0x00007f6130b5689d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112 #7 0x0000000000000000 in ?? () Thread 2 (Thread 0x7f610e58e700 (LWP 26187)): #0 0x00007f6130b4a773 in __GI___poll (fds=<optimized out>, nfds=<optimized out>, timeout=<optimized out>) at ../sysdeps/unix/sysv/linux/poll.c:87 #1 0x00007f612a06bf68 in ?? () from /lib/x86_64-linux-gnu/libglib-2.0.so.0 #2 0x00007f612a06c429 in g_main_context_iteration () from /lib/x86_64-linux-gnu/libglib-2.0.so.0 #3 0x00007f61319c5f3e in QEventDispatcherGlib::processEvents (this=0x1e0c2f0, flags=<optimized out>) at kernel/qeventdispatcher_glib.cpp:424 #4 0x00007f6131999cf2 in QEventLoop::processEvents (this=<optimized out>, flags=...) at kernel/qeventloop.cpp:149 #5 0x00007f6131999ef7 in QEventLoop::exec (this=0x7f610e58db30, flags=...) at kernel/qeventloop.cpp:201 #6 0x00007f61318b127f in QThread::exec (this=<optimized out>) at thread/qthread.cpp:498 #7 0x00007f613197ccbf in QInotifyFileSystemWatcherEngine::run (this=0x1de3970) at io/qfilesystemwatcher_inotify.cpp:248 #8 0x00007f61318b3d05 in QThreadPrivate::start (arg=0x1de3970) at thread/qthread_unix.cpp:331 #9 0x00007f612c5e0efc in start_thread (arg=0x7f610e58e700) at pthread_create.c:304 #10 0x00007f6130b5689d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:112 #11 0x0000000000000000 in ?? () Thread 1 (Thread 0x7f6137589a20 (LWP 26138)): [KCrash Handler] #6 0x00007f6130add601 in _IO_new_fclose (fp=0x7f612e239170) at iofclose.c:60 #7 0x00007f612e023be2 in cv::RBaseStream::close() () from /usr/lib/libhighgui.so.2.1 #8 0x00007f612e023c27 in cv::RBaseStream::~RBaseStream() () from /usr/lib/libhighgui.so.2.1 #9 0x00007f612e025e60 in cv::BmpDecoder::~BmpDecoder() () from /usr/lib/libhighgui.so.2.1 #10 0x00007f612e025f89 in cv::BmpDecoder::~BmpDecoder() () from /usr/lib/libhighgui.so.2.1 #11 0x00007f60ffdba445 in std::vector<cv::Ptr<cv::BaseImageDecoder>, std::allocator<cv::Ptr<cv::BaseImageDecoder> > >::~vector() () from /usr/lib/libopencv_highgui.so.2.3 #12 0x00007f6130ab0821 in __run_exit_handlers (status=0, listp=0x7f6130e0d5a8, run_list_atexit=true) at exit.c:78 #13 0x00007f6130ab08a5 in __GI_exit (status=<optimized out>) at exit.c:100 #14 0x00007f6130a96314 in __libc_start_main (main=0x6b654d <main(int, char**)>, argc=1, ubp_av=0x7fffc112e368, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fffc112e358) at libc-start.c:258 #15 0x000000000048dd89 in _start ()
Crash is in OpenCV library. It's probably the same problem than report #268267 Gilles Caulier
That problem could very well be present, Giles. However, I have both the packaged version and my own build installed. In the packaged version, I don't get the crash, but I still have the same problem I'm reporting here.
The crash is not in digiKam, but in OpenCV. As i said, look in report #268267. The problem is in OpenCV, which handle libdc1394 when Gphoto2 do it too. There is an init issue with this interface. digiKam do not play with libdc1394. It's an upstream issue in fact... Gilles Caulier
@Giles: I'm not disputing that the crash in the backtrace above is in OpenCV. But I'm reporting that images do not show up, which happens regardless of whether it crashes or not. The crash is a separate issue, and not the main point of this bug report. Unless/until proved otherwise, I'm going to suggest that this is an unrelated issue - the issue discussed in this thread has been happening for months. OpenCV only crashed within the last week.
Created attachment 66205 [details] Terminal output from video 1 (29/11/11)
Created attachment 66206 [details] backtrace from Video 1 (29/11/11)
Please see the video I have uploaded at http://youtu.be/mHvleOhJyHg, showing the fault. Attached are the output and backtrace generated in the process.
Thanks for your help. So you have three collections, 1 and 3 works with recursive-listing, 2 does never. All child albums of collection 2 work. It seems there is one picture in the topmost folder of collection 2. What is displayed in the lower left corner when collection 2 fails to list?
Created attachment 68324 [details] Perl script to make enough folders to demonstrate bug. First and only argument = target folder in which to create 1000 subfolders.
Update!! Sorry it's taken me so long to get around to looking into this, but I finally have the exact problem: An album will not display if the total number of folders/subfolders below it (and including itself) is >= 1000. This is reproducible every time for me. Others should be able to reproduce it by creating a folder with 1000 subfolders (attached script may help, comments on the script are welcome too! - run with `./makeLotsOfDirs "/path/to/testingDirectory"` ) Place any digikam-compatible image file(s) in any subfolder(s), and add the folder as an album in digikam. Select the album root and turn on 'Include Album Sub-Tree'. No images are displayed. Delete 2 or more subfolders and refresh the digikam view... The image(s) is/are displayed. Try adding more subfolders again, refresh the digikam view... No images are displayed once the total number of folders/subfolders in the album is >= 1000. Now, why this should happen (and why 1000, not 1024), I don't know. Anyone more familiar with the workings of digikam care to take a guess? Here's hoping this helps on the way to fixing it! Rob
Thanks for your work. A bug is so easy to fix once you can reproduce... http://www.sqlite.org/limits.html "To prevent excessive memory allocations, the maximum value of a host parameter number is SQLITE_MAX_VARIABLE_NUMBER, which defaults to 999." As you may guess, we give each album as a host parameter. Apparently, SQLite imposes us this limit, at least here MySQL would allow more: http://stackoverflow.com/questions/6581573/what-are-the-max-number-of-allowable-parameters-per-database-provider-type
Git commit e359e0c2c3886b7c90165f2eb4b6e4fafb9a7536 by Marcel Wiesweg. Committed on 19/02/2012 at 22:26. Pushed by mwiesweg into branch 'master'. When listing albums recursively, we may exceed the maximum number of bound parameters. Split the query in this case M +2 -1 NEWS M +17 -5 libs/database/imagelister.cpp http://commits.kde.org/digikam/e359e0c2c3886b7c90165f2eb4b6e4fafb9a7536
Just wanted to confirm the fix works with my collection, which previously didn't display.