Bug 124013

Summary:	Digikam is overwhelmed by a large (60Gb) album
Product:	[Applications] digikam	Reporter:	Greg <noh.way.jose>
Component:	Database-Scan	Assignee:	Digikam Developers <digikam-bugs-null>
Status:	RESOLVED UPSTREAM
Severity:	normal	CC:	caulier.gilles, fabienne, kde, marcel.wiesweg, thomasdn, ubuntu
Priority:	NOR
Version:	0.8.1
Target Milestone:	---
Platform:	unspecified
OS:	Linux
Latest Commit:		Version Fixed In:	2.5.0
Sentry Crash Report:

Description Greg 2006-03-21 10:13:52 UTC

Version:           0.8.1 (using KDE 3.4.2 Level "b" , SUSE 10.0)
Compiler:          Target: i586-suse-linux
OS:                Linux (i686) release 2.6.13-15.8-default

Digicam is overwhelmed by a large (60Gb) album - folders in the album show up but their contents (images) don't. I have a 160Gb SATA hard drive partitioned in two. One of those partitions is called /photos/ and currently contains about 60Gb. It's an LVM volume so that I can chain some more physical space on to it, when needed. Digicam spends a long time opening, as one might expect, but when it does eventually open, the image list pane is empty, even though all the folders (and their subfolders show up in the tree pane.

Comment 1 Greg 2006-03-21 10:18:41 UTC

I presume that this is a database issue but I can't find any settings to control any aspect of the database. Eg, size limits, location, which database (mySQL, DB2, Cloudscape...), ACL settings, etc

Comment 2 caulier.gilles 2006-03-21 10:29:00 UTC

digikam use sqlite database.

Gilles Caulier

Comment 3 Tom Albers 2006-03-21 11:36:51 UTC

Start digikam from konsole and post the output here.

Comment 4 Greg 2006-03-22 10:45:59 UTC

I've captured a small sample to post. See below. I'm also running digikam and piping the output to a file, so I can see it all. It takes a long time, so I'm just posting this snippet for the moment (Gotta go to work) and I'll post more this evening, if there's anything different in the full trace than this...

Cheers,

Greg
-----------------------------------
digikam: WARNING: sqlite_step error: attempt to write a readonly database on query: REPLACE INTO Images ( caption , datetime, name, dirid )  VALUES ('','2004-03-30T00:05:20','IMG_1710.JPG',162)
digikam: WARNING: sqlite_step error: attempt to write a readonly database on query: REPLACE INTO Images ( caption , datetime, name, dirid )  VALUES ('','2004-03-30T00:06:55','IMG_1711.JPG',162)
digikam: WARNING: sqlite_step error: attempt to write a readonly database on query: REPLACE INTO Images ( caption , datetime, name, dirid )  VALUES ('','2004-03-30T00:25:57','IMG_1712.JPG',162)
------------------------------------
and later
------------------------------------
setmntent: Too many open files
setmntent: Too many open files
setmntent: Too many open files
Couldn't open /opt/kde3/share/config/kdebug.areas
digikam: WARNING: KIPI::PluginLoader:: createInstanceFromLibrary returned 0 for JPEGLossless (kipiplugin_jpeglossless) with error number 3
digikam: WARNING: KLibLoader says: file not found
digikam: WARNING: KIPI::PluginLoader:: createInstanceFromLibrary returned 0 for CDArchiving (kipiplugin_cdarchiving) with error number 3
digikam: WARNING: KLibLoader says: file not found
digikam: WARNING: KIPI::PluginLoader:: createInstanceFromLibrary returned 0 for ImagesGallery (kipiplugin_imagesgallery) with error number 3
digikam: WARNING: KLibLoader says: file not found
digikam: WARNING: KIPI::PluginLoader:: createInstanceFromLibrary returned 0 for SendImages (kipiplugin_sendimages) with error number 3
digikam: WARNING: KLibLoader says: file not found
digikam: WARNING: KIPI::PluginLoader:: createInstanceFromLibrary returned 0 for Flickr Exporter (kipiplugin_flickrexport) with error number 3
digikam: WARNING: KLibLoader says: file not found
digikam: WARNING: KIPI::PluginLoader:: createInstanceFromLibrary returned 0 for Calendar (kipiplugin_calendar) with error number 3
digikam: WARNING: KLibLoader says: file not found
digikam: WARNING: KIPI::PluginLoader:: createInstanceFromLibrary returned 0 for MPEGEncoder (kipiplugin_mpegencoder) with error number 3
digikam: WARNING: KLibLoader says: file not found
digikam: WARNING: KIPI::PluginLoader:: createInstanceFromLibrary returned 0 for AcquireImages (kipiplugin_acquireimages) with error number 3
digikam: WARNING: KLibLoader says: file not found
digikam: WARNING: KIPI::PluginLoader:: createInstanceFromLibrary returned 0 for Remote Gallery Export (kipiplugin_galleryexport) with error number 3
digikam: WARNING: KLibLoader says: file not found
digikam: WARNING: KIPI::PluginLoader:: createInstanceFromLibrary returned 0 for TimeDateAdjust (kipiplugin_timeadjust) with error number 3
digikam: WARNING: KLibLoader says: file not found
digikam: WARNING: KIPI::PluginLoader:: createInstanceFromLibrary returned 0 for FindImages (kipiplugin_findimages) with error number 3
digikam: WARNING: KLibLoader says: file not found
digikam: WARNING: KIPI::PluginLoader:: createInstanceFromLibrary returned 0 for PrintWizard (kipiplugin_printwizard) with error number 3
digikam: WARNING: KLibLoader says: file not found
digikam: WARNING: KIPI::PluginLoader:: createInstanceFromLibrary returned 0 for WallPaper (kipiplugin_wallpaper) with error number 3
digikam: WARNING: KLibLoader says: file not found
digikam: WARNING: KIPI::PluginLoader:: createInstanceFromLibrary returned 0 for SlideShow (kipiplugin_slideshow) with error number 3
digikam: WARNING: KLibLoader says: file not found
digikam: WARNING: KIPI::PluginLoader:: createInstanceFromLibrary returned 0 for BatchProcessImages (kipiplugin_batchprocessimages) with error number 3
digikam: WARNING: KLibLoader says: file not found
digikam: WARNING: KIPI::PluginLoader:: createInstanceFromLibrary returned 0 for RawConverter (kipiplugin_rawconverter) with error number 3
digikam: WARNING: KLibLoader says: file not found
digikam: WARNING: KIPI::PluginLoader:: createInstanceFromLibrary returned 0 for ImagePlugin_Core (digikamimageplugin_core) with error number 3
digikam: WARNING: KLibLoader says: file not found
KTempFile: Error trying to create /home/greg/.kde/share/config/digikamrc.lockXXXXXX.tmp: Too many open files
KTempFile: Error trying to create /home/greg/.kde/share/config/digikamrc.lockXXXXXX.tmp: Too many open files
KTempFile: Error trying to create /home/greg/.kde/share/config/digikamrc.lockXXXXXX.tmp: Too many open files
KTempFile: Error trying to create /home/greg/.kde/share/config/digikamrc.lockXXXXXX.tmp: Too many open files
KTempFile: Error trying to create /home/greg/.kde/share/config/digikamrc.lockXXXXXX.tmp: Too many open files
KTempFile: Error trying to create /home/greg/.kde/share/config/digikamrcXXXXXX.new: Too many open files
KTempFile: Error trying to create /home/greg/.kde/share/config/kdeglobals.lockXXXXXX.tmp: Too many open files
KTempFile: Error trying to create /home/greg/.kde/share/config/kdeglobals.lockXXXXXX.tmp: Too many open files
KTempFile: Error trying to create /home/greg/.kde/share/config/kdeglobals.lockXXXXXX.tmp: Too many open files
KTempFile: Error trying to create /home/greg/.kde/share/config/kdeglobals.lockXXXXXX.tmp: Too many open files
KTempFile: Error trying to create /home/greg/.kde/share/config/kdeglobals.lockXXXXXX.tmp: Too many open files
KTempFile: Error trying to create /home/greg/.kde/share/config/kdeglobalsXXXXXX.new: Too many open files
QImage::pixel: x=0 out of range
digikam: WARNING: KLocale: trying to look up "" in catalog. Fix the program
KTempFile: Error trying to create /tmp/ksocket-greg/digikamXXXXXX.slave-socket: Too many open files
KTempFile: Error trying to create /tmp/ksocket-greg/digikamXXXXXX.slave-socket: Too many open files
digikam: ERROR: : couldn't create slave : Unable to create io-slave: Too many open files
KTempFile: Error trying to create /tmp/ksocket-greg/digikamXXXXXX.slave-socket: Too many open files
digikam: ERROR: : couldn't create slave : Unable to create io-slave: Too many open files
digikam: WARNING: [void AlbumManager::slotResult(KIO::Job*)] Failed to list dates
KTempFile: Error trying to create /tmp/ksocket-greg/digikamXXXXXX.slave-socket: Too many open files
digikam: ERROR: : couldn't create slave : Unable to create io-slave: Too many open files
digikam: WARNING: Failed to list url: Could not start process Unable to create io-slave: Too many open files.
KTempFile: Error trying to create /home/greg/.kde/share/config/digikamrc.lockXXXXXX.tmp: Too many open files
KTempFile: Error trying to create /home/greg/.kde/share/config/digikamrc.lockXXXXXX.tmp: Too many open files
KTempFile: Error trying to create /home/greg/.kde/share/config/digikamrc.lockXXXXXX.tmp: Too many open files
KTempFile: Error trying to create /home/greg/.kde/share/config/digikamrc.lockXXXXXX.tmp: Too many open files
KTempFile: Error trying to create /home/greg/.kde/share/config/digikamrc.lockXXXXXX.tmp: Too many open files
KTempFile: Error trying to create /home/greg/.kde/share/config/digikamrcXXXXXX.new: Too many open files
KTempFile: Error trying to create /home/greg/.kde/share/config/kdeglobals.lockXXXXXX.tmp: Too many open files
KTempFile: Error trying to create /home/greg/.kde/share/config/kdeglobals.lockXXXXXX.tmp: Too many open files
KTempFile: Error trying to create /home/greg/.kde/share/config/kdeglobals.lockXXXXXX.tmp: Too many open files
KTempFile: Error trying to create /home/greg/.kde/share/config/kdeglobals.lockXXXXXX.tmp: Too many open files
KTempFile: Error trying to create /home/greg/.kde/share/config/kdeglobals.lockXXXXXX.tmp: Too many open files
KTempFile: Error trying to create /home/greg/.kde/share/config/kdeglobalsXXXXXX.new: Too many open files
KTempFile: Error trying to create /tmp/ksocket-greg/digikamXXXXXX.slave-socket: Too many open files
digikam: ERROR: : couldn't create slave : Unable to create io-slave: Too many open files
digikam: WARNING: Failed to list url: Could not start process Unable to create io-slave: Too many open files.
KTempFile: Error trying to create /tmp/ksocket-greg/digikamXXXXXX.slave-socket: Too many open files
digikam: ERROR: : couldn't create slave : Unable to create io-slave: Too many open files
digikam: WARNING: Failed to list url: Could not start process Unable to create io-slave: Too many open files.
------------------------------------

Comment 5 Tom Albers 2006-03-22 11:30:55 UTC

"attempt to write a readonly database", could it be that this is correct? Are you using digikam on an nfs mounted folder perhaps?

Comment 6 Greg 2006-03-24 02:15:11 UTC

The folder in question is a partition on an internal drive on the same machine as I'm running digikam. Is it possible to run an NFS volume locally and how would I tell?

my fstab has an entry for /photo/

/dev/sda1            /                    ext3       acl,user_xattr        1 1
/dev/sdb6            /photos              ext3       defaults              1 2
/dev/system/LVhome   /home                reiserfs   acl,user_xattr        1 2
/dev/system/LVopt    /opt                 reiserfs   acl,user_xattr        1 2
/dev/sdb7            /arch                ext3       defaults              1 2
/dev/system/LVtmp    /tmp                 reiserfs   acl,user_xattr        1 2
/dev/system/LVusr    /usr                 reiserfs   acl,user_xattr        1 2
/dev/system/LVvar    /var                 reiserfs   acl,user_xattr        1 2
/dev/hda1            /windows/C           ntfs       ro,users,gid=users,umask=0002,nls=utf8 0 0
/dev/sda5            swap                 swap       defaults              0 0
/dev/sdb1            swap                 swap       defaults              0 0
proc                 /proc                proc       defaults              0 0
sysfs                /sys                 sysfs      noauto                0 0
usbfs                /proc/bus/usb        usbfs      noauto                0 0
devpts               /dev/pts             devpts     mode=0620,gid=5       0 0
/dev/dvdram          /media/dvdram        subfs      noauto,fs=cdfss,ro,procuid,nosuid,nodev,exec,iocharset=utf8 0 0
/dev/cdrecorder      /media/cdrecorder    subfs      noauto,fs=cdfss,ro,procuid,nosuid,nodev,exec,iocharset=utf8 0 0
/dev/fd0             /media/floppy        subfs      noauto,fs=floppyfss,procuid,nodev,nosuid,sync 0 0

Comment 7 Greg 2006-03-24 02:17:29 UTC

I meant "my fstab has an entry for /photos" (sorry for the typo)

Comment 8 Xesc Arbona 2006-03-28 16:22:48 UTC

I have a similar problem. Digikam doesn't load any dikigamimageplugin. There ara no icons nor photos shown, only hierarchy of albums. I have 12GB of photos. 

I'm using 0.8.1-4 Debian version and I've tried to downgrade to dikigam 0.7.2 version, but the problem remains the same.

I've run showfoto (from the same deb package) and it works fine. All the plugins are loaded.

The ouput I get from the console is:

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

kbuildsycoca running...
setmntent: El procés té massa fitxers oberts
setmntent: El procés té massa fitxers oberts
setmntent: El procés té massa fitxers oberts
setmntent: El procés té massa fitxers oberts
setmntent: El procés té massa fitxers oberts
setmntent: El procés té massa fitxers oberts
setmntent: El procés té massa fitxers oberts
setmntent: El procés té massa fitxers oberts
setmntent: El procés té massa fitxers oberts
...
setmntent: El procés té massa fitxers oberts
setmntent: El procés té massa fitxers oberts
setmntent: El procés té massa fitxers oberts
setmntent: El procés té massa fitxers oberts
setmntent: El procés té massa fitxers oberts
Couldn't open /etc/kde3/kdebug.areas
digikam: KIPI::PluginLoader: plugin KameraKlient is in the ignore list for host application
digikam: WARNING: KIPI::PluginLoader:: createInstanceFromLibrary returned 0 for JPEGLossless (kipiplugin_jpeglossless) with error number 3
digikam: WARNING: KLibLoader says: file not found
digikam: WARNING: KIPI::PluginLoader:: createInstanceFromLibrary returned 0 for CDArchiving (kipiplugin_cdarchiving) with error number 3
digikam: WARNING: KLibLoader says: file not found
digikam: WARNING: KIPI::PluginLoader:: createInstanceFromLibrary returned 0 for ImagesGallery (kipiplugin_imagesgallery) with error number 3
digikam: WARNING: KLibLoader says: file not found
digikam: WARNING: KIPI::PluginLoader:: createInstanceFromLibrary returned 0 for SendImages (kipiplugin_sendimages) with error number 3
digikam: WARNING: KLibLoader says: file not found
digikam: WARNING: KIPI::PluginLoader:: createInstanceFromLibrary returned 0 for Flickr Exporter (kipiplugin_flickrexport) with error number 3
digikam: WARNING: KLibLoader says: file not found
digikam: WARNING: KIPI::PluginLoader:: createInstanceFromLibrary returned 0 for Calendar (kipiplugin_calendar) with error number 3
digikam: WARNING: KLibLoader says: file not found
digikam: WARNING: KIPI::PluginLoader:: createInstanceFromLibrary returned 0 for MPEGEncoder (kipiplugin_mpegencoder) with error number 3
digikam: WARNING: KLibLoader says: file not found
digikam: WARNING: KIPI::PluginLoader:: createInstanceFromLibrary returned 0 for AcquireImages (kipiplugin_acquireimages) with error number 3
digikam: WARNING: KLibLoader says: file not found
digikam: WARNING: KIPI::PluginLoader:: createInstanceFromLibrary returned 0 for Remote Gallery Export (kipiplugin_galleryexport) with error number 3
digikam: WARNING: KLibLoader says: file not found
digikam: WARNING: KIPI::PluginLoader:: createInstanceFromLibrary returned 0 for TimeDateAdjust (kipiplugin_timeadjust) with error number 3
digikam: WARNING: KLibLoader says: file not found
digikam: WARNING: KIPI::PluginLoader:: createInstanceFromLibrary returned 0 for FindImages (kipiplugin_findimages) with error number 3
digikam: WARNING: KLibLoader says: file not found
digikam: WARNING: KIPI::PluginLoader:: createInstanceFromLibrary returned 0 for PrintWizard (kipiplugin_printwizard) with error number 3
digikam: WARNING: KLibLoader says: file not found
digikam: WARNING: KIPI::PluginLoader:: createInstanceFromLibrary returned 0 for WallPaper (kipiplugin_wallpaper) with error number 3
digikam: WARNING: KLibLoader says: file not found
digikam: WARNING: KIPI::PluginLoader:: createInstanceFromLibrary returned 0 for SlideShow (kipiplugin_slideshow) with error number 3
digikam: WARNING: KLibLoader says: file not found
digikam: WARNING: KIPI::PluginLoader:: createInstanceFromLibrary returned 0 for BatchProcessImages (kipiplugin_batchprocessimages) with error number 3
digikam: WARNING: KLibLoader says: file not found
digikam: WARNING: KIPI::PluginLoader:: createInstanceFromLibrary returned 0 for SimpleViewer (kipiplugin_simpleviewer) with error number 3
digikam: WARNING: KLibLoader says: file not found
digikam: WARNING: KIPI::PluginLoader:: createInstanceFromLibrary returned 0 for RawConverter (kipiplugin_rawconverter) with error number 3
digikam: WARNING: KLibLoader says: file not found
digikam: WARNING: KIPI::PluginLoader:: createInstanceFromLibrary returned 0 for ImagePlugin_Core (digikamimageplugin_core) with error number 3
digikam: WARNING: KLibLoader says: file not found
digikam: WARNING: KTempFile: Error trying to create /tmp/ksocket-xescRRkyVF/digikamXXXXXX.slave-socket: El proc??s t?? massa fitxers oberts
digikam: WARNING: KTempFile: Error trying to create /tmp/ksocket-xescRRkyVF/digikamXXXXXX.slave-socket: El proc??s t?? massa fitxers oberts
digikam: ERROR: : couldn't create slave : Unable to create io-slave: El procés té massa fitxers oberts
digikam: WARNING: KTempFile: Error trying to create /tmp/ksocket-xescRRkyVF/digikamXXXXXX.slave-socket: El proc??s t?? massa fitxers oberts
digikam: ERROR: : couldn't create slave : Unable to create io-slave: El procés té massa fitxers oberts
digikam: WARNING: [void AlbumManager::slotResult(KIO::Job*)] Failed to list dates
digikam: WARNING: KTempFile: Error trying to create /tmp/ksocket-xescRRkyVF/digikamXXXXXX.slave-socket: El proc??s t?? massa fitxers oberts
digikam: ERROR: : couldn't create slave : Unable to create io-slave: El procés té massa fitxers oberts
digikam: WARNING: Failed to list url: Could not start process Unable to create io-slave: El procés té massa fitxers oberts.
digikam: WARNING: KTempFile: Error trying to create /tmp/ksocket-xescRRkyVF/digikamXXXXXX.slave-socket: El proc??s t?? massa fitxers oberts
digikam: ERROR: : couldn't create slave : Unable to create io-slave: El procés té massa fitxers oberts
digikam: WARNING: KTempFile: Error trying to create /tmp/ksocket-xescRRkyVF/digikamXXXXXX.slave-socket: El proc??s t?? massa fitxers oberts
digikam: ERROR: : couldn't create slave : Unable to create io-slave: El procés té massa fitxers oberts
digikam: WARNING: KTempFile: Error trying to create /tmp/ksocket-xescRRkyVF/digikamXXXXXX.slave-socket: El proc??s t?? massa fitxers oberts
digikam: ERROR: : couldn't create slave : Unable to create io-slave: El procés té massa fitxers oberts
digikam: WARNING: KTempFile: Error trying to create /tmp/ksocket-xescRRkyVF/digikamXXXXXX.slave-socket: El proc??s t?? massa fitxers oberts
digikam: ERROR: : couldn't create slave : Unable to create io-slave: El procés té massa fitxers oberts
digikam: WARNING: KTempFile: Error trying to create /tmp/ksocket-xescRRkyVF/digikamXXXXXX.slave-socket: El proc??s t?? massa fitxers oberts
digikam: ERROR: : couldn't create slave : Unable to create io-slave: El procés té massa fitxers oberts
digikam: WARNING: KTempFile: Error trying to create /tmp/ksocket-xescRRkyVF/digikamXXXXXX.slave-socket: El proc??s t?? massa fitxers oberts
digikam: ERROR: : couldn't create slave : Unable to create io-slave: El procés té massa fitxers oberts
digikam: WARNING: KTempFile: Error trying to create /tmp/ksocket-xescRRkyVF/digikamXXXXXX.slave-socket: El proc??s t?? massa fitxers oberts
digikam: ERROR: : couldn't create slave : Unable to create io-slave: El procés té massa fitxers oberts
digikam: WARNING: KTempFile: Error trying to create /tmp/ksocket-xescRRkyVF/digikamXXXXXX.slave-socket: El proc??s t?? massa fitxers oberts
digikam: ERROR: : couldn't create slave : Unable to create io-slave: El procés té massa fitxers oberts
digikam: WARNING: KTempFile: Error trying to create /tmp/ksocket-xescRRkyVF/digikamXXXXXX.slave-socket: El proc??s t?? massa fitxers oberts
digikam: ERROR: : couldn't create slave : Unable to create io-slave: El procés té massa fitxers oberts
digikam: WARNING: KTempFile: Error trying to create /tmp/ksocket-xescRRkyVF/digikamXXXXXX.slave-socket: El proc??s t?? massa fitxers oberts
digikam: ERROR: : couldn't create slave : Unable to create io-slave: El procés té massa fitxers oberts
digikam: WARNING: KTempFile: Error trying to create /tmp/ksocket-xescRRkyVF/digikamXXXXXX.slave-socket: El proc??s t?? massa fitxers oberts
digikam: ERROR: : couldn't create slave : Unable to create io-slave: El procés té massa fitxers oberts
digikam: WARNING: KTempFile: Error trying to create /tmp/ksocket-xescRRkyVF/digikamXXXXXX.slave-socket: El proc??s t?? massa fitxers oberts
digikam: ERROR: : couldn't create slave : Unable to create io-slave: El procés té massa fitxers oberts
digikam: WARNING: KTempFile: Error trying to create /tmp/ksocket-xescRRkyVF/digikamXXXXXX.slave-socket: El proc??s t?? massa fitxers oberts
digikam: ERROR: : couldn't create slave : Unable to create io-slave: El procés té massa fitxers oberts
digikam: WARNING: KTempFile: Error trying to create /tmp/ksocket-xescRRkyVF/digikamXXXXXX.slave-socket: El proc??s t?? massa fitxers oberts
digikam: ERROR: : couldn't create slave : Unable to create io-slave: El procés té massa fitxers oberts
digikam: WARNING: KTempFile: Error trying to create /tmp/ksocket-xescRRkyVF/digikamXXXXXX.slave-socket: El proc??s t?? massa fitxers oberts
digikam: ERROR: : couldn't create slave : Unable to create io-slave: El procés té massa fitxers oberts
digikam: WARNING: KTempFile: Error trying to create /tmp/ksocket-xescRRkyVF/digikamXXXXXX.slave-socket: El proc??s t?? massa fitxers oberts
digikam: ERROR: : couldn't create slave : Unable to create io-slave: El procés té massa fitxers oberts
digikam: WARNING: KTempFile: Error trying to create /tmp/ksocket-xescRRkyVF/digikamXXXXXX.slave-socket: El proc??s t?? massa fitxers oberts
digikam: ERROR: : couldn't create slave : Unable to create io-slave: El procés té massa fitxers oberts
digikam: WARNING: KTempFile: Error trying to create /tmp/ksocket-xescRRkyVF/digikamXXXXXX.slave-socket: El proc??s t?? massa fitxers oberts
digikam: ERROR: : couldn't create slave : Unable to create io-slave: El procés té massa fitxers oberts
digikam: WARNING: KTempFile: Error trying to create /tmp/ksocket-xescRRkyVF/digikamXXXXXX.slave-socket: El proc??s t?? massa fitxers oberts
digikam: ERROR: : couldn't create slave : Unable to create io-slave: El procés té massa fitxers oberts
digikam: WARNING: KTempFile: Error trying to create /tmp/ksocket-xescRRkyVF/digikamXXXXXX.slave-socket: El proc??s t?? massa fitxers oberts
digikam: ERROR: : couldn't create slave : Unable to create io-slave: El procés té massa fitxers oberts
digikam: WARNING: KTempFile: Error trying to create /tmp/ksocket-xescRRkyVF/digikamXXXXXX.slave-socket: El proc??s t?? massa fitxers oberts
digikam: ERROR: : couldn't create slave : Unable to create io-slave: El procés té massa fitxers oberts
digikam: WARNING: KTempFile: Error trying to create /tmp/ksocket-xescRRkyVF/digikamXXXXXX.slave-socket: El proc??s t?? massa fitxers oberts
digikam: ERROR: : couldn't create slave : Unable to create io-slave: El procés té massa fitxers oberts
digikam: WARNING: KTempFile: Error trying to create /tmp/ksocket-xescRRkyVF/digikamXXXXXX.slave-socket: El proc??s t?? massa fitxers oberts
digikam: ERROR: : couldn't create slave : Unable to create io-slave: El procés té massa fitxers oberts
digikam: WARNING: KTempFile: Error trying to create /tmp/ksocket-xescRRkyVF/digikamXXXXXX.slave-socket: El proc??s t?? massa fitxers oberts
digikam: ERROR: : couldn't create slave : Unable to create io-slave: El procés té massa fitxers oberts
digikam: WARNING: KTempFile: Error trying to create /tmp/ksocket-xescRRkyVF/digikamXXXXXX.slave-socket: El proc??s t?? massa fitxers oberts
digikam: ERROR: : couldn't create slave : Unable to create io-slave: El procés té massa fitxers oberts
digikam: WARNING: KTempFile: Error trying to create /tmp/ksocket-xescRRkyVF/digikamXXXXXX.slave-socket: El proc??s t?? massa fitxers oberts
digikam: ERROR: : couldn't create slave : Unable to create io-slave: El procés té massa fitxers oberts
digikam: WARNING: KTempFile: Error trying to create /tmp/ksocket-xescRRkyVF/digikamXXXXXX.slave-socket: El proc??s t?? massa fitxers oberts
digikam: ERROR: : couldn't create slave : Unable to create io-slave: El procés té massa fitxers oberts
digikam: WARNING: KTempFile: Error trying to create /tmp/ksocket-xescRRkyVF/digikamXXXXXX.slave-socket: El proc??s t?? massa fitxers oberts
digikam: ERROR: : couldn't create slave : Unable to create io-slave: El procés té massa fitxers oberts
digikam: WARNING: KTempFile: Error trying to create /tmp/ksocket-xescRRkyVF/digikamXXXXXX.slave-socket: El proc??s t?? massa fitxers oberts
digikam: ERROR: : couldn't create slave : Unable to create io-slave: El procés té massa fitxers oberts
digikam: WARNING: KTempFile: Error trying to create /tmp/ksocket-xescRRkyVF/digikamXXXXXX.slave-socket: El proc??s t?? massa fitxers oberts
digikam: ERROR: : couldn't create slave : Unable to create io-slave: El procés té massa fitxers oberts
digikam: WARNING: KTempFile: Error trying to create /tmp/ksocket-xescRRkyVF/digikamXXXXXX.slave-socket: El proc??s t?? massa fitxers oberts
digikam: ERROR: : couldn't create slave : Unable to create io-slave: El procés té massa fitxers oberts
digikam: WARNING: KTempFile: Error trying to create /tmp/ksocket-xescRRkyVF/digikamXXXXXX.slave-socket: El proc??s t?? massa fitxers oberts
digikam: ERROR: : couldn't create slave : Unable to create io-slave: El procés té massa fitxers oberts
digikam: WARNING: KTempFile: Error trying to create /tmp/ksocket-xescRRkyVF/digikamXXXXXX.slave-socket: El proc??s t?? massa fitxers oberts
digikam: ERROR: : couldn't create slave : Unable to create io-slave: El procés té massa fitxers oberts
digikam: WARNING: KTempFile: Error trying to create /tmp/ksocket-xescRRkyVF/digikamXXXXXX.slave-socket: El proc??s t?? massa fitxers oberts
digikam: ERROR: : couldn't create slave : Unable to create io-slave: El procés té massa fitxers oberts
digikam: WARNING: KTempFile: Error trying to create /tmp/ksocket-xescRRkyVF/digikamXXXXXX.slave-socket: El proc??s t?? massa fitxers oberts
digikam: ERROR: : couldn't create slave : Unable to create io-slave: El procés té massa fitxers oberts
digikam: WARNING: KTempFile: Error trying to create /tmp/ksocket-xescRRkyVF/digikamXXXXXX.slave-socket: El proc??s t?? massa fitxers oberts
digikam: ERROR: : couldn't create slave : Unable to create io-slave: El procés té massa fitxers oberts
digikam: WARNING: KTempFile: Error trying to create /tmp/ksocket-xescRRkyVF/digikamXXXXXX.slave-socket: El proc??s t?? massa fitxers oberts
digikam: ERROR: : couldn't create slave : Unable to create io-slave: El procés té massa fitxers oberts
digikam: WARNING: KTempFile: Error trying to create /tmp/ksocket-xescRRkyVF/digikamXXXXXX.slave-socket: El proc??s t?? massa fitxers oberts
digikam: ERROR: : couldn't create slave : Unable to create io-slave: El procés té massa fitxers oberts
digikam: WARNING: KTempFile: Error trying to create /tmp/ksocket-xescRRkyVF/digikamXXXXXX.slave-socket: El proc??s t?? massa fitxers oberts
digikam: ERROR: : couldn't create slave : Unable to create io-slave: El procés té massa fitxers oberts
digikam: WARNING: KTempFile: Error trying to create /tmp/ksocket-xescRRkyVF/digikamXXXXXX.slave-socket: El proc??s t?? massa fitxers oberts
digikam: ERROR: : couldn't create slave : Unable to create io-slave: El procés té massa fitxers oberts
digikam: WARNING: KTempFile: Error trying to create /tmp/ksocket-xescRRkyVF/digikamXXXXXX.slave-socket: El proc??s t?? massa fitxers oberts
digikam: ERROR: : couldn't create slave : Unable to create io-slave: El procés té massa fitxers oberts
digikam: WARNING: KTempFile: Error trying to create /home/xesc/.kde/share/config/digikamrc.lockXXXXXX.tmp: El proc??s t?? massa fitxers oberts
digikam: WARNING: KTempFile: Error trying to create /home/xesc/.kde/share/config/digikamrc.lockXXXXXX.tmp: El proc??s t?? massa fitxers oberts
digikam: WARNING: KTempFile: Error trying to create /home/xesc/.kde/share/config/digikamrc.lockXXXXXX.tmp: El proc??s t?? massa fitxers oberts
digikam: WARNING: KTempFile: Error trying to create /home/xesc/.kde/share/config/digikamrc.lockXXXXXX.tmp: El proc??s t?? massa fitxers oberts
digikam: WARNING: KTempFile: Error trying to create /home/xesc/.kde/share/config/digikamrc.lockXXXXXX.tmp: El proc??s t?? massa fitxers oberts
digikam: WARNING: KTempFile: Error trying to create /home/xesc/.kde/share/config/digikamrcXXXXXX.new: El proc??s t?? massa fitxers oberts
digikam: WARNING: KTempFile: Error trying to create /home/xesc/.kde/share/config/kdeglobals.lockXXXXXX.tmp: El proc??s t?? massa fitxers oberts
digikam: WARNING: KTempFile: Error trying to create /home/xesc/.kde/share/config/kdeglobals.lockXXXXXX.tmp: El proc??s t?? massa fitxers oberts
digikam: WARNING: KTempFile: Error trying to create /home/xesc/.kde/share/config/kdeglobals.lockXXXXXX.tmp: El proc??s t?? massa fitxers oberts
digikam: WARNING: KTempFile: Error trying to create /home/xesc/.kde/share/config/kdeglobals.lockXXXXXX.tmp: El proc??s t?? massa fitxers oberts
digikam: WARNING: KTempFile: Error trying to create /home/xesc/.kde/share/config/kdeglobals.lockXXXXXX.tmp: El proc??s t?? massa fitxers oberts
digikam: WARNING: KTempFile: Error trying to create /home/xesc/.kde/share/config/kdeglobalsXXXXXX.new: El proc??s t?? massa fitxers oberts

<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

Ah, "El proc??s t?? massa fitxers oberts" is a message in catalan that means "too many files opened".

Hope this will hope to solve the problem. Just ask if you need more information about my system.

Comment 9 Greg 2006-04-11 09:06:33 UTC

Could there be an ACL (users & groups) permissions fix for this problem? _ If not WHAT?

Please will someone on the team have a go at addressing this problem or suggest a fix. At worst, an indication of when it might make it to the top of the pile would be appreciated (to the nearest season) - 

DON'T flame me for asking!!

I look forward to a polite, quick and helpful response. 

Cheers, 

Greg

Comment 10 Ignacio Serantes 2006-04-23 21:02:19 UTC

I have the same problem but if I remove several directories the programs works fine again.
I have SuSE 9.3 and the problem occurs with 0.8.1 rpm version and with 0.9 cvs version compiled by me.

Comment 11 Marcel Wiesweg 2006-04-23 22:26:54 UTC

How many files does the directory contain?

Please try to set "Scan at start" in the Miscellaneuos settings tab to false, and start again, check if the "too many open files" error still occurs.
If it does, please check that the db has been scanned at least once (open your digikam3.db file with the "sqlite3" tool and type "SELECT value FROM Settings WHERE keyword='Scanned';", should be some date).

If the read-only-database error still occurs, please assert that you have write permissions to your digikam3.db file. Open it with the "sqlite3" tool and type something like
REPLACE INTO Images ( caption , datetime, name, dirid )  VALUES ('','2004-03-30T00:05:20','IMG_1710.JPG',162);
which is taken from the log you posted.

Comment 12 Ignacio Serantes 2006-04-23 23:41:57 UTC

I allways have "scan at start" disabled and I can modify digikam3.db using knoda. The result for the query is one record with the value 2006-04-23T19:40:09.

I'm using symlinks and the yesterday cvs version of digikam 0.9.

I don't know if the next information are useful. I follow the next steps:

1) Scan at start is disabled.
2) I do a scan and the program fails. Not all the albums (directories) are scaned.
3) Exit the program and enter again. The program has no icons, no theme and the searches don't work.
4) Exit the program.
5) I remove some symlinks related to albums.
5) Enter again without scan and the program works again without problem because the program don't detect/open/scan/search/enter? that albums.

If I do a manual scan digikam detects the removed albums.
If I restore de symlinks the programa fails again.

The big numbers are:
The program works with 1031 folders and 17910 images.
The program fails with 1055 folders and 18195 images.

Comment 13 Marc C 2006-09-04 23:59:49 UTC

Hello, First Thanks for this great tool that really rocks !!!


Now the BUT I have an issue : "Too Many Open files" Error

The picture directory has 1327 directories and 18992 files.

I'm using Kubuntu/Dapper and digikam 0.9-beta1 from http://kubuntu.omat.nl/dapper/ , I tried with the dapper one and the problem remain the same.
Qt: 3.3.6
KDE: 3.5.4

below an extract of digikam messages while launched.

Is it possible that the problem is linked to KDE ? I didn't upload new picture since last time it worked.The only thing that changed really is KDE 3.5.4 upgrade from kubuntu.org.

If needed I can run some specific test.

kio (KSycoca): Trying to open ksycoca from /var/tmp/kdecache-oniryx/ksycoca
ScimInputContextPlugin()
kdeui (KDatePicker): KDatePicker::dateChangedSlot: date changed (2006/9/4).
kio (KDirWatch): Available methods: Stat, DNotify
kio (KDirWatch): Added Dir /media/sda7/SavePhotos/photos [KDirWatch-1]
kio (KDirWatch):  Setup DNotify (fd 128) for /media/sda7/SavePhotos/photos

[.... goes well for while ....]

kio (KDirWatch): Added Dir /media/sda7/SavePhotos/photos/03-Grande Bretagne/03-Oxford/Visiting [KDirWatch-1]
setmntent: Too many open files
kio (KDirWatch): Global Poll Freq is now 500 msec
kio (KDirWatch):  Started Polling Timer, freq 500
kio (KDirWatch):  Setup Stat (freq 500) for /media/sda7/SavePhotos/photos/03-Grande Bretagne/03-Oxford/Visiting

[... goes bad for the rest of the directories ...]

kio (KTrader): query for Digikam/ImagePlugin : returning 30 offers
digikam: WARNING: KIPI::PluginLoader:: createInstanceFromLibrary returned 0 for ImagePlugin_Core (digikamimageplugin_core) with error number 3
digikam: WARNING: KLibLoader says: file not found
digikam: WARNING: KTempFile: Error trying to create /home/anona/.kde/share/config/digikamrc.lockXXXXXX.tmp: Too many open files

Comment 14 caulier.gilles 2006-09-05 07:53:26 UTC

You have right. The errors come from KDELibs...but can be relevant of a common part used by KDE in your system.

Gilles Caulie

Comment 15 Marc C 2006-09-05 10:48:16 UTC

Thanks for your quick answer.

How I can identify, check, fix the problem ? 

Or could you redirect me to an IRC channel, mailing-list or people that could help identify the problem in KDELibs ?


Marc

Comment 16 caulier.gilles 2006-09-05 17:45:48 UTC

On freenode IRC server, try to post a message in #kde-devel channel

Gilles

Comment 17 caulier.gilles 2006-09-06 07:07:01 UTC

Another way is to contact the Amarok team. Amarok and digiKam uses sqlite internally. I'm sure that Amarok team can give us a good feedback

Gilles

Comment 18 Marc C 2006-09-07 11:06:14 UTC

Ok, I'll try that. For the time being I downgraded KDE to 3.5.2 and I'm using the 0.9 beta, which is great. But get times to open up, which might be normal with 1327 directories and 18992 files.

Marc

Comment 19 caulier.gilles 2006-09-07 11:12:26 UTC

Try to disable the 'Miscs/Scan new items at startup' option from setup.

To scan new items manually, look in Tools menu.

Note : 0.9.0-beta2 is in the way. Packages will be built by Paco this Friday. I recommend to test the current svn implementation where more that 50 bugs have been fixed since beta1 and camera gui have been really improved. Look into NEWS file from svn for details :

http://websvn.kde.org/trunk/extragear/graphics/digikam/NEWS?rev=581497&view=auto

Gilles

Comment 20 Andreas Huggel 2006-09-07 13:24:50 UTC

As a workaround, check the number of files that can be open at any one time and increase if necessary. See, e.g., here:

http://www.patoche.org/LTT/kernel/00000128.html

Also, if your images are RAW images, use exiv2 from svn to reduce the time needed to scan for metadata by about factor 6 compared to v0.10. For JPEGs this has no effect.

-ahu.

Comment 21 Andreas Huggel 2006-09-07 13:31:01 UTC

Just noticed that this is an old article I referred to. These parameters have moved to /proc/sys/fs/ in the meantime.

Comment 22 Marcel Wiesweg 2006-09-16 17:02:26 UTC

The problem is KDirWatch, and the number of directories.
It seems that there is a certain maximum number of directories (1024 is a good guess). The error occurs when too many directories are added to the KDirWatch instance. I think there are a various methods how KDirWatch is watching the files, and something may have changed from 3.5.2->3.5.3.
I can reproduce the problem here with > ~1000 directories, each only one file.

Comment 23 Arnd Baecker 2007-10-03 07:28:13 UTC

For KDE4, is there an improved KDirWatch which would solve this bug?

Comment 24 caulier.gilles 2007-11-28 16:35:46 UTC

Marcel,

Using more recent KDE3.5.x solve this problem ? KDirWatch have been fixed ?

If no, i recommend to affect this file on kdelibs core component. Fine for you ?

Gilles

Comment 25 Detlef Jockheck 2007-12-11 21:51:09 UTC

Hi,
I just compiled digiKam 0.9.3-rc1 on Suse 9.3 with the hope that it is fixed in a newer version than 0.7.5 (shipped with SuSE). Can this problem be solved by using newes KDirWatch?
Detlef

Comment 26 caulier.gilles 2007-12-11 21:54:31 UTC

Detlef,

I don't know if KDirWatch is fixed. It come from KDE API and i have no feedback from KDE developper about.

The only way is to check code or in bugzilla if something have been fixed recently...

Gilles

Comment 27 caulier.gilles 2008-03-28 18:10:38 UTC

Marcel,

With a recent KDE version, KDirwatch have been fixed ? If not, we need certainly to move this file in KDELibs bugzilla component, else nothing will be done.

Greg,

Are you tested with a more recent KDE ?

Gilles Caulier

Comment 28 caulier.gilles 2008-03-28 19:01:13 UTC

Marcel,

look in KDirWatchPrivate::addEntry() implementation. There is not limitation here:

http://websvn.kde.org/branches/KDE/3.5/kdelibs/kio/kio/kdirwatch.cpp?revision=556220&view=markup

Gilles

Comment 29 Marcel Wiesweg 2008-03-29 18:21:11 UTC

If you look at the man page of inotify_add_watch, which is called by KDirWatch if it is using inotify, you'll see that a  file descriptor is returned. It's a good guess that there is an operating system limit on the number of open files and every watched directory counts as an open file.

Comment 30 caulier.gilles 2008-03-29 19:22:20 UTC

Marcel,

From original post, user said than Ext3 file system is used to host photos.

Greg,

I can see 2 issues to progress here :

- try to use another FS, like ReiserFS (i use it here without any problem)
- try to use FAM notificator instead inotify  (i use it here without any problem).

Can you try on your computer ?

Gilles Caulier

Comment 31 Joerg Kuehne 2008-06-23 21:42:26 UTC

There is a limit of maximum number of open files per process. The bash command
ulimit -Hn displays the hard limit set by root and ulimit -Sn shows the soft limit adjustable by user but not larger than the hard limit. It seems that the default value is 1024 which is not enough for digikam album path with a large number of directories.
Typically the system settings for the hard limit and the default soft limit are made in /etc/security/limits.conf (PAM configuration file)
Add the following lines
*                hard    nofile          8192
*                soft    nofile          8192
to increase the limit to 8192.

Comment 32 Arnd Baecker 2008-11-06 11:55:35 UTC

Detlef, have you tried the suggestion by Joerg to increase the 
maximum number of open files per process?
If yes, this would be a strong indication that KDirWatch is not the problem...

Comment 33 caulier.gilles 2008-12-04 15:51:35 UTC

What news about this entry ?

It still valid using digiKam 0.10.0 with KDE4 desktop ?

Gilles Caulier

Comment 34 Ariel 2009-01-06 21:02:36 UTC

I see the same problem here.  Ubuntu 8.10 with KDE 4.1 (Intrepid Ibex)
Increasing the overall file system limits doesn't help.  They are very high anyway.  The limitation seems to be per-user limits but the suggestion above doesn't work for me:

tcsh> limit descriptors 10000
limit: descriptors: Can't set limit (Operation not permitted)

Also: can't run sudo because 'limit' is (tcsh) shell builtin.
Any better hint on how to increase per-user limits would be helpful.

Here's the output from digikam:

$ digikam                                  
kbuildsycoca running...                                         
Found dcraw version: 8.83                                       
Cannot load metadata using Exiv2  (Error #11: /home/<username>/photos/2009/01-05/m2u00169.mpg: The file contains data of an unknown image type)                      
setmntent: Too many open files                                                  
setmntent: Too many open files                                                  
setmntent: Too many open files                                                  
setmntent: Too many open files                                                  
setmntent: Too many open files                                                  
setmntent: Too many open files                                                  
setmntent: Too many open files                                                  
setmntent: Too many open files                                                  
setmntent: Too many open files                                                  
setmntent: Too many open files                                                  
setmntent: Too many open files                                                  
setmntent: Too many open files                                                  
setmntent: Too many open files                                                  
setmntent: Too many open files                                                  
setmntent: Too many open files                                                  
setmntent: Too many open files

[Note: this line gets repeated many hundreds of times, trimming for brevity]

Couldn't open /etc/kde3/kdebug.areas                                            
QPainter::begin: Cannot paint null pixmap                                       
QPainter::setPen: Will be reset by begin()                                      
QPainter::setBrush: Will be reset by begin()                                    
QPainter::setBrush: Will be reset by begin()                                    
QPainter::setPen: Will be reset by begin()                                      
QPainter::end: Missing begin() or begin() failed                                
QPainter::begin: Cannot paint null pixmap                                       
QPainter::setPen: Will be reset by begin()                                      
QPainter::setBrush: Will be reset by begin()                                    
QPainter::setBrush: Will be reset by begin()                                    
QPainter::setPen: Will be reset by begin()                                      
QPainter::end: Missing begin() or begin() failed                                
QPainter::begin: Cannot paint null pixmap                                       
QPainter::setPen: Will be reset by begin()                                      
QPainter::setBrush: Will be reset by begin()                                    
QPainter::setBrush: Will be reset by begin()                                    
QPainter::setPen: Will be reset by begin()                                      
QPainter::end: Missing begin() or begin() failed                                
QPainter::begin: Cannot paint null pixmap                                       
QPainter::setPen: Will be reset by begin()                                      
QPainter::setBrush: Will be reset by begin()                                    
QPainter::setBrush: Will be reset by begin()                                    
QPainter::setPen: Will be reset by begin()                                      
QPainter::end: Missing begin() or begin() failed                                
QPainter::begin: Cannot paint null pixmap                                       
QPainter::setPen: Will be reset by begin()                                      
QPainter::setBrush: Will be reset by begin()                                    
QPainter::setBrush: Will be reset by begin()                                    
QPainter::setPen: Will be reset by begin()                                      
QPainter::end: Missing begin() or begin() failed                                
QPainter::begin: Cannot paint null pixmap                                       
QPainter::setPen: Will be reset by begin()                                      
QPainter::setBrush: Will be reset by begin()                                    
QPainter::setBrush: Will be reset by begin()                                    
QPainter::setPen: Will be reset by begin()                                      
QPainter::end: Missing begin() or begin() failed                                
KCrash: Application 'digikam' crashing...                                       
Could not find 'drkonqi' executable.                                            
KCrash cannot reach kdeinit, launching directly.                                


Digikam used to work just fine before my upgrade to Ubuntu 8.10
HTH.

Comment 35 Fabienne 2009-03-28 20:10:56 UTC

I had the same problem with ubuntu 8.10 (intrepid).

I've solved it like this:

1) Updating the file  /etc/security/limits.conf

Setting these 2 values:
*		 soft	 nofile		 20000
*		 hard	 nofile		 20000

Just to be sure these values are good for you, issue this command:
$ cat /proc/sys/fs/file-max 
101270 # number of files allowed to be opened at the same time

Cf http://www.linuxplusvalue.be/mylpv.php?id=203

2) rebooting the computer because I don't know how to make it to be taken in account immediately.

Comment 36 caulier.gilles 2009-03-28 20:24:18 UTC

Marcel,

It sound like a settings problem from linux box. Not a digiKam or KDE lib problem.

What do you think about ? 

Gilles

Comment 37 Nicolas L. 2009-07-21 00:18:41 UTC

i think we can close this bugreport as this is not related to a digikam issue but as you explained to a linux issue.

Comment 38 caulier.gilles 2009-07-21 06:10:42 UTC

Agree for me.

Andi, Marcel, fine for you to close this entry ?

Gilles

Comment 39 Andi Clemens 2009-07-21 10:40:53 UTC

Yes, should be fine

Comment 40 Thomas Damgaard 2009-08-23 18:21:36 UTC

I have similar problem on 160GB album.

Comment 41 Andi Clemens 2009-10-08 01:45:47 UTC

Asking again :-)
Close this one?