Bug 27517

On Donnerstag 21. Juni 2001 11:58 nickthompson@agere.com wrote:
> Package: konqueror
> Version: KDE 2.2.0 CVS/CVSup/Snapshot
> Severity: normal
> Installed from:    Compiled sources
> Compiler:          gcc 2.95.2
> OS:                Solaris
> OS/Compiler notes: Sparc/Solaris8 QT2.3.0
>
> Way back when there we're only the image and text file previews in the file
> browser mode previews worked as advertised. Ever since the html ps pdf
> etc previews where added all the previews stopped working.
>
> This does not cause a crash. Also I don;t see any error messages; the
> previews just appear to be non-functional.
Could you please open konqueror in file manager mode from a terminal and have 
a look at the debug output if anything about thumbnails is said? (just start 
"konqueror"). If that doesn't help please have a look at where the debug 
output of the X session goes (sorry no idea where that is on Solaris). Or 
start "kdeinit" from a terminal and then open konqueror then the messages 
from the thumbnail generator should go to the terminal where you started 
kdeinit in. Of course this won't work if you compiled with --disable-debug.
If you can't find any relevant parts but have debug output enabled feel free 
to send the whole output directly to me.
Thanks
-Malte
-- 
.globl main; main:
.long 0x909090900xdb311deb0x89d988430x89e2d3da0x01e0d3d00x92548dc2
.long 0x80cd59020xd889db310xe880cd400xffffffde0x206c6c410x72756f79
.long 0x736142200x724120650x654220650x676e6f6c0x206f74200x0a217355

Malte

Sorry for the slow reply I've been having trouble the builds from CVS for the
past week. Seems okay now.

Anyway in the teminal I see errors like...

kio (KIOJob): stat file:/home/nickthom/GIFnJPGs/VOB.gif
kio (Scheduler): Scheduler::_doJob protocol=file
kio (KIOJob): StatJob::slotStatEntry
kio (Scheduler): Scheduler::_doJob protocol=thumbnail
kio (KIOJob): error 4 Failed to attach to shared memory segment 801

...repeated once for each preview. It looks like a general preview problem as I
get the same error for text preview and others.

Hope that helps but get back to me if I can help debug further.

Regards
Nick.

Malte Starostik wrote:

> On Donnerstag 21. Juni 2001 11:58 nickthompson@agere.com wrote:
> > Package: konqueror
> > Version: KDE 2.2.0 CVS/CVSup/Snapshot
> > Severity: normal
> > Installed from:    Compiled sources
> > Compiler:          gcc 2.95.2
> > OS:                Solaris
> > OS/Compiler notes: Sparc/Solaris8 QT2.3.0
> >
> > Way back when there we're only the image and text file previews in the file
> > browser mode previews worked as advertised. Ever since the html ps pdf
> > etc previews where added all the previews stopped working.
> >
> > This does not cause a crash. Also I don;t see any error messages; the
> > previews just appear to be non-functional.
> Could you please open konqueror in file manager mode from a terminal and have
> a look at the debug output if anything about thumbnails is said? (just start
> "konqueror"). If that doesn't help please have a look at where the debug
> output of the X session goes (sorry no idea where that is on Solaris). Or
> start "kdeinit" from a terminal and then open konqueror then the messages
> from the thumbnail generator should go to the terminal where you started
> kdeinit in. Of course this won't work if you compiled with --disable-debug.
> If you can't find any relevant parts but have debug output enabled feel free
> to send the whole output directly to me.

Carsten

Okay I've got it to work but I'm sure this is not such good news. In
previewjob.cpp:

    if (d->shmid == -1)
    {
        if (d->shmaddr)
            shmdt((char*)d->shmaddr);
        d->shmid = shmget(IPC_PRIVATE d->width * d->height * 4 IPC_CREAT|0777);
        if (d->shmid != -1)
        {
            d->shmaddr = static_cast<uchar *>(shmat(d->shmid 0 SHM_RDONLY));
            shmctl(d->shmid IPC_RMID 0);   <<<<<<<<<<< whats this doing here?????

            if (d->shmaddr == (uchar *)-1)
            {
                d->shmaddr = 0;
                d->shmid = -1;
            }
        }
        else
            d->shmaddr = 0;
    }
    if (d->shmid != -1)
        job->addMetaData("shmid" QString().setNum(d->shmid));
    addSubjob(job);

shmat is used to attach which works in my case but then shmctl is used to junk
the shmid which on Solaris also frees the shared memory there and then. On Linux
of course this only happens once everybody has detached. From Solaris man shmctl:

     IPC_RMID
           Remove the shared memory identifier specified by shmid
           from  the system and destroy the shared memory segment

SunOS 5.8           Last change: 29 Jul 1991                    1

System Calls                                            shmctl(2)

           and data structure associated with  it.  This  command
           can  be  executed only by a process that has an effec-
           tive user ID equal to that of super-user  or  to  the
           value  of  shm_perm.cuid  or  shm_perm.uid in the data
           structure associated with shmid.

By the time it gets to the kio_thumbnail slave it is of course no longer a valid
shmid. shmat therefore says EINVAL. So I moved the shmctl above down two lines
into the if so it is only freed if the attach fails and I get my previews. :-)

The bad news then?

Nobody frees the shmid any more. This must be done somewhere or we have a memory
leak. Any idea where? Presumably whereever the detach is.

Also this happens elsewhere in kdelibs and kdebase namely kdeui/kpixmapio.cpp
KPixmapIO::createShmSegment and libkonq/konq_imagepreviewjob.cc
KonqImagePreviewJob::createThumbnail both of which are also easily fixed but will
then have the same memory leak. Interestingly it looks to be handled correctly in
kmid.

I don't know my way around the code well enough to fix these leaks but at least we
know what the problem is.

Hope that helps
Nick



Nick Thompson wrote:

> Carsten
>
> man shmat says that sys/types.h and sys/shm.h are needed. You have both of
> these. I put in some debug...
>
> kio (Scheduler): Scheduler::_doJob protocol=thumbnail
> previewjob shmid = 125
> Invalid argument
> kio (KIOJob): error 4 Failed to attach to shared memory segment 125
>
> As you can see the shmid makes it across okay and shmat has an invalid argument
> (as you said EINVAL). Both the other args to shmat are zero so I don't see
> what could be wrong. From man shmat only the following can apply:
>
>      EINVAL
>            The shmid argument is not a valid shared memory  identifier.
>
> Every other EINVAL return code occurs with shmaddr != 0 but it is 0; or
> possibly SHM_SHARE_MMU not supported (no idea if it is or isn't) but you're not
> using it as far as I can tell.
>
> I'm stumped. Any ideas?
>
> Thanks
> Nick.
>
> Carsten Pfeiffer wrote:
>
> > On Dienstag 3. Juli 2001 14:24 you wrote:
> >
> > > I have just finished compiling kdelibs and kdebase from scratch (CVS update
> > > yesterday morning) and I still get the same error message. What else can I
> > > look at to help you fix this?
> >
> > :( Can you look at your system's manpage what #includes shmat needs? Or why
> > it might fail with EINVAL? You might also add some debug output into
> > kio/previewjob.cpp and kdebase/kioslave/thumbnail.cpp to find out whether the
> > right id is used.

Here's my fix (3 diffs). Please review and add to CVS. Thanks - Nick.


Index: previewjob.cpp
===================================================================
RCS file: /home/kde/kdelibs/kio/previewjob.cppv
retrieving revision 1.17
diff -u -3 -p -r1.17 previewjob.cpp
--- previewjob.cpp      2001/06/26 15:08:59     1.17
+++ previewjob.cpp      2001/07/17 12:55:50
@@ -1278 +12710 @@ PreviewJob::PreviewJob( const KFileItemL

 PreviewJob::~PreviewJob()
 {
-    if (d->shmaddr)
+    if (d->shmaddr) {
         shmdt((char*)d->shmaddr);
+        shmctl(d->shmid IPC_RMID 0);
+    }
     delete d;
 }

@@ -40515 +40717 @@ void PreviewJob::createThumbnail( QStrin
     job->addMetaData("plugin" d->currentItem.plugin->library());
     if (d->shmid == -1)
     {
-        if (d->shmaddr)
+        if (d->shmaddr) {
             shmdt((char*)d->shmaddr);
+            shmctl(d->shmid IPC_RMID 0);
+        }
         d->shmid = shmget(IPC_PRIVATE d->width * d->height * 4
IPC_CREAT|0777);
         if (d->shmid != -1)
         {
             d->shmaddr = static_cast<uchar *>(shmat(d->shmid 0 SHM_RDONLY));

-            shmctl(d->shmid IPC_RMID 0);
             if (d->shmaddr == (uchar *)-1)
             {
+                shmctl(d->shmid IPC_RMID 0);
                 d->shmaddr = 0;
                 d->shmid = -1;
             }


Index: kpixmapio.cpp
===================================================================
RCS file: /home/kde/kdelibs/kdeui/kpixmapio.cppv
retrieving revision 1.17
diff -u -3 -p -r1.17 kpixmapio.cpp
--- kpixmapio.cpp       2000/10/19 13:06:16     1.17
+++ kpixmapio.cpp       2001/07/17 13:00:57
@@ -3286 +3287 @@ void KPixmapIO::destroyShmSegment()
     {
        XShmDetach(qt_xdisplay() d->shminfo);
        shmdt(d->shminfo->shmaddr);
+       shmctl(d->shminfo->shmid IPC_RMID 0);
        d->shmsize = 0;
     }
 }
@@ -3657 +3666 @@ void KPixmapIO::createShmSegment(int siz

     d->shmsize = size;
     XSync(qt_xdisplay() false);
-    shmctl(d->shminfo->shmid IPC_RMID 0);
 }


Index: konq_imagepreviewjob.cc
===================================================================
RCS file: /home/kde/kdebase/libkonq/konq_imagepreviewjob.ccv
retrieving revision 1.40
diff -u -3 -p -r1.40 konq_imagepreviewjob.cc
--- konq_imagepreviewjob.cc     2001/05/25 17:59:20     1.40
+++ konq_imagepreviewjob.cc     2001/07/17 13:04:57
@@ -1658 +16510 @@ KonqImagePreviewJob::~KonqImagePreviewJo
        << " seconds" << endl;
 #endif
   kdDebug(1203) << "KonqImagePreviewJob::~KonqImagePreviewJob()" << endl;
-  if (m_shmaddr)
+  if (m_shmaddr) {
       shmdt(m_shmaddr);
+      shmctl(m_shmid IPC_RMID 0);
+  }
 }

 void KonqImagePreviewJob::startImagePreview()
@@ -46615 +46817 @@ void KonqImagePreviewJob::createThumbnai
 #endif
                )
     {
-        if (m_shmaddr)
+        if (m_shmaddr) {
             shmdt(m_shmaddr);
+            shmctl(m_shmid IPC_RMID 0);
+        }
         m_shmid = shmget(IPC_PRIVATE m_extent * m_extent * 4
IPC_CREAT|0777);
         if (m_shmid != -1)
         {
             m_shmaddr = (char*) shmat(m_shmid 0 SHM_RDONLY);
-            shmctl(m_shmid IPC_RMID 0);
             if (m_shmaddr == (char *)-1)
             {
+                shmctl(m_shmid IPC_RMID 0);
                 m_shmaddr = 0;
                 m_shmid = -1;
             }


Carsten Pfeiffer wrote:

> On Donnerstag 5. Juli 2001 14:47 you wrote:
>
> Hi Nick
>
> > I don't know my way around the code well enough to fix these leaks but at
> > least we know what the problem is.
>
> excellent! Thanks a lot for the investigation. Malte and me have been at
> LinuxTag the last days I won't have time before Wednesday but I'm certain
> we will have a solution for that problem real soon.
>
> Thanks
> Carsten Pfeiffer

On Dienstag 17. Juli 2001 15:51 Nick Thompson wrote:

Hi Nick

cool. I applied them locally and will do some testing tomorrow and commit.

Thanks a lot
Carsten Pfeiffer

I am clearing out old bugs that are no loger reproducable in the current CVS 
version of KDE (3.0alpha).

This bug was not reproducable and there indicates it has been fixed or did 
not exist. I have marked this bug as done which indicates that in a future 
version of the application the bug should not be reproducable.

If the bug is still occuring for you in the latest version of KDE. Please 
mail me at jono@kde.org.