Bug 203512 - Full-size thumbnails are saved under .thumbnails/normal
Summary: Full-size thumbnails are saved under .thumbnails/normal
Status: RESOLVED FIXED
Alias: None
Product: kde
Classification: I don't know
Component: general (show other bugs)
Version: unspecified
Platform: Debian testing Linux
: NOR normal
Target Milestone: ---
Assignee: Unassigned bugs mailing-list
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2009-08-12 07:54 UTC by Brendon Higgins
Modified: 2009-08-14 21:21 UTC (History)
1 user (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Brendon Higgins 2009-08-12 07:54:34 UTC
Version:            (using KDE 4.3.0)
OS:                Linux
Installed from:    Debian testing/unstable Packages

I was trying to narrow down a problem with the Gimp file selector, and I've discovered it's down to the size of the thumbnails in .thumbnails/normal. They can be huge! It seems that KDE saves full-size images in this folder, and this plays very badly with the Gnome file selector (causes it to hang and chew RAM for a very long time, if not indefinitely).

I do some artwork and some of my images have large dimensions, which is bad news for programs that aren't tailored to loading very large "thumbnail" files (that's an oxymoron, isn't it?). Finding the proposed thumbnails spec via freedesktop.org at http://people.freedesktop.org/~vuntz/thumbnail-spec-cache/ suggests that anything above 128x128 in the "normal" folder is not to spec.

Please respect the spec, and play nice with other users of the .thumbnails folder. At best KDE should not save those files to "normal", but further, KDE should not save fullsize images as thumbnails, especially if they are way beyond a reasonable thumbnail size (that's just crazy!).

Peace,
Brendon
Comment 1 Dario Andres 2009-08-12 21:08:23 UTC
I just checked the standard code that generate thumbnails ("PreviewJob"):

"kdelibs/kio/kio/previewjob.cpp":

    if (bNeedCache)
    {
        if (width <= 128 && height <= 128) cacheWidth = cacheHeight = 128;
        else cacheWidth = cacheHeight = 256;
        thumbPath = thumbRoot + (cacheWidth == 128 ? "normal/" : "large/");
        KStandardDirs::makeDir(thumbPath, 0700);
    }

Also, the directory thumb creator has its cache method:

"kdebase/runtime/kioslave/thumbnail/thumbnail.cpp":

KMD5 md5(QFile::encodeName(fileName.url()));
        const QString thumbName = QFile::encodeName(md5.hexDigest()) + ".png";
        if (m_thumbBasePath.isEmpty()) {
            m_thumbBasePath = QDir::homePath() + "/.thumbnails/";
            KStandardDirs::makeDir(m_thumbBasePath + "normal/", 0700);
            KStandardDirs::makeDir(m_thumbBasePath + "large/", 0700);
        }

As you can see both standard methods honor the standard.
So, it is a different app (or a KDE app using a custom method) which could be causing this.

As a test you could:
- Clean your .thumbnails folder
- Start and use every app you normally use and check after every run if the thumbnails with unusual large are created in normal/

- What other graphics application are you using ?

Regards
Comment 2 Dario Andres 2009-08-12 21:09:43 UTC
Sorry, I didn't pasted the relevant code of "kdebase/runtime/kioslave/thumbnail/thumbnail.cpp":

        QString thumbPath = m_thumbBasePath;
        if ((segmentWidth <= 128) && (segmentHeight <= 128)) {
            cacheSize = 128;
            thumbPath += "normal/";
        } else {
            cacheSize = 256;
            thumbPath += "large/";
        }
Comment 3 Dario Andres 2009-08-13 01:02:57 UTC
Moving to "kde" until we determine which apps generate the thumbs in the wrong place.
Comment 4 Brendon Higgins 2009-08-13 02:27:53 UTC
Then it must be a problem specific to Dolphin. It's certainly the case that enabling Preview mode in Dolphin causes this. I just assumed Dolphin used standard thumbnail generator code, but I guess it doesn't. Feel free to reassign there.
Comment 5 Dario Andres 2009-08-13 02:30:57 UTC
Mh, Dolphin uses an interface to the PreviewJob code I showed to you in comment 1.

Did you tested removing the thumbs cache and browsing some folders with Dolphin that would show this issue ? 
Thanks
Comment 6 Brendon Higgins 2009-08-13 05:35:53 UTC
I'm trying to work out a way to reproduce this. I cleared out .thumbnails and set up a couple of basic 1600x1200 test images in a lone test folder. Now, I haven't managed to find a way that saves the full 1600x1200 image as a thumbnail yet, but I've discovered that if I use dolphin to browse .thumbnails/large, then fullsize copies of those large thumbnail images (with different hashes) will appear in .thumbnails/normal. The two problems with this behaviour, of course, are (1) the image size, and (2) it probably isn't wise to be generating thumbnails for files inside .thumbnails, anyway.

This is very odd considering preview.cpp line 245:
!(*kit).url().directory( KUrl::AppendTrailingSlash ).startsWith(thumbRoot)) &&
Comment 7 Brendon Higgins 2009-08-13 07:53:09 UTC
I think the problem is createSubThumbnail in thumbnail.cpp:
if (subCreator->create(filePath, cacheSize, cacheSize, thumbnail)) {
    // The thumbnail has been created successfully. Store the thumbnail
    // to the cache for future access.
    KTemporaryFile temp;
    temp.setPrefix(thumbPath + "kde-tmp-");
    temp.setSuffix(".png");
    temp.setAutoRemove(false);
    if (temp.open()) {
        tempFileName = temp.fileName();
        savedCorrectly = thumbnail.save(tempFileName, "PNG");
    }
} else {
    return false;
}

This creates a thumbnail for use as a sub-thumbnail for in the generation of folder thumbnails, but it performs no size checking on the image that's created by the sub-thumbnail generator.

ImageCreator::create seems to return the image without scaling. Presumably the assumption is that this is safe because the get method in thumbnail.cpp does this:
if (img.width() > m_width || img.height() > m_height) {
    double imgRatio = (double)img.height() / (double)img.width();
    if (imgRatio > (double)m_height / (double)m_width) {
        img = img.scaled( int(qMax((double)m_height / imgRatio, 1.0)), m_height, Qt::IgnoreAspectRatio, Qt::SmoothTransformation);
    } else {
        img = img.scaled(m_width, int(qMax((double)m_width * imgRatio, 1.0)), Qt::IgnoreAspectRatio, Qt::SmoothTransformation);
    }
}

createSubThumbnail doesn't do that.

If I'm correct, then this is how unscaled images work their way into the thumbnails folder.

Peace,
Brendon
Comment 8 Peter Penz 2009-08-14 21:19:55 UTC
SVN commit 1011482 by ppenz:

Assure that the thumbnails of directory previews don't exceed the cache size. Thanks to Brendon Higgins for the detailed analyses.

BUG: 203512
CCBUG: 202960

 M  +15 -8     thumbnail.cpp  
 M  +6 -0      thumbnail.h  


WebSVN link: http://websvn.kde.org/?view=rev&revision=1011482
Comment 9 Peter Penz 2009-08-14 21:21:25 UTC
SVN commit 1011483 by ppenz:

Backport of SVN commit 1011482: Assure that the thumbnails of directory previews don't exceed the cache size.
Thanks to Brendon Higgins for the detailed analyses.

BUG: 203512
CCBUG: 202960

 M  +15 -8     thumbnail.cpp  
 M  +6 -0      thumbnail.h  


WebSVN link: http://websvn.kde.org/?view=rev&revision=1011483