Bug 234030

No anyone who can confirm this bug? No way.

I had the same issue yesterday - partially downloaded files (potentially with a valid beginning, torrent downloads file parts in random order) caused kio_thumbnail to eat mamory, and it took 10-15 minutes for me to log in in a shell and kill the process, due to slow swapping.
I suggest to set a maximum for memory allocations, if something is requesting too much memory/time just for a thumbnail it should be skipped.

Created attachment 57930 [details]
Corrupted XCF file.

I've the same bug with a corrupted xcf file.
I've disabled all preview in Dolphin settings but kio_thumbnail is still called.
I attach this corrupted file in a compressed format to not freeze your computer if you download it.

Note: I'm running KDE 4.6.1 from Kubuntu PPA.

I've noticed this too.

(In reply to comment #3)
> Created an attachment (id=57930) [details]
> Corrupted XCF file.
> 
> I've the same bug with a corrupted xcf file.
> I've disabled all preview in Dolphin settings but kio_thumbnail is still
> called.
> I attach this corrupted file in a compressed format to not freeze your computer
> if you download it.

Is the Information Panel in Dolphin enabled ? If it is, then Dolphin will try to generate previews regardless of your "Preview" settings.

Created attachment 63357 [details]
Dangerous memory leak

Wow, this happens in seconds and could eat all the ram and swap and cause the system to fail. I think this bug should be marked as a critical.

UPD:
https://bugs.kde.org/show_bug.cgi?id=234030#c7 happened with the corrupted XCF file, attached in the comment #3

Git commit af286ba6d8f8807c160f259b658cda30fdb4e393 by Dawit Alemayehu.
Committed on 08/12/2011 at 08:08.
Pushed by adawit into branch 'KDE/4.7'.

Ignore corrupted XCF images that contain individual tiles whose width
or height is set to a value larger than that of the image itself.

BUG: 234030
FIXED-IN: 4.8.0

M  +9    -0    kimgio/xcf.cpp

http://commits.kde.org/kdelibs/af286ba6d8f8807c160f259b658cda30fdb4e393

Don't you think this should be checked for every corrupted image format?

Also I reported the problem with the video preview. So I think it shouldn't ignore some specific files as there is many formats and it's hard to handle them all. Instead it should detect when something going wrong and stop. It should check the data for validity before process.

(In reply to comment #11)
> Also I reported the problem with the video preview. So I think it shouldn't
> ignore some specific files as there is many formats and it's hard to handle
> them all. Instead it should detect when something going wrong and stop. It
> should check the data for validity before process.

(In reply to comment #10)
> Don't you think this should be checked for every corrupted image format?

Possibly, but I personally do not maintain any of this stuff. I just fixed this one because there was a sample provided.

(In reply to comment #11)
> Also I reported the problem with the video preview. So I think it shouldn't
> ignore some specific files as there is many formats and it's hard to handle
> them all. Instead it should detect when something going wrong and stop. 

Ahh... The bug was not in the thumbnail generator. It was in the QImage plugin that reads the XCF file formats. As such there is no way the thumbnail generator can workaround or detect bugs in a completely different piece of code. 

> It should check the data for validity before process.

How should it exactly do that ? Understand every image, text and other content format intimately by itself ? It cannot. Anyhow, I closed this bug report because there are several others that deal with problems generating thumbail for video files. You yourself seem to have commented on those ; so this is at best simply a duplicate. 

BTW, the fact that dolphin attempts to generate a preview for files that are in the process of being downloaded is an issue of dolphin itself and the fact that ffmpegthumbnailer generator eats all the CPU attempting to generate a preview image in such circumstances is is something that needs to be fixed in the kdemultimedia module, which is where the ffmpegthumbnail generator plugin is located.

Agree. But perhaps it could detect when the process consumes too many memory and CPU time basis on the input file size. For example it's definitely clear when process goes to eat over 10Gb of RAM when processing the 1Mb file. I mean to add something in thumbnail system that will prevent dangerous situations like on the screenshot.

I disagree. Killing a process is not a solution, will make the user experience mysterious slowdowns with no visible result and delay the correct fix. Also, there is no way an algorithm can decide when memory consumption is too high for an arbitrary file. Here's a good example of a 59kb file expanding to over 48mb in memory: http://upload.wikimedia.org/wikipedia/commons/c/c4/All_24-bit_RGB_colors.png
In theory I could write a program that saves one single byte in a file and produces huge outputs by looking up in a table of pre-made video files, for example.
The problem is that memory allocation doesn't fail until you run out of virtual memory. The real fix is that each plugin should take into account bogus data and refrain from loading it if no partial recovery is possible, or stop at once.