Version: 4.7 (using KDE 4.7.0) OS: Linux If KMimeTypeRepository::findFromContent tries to determine MIME from a file that cannot be read, such as on a corrupted optical disc, a read attempt is made in KMimeMagicMatch::match for every available rule, resulting in UI hangs (e.g. file dialogs, dolphin) for tens of minutes. Reproducible: Always Steps to Reproduce: List a directory containing corrupted files in e.g. Dolphin. Actual Results: UI becomes unresponsive for a very long time, usually needing to kill the app. Expected Results: Applications should stay responsive. A typical strace then looks like: open("/media/Fedora-15-x86_64-Live-KDE.iso/isolinux/initrd0.img", O_RDONLY|O_CLOEXEC) = 23 fcntl(23, F_SETFD, FD_CLOEXEC) = 0 fstat(23, {st_mode=S_IFREG|0444, st_size=9577989, ...}) = 0 fstat(23, {st_mode=S_IFREG|0444, st_size=9577989, ...}) = 0 lseek(23, 2089, SEEK_SET) = 2089 fstat(23, {st_mode=S_IFREG|0444, st_size=9577989, ...}) = 0 read(23, 0xcaaa30, 16384) = -1 EIO (Input/output error) read(23, 0xca9c08, 10) = -1 EIO (Input/output error) lseek(23, 0, SEEK_SET) = 0 read(23, 0xcaaa30, 16384) = -1 EIO (Input/output error) read(23, 0xc9ffe8, 2) = -1 EIO (Input/output error) lseek(23, 0, SEEK_SET) = 0 read(23, 0xcaaa30, 16384) = -1 EIO (Input/output error) read(23, 0xc9ffe8, 3) = -1 EIO (Input/output error) (repeated hundreds of times for each file) As KMimeMagicMatch::match returns a bool and cannot easily signal such a broken file, I alleviated this problem by pre-reading enough data before matching any rules. The maximum amount of data ever needed were the first 1029 bytes of each file, so I read 2K, leaving a buffer for future rules. I also had the file opened unbuffered in both KMimeType::findByUrlHelper and KMimeType::findByFileContent. It was trying to needlesly read whole 16K for the buffer, increasing the chance for the EIO (as some files fail after the 2K needed). If the pre-read fails, matching by content fails also. This cuts the EIO's to at most one per corrupted file, shrinking the wait to mere seconds. For the proposed fix see attached patch (changing 2 files in kdecore/services).
Created attachment 62988 [details] proposed patch Gentoo won't tether. Android browser won't upload files. Waiting times after my fix are comparable to those in Nautilus (Ubuntu).
Thanks a lot for the patch! Could you please put the patch to https://git.reviewboard.kde.org and put faure@kde.org to the recipients-list? I'm quite sure that this patch will be "forgotten" here in bugs.kde.org within hundreds of other reports - on git.reviewboard.kde.org it usually takes only a few days until the patch can get committed.
Well, I registered and created a review request (102382), but uploading a patch to the system seems to require the correct commit hashes, which in turn requires a git clone of the lot of kdelibs, which is not so great over my current connection. I believe Mr. Faure could apply the patch as-is in his environment. I have confirmed it will apply cleanly to the current master. Sorry for the inconvenience (mine would be greater).
No problem, I've created a review request on https://git.reviewboard.kde.org/r/102391/ and will take care to push it if David found the time to review it.
Thanks. I have commented in the review request. Would the problem be fixed for you without the use of Unbuffered? (Alternatively we could make sure to always update the cache, in the code, when reading data after the current cache size, and then we could use Unbuffered)
Git commit 880a4e8512343462353f69ddecb63ae1748b9902 by Peter Penz. Committed on 20/08/2011 at 19:14. Pushed by ppenz into branch 'KDE/4.7'. Don't hang when determining MIME type of corrupted files The patch has been provided by Miroslav Ľos. BUG: 280446 M +8 -0 kdecore/services/kmimetyperepository.cpp M +2 -2 kdecore/services/kmimetype.cpp http://commits.kde.org/kdelibs/880a4e8512343462353f69ddecb63ae1748b9902
Git commit 7705b25883c76affcc608e157e0d003de98ad531 by Peter Penz. Committed on 20/08/2011 at 19:14. Pushed by ppenz into branch 'KDE/4.7'. Don't hang when determining MIME type of corrupted files The patch has been provided by Miroslav Ľos. BUG: 280446 M +8 -0 kdecore/services/kmimetyperepository.cpp M +2 -2 kdecore/services/kmimetype.cpp http://commits.kde.org/kdelibs/7705b25883c76affcc608e157e0d003de98ad531
Sorry, I accidentally have pushed the patch. I had some troubles with the history then in git (-> 2 commits listed here) but the patch is reverted now.
Git commit 6606337b9bdf98111276d0fd3d803a644462cb28 by David Faure. Committed on 19/10/2011 at 09:35. Pushed by dfaure into branch 'KDE/4.7'. Read the first 16K of the file upfront, to minimize seeks and reads. Patch by Miroslav Ľos - thanks! BUG: 280446 FIXED-IN: 4.7.3 REVIEW: 102391 M +8 -0 kdecore/services/kmimetyperepository.cpp http://commits.kde.org/kdelibs/6606337b9bdf98111276d0fd3d803a644462cb28