Bug 185267 - Cutting many files from removable media fails
Summary: Cutting many files from removable media fails
Status: RESOLVED FIXED
Alias: None
Product: dolphin
Classification: Applications
Component: general (show other bugs)
Version: 16.12.2
Platform: Debian testing Unspecified
: NOR normal
Target Milestone: ---
Assignee: Peter Penz
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2009-02-22 19:48 UTC by Daniel Nicoletti
Modified: 2023-12-15 09:38 UTC (History)
6 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments
dolphin icons (53.33 KB, image/png)
2009-04-08 19:48 UTC, Daniel Nicoletti
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Daniel Nicoletti 2009-02-22 19:48:41 UTC
Version:            (using KDE 4.2.0)
Installed from:    Debian testing/unstable Packages

Today my g/f was CUTING 499 files from a 8gb pen drive and it simple fails, when ctrl + x (or clicking cut) it showed "paste 499" files and immediately after it changed to "Paste", and this paste was really old, it was a copy from a network computer, BUT COPING those 499 files worked fine, and if we CUT just a few files it works perfectly. So a high number of files in CUT seems to be the problem.

Thanks.
Comment 1 FiNeX 2009-02-23 22:15:43 UTC
Hi, could you exactly explain how we could reproduce it "step by step"? Many thanks for your feedback!
Comment 2 Daniel Nicoletti 2009-02-24 00:17:10 UTC
Well it pretty simple, you can do the following:
mkdir test && cd test
touch $(seq 500) # or more
get there through dolphin and
CRTL + A
CTRL + X
This normally makes dolphin to froze a bit.. (or a lot
if you have tons of files)
After it unfroze it will show "Paste 500 files" and
suddenly to "Paste One File" (this is the files i CTRL + C before)
IMHO this seems to be some kind of mem leak as CTRL + C work faster AND Works :D

Thanks
Comment 3 David Faure 2009-03-07 01:13:31 UTC
Dario Andres said dolphin was using 100% CPU for a long time and attached gdb to it and got this backtrace.

#1  0xb72edced in KUrl::List::fromMimeData (mimeData=0x9c8a5a8, metaData=0x0)
    at /home/kde-devel/kde/src/KDE/kdelibs/kdecore/io/kurl.cpp:317
#2  0xb7c94b73 in KFilePreviewGenerator::Private::isCutItem (this=0x9a573c0, item=@0x9dc7930)
    at /home/kde-devel/kde/src/KDE/kdelibs/kfile/kfilepreviewgenerator.cpp:613
#3  0xb7c95c14 in KFilePreviewGenerator::Private::applyCutItemEffect (this=0x9a573c0)
    at /home/kde-devel/kde/src/KDE/kdelibs/kfile/kfilepreviewgenerator.cpp:642
#4  0xb7c960ff in KFilePreviewGenerator::Private::generatePreviews (this=0x9a573c0, items=@0xbf94ba28)
    at /home/kde-devel/kde/src/KDE/kdelibs/kfile/kfilepreviewgenerator.cpp:404
#5  0xb7c96340 in KFilePreviewGenerator::Private::generatePreviews (this=0x9a573c0, topLeft=@0xbf94bbd8,
    bottomRight=@0xbf94bbd8) at /home/kde-devel/kde/src/KDE/kdelibs/kfile/kfilepreviewgenerator.cpp:436
#6  0xb7c968fc in KFilePreviewGenerator::qt_metacall (this=0x9a57238, _c=QMetaObject::InvokeMetaMethod,
    _id=1, _a=0xbf94bb28) at /home/kde-devel/kde/build/KDE/kdelibs/kfile/kfilepreviewgenerator.moc:80
#7  0xb691b1a9 in QMetaObject::activate () from /usr/lib/libQtCore.so.4
#8  0xb691b752 in QMetaObject::activate () from /usr/lib/libQtCore.so.4
#9  0xb6951f49 in QAbstractItemModel::dataChanged () from /usr/lib/libQtCore.so.4
#10 0xb7dab8c5 in KDirModel::itemChanged (this=0x9941070, index=@0x9941070)
    at /home/kde-devel/kde/src/KDE/kdelibs/kio/kio/kdirmodel.cpp:618
#11 0xb7de122d in KMimeTypeResolverPrivate::_k_slotProcessMimeIcons (this=0x9ac6828)
    at /home/kde-devel/kde/src/KDE/kdelibs/kio/kio/kmimetyperesolver.cpp:167

Peter had warned me that r926922 might be a bad idea, and he was right :-)
But instead of reverting I'll debug this next week to find out how we could make KFilePreviewGenerator faster. Obviously it's decoding the cut-list-of-urls every single time (i.e. for every icon whose mimetype is determined), this has got to be slow, I guess it could just cache that until the clipboard changes.
Also the relation with cut'ing actually failing isn't clear. More debugging to be done - next week.
Comment 4 Daniel Nicoletti 2009-03-07 01:49:17 UTC
David i said failing cause here it really fails, i cut ... wait... wait.. and then there are no items selected to paste.. (only the old ctrl + c) :P

that's why i said it failed.
Comment 5 FiNeX 2009-03-07 10:52:45 UTC
Probably it happened to me too a couple of times on the past weeks.
Comment 6 David Faure 2009-03-20 02:08:11 UTC
SVN commit 941585 by dfaure:

Improve performance when marking a large amount of files as "cut". For Ctrl+A Ctrl+X on 5000 files:
- before this commit: 300s
- after not decoding the full clipboard for every item: 6-8s
- after also looking up into a QSet instead of a QList: 4-5s
(Cutting only one item in the 5000 takes 0.03s with the QList and 0.05s with the 
QSet because of the time needed to build the set, but that's negligible)
CCMAIL: kde-optimize@kde.org
CCBUG: 185267
(but this only fixes performance, not the fact that X says "clipboard too big"
and truncates the list when pasting!)


 M  +5 -10     kfilepreviewgenerator.cpp  


WebSVN link: http://websvn.kde.org/?view=rev&revision=941585
Comment 7 David Faure 2009-03-20 02:38:46 UTC
SVN commit 941599 by dfaure:

Backport speed improvement from r941585, but things are still slow in this branch due to Qt-4.4.
An optimization was added in 4.5's QItemSelectionModelPrivate::_q_layoutChanged:
"special case for when all indexes are selected".
CCBUG: 185267


 M  +5 -10     kfilepreviewgenerator.cpp  


WebSVN link: http://websvn.kde.org/?view=rev&revision=941599
Comment 8 David Faure 2009-03-20 02:46:18 UTC
The real problem is this:
QClipboard::setData: Cannot set X11 selection owner for CLIPBOARD
QClipboard: Cannot transfer data, no data available

it's too much data for the X11 clipboard, it seems :(
Comment 9 Daniel Nicoletti 2009-03-22 13:44:25 UTC
Great and thanks for fixing these, shouldn't we file a bug report against X11 or something? seems that they should follow some of the perl's rules to not have unnecessary limits :P
Comment 10 David Faure 2009-03-23 15:11:41 UTC
Not sure my analysis was correct. In further tests it worked, and Maksim commented to my commit log saying:

"That sounds like a Qt bug to me, actually --- not switching to INCR
transfer or honoring property size limit. The only limit in X is, AFAIK,
due to 32-bitness of INCR (and that can be potentially bypassed by a
cross-desktop protocol too, if needed) --- after all, X doesn't actually
have a clipboard."
Comment 11 Daniel Nicoletti 2009-04-08 19:46:39 UTC
David, it works much better in 4.2.2 and qt4.5 but when ctrl + x,
dolphin cuts the files almost imediatly and if i switch to another folder it
pastes pretty fast but.
if i remeain in the same folder it gets dam slow, dolphin gets 60% of cpu
and icons get pretty strange, first they get a bit grey, and when moving the mouse over they, their icons simple disapear.. there was some debug output.
i hope this helps somehow:

WARNING: field 'maxLineLength' is not defined in any rdfs ontology database.
WARNING: field 'line ending format' is not defined in any rdfs ontology database.
WARNING: field 'dds volume depth' is not defined in any rdfs ontology database.
WARNING: field 'dds mipmap count' is not defined in any rdfs ontology database.
WARNING: field 'dds image type' is not defined in any rdfs ontology database.
WARNING: field 'font.family' is not defined in any rdfs ontology database.
WARNING: field 'font.weight' is not defined in any rdfs ontology database.
WARNING: field 'font.slant' is not defined in any rdfs ontology database.
WARNING: field 'font.width' is not defined in any rdfs ontology database.
WARNING: field 'font.spacing' is not defined in any rdfs ontology database.
WARNING: field 'font.foundry' is not defined in any rdfs ontology database.
WARNING: field 'content.version' is not defined in any rdfs ontology database.
WARNING: field 'document.stats.image_count' is not defined in any rdfs ontology database.
WARNING: field 'document.stats.image_name' is not defined in any rdfs ontology database.
WARNING: field 'document.stats.image_shared_rows' is not defined in any rdfs ontology database.
WARNING: field 'content.thumbnail' is not defined in any rdfs ontology database.
WARNING: field 'ole.category' is not defined in any rdfs ontology database.
WARNING: field 'ole.presentationtarget' is not defined in any rdfs ontology database.
WARNING: field 'ole.manager' is not defined in any rdfs ontology database.
WARNING: field 'ole.company' is not defined in any rdfs ontology database.
WARNING: field 'document.stats.table_count' is not defined in any rdfs ontology database.
WARNING: field 'document.stats.object_count' is not defined in any rdfs ontology database.
WARNING: field 'http://rdf.openmolecules.net/0.9#moleculeCount' is not defined in any rdfs ontology database.
dolphin(6912) KFileMetaInfoPrivate::init: KUrl("file:///home/daniel/blka/8")
dolphin(6912) MetaDataWidget::setFile: KUrl("file:///home/daniel/blka/8")
dolphin(6912) MetaDataWidget::setFile: KUrl("file:///home/daniel/blka")
dolphin(6912) KFileMetaInfoPrivate::init: KUrl("file:///home/daniel/blka/4")
dolphin(6912) MetaDataWidget::setFile: KUrl("file:///home/daniel/blka/4")
dolphin(6912) KFileMetaInfoPrivate::init: KUrl("file:///home/daniel/blka/14")
dolphin(6912) MetaDataWidget::setFile: KUrl("file:///home/daniel/blka/14")
Comment 12 Daniel Nicoletti 2009-04-08 19:48:38 UTC
Created attachment 32704 [details]
dolphin icons

here dolphin after Ctrl + x in the same folter, note that
if i switch to any other dir it works fast and as expected.
Comment 13 Peter Penz 2010-09-02 20:33:54 UTC
Is this issue still reproducible with KDE SC 4.5.0 and Qt 4.6/4.7? At least the things from comment #11 should not happen anymore...
Comment 14 Julian Steinmann 2018-03-18 14:32:31 UTC
I tried to reproduce this with Dolphin 17.12.3 and failed, but I encountered another bug instead:
If you 
1. create 500 files with e.g. touch $(seq 500)
2. cut them inside Dolphin
3. and paste them into another folder,
the plasmashell does not respond anymore (so I cannot click any icons in the application bar and so on). Can someone try to reproduce this? It might also be related to Dolphin, but I wasn't able to triage this behavior further.
Comment 15 Patrick Silva 2018-03-20 16:35:39 UTC
(In reply to Julian Schraner from comment #14)

> 1. create 500 files with e.g. touch $(seq 500)
> 2. cut them inside Dolphin
> 3. and paste them into another folder,
> the plasmashell does not respond anymore (so I cannot click any icons in the
> application bar and so on). Can someone try to reproduce this? It might also
> be related to Dolphin, but I wasn't able to triage this behavior further.

known problem
https://bugs.kde.org/show_bug.cgi?id=378935
Comment 16 Proletario Libertario 2018-05-24 09:16:26 UTC
Something similar happens to me, when I move files, cut and paste, from one directory to another, it does not matter if it is between two partitions, the same or different disks, the files are transferred but the dolphin from where I cut them hangs. Does not respond any Dolphin window, if I open a CTRL + ALT + F2 console as root and check the processes I see that two Plasmashell processes one at 99% and another something less than approximately 89%, from here it is impossible to do anything, except to restart the computer, however the files have been transferred correctly.

If I do the same only through console commands, without opening dolphin, for example, from anywhere I do "mv * / media / data / roms /" all files are transferred without problems, while cut and paste in dolphin hangs.

It only happens when it is a large number of files using dolphin, for example 700 files.

Kubuntu 18.04
plasmashell 5.12.4
kernel 4.15.0-22
dolphin 17.12.3
Comment 17 Akseli Lahtinen 2023-12-15 09:38:52 UTC
Can't reproduce, seems to be fixed.