Bug 113648

data loss -> increasing severity

Can you reproduce this problem with other ioslaves than camera:/ ?

No, I cannot reproduce this with other kioslaves.

"data loss" is supposed to be "grave" in Bugzilla.

Are you sure it's camera:/ not kamera:/ ?

yes, it is camera:/

which camera is it exactly?

I do not think that matters, as stated, gphoto2 gets the movies perfectly. Problem source seems to be the size of 20MB per movie (it works well for smaller stuff).
It is a Kodak EasyShare V550.

camera:/ has nothing to do with gphoto2
(kamera:/ does)

incorrect, camera:/ is the kioslave based on libgphoto2.

kamera:/ is unknown to me.

I just noticed, the camera.protocol from kmobile is not installed. So it's indeed Marcus to blame ;)

$ locate camera.protocol
/home/thiago/programs/src/kde3/KDE/kdepim/kmobile/kioslave/camera.protocol

$ locate kamera.protocol
/home/thiago/programs/src/kde3/KDE/kdegraphics/kamera/kioslave/kamera.protocol

Hence the conclusion.

hmm, what libgphoto2 (or SUSE ) version? 

SUSE Linux 10.0

scorpio:~ # rpm -qa| grep gphoto
gphoto-2.1.99head-3
libgphoto2-2.1.99.head.0-12
scorpio:~ #

hmm.

pretty difficult to find out without having such a kamera :(

if you copy out the images, do they have size 0 too?

yes.

can you try normal gphoto2 -P download? does this work?

you could also compile the kamera slave with debug and attach the debugoutput here, so we can see if there are visible problems :(

I have compiled kdelibs with debugging symbols :)

I can confirm this. But it has nothing to do with moving. It also happens when you just copy.
I have a Fujifilm Finepix F30 here and it happened yesterday with a movie of 19MB size. Now I have reproduced it with a 200MB movie.

"gphoto2 -P" loads it fine and displays the progress status during copying - in contrast to the kioslave, which stayed all the time at 0% until it silently closed and left the 0 byte file.

Please tell me/us how we can debug this. Should I turn DEBUG on when compiling the Ubuntu kdegraphics package? How should we invoke the ioslave?

I pledge for entirely removing this kioslave. I am still sad because it ate my favorite holiday movie. Better have nothing than a data-eater.

Bug reported in Ubuntu: https://bugs.launchpad.net/ubuntu/+source/kdegraphics/+bug/111036

Marcus, have you tried reproducing it?

The basic problem here seems to be that the original source file gets deleted (when moving), although the copy process before (silently) failed.

Here's what's happening: the camera get initialised and then goes into copy mode. It even seems to copy for about 2.5 minutes (according to the camera's led, which is blinking), but then the "Copy File(s) Progress" window just gets closed and a 0 byte file is left.

will have to look at it again. As I tried it it worked. :/

i bet it is triggering a timeout. :/

confirmed with a Canon IXUS Wireless, libgphoto2 just before 2.3.0

*** Bug 118418 has been marked as a duplicate of this bug. ***

fixed for 3_5 branch in r660816

the reason is simple;

kio/connection.cpp discards data blobs larger than 16MB silently. We wrote
a data blob of xxxx MB....

I am now splitting up this datablob into 1 MB pieces, just to be on the safe side.

Created attachment 20475 [details]
xx.pat

committed, untested to TRUNK in r660823

so this bug should have happened onlyu for files >= 16MB.

If it happened for smnaller files, please reopen.

Thanks a lot for fixing it!
I can confirm that the patch works with a 200MB file.

But: IMHO the really bad thing happening here is, that the source file gets deleted, though the first part in the two-step "move process" (copy+delete) failed - and there had been no error dialog.
Is this a bug with using the kioslave system in this specific case or a more general design flaw? Should we file a new bug about this?

it was silently discarded, without any error report.

so I think it is a problem in the KIO slave support in kdelibs.

(issue is:

any data blob larger than 16MB is discarded. See connection.cpp:

bool Connection::sendnow( int _cmd, const QByteArray &data )
{   
    if (f_out == 0) {
        return false;
    }

    if (data.size() > 0xffffff)
        return false;



The error is not returning back to data()

On Friday 04 May 2007, Marcus Meissner wrote:
> any data blob larger than 16MB is discarded. See connection.cpp:

Yes. The kioslave needs to chunk it.

Marcus, you did a great job analyzing this. Thanks. Can we maybe
-    if (data.size() > 0xffffff) 
+    if (data.size() > 0xffffffff) 
I mean, 16 MB is really little for my movies.
 

tstaerk: well, some lines below it does %06 printf of the size, so I guess there is a hard limit.

the camera slave is doing chunked writes now, so the issue is mostly addressed.

> Marcus, you did a great job analyzing this. Thanks. Can we maybe
> -    if (data.size() > 0xffffff) 
> +    if (data.size() > 0xffffffff) 
> I mean, 16 MB is really little for my movies.

The limit in KIO isn't a file size limit - it's a communication limit, i.e. how big should a chunk of data be.
More than 16MB at a time really kills the system, the slave (and the application) have to chunk the 
data they're sending (as documented in the dataReq signal for instance).

Marcussays the slave is doing chunked writes now; closing report.

I cannot verify the solution. camera:/ no longer shows anything for me.

I can now reproduce this bug on my KDE 3.5.5. r660823 is between 3.5.6 and 3.5.7, so, upgrading to test if the patch resolves the issue.

r660823 is in kdegraphics only, to verify, we need to upgrade this. To be able to access camera:/// in konqueror, you need the package kdegraphics installed.

camera:/// shows nothing, system:/media/camera now shows the same as camera:/// before. Continueing, I am now talking about system:/media/camera.

I confirm this bug is fixed, set its state to VERIFIED and thank all participants for their good work.

I set this bug to closed (although I do not understand why this is necessary).