Bug 322550

Can't reproduce in Debian sid with KDE SC 4.10.5.
I can copy folders with subdirectories just fine.

Are you experiencing this issue every time? It might be a connection problem.

Thank you for your reply.

Yes, I am experiencing this error every time.

I did check this right just now and found out, that I don't get this error with the "webdavs" protocol.
But with the "webdav" protocol, I get this error.

At least I can now copy my files with webdavs.

Could this be because your owncloud server is configured to accept SSL connections only? Otherwise, it does not make sense since the webdavs and webdav protocols use the exact same code base in kio_http.

Awaiting answer on comment #3.

To further investigate this issue, KDE developers need the information requested in comment #3. If you can provide it, or need help with finding that information, please add a comment.

Same here.

Setup: NAS with webdavs (no webdav w/out ssl)

When trying to copy from PC to NAS, it will copy a few folders and then report this error.

(In reply to comment #6)
> Same here.
> 
> Setup: NAS with webdavs (no webdav w/out ssl)
> 
> When trying to copy from PC to NAS, it will copy a few folders and then
> report this error.

Happens both in Dolphin and Konqueror -- probably bug in libs?!

If you need any further info, let me know.

The needed information is requested in comment #3. If you can provide it, please add a comment.

This error occurs both with webdav as well as webdavs.

Replying to comments #3 #5 & #8:
Original bug report says problem exists with webdav://
Confirming on comment #2 that it seems to work with webdavs://
Comment #6 says that problem occurs with webdavs://
Comment #7 says it happens in konquerer and dolphin
So it seems to occur on both protocols when using kio slave based applications.
What information do you need in addition to consider comment #3 as answered then?

Some additional information from my side:
I get the very same error when using webdavs:// with german cloud provider goneo.de.
This is what their server says it is using: SabreDAV 1.8.5-stable (c)2007-2013
So it seems this is happening not only with owncloud.
Reproducible only with many deeply nested folders.
Version used: Qt: 4.8.2, KDE: 4.12.3, Dolphin: 4.12.3
According to this forum entry: http://forum.ubuntuusers.de/topic/homeserver-webdav-unerwarteter-fehler-200-dolp/#post-4931507
Nautilus seems to work just fine.

Dawit, could you clarify if any more information (and if yes, which information) is needed to understand this issue?

(In reply to comment #11)
> Dawit, could you clarify if any more information (and if yes, which
> information) is needed to understand this issue?

Well I do not need any additional information. I will try again to reproduce the problem on the test owncloud webdav server.

I tested this again with the owncloud demo server (webdav://demo.owncloud.org/remote.php/webdav/). I created the exact directory structure shown above and tried to copy it and move it and both worked just fine. So I am unable to reproduce this problem.

Oh and for the record this is using KDE 4.13.1. Though there are no changes in KDE 4.13 that should affect this IIRC.

Hello Dawit,

sorry to bother you but this is still persisting as of today.

Here is my setup: running webdavs on my Synology NAS -- whenever I try to copy nested folders I get the mentioned error.

Since you cannot reproduce this, what can I do to debug it? How can I get detailed information about the protocol between client and server?

Cheers

May be this is not even related to copying nested folders but is somehow depending on the total number of folders to be copied?!

Topfolder
|-- Subfolder
    |-- Subsubfolder1
    |-- Subsubfolder2
    |-- Subsubfolder3
    |-- Subsubfolder4
    |-- Subsubfolder5
    |-- Subsubfolder6
    |-- Subsubfolder7

The first time Topfolder and Subfolder got copied before the error occured. After this I marked all Subsubfolders and copied them: Subsubfolder1 and Subsubfolder2 got copied before the error occured...

So it seems as only two folders can be copied at once.

(In reply to nik.knatterton from comment #16)
> May be this is not even related to copying nested folders but is somehow
> depending on the total number of folders to be copied?!
> 
> Topfolder
> |-- Subfolder
>     |-- Subsubfolder1
>     |-- Subsubfolder2
>     |-- Subsubfolder3
>     |-- Subsubfolder4
>     |-- Subsubfolder5
>     |-- Subsubfolder6
>     |-- Subsubfolder7
> 
> The first time Topfolder and Subfolder got copied before the error occured.
> After this I marked all Subsubfolders and copied them: Subsubfolder1 and
> Subsubfolder2 got copied before the error occured...
> 
> So it seems as only two folders can be copied at once.

Is there by any chance a limit on the number of simuntaneous connections you can have the server you are trying to copy from? That is the only question I did not ask so far. Anyways, can you please follow the instructions to obtain the debug output from kio_http from the link below and post the results here: 

https://techbase.kde.org/Development/Tutorials/Debugging/Debugging_IOSlaves#How_to_get_debug_output

Be sure to remove any personally identifying information from the generated output like authorization information etc.

(In reply to Dawit Alemayehu from comment #17)
> (In reply to nik.knatterton from comment #16)
> > May be this is not even related to copying nested folders but is somehow
> > depending on the total number of folders to be copied?!
> > 
> > Topfolder
> > |-- Subfolder
> >     |-- Subsubfolder1
> >     |-- Subsubfolder2
> >     |-- Subsubfolder3
> >     |-- Subsubfolder4
> >     |-- Subsubfolder5
> >     |-- Subsubfolder6
> >     |-- Subsubfolder7
> > 
> > The first time Topfolder and Subfolder got copied before the error occured.
> > After this I marked all Subsubfolders and copied them: Subsubfolder1 and
> > Subsubfolder2 got copied before the error occured...
> > 
> > So it seems as only two folders can be copied at once.
> 
> Is there by any chance a limit on the number of simuntaneous connections you
> can have the server you are trying to copy from? That is the only question I
> did not ask so far. Anyways, can you please follow the instructions to
> obtain the debug output from kio_http from the link below and post the
> results here: 
> 
> https://techbase.kde.org/Development/Tutorials/Debugging/
> Debugging_IOSlaves#How_to_get_debug_output
> 
> Be sure to remove any personally identifying information from the generated
> output like authorization information etc.

I think I forgot to mention that you also needs to install the kdelibs debug packages for your specific distro:

https://techbase.kde.org/Development/Tutorials/Debugging/How_to_create_useful_crash_reports#Preparing_your_KDE_packages

Created attachment 88603 [details]
Debug file

Hi Dawit,

thanks for taking the time!

Here is what I did:
- created a folder structure as described in comment#16 on my local pc
- authorized at the webdavs server
- deleted the debug file (to have a clean start)
- copied the Topfolder to the webdavs directory
- opened debug file right after the message appeared

Remark: I changed some values like the IP and folder names in the debug file -- this should not make a difference, though.

Cheers
Nik

(In reply to nik.knatterton from comment #19)
> Created attachment 88603 [details]
> Debug file
> 
> Hi Dawit,
> 
> thanks for taking the time!
> 
> Here is what I did:
> - created a folder structure as described in comment#16 on my local pc
> - authorized at the webdavs server
> - deleted the debug file (to have a clean start)
> - copied the Topfolder to the webdavs directory
> - opened debug file right after the message appeared
> 
> Remark: I changed some values like the IP and folder names in the debug file
> -- this should not make a difference, though.
> 
> Cheers
> Nik

The server does not seem to have any issues creating the top level folder

kio_http(4019)/kio_http_debug HTTPProtocol::mkdir: KUrl("webdavs://192.168.1.123/Topfolder")
kio_http(4019)/kio_http_debug HTTPProtocol::maybeSetRequestUrl: KUrl("webdavs://192.168.1.123/Topfolder")
kio_http(4019)/kio_http_debug HTTPProtocol::resetSessionSettings: Window Id = "60817430"
kio_http(4019)/kio_http_debug HTTPProtocol::resetSessionSettings: ssl_was_in_use = ""
kio_http(4019)/kio_http_debug HTTPProtocol::proceedUntilResponseHeader:
kio_http(4019)/kio_http_debug HTTPProtocol::sendQuery:
kio_http(4019)/kio_http_debug HTTPProtocol::httpShouldCloseConnection:
kio_http(4019)/kio_http_debug HTTPProtocol::authenticationHeader: creating www authentcation header from cached info
kio_http(4019)/kio_http_debug HTTPProtocol::sendQuery: sent it!
kio_http(4019)/kio_http_debug HTTPProtocol::readResponseHeader:
kio_http(4019)/kio_http_debug HTTPProtocol::readResponseHeader: wasAuthError= false isAuthError= false sameAuthError= false
kio_http(4019)/kio_http_debug HTTPProtocol::readResponseHeader:  -- full response:
"HTTP/1.1 201 Created
Date: Sun, 07 Sep 2014 12:02:48 GMT
Server: Apache
Location: https://192.168.1.123/Topfolder
Content-Length: 191
Keep-Alive: timeout=5, max=98
Connection: Keep-Alive
Content-Type: text/html; charset=ISO-8859-1"

for some reason it returns a bizzare response when we attempt to create a subfolder

kio_http(4019)/kio_http_debug HTTPProtocol::mkdir: KUrl("webdavs://192.168.1.123/Topfolder/Subfolder")
kio_http(4019)/kio_http_debug HTTPProtocol::maybeSetRequestUrl: KUrl("webdavs://192.168.1.123/Topfolder/Subfolder")
kio_http(4019)/kio_http_debug HTTPProtocol::resetSessionSettings: Window Id = "60817430"
kio_http(4019)/kio_http_debug HTTPProtocol::resetSessionSettings: ssl_was_in_use = ""
kio_http(4019)/kio_http_debug HTTPProtocol::proceedUntilResponseHeader:
kio_http(4019)/kio_http_debug HTTPProtocol::sendQuery:
kio_http(4019)/kio_http_debug HTTPProtocol::httpShouldCloseConnection:
kio_http(4019)/kio_http_debug HTTPProtocol::authenticationHeader: creating www authentcation header from cached info
kio_http(4019)/kio_http_debug HTTPProtocol::sendQuery: sent it!
kio_http(4019)/kio_http_debug HTTPProtocol::readResponseHeader:
kio_http(4019)/kio_http_debug HTTPProtocol::readResponseHeader: DO NOT WANT. 50

The "DO NOT WANT.50" is not a valid response. Are you able to copy the same directory structure using another environment or another a non-kde client?

I just tried PCManFM 1.2.0 running from within a virtual machine (lubuntu 14.04).

Copying the folder structure works without any problem (webdavs). I do not know where to look for a log file, though.

Today I've had the chance to cross check with a different WebDAV implementation (mydrive.ch). What am I to say? It works without problems...

So obviously the implementation of Synology's DSM WebDAV seems to have some error which is being ignored by other clients (like PCManFM) but not kio_http.

I will now see whether I can get someone from Synology to deal with that -- unless you have another solution.

Thank you anyway for helping me tracing this down!

(In reply to nik.knatterton from comment #22)
> Today I've had the chance to cross check with a different WebDAV
> implementation (mydrive.ch). What am I to say? It works without problems...
> 
> So obviously the implementation of Synology's DSM WebDAV seems to have some
> error which is being ignored by other clients (like PCManFM) but not
> kio_http.
> 
> I will now see whether I can get someone from Synology to deal with that --
> unless you have another solution.
> 
> Thank you anyway for helping me tracing this down!

My suggestion to you is to enable debug area 7103 (does not seem to be have enabled in the log file you posted) so that you can get the request headers and ask on the Synology support why you get such invalid response. It would also help if you also posted the new debug output here so I can see the request sent to the server. 

FYI, in the log file if you only see debug message that starts with kio_http_debug and not kio_http, then only debug area 7113 is enabled. If you enable 7103 and tell it to send the log out to the same file, then you get output for both debug areas in one single file.

Here you go -- if you need more, let me know. I gave the Synology support the link to this bug ticket.

######################################################
kio_http(3986) HTTPProtocol::sendQuery: ============ Sending Header:
kio_http(3986) HTTPProtocol::sendQuery: "MKCOL /Topfolder/Subfolder HTTP/1.1"
kio_http(3986) HTTPProtocol::sendQuery: "Host: 192.168.1.123"
kio_http(3986) HTTPProtocol::sendQuery: "Connection: keep-alive"
kio_http(3986) HTTPProtocol::sendQuery: "User-Agent: Mozilla/5.0 (X11; Linux x86_64) KHTML/4.13.3 (like Gecko) Konqueror/4.13"
kio_http(3986) HTTPProtocol::sendQuery: "Pragma: no-cache"
kio_http(3986) HTTPProtocol::sendQuery: "Cache-control: no-cache"
kio_http(3986) HTTPProtocol::sendQuery: "Accept: text/html, text/*;q=0.9, image/jpeg;q=0.9, image/png;q=0.9, image/*;q=0.9, */*;q=0.8"
kio_http(3986) HTTPProtocol::sendQuery: "Accept-Encoding: gzip, deflate, x-gzip, x-deflate"
kio_http(3986) HTTPProtocol::sendQuery: "Accept-Charset: utf-8,*;q=0.5"
kio_http(3986) HTTPProtocol::sendQuery: "Accept-Language: de,en-US;q=0.9,en;q=0.8"
kio_http(3986) HTTPProtocol::sendQuery: "Authorization: Basic ffjhsdgfiugefseifgIEFU=="
kio_http(3986)/kio_http_debug HTTPProtocol::sendQuery: sent it!
kio_http(3986)/kio_http_debug HTTPProtocol::readResponseHeader:
kio_http(3986) HTTPProtocol::readResponseHeader: ============ Received Status Response:
kio_http(3986) HTTPProtocol::readResponseHeader: "!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">"
kio_http(3986)/kio_http_debug HTTPProtocol::readResponseHeader: DO NOT WANT. 50
kio_http(3986)/kio_http_debug HTTPProtocol::proceedUntilResponseHeader: Previous Response: 0
kio_http(3986)/kio_http_debug HTTPProtocol::proceedUntilResponseHeader: Current Response: 200
kio_http(3987)/kio_http_debug HTTPProtocol::authenticationHeader: creating www authentcation header from cached info
######################################################

I was able to reproduce this problem when I was finally able to setup up my own webdav server with apache + mod_dav plugin.

Please note that the issue is very specific to apache with the mod_dav plugin. The problem does not exist with sabredav (which is what owncloud uses).

(In reply to Dawit Alemayehu from comment #25)
> I was able to reproduce this problem when I was finally able to setup up my
> own webdav server with apache + mod_dav plugin.

https://git.reviewboard.kde.org/r/120439/

Great! I am looking forward to the update in Kubuntu. :)

Just because I am curious: why is there a different behaviour depending on the DAV-implementation?

Thank you, Dawit!

Created attachment 88904 [details]
attachment-30756-0.html

On Tue, Sep 30, 2014 at 9:23 AM, <nik.knatterton@web.de> wrote:

> https://bugs.kde.org/show_bug.cgi?id=322550
>
> --- Comment #28 from nik.knatterton@web.de ---
> Great! I am looking forward to the update in Kubuntu. :)
>
> Just because I am curious: why is there a different behaviour depending on
> the
> DAV-implementation?
>

When you send MKCOL (read: create directory) request to Apache + mod_dav,
it responds with 201 Created + plus some content. Our webdav kioslave
however is not interested in the content the server sends back so it simply
ignores it. Unfortunately, apache does not like that and it will reject any
further requests you make to it. The solution is to simply read and discard
the entire response from the server including the content.  It is literally
a one line change in our code base.

Git commit 0f6faa53d6860c8e8d6a947b02aa24dcbdf753b5 by Dawit Alemayehu.
Committed on 30/09/2014 at 12:44.
Pushed by adawit into branch 'KDE/4.14'.

When creating a folder through webdav, read and discard the content sent by the web sever.
FIXED-IN: 4.14.2
REVIEW: 120439

M  +1    -1    kioslave/http/http.cpp

http://commits.kde.org/kdelibs/0f6faa53d6860c8e8d6a947b02aa24dcbdf753b5

Git commit 446dda272c109977587307c572f1965acb64ebd2 by Dawit Alemayehu.
Committed on 07/10/2014 at 11:40.
Pushed by adawit into branch 'master'.

frameworks port of commit 0f6faa5:

When creating a folder through webdav, read and discard the content
sent by the web sever.

M  +1    -1    src/ioslaves/http/http.cpp

http://commits.kde.org/kio/446dda272c109977587307c572f1965acb64ebd2