Bug 322550 - Get error if I try to copy a directory with one or more subdirectory to my Owncloud-server
Summary: Get error if I try to copy a directory with one or more subdirectory to my Ow...
Status: RESOLVED FIXED
Alias: None
Product: kio
Classification: Frameworks and Libraries
Component: webdav (show other bugs)
Version: 4.10.4
Platform: Ubuntu Linux
: NOR normal
Target Milestone: ---
Assignee: kdelibs bugs
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2013-07-18 21:47 UTC by Michael Neufing
Modified: 2014-10-10 02:11 UTC (History)
6 users (show)

See Also:
Latest Commit:
Version Fixed In: 4.14.2


Attachments
Debug file (18.86 KB, text/plain)
2014-09-07 12:19 UTC, nik.knatterton
Details
attachment-30756-0.html (1.25 KB, text/html)
2014-09-30 18:36 UTC, Dawit Alemayehu
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Michael Neufing 2013-07-18 21:47:06 UTC
When I try to copy a directory to my Owncloud embedded webdav server, that contains one or more subdirectories, I get an error "An unexpected error 200 occurred while attempting to create the requested folder.

For example, when I try to copy the following directory structure:

RootDir
├── ChildDir1
│   ├── ChildDir2
│   │   └── FileInChild2.txt
│   └── FileInChild1.txt
└── FileInRoot.txt

The directories RootDir and ChildDir1 are copied and then I get the error message.
All other directories or files don't get copied.

The original German error message is: "Bei der Aktion „angegebenen Ordner erstellen“ ist der unerwartete Fehler 200 aufgetreten."

Reproducible: Always

Steps to Reproduce:
1. Create the directory structure as mentioned in details
2. Connect to an Owncloud WebDAV server with the "webdav://"-KIO slave (using i.g. Dolphin)
3. Try to copy the directory structure to the WebDAV
Actual Results:  
You get the error, I mentioned in the details.

Expected Results:  
The directory structure should be created and the containing files should be copied without any error.

I have tried the same with the Windows Exporer and with Nautilus on a fresh installed Ubuntu 13.04-VM.

These two file managers are working as expected. They create the directory structure and copy the files without any problems.
Comment 1 Jon Ander Peñalba 2013-07-20 18:30:16 UTC
Can't reproduce in Debian sid with KDE SC 4.10.5.
I can copy folders with subdirectories just fine.

Are you experiencing this issue every time? It might be a connection problem.
Comment 2 Michael Neufing 2013-07-20 18:42:30 UTC
Thank you for your reply.

Yes, I am experiencing this error every time.

I did check this right just now and found out, that I don't get this error with the "webdavs" protocol.
But with the "webdav" protocol, I get this error.

At least I can now copy my files with webdavs.
Comment 3 Dawit Alemayehu 2013-10-12 15:03:23 UTC
Could this be because your owncloud server is configured to accept SSL connections only? Otherwise, it does not make sense since the webdavs and webdav protocols use the exact same code base in kio_http.
Comment 4 Dawit Alemayehu 2014-01-05 07:52:11 UTC
Awaiting answer on comment #3.
Comment 5 Christoph Feck 2014-01-25 21:25:50 UTC
To further investigate this issue, KDE developers need the information requested in comment #3. If you can provide it, or need help with finding that information, please add a comment.
Comment 6 nik.knatterton 2014-01-30 09:55:43 UTC
Same here.

Setup: NAS with webdavs (no webdav w/out ssl)

When trying to copy from PC to NAS, it will copy a few folders and then report this error.
Comment 7 nik.knatterton 2014-02-03 17:21:34 UTC
(In reply to comment #6)
> Same here.
> 
> Setup: NAS with webdavs (no webdav w/out ssl)
> 
> When trying to copy from PC to NAS, it will copy a few folders and then
> report this error.

Happens both in Dolphin and Konqueror -- probably bug in libs?!

If you need any further info, let me know.
Comment 8 Christoph Feck 2014-02-08 23:24:37 UTC
The needed information is requested in comment #3. If you can provide it, please add a comment.
Comment 9 nik.knatterton 2014-02-09 14:32:01 UTC
This error occurs both with webdav as well as webdavs.
Comment 10 ktp1976 2014-04-17 16:11:06 UTC
Replying to comments #3 #5 & #8:
Original bug report says problem exists with webdav://
Confirming on comment #2 that it seems to work with webdavs://
Comment #6 says that problem occurs with webdavs://
Comment #7 says it happens in konquerer and dolphin
So it seems to occur on both protocols when using kio slave based applications.
What information do you need in addition to consider comment #3 as answered then?

Some additional information from my side:
I get the very same error when using webdavs:// with german cloud provider goneo.de.
This is what their server says it is using: SabreDAV 1.8.5-stable (c)2007-2013
So it seems this is happening not only with owncloud.
Reproducible only with many deeply nested folders.
Version used: Qt: 4.8.2, KDE: 4.12.3, Dolphin: 4.12.3
According to this forum entry: http://forum.ubuntuusers.de/topic/homeserver-webdav-unerwarteter-fehler-200-dolp/#post-4931507
Nautilus seems to work just fine.
Comment 11 Christoph Feck 2014-05-04 21:44:14 UTC
Dawit, could you clarify if any more information (and if yes, which information) is needed to understand this issue?
Comment 12 Dawit Alemayehu 2014-05-04 22:49:48 UTC
(In reply to comment #11)
> Dawit, could you clarify if any more information (and if yes, which
> information) is needed to understand this issue?

Well I do not need any additional information. I will try again to reproduce the problem on the test owncloud webdav server.
Comment 13 Dawit Alemayehu 2014-05-24 14:18:12 UTC
I tested this again with the owncloud demo server (webdav://demo.owncloud.org/remote.php/webdav/). I created the exact directory structure shown above and tried to copy it and move it and both worked just fine. So I am unable to reproduce this problem.
Comment 14 Dawit Alemayehu 2014-05-24 14:19:03 UTC
Oh and for the record this is using KDE 4.13.1. Though there are no changes in KDE 4.13 that should affect this IIRC.
Comment 15 nik.knatterton 2014-09-06 12:09:44 UTC
Hello Dawit,

sorry to bother you but this is still persisting as of today.

Here is my setup: running webdavs on my Synology NAS -- whenever I try to copy nested folders I get the mentioned error.

Since you cannot reproduce this, what can I do to debug it? How can I get detailed information about the protocol between client and server?

Cheers
Comment 16 nik.knatterton 2014-09-06 12:16:30 UTC
May be this is not even related to copying nested folders but is somehow depending on the total number of folders to be copied?!

Topfolder
|-- Subfolder
    |-- Subsubfolder1
    |-- Subsubfolder2
    |-- Subsubfolder3
    |-- Subsubfolder4
    |-- Subsubfolder5
    |-- Subsubfolder6
    |-- Subsubfolder7

The first time Topfolder and Subfolder got copied before the error occured. After this I marked all Subsubfolders and copied them: Subsubfolder1 and Subsubfolder2 got copied before the error occured...

So it seems as only two folders can be copied at once.
Comment 17 Dawit Alemayehu 2014-09-06 14:40:03 UTC
(In reply to nik.knatterton from comment #16)
> May be this is not even related to copying nested folders but is somehow
> depending on the total number of folders to be copied?!
> 
> Topfolder
> |-- Subfolder
>     |-- Subsubfolder1
>     |-- Subsubfolder2
>     |-- Subsubfolder3
>     |-- Subsubfolder4
>     |-- Subsubfolder5
>     |-- Subsubfolder6
>     |-- Subsubfolder7
> 
> The first time Topfolder and Subfolder got copied before the error occured.
> After this I marked all Subsubfolders and copied them: Subsubfolder1 and
> Subsubfolder2 got copied before the error occured...
> 
> So it seems as only two folders can be copied at once.

Is there by any chance a limit on the number of simuntaneous connections you can have the server you are trying to copy from? That is the only question I did not ask so far. Anyways, can you please follow the instructions to obtain the debug output from kio_http from the link below and post the results here: 

https://techbase.kde.org/Development/Tutorials/Debugging/Debugging_IOSlaves#How_to_get_debug_output

Be sure to remove any personally identifying information from the generated output like authorization information etc.
Comment 18 Dawit Alemayehu 2014-09-06 15:33:33 UTC
(In reply to Dawit Alemayehu from comment #17)
> (In reply to nik.knatterton from comment #16)
> > May be this is not even related to copying nested folders but is somehow
> > depending on the total number of folders to be copied?!
> > 
> > Topfolder
> > |-- Subfolder
> >     |-- Subsubfolder1
> >     |-- Subsubfolder2
> >     |-- Subsubfolder3
> >     |-- Subsubfolder4
> >     |-- Subsubfolder5
> >     |-- Subsubfolder6
> >     |-- Subsubfolder7
> > 
> > The first time Topfolder and Subfolder got copied before the error occured.
> > After this I marked all Subsubfolders and copied them: Subsubfolder1 and
> > Subsubfolder2 got copied before the error occured...
> > 
> > So it seems as only two folders can be copied at once.
> 
> Is there by any chance a limit on the number of simuntaneous connections you
> can have the server you are trying to copy from? That is the only question I
> did not ask so far. Anyways, can you please follow the instructions to
> obtain the debug output from kio_http from the link below and post the
> results here: 
> 
> https://techbase.kde.org/Development/Tutorials/Debugging/
> Debugging_IOSlaves#How_to_get_debug_output
> 
> Be sure to remove any personally identifying information from the generated
> output like authorization information etc.

I think I forgot to mention that you also needs to install the kdelibs debug packages for your specific distro:

https://techbase.kde.org/Development/Tutorials/Debugging/How_to_create_useful_crash_reports#Preparing_your_KDE_packages
Comment 19 nik.knatterton 2014-09-07 12:19:58 UTC
Created attachment 88603 [details]
Debug file

Hi Dawit,

thanks for taking the time!

Here is what I did:
- created a folder structure as described in comment#16 on my local pc
- authorized at the webdavs server
- deleted the debug file (to have a clean start)
- copied the Topfolder to the webdavs directory
- opened debug file right after the message appeared

Remark: I changed some values like the IP and folder names in the debug file -- this should not make a difference, though.

Cheers
Nik
Comment 20 Dawit Alemayehu 2014-09-08 00:08:28 UTC
(In reply to nik.knatterton from comment #19)
> Created attachment 88603 [details]
> Debug file
> 
> Hi Dawit,
> 
> thanks for taking the time!
> 
> Here is what I did:
> - created a folder structure as described in comment#16 on my local pc
> - authorized at the webdavs server
> - deleted the debug file (to have a clean start)
> - copied the Topfolder to the webdavs directory
> - opened debug file right after the message appeared
> 
> Remark: I changed some values like the IP and folder names in the debug file
> -- this should not make a difference, though.
> 
> Cheers
> Nik

The server does not seem to have any issues creating the top level folder

kio_http(4019)/kio_http_debug HTTPProtocol::mkdir: KUrl("webdavs://192.168.1.123/Topfolder")
kio_http(4019)/kio_http_debug HTTPProtocol::maybeSetRequestUrl: KUrl("webdavs://192.168.1.123/Topfolder")
kio_http(4019)/kio_http_debug HTTPProtocol::resetSessionSettings: Window Id = "60817430"
kio_http(4019)/kio_http_debug HTTPProtocol::resetSessionSettings: ssl_was_in_use = ""
kio_http(4019)/kio_http_debug HTTPProtocol::proceedUntilResponseHeader:
kio_http(4019)/kio_http_debug HTTPProtocol::sendQuery:
kio_http(4019)/kio_http_debug HTTPProtocol::httpShouldCloseConnection:
kio_http(4019)/kio_http_debug HTTPProtocol::authenticationHeader: creating www authentcation header from cached info
kio_http(4019)/kio_http_debug HTTPProtocol::sendQuery: sent it!
kio_http(4019)/kio_http_debug HTTPProtocol::readResponseHeader:
kio_http(4019)/kio_http_debug HTTPProtocol::readResponseHeader: wasAuthError= false isAuthError= false sameAuthError= false
kio_http(4019)/kio_http_debug HTTPProtocol::readResponseHeader:  -- full response:
"HTTP/1.1 201 Created
Date: Sun, 07 Sep 2014 12:02:48 GMT
Server: Apache
Location: https://192.168.1.123/Topfolder
Content-Length: 191
Keep-Alive: timeout=5, max=98
Connection: Keep-Alive
Content-Type: text/html; charset=ISO-8859-1"

for some reason it returns a bizzare response when we attempt to create a subfolder

kio_http(4019)/kio_http_debug HTTPProtocol::mkdir: KUrl("webdavs://192.168.1.123/Topfolder/Subfolder")
kio_http(4019)/kio_http_debug HTTPProtocol::maybeSetRequestUrl: KUrl("webdavs://192.168.1.123/Topfolder/Subfolder")
kio_http(4019)/kio_http_debug HTTPProtocol::resetSessionSettings: Window Id = "60817430"
kio_http(4019)/kio_http_debug HTTPProtocol::resetSessionSettings: ssl_was_in_use = ""
kio_http(4019)/kio_http_debug HTTPProtocol::proceedUntilResponseHeader:
kio_http(4019)/kio_http_debug HTTPProtocol::sendQuery:
kio_http(4019)/kio_http_debug HTTPProtocol::httpShouldCloseConnection:
kio_http(4019)/kio_http_debug HTTPProtocol::authenticationHeader: creating www authentcation header from cached info
kio_http(4019)/kio_http_debug HTTPProtocol::sendQuery: sent it!
kio_http(4019)/kio_http_debug HTTPProtocol::readResponseHeader:
kio_http(4019)/kio_http_debug HTTPProtocol::readResponseHeader: DO NOT WANT. 50

The "DO NOT WANT.50" is not a valid response. Are you able to copy the same directory structure using another environment or another a non-kde client?
Comment 21 nik.knatterton 2014-09-16 18:09:32 UTC
I just tried PCManFM 1.2.0 running from within a virtual machine (lubuntu 14.04).

Copying the folder structure works without any problem (webdavs). I do not know where to look for a log file, though.
Comment 22 nik.knatterton 2014-09-19 12:28:37 UTC
Today I've had the chance to cross check with a different WebDAV implementation (mydrive.ch). What am I to say? It works without problems...

So obviously the implementation of Synology's DSM WebDAV seems to have some error which is being ignored by other clients (like PCManFM) but not kio_http.

I will now see whether I can get someone from Synology to deal with that -- unless you have another solution.

Thank you anyway for helping me tracing this down!
Comment 23 Dawit Alemayehu 2014-09-19 12:46:58 UTC
(In reply to nik.knatterton from comment #22)
> Today I've had the chance to cross check with a different WebDAV
> implementation (mydrive.ch). What am I to say? It works without problems...
> 
> So obviously the implementation of Synology's DSM WebDAV seems to have some
> error which is being ignored by other clients (like PCManFM) but not
> kio_http.
> 
> I will now see whether I can get someone from Synology to deal with that --
> unless you have another solution.
> 
> Thank you anyway for helping me tracing this down!

My suggestion to you is to enable debug area 7103 (does not seem to be have enabled in the log file you posted) so that you can get the request headers and ask on the Synology support why you get such invalid response. It would also help if you also posted the new debug output here so I can see the request sent to the server. 

FYI, in the log file if you only see debug message that starts with kio_http_debug and not kio_http, then only debug area 7113 is enabled. If you enable 7103 and tell it to send the log out to the same file, then you get output for both debug areas in one single file.
Comment 24 nik.knatterton 2014-09-19 18:03:44 UTC
Here you go -- if you need more, let me know. I gave the Synology support the link to this bug ticket.

######################################################
kio_http(3986) HTTPProtocol::sendQuery: ============ Sending Header:
kio_http(3986) HTTPProtocol::sendQuery: "MKCOL /Topfolder/Subfolder HTTP/1.1"
kio_http(3986) HTTPProtocol::sendQuery: "Host: 192.168.1.123"
kio_http(3986) HTTPProtocol::sendQuery: "Connection: keep-alive"
kio_http(3986) HTTPProtocol::sendQuery: "User-Agent: Mozilla/5.0 (X11; Linux x86_64) KHTML/4.13.3 (like Gecko) Konqueror/4.13"
kio_http(3986) HTTPProtocol::sendQuery: "Pragma: no-cache"
kio_http(3986) HTTPProtocol::sendQuery: "Cache-control: no-cache"
kio_http(3986) HTTPProtocol::sendQuery: "Accept: text/html, text/*;q=0.9, image/jpeg;q=0.9, image/png;q=0.9, image/*;q=0.9, */*;q=0.8"
kio_http(3986) HTTPProtocol::sendQuery: "Accept-Encoding: gzip, deflate, x-gzip, x-deflate"
kio_http(3986) HTTPProtocol::sendQuery: "Accept-Charset: utf-8,*;q=0.5"
kio_http(3986) HTTPProtocol::sendQuery: "Accept-Language: de,en-US;q=0.9,en;q=0.8"
kio_http(3986) HTTPProtocol::sendQuery: "Authorization: Basic ffjhsdgfiugefseifgIEFU=="
kio_http(3986)/kio_http_debug HTTPProtocol::sendQuery: sent it!
kio_http(3986)/kio_http_debug HTTPProtocol::readResponseHeader:
kio_http(3986) HTTPProtocol::readResponseHeader: ============ Received Status Response:
kio_http(3986) HTTPProtocol::readResponseHeader: "!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">"
kio_http(3986)/kio_http_debug HTTPProtocol::readResponseHeader: DO NOT WANT. 50
kio_http(3986)/kio_http_debug HTTPProtocol::proceedUntilResponseHeader: Previous Response: 0
kio_http(3986)/kio_http_debug HTTPProtocol::proceedUntilResponseHeader: Current Response: 200
kio_http(3987)/kio_http_debug HTTPProtocol::authenticationHeader: creating www authentcation header from cached info
######################################################
Comment 25 Dawit Alemayehu 2014-09-30 12:40:58 UTC
I was able to reproduce this problem when I was finally able to setup up my own webdav server with apache + mod_dav plugin.
Comment 26 Dawit Alemayehu 2014-09-30 12:42:06 UTC
Please note that the issue is very specific to apache with the mod_dav plugin. The problem does not exist with sabredav (which is what owncloud uses).

(In reply to Dawit Alemayehu from comment #25)
> I was able to reproduce this problem when I was finally able to setup up my
> own webdav server with apache + mod_dav plugin.
Comment 27 Dawit Alemayehu 2014-09-30 12:55:20 UTC
https://git.reviewboard.kde.org/r/120439/
Comment 28 nik.knatterton 2014-09-30 13:23:46 UTC
Great! I am looking forward to the update in Kubuntu. :)

Just because I am curious: why is there a different behaviour depending on the DAV-implementation?

Thank you, Dawit!
Comment 29 Dawit Alemayehu 2014-09-30 18:36:40 UTC
Created attachment 88904 [details]
attachment-30756-0.html

On Tue, Sep 30, 2014 at 9:23 AM, <nik.knatterton@web.de> wrote:

> https://bugs.kde.org/show_bug.cgi?id=322550
>
> --- Comment #28 from nik.knatterton@web.de ---
> Great! I am looking forward to the update in Kubuntu. :)
>
> Just because I am curious: why is there a different behaviour depending on
> the
> DAV-implementation?
>

When you send MKCOL (read: create directory) request to Apache + mod_dav,
it responds with 201 Created + plus some content. Our webdav kioslave
however is not interested in the content the server sends back so it simply
ignores it. Unfortunately, apache does not like that and it will reject any
further requests you make to it. The solution is to simply read and discard
the entire response from the server including the content.  It is literally
a one line change in our code base.
Comment 30 Dawit Alemayehu 2014-10-01 05:15:24 UTC
Git commit 0f6faa53d6860c8e8d6a947b02aa24dcbdf753b5 by Dawit Alemayehu.
Committed on 30/09/2014 at 12:44.
Pushed by adawit into branch 'KDE/4.14'.

When creating a folder through webdav, read and discard the content sent by the web sever.
FIXED-IN: 4.14.2
REVIEW: 120439

M  +1    -1    kioslave/http/http.cpp

http://commits.kde.org/kdelibs/0f6faa53d6860c8e8d6a947b02aa24dcbdf753b5
Comment 31 Dawit Alemayehu 2014-10-10 02:11:18 UTC
Git commit 446dda272c109977587307c572f1965acb64ebd2 by Dawit Alemayehu.
Committed on 07/10/2014 at 11:40.
Pushed by adawit into branch 'master'.

frameworks port of commit 0f6faa5:

When creating a folder through webdav, read and discard the content
sent by the web sever.

M  +1    -1    src/ioslaves/http/http.cpp

http://commits.kde.org/kio/446dda272c109977587307c572f1965acb64ebd2