Bug 354633 - [ntfs] ktorrent freezes very often
Summary: [ntfs] ktorrent freezes very often
Status: RESOLVED FIXED
Alias: None
Product: ktorrent
Classification: Applications
Component: general (show other bugs)
Version: 4.3.1
Platform: Kubuntu Linux
: NOR major
Target Milestone: ---
Assignee: Joris Guisson
URL:
Keywords:
: 356559 357160 (view as bug list)
Depends on:
Blocks:
 
Reported: 2015-10-31 07:15 UTC by Amichai Rothman
Modified: 2016-05-29 17:33 UTC (History)
11 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments
Backtrace (2.60 KB, text/plain)
2016-02-24 10:29 UTC, Igor Poboiko
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Amichai Rothman 2015-10-31 07:15:57 UTC
Ever since the upgrade to Kubuntu 15.10 (KDE 5.4.0? The KDE About dialog no longer shows the version!) KTorrent has become almost unusable. When it's downloading a torrent, usually at ~1mb/s, after a few seconds the whole GUI freezes and becomes totally unresponsive for 10-30 seconds. Minimizing+maximizing the window results in a blank white window. During this time htop shows one CPU core at 100% 'wait' state. After this it unfreezes and works normally for about 5-10 seconds, and then freezes again. It continues this freeze-unfreeze loop (mostly frozen) until the torrent download is complete.

This did not happen before the upgrade. To my understanding, the KTorrent version itself has not changed in a while - so perhaps this is some kind of new incompatibility with KDE5 or newer kernel or some other updated component.


Reproducible: Always
Comment 1 Frederic Mohr 2015-11-21 10:45:16 UTC
Same proplem here, ever since upgrade from Kubuntu 15.04 to 15.10 ktorrent freezes whenever I open the gui. It could be due to the NTFS disk I'm saving to, I think before I had SSD/ext4 in an LVM container running. I'll try to revert it to see if it changes anything.
Comment 2 Amichai Rothman 2015-11-22 16:33:16 UTC
Mine is also an NTFS disk. Perhaps there was some regression with the driver, or maybe the disk is starting to fail. In any case, I've configured ktorrent to use a different disk (btrfs) and so far haven't seen the stalls. I'll update if I see them again.
Comment 3 Frederic Mohr 2015-11-22 18:14:03 UTC
Seems to be the problem, I changed /media/fmohr/Backup/ to /home/fmohr/Backup in .kde/share/config/ktorrentrc and after a while ktorrent stopped freezing altogether. I'll try to change the final move destination back to the ntfs disk, maybe that's a working compromise.
Comment 4 mrl586 2015-11-28 17:02:24 UTC
If you upgrade ntfs-3g driver same version as in xenial, ktorrent doesn't freeze so often.
Comment 5 Frederic Mohr 2015-11-28 18:56:14 UTC
But that would mean I'd have to use another repo...

$ apt-cache policy ntfs-3g
ntfs-3g:
  Installed: 1:2014.2.15AR.3-3
  Candidate: 1:2014.2.15AR.3-3
  Version table:
 *** 1:2014.2.15AR.3-3 0
        500 http://de.archive.ubuntu.com/ubuntu/ wily/main amd64 Packages
        100 /var/lib/dpkg/status
Comment 6 Nick Shaforostoff 2016-02-14 22:29:08 UTC
please retry with ktorrent for KDE5 Frameworks and also compare ntfs saving with any other bittorrent client like transmission
Comment 7 Igor Poboiko 2016-02-23 18:21:39 UTC
I also experience this with KTorrent 4.3.1, while Transmission 2.84 works fine. I use ntfs-3g 2015.3.14.

(In reply to Nick Shaforostoff from comment #6)
> please retry with ktorrent for KDE5 Frameworks and also compare ntfs saving
> with any other bittorrent client like transmission

There is no stable release for KF5 yet, right? Going to try recent build from Git and see if it works.
Comment 8 Igor Poboiko 2016-02-24 10:29:12 UTC
Created attachment 97384 [details]
Backtrace

This bug is still present in the latest version from Git.
I also ran KTorrent via GDB and interrupted it during the freeze and obtained the attached backtrace. Hope this helps.
Comment 9 Nick Shaforostoff 2016-02-24 12:26:54 UTC
thanks. i think the problem may be resolved by moving bt::Downloader::pieceReceived call to non-main thread, give me few days to implement that...
Comment 10 Nick Shaforostoff 2016-02-24 23:33:10 UTC
while we're on it, could you please try mounting your ntfs partition with 'big_writes' option?

http://superuser.com/questions/613869/ntfs-write-speed-really-slow-15mb-s-on-ubuntu
Comment 11 Igor Poboiko 2016-02-25 08:43:21 UTC
(In reply to Nick Shaforostoff from comment #10)
> while we're on it, could you please try mounting your ntfs partition with
> 'big_writes' option?
> 
> http://superuser.com/questions/613869/ntfs-write-speed-really-slow-15mb-s-on-
> ubuntu

It didn't help that much. Maybe it started to freeze less often, but noticeably.
Comment 12 Nick Shaforostoff 2016-02-27 21:57:37 UTC
what is really scary is that i could reproduce this issue with my ssd drive just by downloading one 'House of Cards' episode. It seems that each time a chunk is written, the whole file is being rewritten on the disk. googling for 'transmission ntfs' bring similar results.

anyways, i suggest using a partition with different filesystem and then optionally move finished files to ntfs one
Comment 13 Nick Shaforostoff 2016-02-27 21:57:56 UTC
*** Bug 356559 has been marked as a duplicate of this bug. ***
Comment 14 Amichai Rothman 2016-02-28 09:08:46 UTC
Just wanted to give more specific info that I now have: It happens on two different HDD NTFS drives, so it's less likely to be related to failed disk, and does not happen on btrfs HDD or ext4 SSD.

It appears to happen more (not 100% sure) with faster download speeds (MB/s as opposed to KB/s).

Also, both drives are relatively full - iirc at some point NTFS starts writing data blocks to the MFT when it gets crowded, I don't know at what threshold but maybe it's somehow related? Can others check how full their problematic drives are to potentially reinforce or rule out this lead?
Comment 15 Igor Poboiko 2016-02-28 11:09:57 UTC
Well, I've got SSD drive with 11G out of 131G being free on my ntfs partition.

And in my case Transmission did not have this bug. Well, at least the interface didn't become unusable. So I guess moving IO jobs out of UI thread to separate one might at least solve the UI problem.
Comment 16 Amichai Rothman 2016-02-28 12:13:44 UTC
I certainly agree - separating business logic from UI threads is the proper design in any case, and should solve the responsiveness problem - but not necessarily the root cause of the long IO pauses themselves.

According to https://support.microsoft.com/en-us/kb/174619, 12.5% of disk space is allocated to the MFT by default, and is used to store data only if the rest of the free space is exhausted. For a 131G disk that's about 16G, so it looks like your case is consistent with the theory so far.
Comment 17 yodelu 2016-03-06 22:20:34 UTC
Bug confirmed (pclinuxos kde 32bit).
It seems that KT cannot (handle) reserve space on hdd when start the download on a NTFS partition.
That is happening with big torrents greater than 4GB, ,for example this one bellow , about 7 GB, will never work whenever i choose to reserve space OR full reserve space, before starting the download. Moreover, KT it just constantly freeze with reserve space on (full or quick). 
http://vault.centos.org/7.0.1406/isos/x86_64/CentOS-7.0-1406-x86_64-Everything.torrent
As a remark,  if choose to not use reserve space feature at all, then the KT reserve spaces anyway..
Comment 18 yodelu 2016-03-06 22:31:50 UTC
forgot to mention:
 - vuze, transmission and qtorrent are working fine..
- the ntfs partition is about 500GB and it is almost empty one ( a new HDD in fact )
Comment 19 Philip 2016-04-06 06:08:11 UTC
*** Bug 357160 has been marked as a duplicate of this bug. ***
Comment 20 muuhix 2016-04-06 21:03:37 UTC
Confirming this on: openSUSE tumbleweed 64bit AMD.
ntfs-3g ver. 2016.2.22-43.1
ktorrent ver. 4.3.1-12.1


2.1TiB free out of 3Tib on disk so its not MFT issue. Apparently the disk itself is not too busy because I can still read and write large files at normal speeds.
Comment 21 Nick Shaforostoff 2016-04-16 23:12:05 UTC
can someone confirm that commenting 'if (!OpenFileAllowed())' line (making the next 'return 0;' being executed unconditionally) in cachefile.cpp makes ntfs download work fine finally?
Comment 22 Nick Shaforostoff 2016-04-16 23:29:07 UTC
Git commit bc899d7449bd605421556f034c20879b595da88a by Nick Shaforostoff.
Committed on 16/04/2016 at 23:27.
Pushed by shaforo into branch '2.0'.

avoid mmaping on ntfs partitions

M  +3    -1    src/diskio/cachefile.cpp
M  +1    -1    src/torrent/torrentstats.cpp

http://commits.kde.org/libktorrent/bc899d7449bd605421556f034c20879b595da88a
Comment 23 Igor Poboiko 2016-04-17 14:24:08 UTC
(In reply to Nick Shaforostoff from comment #21)
> can someone confirm that commenting 'if (!OpenFileAllowed())' line (making
> the next 'return 0;' being executed unconditionally) in cachefile.cpp makes
> ntfs download work fine finally?

Cool! I confirm that KTorrent 5.0 with patch works just perfect.
Comment 24 Ruslan Kabatsayev 2016-05-29 17:33:46 UTC
Could this fix be somehow backported to KTorrent 4.3? I'm experiencing the same problem on Kubuntu 14.04 LTS with this version and NTFS destination.