Bug 315239 - ktorrent-4.3.1 does not gunzip IP Filter files anymore
Summary: ktorrent-4.3.1 does not gunzip IP Filter files anymore
Status: RESOLVED FIXED
Alias: None
Product: ktorrent
Classification: Applications
Component: general (show other bugs)
Version: 4.3.1
Platform: Gentoo Packages Linux
: NOR normal
Target Milestone: ---
Assignee: Joris Guisson
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2013-02-15 21:40 UTC by Denis M. (Phr33d0m)
Modified: 2016-01-19 18:54 UTC (History)
21 users (show)

See Also:
Latest Commit:
Version Fixed In:
Sentry Crash Report:


Attachments
snapshot of the error.png (114.24 KB, image/png)
2013-02-15 21:40 UTC, Denis M. (Phr33d0m)
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Denis M. (Phr33d0m) 2013-02-15 21:40:21 UTC
Hello, with version 4.3.0 this wasn't present.
After upgrading to 4.3.1, when ktorrent tries to download an IP filter file from (for example) the default location "http://list.iblocklist.com/?list=bt_level1&fileformat=p2p&archiveformat=gz" it successfully downloads the file but does not decompress it.


Reproducible: Always

Steps to Reproduce:
1. KTorrent downloads the IP filter list
2. It "forgets" to decompress (gunzip) it 
3. It renames(?) the file to level1.txt but the file is still compressed
Actual Results:  
KTorrent shows an error message.


If I manually go to the ~/.kde4/share/apps/ktorrent/ and rename the level1.txt file to level1.txt.gz and do `gunzip level1.txt.gz` the file gets decompressed and ktorrent recognizes it as valid file.

I attach a screenshot of the error message and what the level1.txt file looks like before doing my trick said above.
Comment 1 Denis M. (Phr33d0m) 2013-02-15 21:40:46 UTC
Created attachment 77345 [details]
snapshot of the error.png
Comment 2 Joris Guisson 2013-02-18 16:59:58 UTC
Seems to work fine here
Comment 3 Lukas Schneiderbauer 2013-02-20 11:10:07 UTC
I get the same error-popup after downloading from the default location (didn't try other sources). Btw the level1.txt file gets deleted after confirming the popup-message.
Comment 4 Denis M. (Phr33d0m) 2013-02-20 22:34:40 UTC
(In reply to comment #3)
> Btw the level1.txt file gets deleted after confirming the popup-message.
Yup, same here. I forgot to mention that in my description message above.
Comment 5 Sebastian Żebrowski 2013-02-22 23:13:22 UTC
I cannot update ipfilter list, getting following error message:
"There are no IP addresses to convert in /home/user/.kde4/share/apps/ktorrent/level1.txt"

Should I create new bug report or is it connected to this bug?
Comment 6 Denis M. (Phr33d0m) 2013-02-22 23:23:23 UTC
(In reply to comment #5)
> Should I create new bug report or is it connected to this bug?
This is the same bug yes, see the screenshot in comment 1.

Good to see I wasn't the only one getting this issue.
Comment 7 Joris Guisson 2013-03-11 18:35:21 UTC
Must be the file type detection which is going wrong and thinking it is a text file, while it is a zip file.
Comment 8 Joris Guisson 2013-03-11 18:51:41 UTC
Git commit cb643c530b0b8a906032f0f2d8947a4cee6553bc by Joris Guisson.
Committed on 11/03/2013 at 19:51.
Pushed by guisson into branch 'master'.

Use file contents to figure out which type of blocklist we are dealing with in IP filter

M  +1    -0    ChangeLog
M  +16   -18   plugins/ipfilter/downloadandconvertjob.cpp

http://commits.kde.org/ktorrent/cb643c530b0b8a906032f0f2d8947a4cee6553bc
Comment 9 Lukas Schneiderbauer 2013-03-11 23:31:41 UTC
I applied this patch to the released ktorrent-4.3.1. The behaviour described above is unchanged. The bug still is not fixed!
Comment 10 Denis M. (Phr33d0m) 2013-03-12 01:36:25 UTC
(In reply to comment #9)
> I applied this patch to the released ktorrent-4.3.1. The behaviour described
> above is unchanged. The bug still is not fixed!
This is not entirely true.

The supplied patch works, BUT only for ZIP archive formats.
So if you select to use http://list.iblocklist.com/?list=bt_level3&fileformat=p2p&archiveformat=zip it will work. Although if you select archiveformat=gz or 7z it won't work showing an error window which says: "Cannot determine file type of http://list.iblocklist.com/?list=bt_level3&fileformat=p2p&archiveformat=gz"

I allow myself to change the bug status back to unconfirmed.
Comment 11 Lukas Schneiderbauer 2013-03-12 16:06:22 UTC
(In reply to comment #10)
> The supplied patch works, BUT only for ZIP archive formats.
> So if you select to use
> http://list.iblocklist.com/?list=bt_level3&fileformat=p2p&archiveformat=zip
> it will work.
It doesn't work for me. Although the error message after downloading is different:
"Cannot find blocklist in zip file /home/vootey/.kde4/share/apps/ktorrent/level1.zip."

> Although if you select archiveformat=gz or 7z it won't work
> showing an error window which says: "Cannot determine file type of
> http://list.iblocklist.com/?list=bt_level3&fileformat=p2p&archiveformat=gz"
For me the same old error message:
"There are no IP addresses to convert in /home/vootey/.kde4/share/apps/ktorrent/level1.txt"
Comment 12 Tomas Coufal 2013-03-31 15:16:10 UTC
*** This bug has been confirmed by popular vote. ***
Comment 13 Joris Guisson 2013-04-18 17:20:51 UTC
I have tested both the gz and zip urls, they both work on the master branch.
Comment 14 Andy 2013-05-14 16:37:58 UTC
Hi, 
this error still occours.

KTorrent Version 4.3.1
Archlinux 64bit

It happens with both:

http://list.iblocklist.com/?list=bt_level3&fileformat=p2p&archiveformat=zip

and

http://list.iblocklist.com/?list=bt_level3&fileformat=p2p&archiveformat=gz

Greetings,
Andy
Comment 15 Tom Chiverton 2013-05-14 20:34:11 UTC
Snap. v4.3.1 still has the bug, Kubuntu 13.04.
Comment 16 Paulo Fidalgo 2013-05-16 22:29:15 UTC
Sill happens Version 4.3.1 on Fedora 18.
Comment 17 Jekyll Wu 2013-05-17 03:41:33 UTC
Joris, which version will the master branch use when it is released ? Please set the "Version Fixed In" field to avoid confusions.
Comment 18 Joris Guisson 2013-05-30 16:41:48 UTC
OK, it seems I can reproduce it now to, probably since my upgrade to raring.
Comment 19 Mathias Dietrich 2013-07-15 15:21:16 UTC
As a workaround you can unzip the blocklist locally and enter the filepath to the text-blocklist in ktorrent
Comment 20 Joris Guisson 2013-08-01 10:51:56 UTC
Git commit 09421b0222c7552ddcfecbd8c05d8c069a05ba5f by Joris Guisson.
Committed on 01/08/2013 at 10:51.
Pushed by guisson into branch 'master'.

Fix gzip not working in ipfilter plugin due to wrong mimetype

M  +1    -0    ChangeLog
M  +15   -3    plugins/ipfilter/downloadandconvertjob.cpp

http://commits.kde.org/ktorrent/09421b0222c7552ddcfecbd8c05d8c069a05ba5f
Comment 21 Joris Guisson 2013-08-01 10:55:44 UTC
gzip is back to normal, apparently the mimetype changed from application/x-gzip to application/gzip in some recent version of KDE. 

zip is also working back, without any modifications. Not sure what changed, must be a bugfix in kdelibs which fixed it.
Comment 22 Joris Guisson 2013-08-01 11:20:03 UTC
Git commit 3dfcbeb9765e99f9b99951134b17268eb7a617d2 by Joris Guisson.
Committed on 01/08/2013 at 11:19.
Pushed by guisson into branch '4.3'.

Backport ipfilter extraction fixes from master branch to 4.3 branch

M  +3    -0    ChangeLog
M  +5    -17   plugins/ipfilter/downloadandconvertjob.cpp

http://commits.kde.org/ktorrent/3dfcbeb9765e99f9b99951134b17268eb7a617d2
Comment 23 nfettinger 2013-08-04 21:41:41 UTC
I am still receiving this error.
Cannot determine file type of http://list.iblocklist.com/?list=bt_level1&fileformat=p2p&archiveformat=gz

Using developmental branch pulled 13/08/02
Qt: 4.8.1
KDE Development Platform: 4.8.5 (4.8.5)
KTorrent: 4.3dev
Comment 24 Eric Griffith 2013-08-15 19:23:01 UTC
Fedora 19 KDE x64, Ktorrent 4.3.1

For completeness... the following URL's error out: 

http://list.iblocklist.com/?list=bt_level1&fileformat=p2p&archiveformat=7z
http://list.iblocklist.com/?list=bt_level1&fileformat=p2p&archiveformat=gz
http://list.iblocklist.com/?list=bt_level1&fileformat=p2p&archiveformat=zip
http://list.iblocklist.com/?list=bt_level1&fileformat=p2p&archiveformat=

Joris, whats KTorrent's release schedule like? 4.3.1 was was in January, 4.3.0 was in September... no offense but 4 months for a bug fix release is kind of.... unheard of. And we're now at 7 months without a 4.3.2 release. So can you tag a 4.3.2 release so this, and other, bugfixes can be pushed to users? Release early, release often.
Comment 25 Michael 2013-09-17 09:24:17 UTC
KTorrent 4.3.1, Kubuntu 13.04, KDE 4.11.1, x64

I'm not sure I understand why the error message differs from what I'm seeing, and what I'm noticing is different from Phr33d0m's original post.

If I go into KTorrent's Settings -> IP Filter, and enable the "Use PeerGuardian filter", click the "Download/Convert" button, I get this familiar message:

"There are no IP addresses to convert in /home/user/.kde/share/apps/ktorrent/level1.txt"

However, if leave that error message visible and open the level1.txt file, I see a text file with 235,295 lines, and they seem valid, like:

SMSHoax FakeAV Fraud Trojan:1.179.144.98-1.179.144.98
Detected AP2P on TOT Public Co. proxy:1.179.147.2-1.179.147.2
Detected AP2P on ChinaNet Beijing proxy:1.202.96.52-1.202.96.52

So there does appear to be IP addresses in it, in contrast to the message.

After I click to dismiss the "error message", the level1.txt file gets deleted.

This is the URL for the IP filter file: http://list.iblocklist.com/?list=bt_level1&fileformat=p2p&archiveformat=gz

So the error message doesn't seem to match what I'm seeing. Does this help in resolving this issue?
Comment 26 Eric Griffith 2013-09-17 15:11:28 UTC
Michael, 

As far as I can tell so far, this is the issue:

Your text editor probably realizes the text file is compressed and uncompresses it on the fly. This is an issue where KTorrent is handed a compressed file, and doesn't realize its compressed and tries to use it as-is. Leading to garbage input, which makes KTorrent think there's nothing valid.

What SHOULD happen is KTorrent realizes its compressed, passes it through the unzip de-compression utility. And then gets handed back the uncompressed version of the text file, which is valid input and everyone's happen.
Comment 27 rockonthemoonfm 2013-10-17 21:44:41 UTC
here (arch 64 ktorrent 4.3.1 kde 4.11.2) bug is still present with the well known warning "There are no IP addresses to convert in XYZ"

compressed ip lists do get decompressed, but do not get loaded.

loading archives in formats that my system doesn't support, produces the same warning.

even if i decompress a list manually, ktorrent produces the same warning while loading it.
Comment 28 dcg 2013-11-25 14:26:08 UTC
I can confirm the same as previous comment here, arch 64 ktorrent 4.3.1 KDE 4.11.3. 

It makes me suspicious that both us are using Arch. How does the KDE mime system works? How could it not be detecting the proper file type?
Comment 29 Tom Chiverton 2013-11-25 18:36:30 UTC
Nah, it's not just Arch. Still bust in Kubuntu's Ktorrent 4.3.1 using http://list.iblocklist.com/?list=bt_level1&fileformat=p2p&archiveformat=gz which is fine on the command line after wget'ing :

$ file index.html\?list\=bt_level1
index.html?list=bt_level1: gzip compressed data, was "bt_level1.txt", from Unix, last modified: Mon Nov 25 04:11:48 2013, max compression

And using that file uncompressed in the dialogue works too.

Maybe the fixed version isn't being pushed to the repositories.
Comment 30 Tom Chiverton 2013-12-21 17:36:13 UTC
KTorrent 4.3.1 in KDE 4.11.3 to be precise.
Comment 31 João Bóia 2013-12-26 15:40:09 UTC
Issue still present in Ktorrent 4.3.1 with KDE platform 4.12
Comment 32 bogdan 2014-02-05 18:09:27 UTC
Still present in Opensuse 13.1 KDE 4.12.2 , ktorrent  4.3.1
If it is Fixed where did those fixes go?
Comment 33 Harald Sitter 2014-02-10 08:57:11 UTC
From what I can see the fixed version has not yet been released. Please wait for KTorrent >= 4.3.2.

TIA
Comment 34 cameron0x00 2014-02-12 21:57:22 UTC
This issue still exist. And it has for about a year. Automatically downloading and converting still does not work. Manually converting does work. So, do we need to wait for a new version or can a patch just be pushed out? 

I'm using: KTorrent 4.3.1; KDE Dev Platform 4.11.5
Comment 35 Harald Sitter 2014-02-12 22:14:06 UTC
That's what I said, isn't it?
Comment 36 ximi 2014-03-04 10:36:40 UTC
Hello!
I'm on debian sid with ktorrent 4.3.1-2 installed from a while. I never had this problem until recently, but ktorrent has not been upgraded. So, I guess it is a bug from a recent update of KDE (I'm on KDE 4.11.6-3)
Comment 37 Tom Chiverton 2014-03-30 13:05:34 UTC
Still not fixed in Ktorrent Version 4.3.1 (Using KDE Development Platform 4.12.95) which is in Ubuntu 14.04 LTS beta 2

The commit to fix this was six months ago, so it should surely have been picked up in all the downstreams by now ?
Comment 38 fademind 2014-03-30 14:31:11 UTC
In Kubuntu 14.04 bug present without any changes>

==============================================
cat /home/tomasz/.kde/share/apps/ktorrent/log

niedz. mar 30 16:26:52 2014: Bound to ::
niedz. mar 30 16:26:52 2014: Cannot bind to port 0.0.0.0:6881 : Adres jest już w użyciu
niedz. mar 30 16:26:52 2014: Bound to TCP port 6881
niedz. mar 30 16:26:52 2014: Cannot open /home/tomasz/.kde/share/apps/ktorrent/groups : Nie ma takiego pliku ani katalogu
niedz. mar 30 16:26:52 2014: Failed to open /home/tomasz/.kde/share/apps/ktorrent/magnets : No such file or directory
niedz. mar 30 16:26:52 2014: Started update timer
niedz. mar 30 16:26:52 2014: Suppressing sleep
niedz. mar 30 16:26:52 2014: Qt Warning: Bus::open: Can not get ibus-daemon's address. 
niedz. mar 30 16:26:52 2014: Qt Debug: IBusInputContext::createInputContext: no connection to ibus-daemon 
niedz. mar 30 16:26:52 2014: Stopped update timer
niedz. mar 30 16:26:52 2014: Stopped suppressing sleep
niedz. mar 30 16:26:53 2014: No torrents to migrate found 
niedz. mar 30 16:27:13 2014: Cannot open /home/tomasz/.kde/share/apps/ktorrent/level1.dat: No such file or directory
niedz. mar 30 16:27:13 2014: Cannot open /home/tomasz/.kde/share/apps/ktorrent/level1.dat: No such file or directory
niedz. mar 30 16:27:17 2014: Qt Warning: Object::connect: No such signal org::freedesktop::UPower::DeviceAdded(QDBusObjectPath)
niedz. mar 30 16:27:17 2014: Qt Warning: Object::connect: No such signal org::freedesktop::UPower::DeviceRemoved(QDBusObjectPath)
niedz. mar 30 16:27:48 2014: Loading /home/tomasz/.kde/share/apps/ktorrent/level1.txt ...
niedz. mar 30 16:27:48 2014: Loaded 0 lines
niedz. mar 30 16:27:53 2014: Cannot open /home/tomasz/.kde/share/apps/ktorrent/level1.dat: No such file or directory
Comment 39 Harald Sitter 2014-03-31 07:29:52 UTC
(In reply to comment #33)
> From what I can see the fixed version has not yet been released. Please wait
> for KTorrent >= 4.3.2.

^ autoquote
Comment 40 Tom Chiverton 2014-03-31 17:53:41 UTC
That was my point Harald. The fix was made half a year ago, but still not rolled out. Why, and how can we speed it up  ?
Comment 41 Harald Sitter 2014-03-31 22:19:26 UTC
I do not know, someone should like look into that ;)
Comment 42 Christo 2014-10-21 20:08:05 UTC
opensuse 13.1 .. kt 4.3.1 .. my filename is level1.txt .. but same issue as described above .. i wonder if KT is an active app ;-)
Comment 43 Christo 2014-11-18 11:57:38 UTC
for Opensuse 13.1 there is a patched version available:

https://build.opensuse.org/package/show?project=KDE%3AExtra&package=ktorrent
Comment 44 Tom Chiverton 2016-01-16 20:20:53 UTC
Still broken in Kubuntu 4.14.4. Clearly updates are not going out to distributions correctly.
Comment 45 Paulo Fidalgo 2016-01-19 18:36:50 UTC
(In reply to Tom Chiverton from comment #44)
> Still broken in Kubuntu 4.14.4. Clearly updates are not going out to
> distributions correctly.

You need to report with your distro. Here in Fedora 23 everything is working right.