Version: 2.0 (using KDE 3.5.4 Level "a" , unofficial build of SUSE ) Compiler: Target: i586-suse-linux OS: Linux (i686) release 2.6.16.13-4-default Downloading KNOPPIX_V5.0.1DVD-2006-06-01-EN, which contains an iso image over 4GB in size. The download completed succesful, apparently, but checking with md5sum revealed a mismatch. Performing a data integrity check on the completely downloaded iso showed 62 wrong pieces, at both the very beginning and very end of the iso file, although not contiguous. The refetch of these pieces by kTorrent did not resolve the problem. Fetching the iso, only the corrupt pieces, with another torrent client (Azureus) ran without problems, md5sum validated. Additional info: the content of the torrent was placed on a Reiserfs filesystem with plenty of free space, so that can not be the cause of the corruption.
We'll try to reproduce this with the knoppix torrent (this can take a while however with a 4 GB torrent)
Hmm, i have enough bandwidth, i'll download this torrent with the svn version to see if it gives me the same troubles. (the torrent will finish in a few hours)
Same bug here: Chunks at the beginning and at the end get removed on manual rehashing in KTorrent after the completion in KTorrent. Completing with KTorrent and rehashing in another client (utorrent) gives me the same picture with corrupted chunks. Torrent content: total 4200108 40 KNOPPIX-CHANGELOG.txt 12 knoppix-cheatcodes.txt 4200048 KNOPPIX_V5.0.1DVD-2006-06-01-EN.iso 4 KNOPPIX_V5.0.1DVD-2006-06-01-EN.iso.md5 4 KNOPPIX_V5.0.1DVD-2006-06-01-EN.iso.md5.asc I think we got ourself an unique torrent here.
I have a hunch that the 2 small files in the beginning somehow cause the offset in the big file to be miscalculated.
*** Bug 132871 has been marked as a duplicate of this bug. ***
It doesn't seem to be the two files, the offsets appear to be correct. I'm thinking more and more that this is some kind of integer overflow. (the dupe is also about a > 4 GB torrent)
The corrupt chunks, where they consecutive ?
SVN commit 576290 by guisson: Fixed data corruption bug. BUG: 132443 M +6 -2 multifilecache.cpp --- trunk/extragear/network/ktorrent/libktorrent/torrent/multifilecache.cpp #576289:576290 @@ -337,19 +337,23 @@ c->setStatus(Chunk::ON_DISK); return; } - else if (tflist.count() == 1 && c->getStatus() == Chunk::BUFFERED) + /* else if (tflist.count() == 1 && c->getStatus() == Chunk::BUFFERED) { // buffered chunks are slightly more difficult CacheFile* fd = files.find(tflist[0]); if (!fd) return; - Uint64 off = c->getIndex() * tor.getChunkSize(); + const TorrentFile & f = tor.getFile(tflist[0]); + + Uint64 off = FileOffset(c,f,tor.getChunkSize()); + //c->getIndex() * tor.getChunkSize(); fd->write(c->getData(),c->getSize(),off); c->clear(); c->setStatus(Chunk::ON_DISK); return; } + */ // Out() << "Writing to " << tflist.count() << " files " << endl; Uint64 written = 0; // number of bytes written
[quote] The corrupt chunks, where they consecutive ? [/quote] The first time i think so, yes. After svn up it seems not fixed here, i'll wipe, clean && co to be sure... I'll need a few hours to get it complete.
Nope, problem still present. Data lost after rehashing looks not consecutive this time. The weird thing is, opening this torrent with azureus gets 3 iso's listed(in the box where we should deside what we want to download, before the hashing) instead of one with the same name but only one with the correct filesize.(?)
Typo's above, plz don't read it, this one should be readable. Nope, problem still present.Data lost after rehashing looks not consecutive this time. The weird thing is, when i open this torrent with azureus, there are 3 iso's listed with all 3 a different filesize(in the box where we should deside what we want to download, before the hashing). You should expect only one iso there, right?
Damn, still something else then. Btw Martijn, I would make the column displaying the filenames in AZ a bit bigger ...
Better reopen the bug
[quote] Btw Martijn, I would make the column displaying the filenames in AZ a bit bigger ... [/quote] Yeah you're right, it was too beatiful cut down at *.iso to notice in a quicky. Anyway, stupid me... :o PS. For what it's worth: btype doesn't seem to open the torrent (ERROR: Caught exception: Invalid integer)
I have noticed that to, it probably can't handle the 64 bit integers. Though my version is pretty old.
SVN commit 576754 by guisson: This time it is fixed. BUG: 132443 M +1 -1 cachefile.cpp M +1 -0 chunkmanager.cpp --- trunk/extragear/network/ktorrent/libktorrent/torrent/cachefile.cpp #576753:576754 @@ -149,7 +149,7 @@ // off is not a multiple of the page_size // so we play around a bit Uint32 diff = (off % page_size); - Uint32 noff = off - diff; + Uint64 noff = off - diff; // Out() << "Offsetted mmap : " << diff << endl; char* ptr = (char*)mmap(0, size + diff, mmap_flag, MAP_SHARED, fd, noff); if (ptr == MAP_FAILED) --- trunk/extragear/network/ktorrent/libktorrent/torrent/chunkmanager.cpp #576753:576754 @@ -840,6 +840,7 @@ } else if (!ok_chunks.get(i) && bitset.get(i)) { + Out() << "Previously OK chunk " << i << " is corrupt !!!!!" << endl; // We think we have a chunk, but we don't bitset.set(i,false); if (c->getStatus() == Chunk::ON_DISK)
Just stats purpose, now a single file torrent with 32.41GB data downloads fine too.