Bug 132443

We'll try to reproduce this with the knoppix torrent (this can take a while however with a 4 GB torrent)

Hmm, i have enough bandwidth, i'll download this torrent with the svn version to see if it gives me the same troubles. (the torrent will finish in a few hours)

Same bug here:
Chunks at the beginning and at the end get removed on manual rehashing in KTorrent after the completion in KTorrent.
Completing with KTorrent and rehashing in another client (utorrent) gives me the same picture with corrupted chunks.

Torrent content:
total 4200108
     40 KNOPPIX-CHANGELOG.txt
     12 knoppix-cheatcodes.txt
4200048 KNOPPIX_V5.0.1DVD-2006-06-01-EN.iso
      4 KNOPPIX_V5.0.1DVD-2006-06-01-EN.iso.md5
      4 KNOPPIX_V5.0.1DVD-2006-06-01-EN.iso.md5.asc

I think we got ourself an unique torrent here.

I have a hunch that the 2 small files in the beginning somehow cause the offset in the big file to be miscalculated.

*** Bug 132871 has been marked as a duplicate of this bug. ***

It doesn't seem to be the two files, the offsets appear to be correct. I'm thinking more and more that this is some kind of integer overflow. (the dupe is also about a > 4 GB torrent)

The corrupt chunks, where they consecutive ?

SVN commit 576290 by guisson:

Fixed data corruption bug.

BUG: 132443 



 M  +6 -2      multifilecache.cpp  


--- trunk/extragear/network/ktorrent/libktorrent/torrent/multifilecache.cpp #576289:576290
@@ -337,19 +337,23 @@
 			c->setStatus(Chunk::ON_DISK);
 			return;
 		}
-		else if (tflist.count() == 1 && c->getStatus() == Chunk::BUFFERED)
+	/*	else if (tflist.count() == 1 && c->getStatus() == Chunk::BUFFERED)
 		{
 			// buffered chunks are slightly more difficult
 			CacheFile* fd = files.find(tflist[0]);
 			if (!fd)
 				return;
 			
-			Uint64 off = c->getIndex() * tor.getChunkSize();
+			const TorrentFile & f = tor.getFile(tflist[0]);
+			
+			Uint64 off = FileOffset(c,f,tor.getChunkSize()); 
+					//c->getIndex() * tor.getChunkSize();
 			fd->write(c->getData(),c->getSize(),off);
 			c->clear();
 			c->setStatus(Chunk::ON_DISK);
 			return;
 		}
+	*/
 		
 	//	Out() << "Writing to " << tflist.count() << " files " << endl;
 		Uint64 written = 0; // number of bytes written

[quote]
The corrupt chunks, where they consecutive ?
[/quote]

The first time i think so, yes.

After svn up it seems not fixed here, i'll wipe, clean && co to be sure...
I'll need a few hours to get it complete.

Nope, problem still present. Data lost after rehashing looks not consecutive this time. The weird thing is, opening this torrent with azureus gets 3 iso's listed(in the box where we should deside what we want to download, before the hashing) instead of one with the same name but only one with the correct filesize.(?)

Typo's above, plz don't read it, this one should be readable.

Nope, problem still present.Data lost after rehashing looks not consecutive this time.
The weird thing is, when i open this torrent with azureus, there are 3 iso's listed with all 3 a different filesize(in the box where we should deside what we want to download, before the hashing). You should expect only one iso there, right?

Damn, still something else then.

Btw Martijn, I would make the column displaying the filenames in AZ a bit bigger ...

Better reopen the bug

[quote]
Btw Martijn, I would make the column displaying the filenames in AZ a bit bigger ... [/quote]

Yeah you're right, it was too beatiful cut down at *.iso to notice in a quicky. Anyway, stupid me... :o

PS. For what it's worth: btype doesn't seem to open the torrent (ERROR: Caught exception: Invalid integer)

I have noticed that to, it probably can't handle the 64 bit integers. Though my version is pretty old.

SVN commit 576754 by guisson:

This time it is fixed.

BUG: 132443



 M  +1 -1      cachefile.cpp  
 M  +1 -0      chunkmanager.cpp  


--- trunk/extragear/network/ktorrent/libktorrent/torrent/cachefile.cpp #576753:576754
@@ -149,7 +149,7 @@
 			// off is not a multiple of the page_size
 			// so we play around a bit
 			Uint32 diff = (off % page_size);
-			Uint32 noff = off - diff;
+			Uint64 noff = off - diff;
 		//	Out() << "Offsetted mmap : " << diff << endl;
 			char* ptr = (char*)mmap(0, size + diff, mmap_flag, MAP_SHARED, fd, noff);
 			if (ptr == MAP_FAILED) 
--- trunk/extragear/network/ktorrent/libktorrent/torrent/chunkmanager.cpp #576753:576754
@@ -840,6 +840,7 @@
 			}
 			else if (!ok_chunks.get(i) && bitset.get(i))
 			{
+				Out() << "Previously OK chunk " << i << " is corrupt !!!!!" << endl;
 				// We think we have a chunk, but we don't
 				bitset.set(i,false);
 				if (c->getStatus() == Chunk::ON_DISK)

Just stats purpose, now a single file torrent with 32.41GB data downloads fine too.