Bug 66790 - ftp slave does not hanlde large files (>2GB) correctly
Summary: ftp slave does not hanlde large files (>2GB) correctly
Status: RESOLVED FIXED
Alias: None
Product: kio
Classification: Frameworks and Libraries
Component: sftp (show other bugs)
Version: unspecified
Platform: Debian testing Linux
: NOR normal
Target Milestone: ---
Assignee: Lucas Fisher
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2003-10-28 20:21 UTC by Riku Voipio
Modified: 2006-04-27 11:05 UTC (History)
3 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments
ftp_largefile.diff (2.83 KB, text/x-diff)
2003-10-28 20:47 UTC, David Faure
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Riku Voipio 2003-10-28 20:21:22 UTC
Version:            (using KDE KDE 3.1.4)
Installed from:    Debian testing/unstable Packages

Originally submitted to debian BTS #158841 by Eduard Bloch against 3.0.3

While the filebrowser itself seems to be ready for Large File
Support, the FTP module does not. It keeps reporting 2GB size on large
files.

Verified to still be a problem with 3.1.4, file size is shown wrong altough
if copied the entire file get's moved correctly
Comment 1 Riku Voipio 2003-10-28 20:25:19 UTC
looks like I targetted kio_ftp too early, atleast kio_smb and kio_sftp
have the same problem.
Comment 2 David Faure 2003-10-28 20:47:20 UTC
Subject: Re:  New: ftp slave does not hanlde large files (>2GB) correctly

> While the filebrowser itself seems to be ready for Large File
> Support, the FTP module does not. It keeps reporting 2GB size on large
> files.
> 
> Verified to still be a problem with 3.1.4, file size is shown wrong altough
> if copied the entire file get's moved correctly

Does this help? (untested, apart from compilation) 



Created an attachment (id=2918)
ftp_largefile.diff
Comment 3 Thiago Macieira 2003-10-28 23:30:32 UTC
Just confirming, for the atoi(3) usage is the source of the problem. I wonder what portable function should be used; atoll is C99 or GNU extension to C89.
Comment 4 Riku Voipio 2003-10-29 00:03:32 UTC
Thanks for 
Comment 5 Riku Voipio 2003-11-08 20:38:39 UTC
This fixes appears to fix the problem in the Ftp part. The Ftp folder window shows the 2.7Gb file size correctly. 

However, progress dialog still thinks the file is 2.0Gb. Which is probably a problem in that dialog rather than in the kioslave, since when copying the same
file using file:/ protocoll causes progress dialog think the file is 16.7TB =)

Notice that I am testing using 3.1.4, so this issue might be fixed in HEAD.
Comment 6 David Faure 2003-11-15 03:54:25 UTC
Waldo, can you review http://bugs.kde.org/attachment.cgi?id=2918&action=view so that I can commit it? Thanks.
Looks like we still have to fix the progress dialog though....
Comment 7 Waldo Bastian 2003-11-15 12:33:27 UTC
http.cc uses the following macro instead of atoll()

#ifdef HAVE_STRTOLL
#define STRTOLL strtoll
#else
#define STRTOLL strtol
#endif

I recommend to do the same here.
Comment 8 David Faure 2003-11-16 01:45:19 UTC
Subject: kdelibs/kioslave/ftp

CVS commit by faure: 

>2GB files ("long long") support in kio_ftp.
With input from Waldo. Partially fixes 66790, looks like the rest is for kio or uiserver.
CCMAIL: 66790@bugs.kde.org


  M +15 -9     ftp.cc   1.190
  M +5 -5      ftp.h   1.49


--- kdelibs/kioslave/ftp/ftp.cc  #1.189:1.190
@@ -64,6 +64,12 @@
 #define FTP_PASSWD  QString::fromLatin1("anonymous@")
 
+#ifdef HAVE_STRTOLL
+#define STRTOLL strtoll
+#else
+#define STRTOLL strtol
+#endif
 
-size_t Ftp::UnknownSize = (size_t)-1;
+
+KIO::filesize_t Ftp::UnknownSize = (KIO::filesize_t)-1;
 
 using namespace KIO;
@@ -1876,5 +1882,5 @@ FtpEntry* Ftp::ftpParseDir( char* buffer
                     de.owner    = QString::fromLatin1(p_owner);
                     de.group    = QString::fromLatin1(p_group);
-                    de.size     = atoi(p_size);
+                    de.size     = STRTOLL(p_size, 0, 10);
                     QCString tmp( p_name );
                     // Some sites put more than one space between the date and the name
@@ -2016,13 +2022,13 @@ void Ftp::get( const KURL & url )
   if ( strlen( rspbuf ) > 4 && m_size == UnknownSize ) {
     const char * p = strrchr( rspbuf, '(' );
-    if ( p != 0L ) m_size = atol( p + 1 );
+    if ( p != 0L ) m_size = STRTOLL( p + 1, 0, 10 );
   }
 
-  size_t bytesLeft = 0;
+  KIO::filesize_t bytesLeft = 0;
   if ( m_size != UnknownSize )
     bytesLeft = m_size - offset;
 
   kdDebug(7102) << "Ftp::get starting with offset=" << offset << endl;
-  int processed_size = offset;
+  KIO::fileoffset_t processed_size = offset;
 
   char buffer[ 2048 ];
@@ -2340,16 +2346,16 @@ bool Ftp::ftpSize( const QString & path,
   }
 
-  m_size = atol(rspbuf+4); // skip leading "213 " (response code)
+  m_size = STRTOLL(rspbuf+4, 0, 10); // skip leading "213 " (response code)
   return true;
 }
 
 
-size_t Ftp::ftpRead(void *buffer, long len)
+KIO::filesize_t Ftp::ftpRead(void *buffer, long len)
 {
-  size_t n = KSocks::self()->read( sData, buffer, len );
+  KIO::filesize_t n = KSocks::self()->read( sData, buffer, len );
   return n;
 }
 
-size_t Ftp::ftpWrite(void *buffer, long len)
+KIO::filesize_t Ftp::ftpWrite(void *buffer, long len)
 {
   return( KSocks::self()->write( sData, buffer, len ) );

--- kdelibs/kioslave/ftp/ftp.h  #1.48:1.49
@@ -45,5 +45,5 @@ struct FtpEntry
   QString link;
 
-  long size;
+  KIO::filesize_t size;
   mode_t type;
   mode_t access;
@@ -203,6 +203,6 @@ private:
   int ftpAcceptConnect();
 
-  size_t ftpRead( void *buffer, long len );
-  size_t ftpWrite( void *buffer, long len );
+  KIO::filesize_t ftpRead( void *buffer, long len );
+  KIO::filesize_t ftpWrite( void *buffer, long len );
 
   bool ftpChmod( const QString & path, int permissions );
@@ -275,6 +275,6 @@ private: // data members
   bool m_bPersistent;
 
-  size_t m_size;
-  static size_t UnknownSize;
+  KIO::filesize_t m_size;
+  static KIO::filesize_t UnknownSize;
 
   enum


Comment 9 Thiago Macieira 2003-12-01 04:52:18 UTC
Is this fixed?
Comment 10 Riku Voipio 2003-12-01 09:30:12 UTC
For ftp yes, but as I later noticed, this still seems to be a issue on atleast kio_smb, kio_sftp and in the "copying file" dialog. I don't have 3.2 betas or cvs installed, so this issue could be fixed there already.
Comment 11 Matt Rogers 2004-01-28 06:43:13 UTC
the "copying file dialog" works for me, and I just fixed an issue with the progress dialog used when the "Show all network operations in one window" setting is checked in Konqueror. I might be able to get around to checking kio_smb and kio_sftp soon.
Comment 12 Stephan Kulow 2004-11-12 12:12:42 UTC
I fixed this for smb differently
Comment 13 Dawit Alemayehu 2005-05-03 22:18:58 UTC
And this has been fixed for kio_sftp recently as well. It should be in the future 3.4.X release. See 

http://bugs.kde.org/show_bug.cgi?id=103729