Created attachment 68630 [details] One of corrupted extracted files Version: unspecified (using KDE 4.7.4) OS: Linux I have many problems building and compiling some programs, and i see that there are corrupt data at the final of a lot of the extracted files. At some point creates non-text chars, and it only occurs if the working partition is ntfs (I test in a ext4 partition and doesn't happen) and when i extract a lot of files (one to one works well) I test too if it happens with zip files, but only occurs with tar.gz files. With tar console command i haven't any problem. Reproducible: Always Steps to Reproduce: - Have a tar.gz with some files inside (text files for test) - copy and extract it somewhere into a ntfs partition - view the files with kate (or joe, or whatever is applicable) Actual Results: Some extracted files have corrupted data at some point in the final of the file, with non-text chars In a example text file in kate: this is a test file. this is a test file. this is a test file. this is a test file this is a test file. this is a test file. this is a test file. this is a test file this is a test file. this is a test file. th******************** ****************************************************************************** ****************************************************************************** (*): * are the square non-text symbol Expected Results: All extracted files are ok without problems The example text file in kate: this is a test file. this is a test file. this is a test file. this is a test file this is a test file. this is a test file. this is a test file. this is a test file this is a test file. this is a test file. this is a test file. this is a test file this is a test file. this is a test file. this is a test file. this is a test file With Ark 2.17 in KUbuntu 11.10 Oneiric Ocelot. KDE 4.7.4 I attached tests files corrupted in my tests
Created attachment 68631 [details] Other corrupted text file
Created attachment 68632 [details] tar.gz with the testfiles intact
In the Attached files, are a tar.gz i made for the tests, with testfile1 testfile2 and testfile3 text files with the same pattern inside. Only 2 and 3 have corrupted data, 1 is intact
Can you check if the problem also happens if you use the "bsdtar" tool to extract files in an NTFS partition?
$ bsdtar -xf testfiles.tar.gz testfiles/testfile1.txt: Can't update time for testfiles/testfile1.txt testfiles/testfile2.txt: Can't update time for testfiles/testfile2.txt testfiles/testfile3.txt: Can't update time for testfiles/testfile3.txt bsdtar: Error exit delayed from previous errors. Also, the files are OK without problems. With Ark still fails.
I've just created an ntfs partition on a file here and mounted it, and both Ark and bsdtar worked fine. I ran ark on the command-line with 'ark -bea testfiles.tar.gz' and bsdtar with the same parameters you used (bsdtar didn't even complain about updating time). This is an up-to-date Arch Linux system with KDE 4.8.0. Can you make sure this is really related to NTFS (ie. try out a few other file systems)? If it is, perhaps it's related to the kernel you are using?
Yes, it's true. I test with a ntfs partition created in a usb flash drive, and doesn't have the problem. But i have the problem with all ntfs partitions in my hard disk. I have two, one for documents and another with a windows installation. With both have the problem. I tested instead with a ext4 in hard disk (linux root) and a fat32 usb flash drive without the problem. I have the 3.0.0-16 kernel version, with KUbuntu KDE 4.7.4 in KUbuntu 11.10. Copy the fstab and mtab (respectively): UUID=164190c6-8ee4-46e8-9217-673dac9ca1fe / ext4 errors=remount-ro 0 1 # /mnt/documentos was on /dev/sda7 during installation UUID=61BC4E8ECAA0CC64 /mnt/documentos ntfs defaults,umask=007,gid=46 0 0 # /mnt/windows was on /dev/sda1 during installation UUID=6ED4B176D4B140DF /mnt/windows ntfs defaults,umask=007,gid=46 0 0 /dev/sda6 none swap sw 0 0 /dev/sda5 / ext4 rw,errors=remount-ro,commit=0 0 0 proc /proc proc rw,noexec,nosuid,nodev 0 0 sysfs /sys sysfs rw,noexec,nosuid,nodev 0 0 fusectl /sys/fs/fuse/connections fusectl rw 0 0 none /sys/kernel/debug debugfs rw 0 0 none /sys/kernel/security securityfs rw 0 0 udev /dev devtmpfs rw,mode=0755 0 0 devpts /dev/pts devpts rw,noexec,nosuid,gid=5,mode=0620 0 0 tmpfs /run tmpfs rw,noexec,nosuid,size=10%,mode=0755 0 0 none /run/lock tmpfs rw,noexec,nosuid,nodev,size=5242880 0 0 none /run/shm tmpfs rw,nosuid,nodev 0 0 /dev/sda7 /mnt/documentos fuseblk rw,nosuid,nodev,allow_other,blksize=4096,default_permissions 0 0 /dev/sda1 /mnt/windows fuseblk rw,nosuid,nodev,allow_other,blksize=4096,default_permissions 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,noexec,nosuid,nodev 0 0
"ark -bea testfiles.tar.gz", bsdtar and extracting from desktop do the same, corrupted text files. If this problem is fixed in KDE 4.8.0, i don't have any problem with that, when i update the system willn't have the problem anymore, but i search it in bugs.kde and i don't find any reference, because that i posted it. I don't know where is the problem, only it's related with my ntfs partition on my hard disk. I can search and upload files or information you please
(In reply to comment #8) > "ark -bea testfiles.tar.gz", bsdtar and extracting from desktop do the same, > corrupted text files. I'm a bit confused here because you said in comment #5 that bsdtar had worked fine. > If this problem is fixed in KDE 4.8.0, i don't have any problem with that, when > i update the system willn't have the problem anymore, but i search it in > bugs.kde and i don't find any reference, because that i posted it. Nah, there's been no commit to 4.8.0 to this part of the code. > I don't know where is the problem, only it's related with my ntfs partition on > my hard disk. I can search and upload files or information you please Let's try to rule out some other possibilities: * Can you try to extract these files from a zip, rar, 7z and or gzip (.gz, not .tar.gz) archive, so that we can check if this is a problem which only affects the libarchive plugin? * Can you test an NTFS partition the same way I did: $ mkdir /tmp/ntfs $ dd if=/dev/zero of=/tmp/ntfs.img bs=1M count=100 $ mkfs.ntfs -f -F /tmp/ntfs.img $ sudo mount /tmp/ntfs.img /tmp/ntfs -t ntfs -o loop $ cd /tmp/ntfs $ # extract the archives here Does it also produce bad results? * One last thing: in your original comment you mentioned extracting a single a file at a time did not produce problematic files. Is that still true?
(In reply to comment #9) > > "ark -bea testfiles.tar.gz", bsdtar and extracting from desktop do the same, > > corrupted text files. > > I'm a bit confused here because you said in comment #5 that bsdtar had worked > fine. Sorry, it's a mistake. bsdtar works fine, the others don't. > * One last thing: in your original comment you mentioned extracting a single a > file at a time did not produce problematic files. Is that still true? I will test the other possibilities, but for this, i can say: if you see the files in tar.gz there are testfile1, testfile2, testfile3. Always have testfile2 and 3 corrupted, but testfile1 is perfect. The same with extracting one file drag and dropping, or otherwise, this file haven't any problem because is the first (and only) file, it seems.
(In reply to comment #9) > * Can you try to extract these files from a zip, rar, 7z and or gzip (.gz, not > .tar.gz) archive, so that we can check if this is a problem which only affects > the libarchive plugin? I have tested 7z, jar, rar, tar, tar.bz, tar.gz, tar.lzma, tar.xz, zip (All formats i can create with Ark): - 7z, jar, rar and zip work well. - All tar based files work wrong (tar, tar.bz, tar.gz, tar.lzma, tar.xz) > * Can you test an NTFS partition the same way I did: > $ mkdir /tmp/ntfs > $ dd if=/dev/zero of=/tmp/ntfs.img bs=1M count=100 > $ mkfs.ntfs -f -F /tmp/ntfs.img > $ sudo mount /tmp/ntfs.img /tmp/ntfs -t ntfs -o loop > $ cd /tmp/ntfs > $ # extract the archives here > Does it also produce bad results? When i formated the usb flash disk with the KDE partition manager i don't have the problem. If you want, i do these steps, but i prefer don't mess with the dd commands and others if not strictly necessary. But, i found some important information, i guess. The two ntfs partitions in hard disk are created and formated from Windows (XP) before install KUbuntu. In these partitions when i use bsdtar for extract tar files (without the problem but) i have those messages: $ bsdtar -xf testfiles.tar testfiles/testfile1.txt: Can't update time for testfiles/testfile1.txt testfiles/testfile2.txt: Can't update time for testfiles/testfile2.txt testfiles/testfile3.txt: Can't update time for testfiles/testfile3.txt bsdtar: Error exit delayed from previous errors. (Anyway, the files are ok) With the ntfs partition created by KDE partition manager in the usb flash disk it doesn't have any message. I will test to format from windows and test again in case that there is any kind of difference that causes this. When i copy in Dolphin a lot of files, in some case i have popups with "I can't change permissions", and i think it's related, but i don't have any corrupted data, all the copied files are ok. Only with Ark have corrupted data, at least that i have seen and so far i haven't had problems with KDE or other applications related.
(In reply to comment #11) > But, i found some important information, i guess. The two ntfs partitions in > hard disk are created and formated from Windows (XP) before install KUbuntu. In > these partitions when i use bsdtar for extract tar files (without the problem > but) i have those messages: > > $ bsdtar -xf testfiles.tar > testfiles/testfile1.txt: Can't update time for testfiles/testfile1.txt > testfiles/testfile2.txt: Can't update time for testfiles/testfile2.txt > testfiles/testfile3.txt: Can't update time for testfiles/testfile3.txt > bsdtar: Error exit delayed from previous errors. > > (Anyway, the files are ok) > > With the ntfs partition created by KDE partition manager in the usb flash disk > it doesn't have any message. I will test to format from windows and test again > in case that there is any kind of difference that causes this. I used format tool of Windows XP, and i don't have any problem with that in linux. When i made the partitions and formated them, i used the booteable application in Windows XP CD. The only solution to test this is use other computer and see if it do the same. In addition, with the command "ark -bea ..." i don't have any messages in all cases, it's a difference with "bsdtar -xf ...", that throws me the messages i posted before, and, with the ntfs usb flash disk formated in windows don't throws me too.
(In reply to comment #12) > I used format tool of Windows XP, and i don't have any problem with that in > linux. When i made the partitions and formated them, i used the booteable > application in Windows XP CD. The only solution to test this is use other > computer and see if it do the same. I'm running out of ideas here too... You could try out the dd thing I suggested in comment #9 (it'll just create the partition on a regular file, so you shouldn't need to worry), and perhaps check if other Kubuntu users (or forums.kde.org users) experience the same problem -- it may be related to the way Windows creates NTFS partitions, as I created some NTFS partitions on file with varying parameters and everything has always worked fine. > In addition, with the command "ark -bea ..." i don't have any messages in all > cases, it's a difference with "bsdtar -xf ...", that throws me the messages i > posted before, and, with the ntfs usb flash disk formated in windows don't > throws me too. Please make sure you have installed any Ark debug package Kubuntu might have, then run "kdebugdialog" and check if Ark's checkbox(es) is/are checked. Doing so should make Ark produce more debugging output on the terminal.
I will test all in another computer, i want to reproduce the problem. If i fail, i will test in the main computer.
I reproduce the bug in another computer: Firstly i have an empty hard disk. - With Windows XP Live CD, i created 4 partitions (in order): 18GB for WinXP, 18GB for Linux root, 1,5GB for swap and 45GB for an empty data partition. I only formatted the winxp partition to ntfs with it for install, the others only created the partitions. - Once in WinXP, i formatted the 45GB partition to ntfs. - In install KUbuntu 11.10 with live usb created with unetbootin. Linux root partition (/) formatted to ext4, and 1,5GB to swap. The others are placed in /mnt/windows (WinXP, ntfs) and /mnt/documentos (Data partition, ntfs) without formatting. - In Kubuntu, i tested the tar.gz file and both have again the problem. For test, i formatted with KDE Partition Manager, inside Kubuntu of course, the Data partition (mnt/documentos) again to ntfs. It still has the problem after that. Both partitions (and both computers) have the behaviour when i paste a file to these partitions, dolphin throws me a message "Can't change permissions of *file*". I will test other situations, creating other ntfs partition how you said, using debug symbols, and creating all partitions from linux directly, or other tests you can say me.
(In reply to comment #9) > * Can you test an NTFS partition the same way I did: > $ mkdir /tmp/ntfs > $ dd if=/dev/zero of=/tmp/ntfs.img bs=1M count=100 > $ mkfs.ntfs -f -F /tmp/ntfs.img > $ sudo mount /tmp/ntfs.img /tmp/ntfs -t ntfs -o loop > $ cd /tmp/ntfs > $ # extract the archives here > Does it also produce bad results? I test this and don't produce bad results. Any other test? (In reply to comment #13) > Please make sure you have installed any Ark debug package Kubuntu might > have, then run "kdebugdialog" and check if Ark's checkbox(es) is/are > checked. Doing so should make Ark produce more debugging output on the > terminal. I am using kdebugdialog, but i don't have any different messages with ark in the differents partitions (and no differences when i have the problem or not). Only have some messages with kate when it tries to read the rare characters and don't recognize the format. kdebugdialog for "ark -bea testfiles.tar.gz" (In all partitions, it's the same): ark(2223)/kdecore (KSycoca) KSycocaPrivate::openDatabase: Trying to open ksycoca from "/var/tmp/kdecache-thywalls/ksycoca4" ark(2223)/kdecore (trader) KMimeTypeTrader::query: query for mimeType "application/x-compressed-tar" , "Kerfuffle/Plugin" : returning 1 offers
Sorry, kdebugdialog aren't proper configurated, in the list are two "Ark" and only was selected one. Yes, there are errors in the messages: ark(4429)/kdecore (KSycoca) KSycocaPrivate::openDatabase: Trying to open ksycoca from "/var/tmp/kdecache-thywalls/ksycoca4" ark(4429)/kdecore (trader) KMimeTypeTrader::query: query for mimeType "application/x-compressed-tar" , "Kerfuffle/Plugin" : returning 1 offers ark(4429) Kerfuffle::factory: Loading plugin "kerfuffle_libarchive" ark(4429) Kerfuffle::ReadOnlyArchiveInterface::ReadOnlyArchiveInterface: ark(4429) main: Setting autosubfolder ark(4429) main: By autodestination, setting path to "/mnt/documentos" ark(4429) main: Entering application loop ark(4429) LibArchiveInterface::list: ark(4429) Kerfuffle::Job::onFinished: true ark(4429) BatchExtract::addExtraction: "Registering job from archive /mnt/documentos/testfiles.tar.gz, to /mnt/documentos, preservePaths 1" ark(4429) KSharedUiServerProxy::KSharedUiServerProxy: kuiserver registered ark(4429) BatchExtract::slotStartJob: Starting first job ark(4429) Kerfuffle::ExtractJob::doWork: Starting extraction with selected files: () Destination dir: "/mnt/documentos" Options: QHash(("PreservePaths", QVariant(bool, true) ) ) ark(4429) LibArchiveInterface::copyFiles: Changing current directory to "/mnt/documentos" ark(4429) LibArchiveInterface::copyFiles: Set root node "" ark(4429) LibArchiveInterface::copyFiles: Writing "" to testfiles/ ark(4429) LibArchiveInterface::copyFiles: Writing "testfile1.txt" to testfiles/testfile1.txt ark(4429) LibArchiveInterface::copyFiles: Writing "testfile2.txt" to testfiles/testfile2.txt ark(4429) LibArchiveInterface::copyData: Error while extracting... (Empty error message) (error nb = 1 ) ark(4429) LibArchiveInterface::copyFiles: Writing "testfile3.txt" to testfiles/testfile3.txt ark(4429) LibArchiveInterface::copyData: Error while extracting... (Empty error message) (error nb = 1 ) ark(4429) Kerfuffle::Job::onFinished: true ark(4429) BatchExtract::slotResult: ark(4429) BatchExtract::slotResult: Finished, emitting the result
New information: Testing some things about partitions, mounting... i found a behaviour. I have tested all with KDE Partition Manager first, and testing after with a more low level utils. When i start linux, i have the problem. But if with KDE Partition Manager, i umount and mount again the partitions, the problem is gone. When i umount/mount if i don't erase the files, they have the problem, but if i extract again the files they haven't. But when i restart the computer the problem is again here. In addition, when i restart, if i extracted the files ok before restart, the files are ok, but if i extract them again, they are wrong. Testing again, if i don't use KDE Partition Manager and use command line umount and mount (sudo umount /mnt/XXXX, sudo mount /mnt/XXXX) the problem is NOT gone. The partitions in fstab: # /mnt/documentos was on /dev/sda7 during installation UUID=D2B478B2B4789AA5 /mnt/documentos ntfs defaults,umask=007,gid=46 0 0 # /mnt/windows was on /dev/sda1 during installation UUID=448026AD8026A580 /mnt/windows ntfs defaults,umask=007,gid=46 0 0 The partitions in mtab after reboot /dev/sda7 /mnt/documentos fuseblk rw,nosuid,nodev,allow_other,blksize=4096,default_permissions 0 0 /dev/sda1 /mnt/windows fuseblk rw,nosuid,nodev,allow_other,blksize=4096,default_permissions 0 0 The partitions in mtab AFTER umount/mount with KDE Partition Manager: /dev/sda1 /mnt/windows fuseblk rw,nosuid,nodev,allow_other,blksize=4096 0 0 /dev/sda7 /mnt/documentos fuseblk rw,nosuid,nodev,allow_other,blksize=4096 0 0 I have pass a testing with KDE Partition Manager and this is the results: Comprobar y reparar la partición ‘/dev/sda1’ (14,00 GiB, ntfs) Trabajo: Comprobar sistema de archivos en la partición ‘/dev/sda1’ Orden: ntfsresize -P -i -f -v /dev/sda1 ntfsresize v2011.4.12AR.4 (libntfs-3g) Device name : /dev/sda1 NTFS volume version: 3.1 Cluster size : 4096 bytes Current volume size: 15035777536 bytes (15036 MB) Current device size: 15035779584 bytes (15036 MB) Checking for bad sectors ... ntfs_mst_post_read_fixup: magic: 0x00000000 size: 1024 usa_ofs: 0 usa_count: 65535: Invalid argument Record 16 has no FILE magic (0x0) ntfs_mst_post_read_fixup: magic: 0x00000000 size: 1024 usa_ofs: 0 usa_count: 65535: Invalid argument Record 17 has no FILE magic (0x0) ntfs_mst_post_read_fixup: magic: 0x00000000 size: 1024 usa_ofs: 0 usa_count: 65535: Invalid argument Record 18 has no FILE magic (0x0) ntfs_mst_post_read_fixup: magic: 0x00000000 size: 1024 usa_ofs: 0 usa_count: 65535: Invalid argument Record 19 has no FILE magic (0x0) ntfs_mst_post_read_fixup: magic: 0x00000000 size: 1024 usa_ofs: 0 usa_count: 65535: Invalid argument Record 20 has no FILE magic (0x0) ntfs_mst_post_read_fixup: magic: 0x00000000 size: 1024 usa_ofs: 0 usa_count: 65535: Invalid argument Record 21 has no FILE magic (0x0) ntfs_mst_post_read_fixup: magic: 0x00000000 size: 1024 usa_ofs: 0 usa_count: 65535: Invalid argument Record 22 has no FILE magic (0x0) ntfs_mst_post_read_fixup: magic: 0x00000000 size: 1024 usa_ofs: 0 usa_count: 65535: Invalid argument Record 23 has no FILE magic (0x0) ntfs_mst_post_read_fixup: magic: 0x00000000 size: 1024 usa_ofs: 0 usa_count: 65535: Invalid argument Record 16 has no FILE magic (0x0) ntfs_mst_post_read_fixup: magic: 0x00000000 size: 1024 usa_ofs: 0 usa_count: 65535: Invalid argument Record 17 has no FILE magic (0x0) ntfs_mst_post_read_fixup: magic: 0x00000000 size: 1024 usa_ofs: 0 usa_count: 65535: Invalid argument Record 18 has no FILE magic (0x0) ntfs_mst_post_read_fixup: magic: 0x00000000 size: 1024 usa_ofs: 0 usa_count: 65535: Invalid argument Record 19 has no FILE magic (0x0) ntfs_mst_post_read_fixup: magic: 0x00000000 size: 1024 usa_ofs: 0 usa_count: 65535: Invalid argument Record 20 has no FILE magic (0x0) ntfs_mst_post_read_fixup: magic: 0x00000000 size: 1024 usa_ofs: 0 usa_count: 65535: Invalid argument Record 21 has no FILE magic (0x0) ntfs_mst_post_read_fixup: magic: 0x00000000 size: 1024 usa_ofs: 0 usa_count: 65535: Invalid argument Record 22 has no FILE magic (0x0) ntfs_mst_post_read_fixup: magic: 0x00000000 size: 1024 usa_ofs: 0 usa_count: 65535: Invalid argument Record 23 has no FILE magic (0x0) Checking filesystem consistency ... Accounting clusters ... Space in use : 4284 MB (28.5%) Collecting resizing constraints ... Estimating smallest shrunken size supported ... File feature Last used at By inode $MFT : 3239 MB 0 Multi-Record : 7518 MB 11311 $MFTMirr : 7518 MB 1 Compressed : 7518 MB 13916 Sparse : 5268 MB 16771 Ordinary : 7519 MB 10 You might resize at 4283523072 bytes or 4284 MB (freeing 10752 MB). Please make a test run using both the -n and -s options before real resizing! Comprobar sistema de archivos en la partición ‘/dev/sda1’: Correcto Trabajo: Maximizar el sistema de archivos en ‘/dev/sda1’ para rellenar la partición El sistema de archivos en la partición ‘/dev/sda1’ ya tiene la longitud pedida de 29366757 sectores. Maximizar el sistema de archivos en ‘/dev/sda1’ para rellenar la partición: Correcto Comprobar y reparar la partición ‘/dev/sda1’ (14,00 GiB, ntfs): Correcto
With this last test, the problem wasn't directly a problem with Ark, but it's one of few programs (or only one) affected. Ark have some problem with recreating tar estructure in these ntfs file system for some reason. I don't know, the problem is rather annoying and strange...
Hi Lukas. Do you still have this problem? Have you tried with a more recent version of Ark and, more importantly, of KDE Partition Manager?
Closing for now... Please reopen if can answer my questions above.
Dear Bug Submitter, This bug has been in NEEDSINFO status with no change for at least 15 days. Please provide the requested information as soon as possible and set the bug status as REPORTED. Due to regular bug tracker maintenance, if the bug is still in NEEDSINFO status with no change in 30 days, the bug will be closed as RESOLVED > WORKSFORME due to lack of needed information. For more information about our bug triaging procedures please read the wiki located here: https://community.kde.org/Guidelines_and_HOWTOs/Bug_triaging If you have already provided the requested information, please set the bug status as REPORTED so that the KDE team knows that the bug is ready to be confirmed. Thank you for helping us make KDE software even better for everyone!
Dear Bug Submitter, This bug has been in NEEDSINFO status with no change for at least 30 days. The bug is now closed as RESOLVED > WORKSFORME due to lack of needed information. For more information about our bug triaging procedures please read the wiki located here: https://community.kde.org/Guidelines_and_HOWTOs/Bug_triaging Thank you for helping us make KDE software even better for everyone!