Created attachment 92882 [details]
Corrupt files when extracted with Ark 2.19, but OK using unzip 6.00 or other zip tool

I cannot reproduce this with current frameworks branch and unzip 6.00.

I checked the md5sums of the files extracted with Ark using clizip-plugin and unzip, respectively, and found no differences.

Created attachment 92887 [details]
The corrupted files extracted by Ark 2.19

That's weird. Here are the corrupt files extracted by Ark 2.19 on my system.

Is it possible because of difference of the underlying libraries? Ark doesn't seem to use clizip-plugin or similar, it uses libarchive13 v3.1.2-7ubuntu2.1.

Or perhaps kernel or my processor?

ceefour@netadm:~ > aptitude show ark
Package: ark                             
State: installed
Automatically installed: no
Version: 4:4.14.2-0ubuntu1~ubuntu14.04~ppa1                                                                                   
Priority: optional                                                                                                            
Section: utils                                                                                                                
Maintainer: Kubuntu Developers <kubuntu-devel@lists.ubuntu.com>
Architecture: amd64
Uncompressed Size: 1.308 k
Depends: libarchive13, libc6 (>= 2.14), libkdecore5, libkdeui5, libkfile4, libkhtml5, libkio5, libkonq5abi1, libkparts4,
         libkpty4, libqt4-dbus (>= 4:4.5.3), libqtcore4 (>= 4:4.7.0~beta1), libqtgui4 (>= 4:4.5.3), libstdc++6 (>= 4.1.1)
Recommends: bzip2, p7zip-full, unzip, zip
Suggests: rar, unrar | unrar-free
Conflicts: ark
Description: archive utility
 Ark manages various archive formats, including tar, gzip, bzip2, rar and zip, as well as CD-ROM images.  Ark can be used to
 browse, extract, create, and modify archives. 
 
 This package is part of the KDE SC utilities module.
Homepage: http://www.kde.org/

ceefour@netadm:~ > aptitude show libarchive13
Package: libarchive13                    
State: installed
Automatically installed: no
Multi-Arch: same
Version: 3.1.2-7ubuntu2.1
Priority: optional
Section: libs
Maintainer: Ubuntu Developers <ubuntu-devel-discuss@lists.ubuntu.com>
Architecture: amd64
Uncompressed Size: 710 k
Depends: libattr1 (>= 1:2.4.46-8), libbz2-1.0, libc6 (>= 2.16), liblzma5 (>= 5.1.1alpha+20120614), liblzo2-2, libnettle4 (>=
         2.3), libxml2 (>= 2.7.4), zlib1g (>= 1:1.1.4)
PreDepends: multiarch-support
Suggests: lrzip
Breaks: libarchive13 (!= 3.1.2-7ubuntu2.1)
Replaces: libarchive13 (< 3.1.2-7ubuntu2.1)
Description: Multi-format archive and compression library (shared library)
 The libarchive library provides a flexible interface for reading and writing archives in various formats such as tar and
 cpio. libarchive also supports reading and writing archives compressed using various compression filters such as gzip and
 bzip2. The library is inherently stream-oriented; readers serially iterate through the archive, writers serially add things
 to the archive. 
 
 Archive formats supported are: 
 
 * tar (read and write, including GNU extensions) 
 * pax (read and write, including GNU and star extensions) 
 * cpio (read and write, including odc and newc variants) 
 * iso9660 (read and write, including Joliet and Rockridge extensions, with some limitations) 
 * zip (read only, with some limitations, uses zlib) 
 * mtree (read and write) 
 * shar (write only) 
 * ar (read and write, including BSD and GNU/SysV variants) 
 * empty (read only; in particular, note that no other format will accept an empty file) 
 * raw (read only) 
 * xar (read only) 
 * rar (read only, with some limitations) 
 * 7zip (read and write, with some limitations) 
   
 Filters supported are: 
 
 * gzip (read and write, uses zlib) 
 * bzip2 (read and write, uses bzlib) 
 * compress (read and write, uses an internal implementation) 
 * uudecode (read only) 
 * separate command-line compressors with fixed-signature auto-detection 
 * xz and lzma (read and write using liblzma) 
   
 This package provides the libarchive shared library.
Homepage: http://www.libarchive.org/

ceefour@netadm:~ > uname -a
Linux netadm.dev 3.13.0-37-generic #64-Ubuntu SMP Mon Sep 22 21:28:38 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux

ceefour@netadm:~ > cat /proc/cpuinfo
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 42
model name      : Intel(R) Core(TM) i5-2430M CPU @ 2.40GHz
stepping        : 7
microcode       : 0x17
cpu MHz         : 1800.000
cache size      : 3072 KB
physical id     : 0
siblings        : 4
core id         : 0
cpu cores       : 2
apicid          : 0
initial apicid  : 0
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx lahf_lm ida arat epb xsaveopt pln pts dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips        : 4789.14
clflush size    : 64
cache_alignment : 64
address sizes   : 36 bits physical, 48 bits virtual
power management:

Ark should prefer the clizip plugin over the libarchive plugin when handling zip archives. Can run Ark from console, extract your zip file, and attach the output here.

I'm not sure if this output is useful:

ceefour@netadm:/data/Downloads > ark Photos_downloaded_by_AirDroid_corrupted.zip 
Bus::open: Can not get ibus-daemon's address. 
IBusInputContext::createInputContext: no connection to ibus-daemon 
Object::connect: No such signal org::freedesktop::UPower::DeviceAdded(QDBusObjectPath)
Object::connect: No such signal org::freedesktop::UPower::DeviceRemoved(QDBusObjectPath)
ark(527)/kdeui (kdelibs) KXMLGUIClient::~KXMLGUIClient: 0x17a6808 deleted without having been removed from the factory first. This will leak standalone popupmenus and could lead to crashes. 

let me know if there is more I need to do.

I'm not sure if I'm using clizip or libarchive.. how do I know?
and should I file this bug at libarchive ? please advise.

No, it doesn't contain useful info. Can you run kdebugdialog and make sure all entries containing "ark" or "kerfuffle" are checked, and then redo the steps mentioned in Comment 4? Make sure you extract the archive before copying the output...

Here's the ark console with kdebugdialog enabled for "ark" (however I can't find kerfuffle):

ceefour@netadm:/data/Downloads > ark Photos_downloaded_by_AirDroid_corrupted.zip    
ark(5629)/kdecore (kdelibs) KTempDir::create: KTempDir: Temporary directory created : "/tmp/kde-ceefour/arkziTYdd/"
ark(5629)/kdecore (KSycoca) KSycocaPrivate::openDatabase: Trying to open ksycoca from "/var/tmp/kdecache-ceefour/ksycoca4"
ark(5629)/kdecore (trader) KMimeTypeTrader::query: query for mimeType  "application/zip" ,  "Kerfuffle/Plugin"  : returning  1  offers
ark(5629)/kdecore (K*TimeZone*) KSystemTimeZonesPrivate::instance: instance(): ... initialised
ark(5629)/kdecore (K*TimeZone*) KSystemTimeZonesPrivate::readConfig: readConfig(): local zone= "Asia/Jakarta"
ark(5629)/kdecore (K*TimeZone*) KSystemTimeZonesPrivate::readZoneTab: readZoneTab( "/usr/share/zoneinfo/zone.tab" )
Bus::open: Can not get ibus-daemon's address. 
IBusInputContext::createInputContext: no connection to ibus-daemon 
ark(5629)/kio (bookmarks) KBookmarkManager::KBookmarkManager: starting KDirWatch for  "/home/ceefour/.local/share/user-places.xbel"
Object::connect: No such signal org::freedesktop::UPower::DeviceAdded(QDBusObjectPath)
Object::connect: No such signal org::freedesktop::UPower::DeviceRemoved(QDBusObjectPath)
ark(5629)/kio (KDirListerCache) KDirListerCache::listDir: Listing directory: KUrl("trash:/")
ark(5629)/kio (Scheduler) KIO::SchedulerPrivate::doJob: KIO::SimpleJob(0x1610340)
ark(5629)/kio (Scheduler) KIO::SchedulerPrivate::protoQ: creating ProtoQueue instance for "trash"
ark(5629)/kio (Scheduler) KIO::ProtoQueue::ProtoQueue: m_maxConnectionsTotal: 2 m_maxConnectionsPerHost: 2
ark(5629)/kio (KDirListerCache) KDirListerCache::listDir: Listing directory: KUrl("file:///")
ark(5629)/kio (Scheduler) KIO::SchedulerPrivate::doJob: KIO::SimpleJob(0x16057a0)
ark(5629)/kio (Scheduler) KIO::SchedulerPrivate::protoQ: creating ProtoQueue instance for "file"
ark(5629)/kio (Scheduler) KIO::ProtoQueue::ProtoQueue: m_maxConnectionsTotal: 5 m_maxConnectionsPerHost: 5
ark(5629)/kio (KDirListerCache) KDirListerCache::stopListingUrl: KDirLister(0x1614ff0)  url= KUrl("file:///")
ark(5629)/kio (Scheduler) KIO::SchedulerPrivate::cancelJob: KIO::ListJob(0x16057a0) QObject(0x0)
ark(5629)/kio (Scheduler) KIO::SchedulerPrivate::jobFinished: KIO::ListJob(0x16057a0) QObject(0x0)
ark(5629)/kio (KDirListerCache) KDirListerCache::slotResult: finished listing KUrl("file:///")
ark(5629)/kio (KDirListerCache) KDirListerCache::listDir: Listing directory: KUrl("file:///")
ark(5629)/kio (Scheduler) KIO::SchedulerPrivate::doJob: KIO::SimpleJob(0x161f510)
ark(5629)/kio (KIOJob) KDirModel::indexForUrl: KUrl("file:///data/Downloads") not found
ark(5629)/kio (KDirModel) KDirModel::expandToUrl: Remembering to emit expand after listing the root url
ark(5629)/kio (KIOJob) KDirModel::indexForUrl: KUrl("file:///data/Downloads") not found
ark(5629)/kio (KDirModel) KDirModel::expandToUrl: Remembering to emit expand after listing the root url
ark(5629)/kio (KDirListerCache) KDirListerCache::listDir: Entry currently being listed: KUrl("trash:/") by (KDirLister(0x13ce500) )
ark(5629)/kio (KDirListerCache) KDirListerCache::stopListingUrl: KDirLister(0x1399fc0)  url= KUrl("trash:/")
ark(5629)/kio (Slave) KIO::Slave::createSlave: createSlave "trash" for KUrl("trash:/")
ark(5629)/kio (KIOConnection) KIO::ConnectionServer::listenForRemote: Listening on  "local:/tmp/ksocket-ceefour/arkPE5629.slave-socket"
ark(5629)/kio (Slave) KIO::Slave::createSlave: createSlave "file" for KUrl("file:///")
ark(5629)/kio (KIOConnection) KIO::ConnectionServer::listenForRemote: Listening on  "local:/tmp/ksocket-ceefour/arkbZ5629.slave-socket"
ark(5629)/kio (KDirModel) KDirModelPrivate::_k_slotNewItems: Listing found KUrl("file:///data") which is a parent of fetched url KUrl("file:///data/Downloads")
ark(5629)/kio (KDirListerCache) KDirListerCache::listDir: Listing directory: KUrl("file:///data")
ark(5629)/kio (Scheduler) KIO::SchedulerPrivate::doJob: KIO::SimpleJob(0x1737320)
ark(5629)/kio (KDirModel) KDirModelPrivate::_k_slotNewItems: Listing found KUrl("file:///data") which is a parent of fetched url KUrl("file:///data/Downloads")
ark(5629)/kio KNFSShare::KNFSSharePrivate::findExportsFile: Could not find exports file! /etc/exports doesn't exist. Configure it in share/config/knfsshare, [General], exportsFile=....
ark(5629)/kio (Slave) KIO::Slave::createSlave: createSlave "file" for KUrl("file:///data")
ark(5629)/kio (KIOConnection) KIO::ConnectionServer::listenForRemote: Listening on  "local:/tmp/ksocket-ceefour/arktQ5629.slave-socket"
ark(5629)/kio (Scheduler) KIO::SchedulerPrivate::jobFinished: KIO::ListJob(0x161f510) KIO::Slave(0x17554d0)
ark(5629)/kio (KDirListerCache) KDirListerCache::slotResult: finished listing KUrl("file:///")
ark(5629)/kio (KDirModel) KDirModelPrivate::_k_slotNewItems: Listing found KUrl("file:///data/Downloads") which is a parent of fetched url KUrl("file:///data/Downloads")
ark(5629)/kio (KDirModel) KDirModelPrivate::_k_slotNewItems: Listing found KUrl("file:///data/Downloads") which is a parent of fetched url KUrl("file:///data/Downloads")
ark(5629)/kio (Scheduler) KIO::SchedulerPrivate::jobFinished: KIO::ListJob(0x1737320) KIO::Slave(0x1755310)
ark(5629)/kio (KDirListerCache) KDirListerCache::slotResult: finished listing KUrl("file:///data")
ark(5629)/kio (Scheduler) KIO::SchedulerPrivate::jobFinished: KIO::ListJob(0x1610340) KIO::Slave(0x165b6a0)
ark(5629)/kio (KDirListerCache) KDirListerCache::slotResult: finished listing KUrl("trash:/")
ark(5629)/kio (Scheduler) KIO::SchedulerPrivate::doJob: KIO::SimpleJob(0x1724700)
ark(5629)/kio (Scheduler) KIO::SchedulerPrivate::jobFinished: KIO::StatJob(0x1724700) KIO::Slave(0x1755310)
ark(5629)/kio (Scheduler) KIO::SchedulerPrivate::doJob: KIO::SimpleJob(0x1708230)
ark(5629)/kio (Scheduler) KIO::SchedulerPrivate::jobFinished: KIO::MkdirJob(0x1708230) KIO::Slave(0x1755310)
ark(5629)/kio (Scheduler) KIO::SchedulerPrivate::doJob: KIO::SimpleJob(0x16866e0)
ark(5629)/kio (KDirListerCache) KDirListerCache::slotFilesAdded: KUrl("file:///data/Downloads")
ark(5629)/kio (KDirListerCache) KDirListerCache::updateDirectory: KUrl("file:///data/Downloads")
ark(5629)/kio (Scheduler) KIO::SchedulerPrivate::jobFinished: KIO::StatJob(0x16866e0) KIO::Slave(0x1755310)
ark(5629)/kio (Scheduler) KIO::SchedulerPrivate::doJob: KIO::SimpleJob(0x1713210)
ark(5629)/kio (Scheduler) KIO::SchedulerPrivate::jobFinished: KIO::StatJob(0x1713210) KIO::Slave(0x1755310)
ark(5629)/kdecore (KConfigSkeleton) KCoreConfigSkeleton::writeConfig:
ark(5629)/kio (Scheduler) KIO::SchedulerPrivate::doJob: KIO::SimpleJob(0x17bea00)
ark(5629)/kio (Scheduler) KIO::SchedulerPrivate::jobFinished: KIO::StatJob(0x17bea00) KIO::Slave(0x1755310)
ark(5629)/kio (KDirListerCache) KDirListerCache::stopListingUrl: KDirLister(0x13ce500)  url= KUrl("trash:/")
ark(5629)/kio (KDirListerCache) KDirListerCache::forgetDirs: KDirLister(0x13ce500) item moved into cache: KUrl("trash:/")
ark(5629)/kio (KDirListerCache) KDirListerCache::stopListingUrl: KDirLister(0x1614ff0)  url= KUrl("file:///")
ark(5629)/kio (KDirListerCache) KDirListerCache::stopListingUrl: KDirLister(0x1614ff0)  url= KUrl("file:///data")
ark(5629)/kio (KDirListerCache) KDirListerCache::forgetDirs: KDirLister(0x1614ff0) item moved into cache: KUrl("file:///")
ark(5629)/kio (KDirListerCache) KDirListerCache::forgetDirs: KDirLister(0x1614ff0) item moved into cache: KUrl("file:///data")
ark(5629)/kio (KDirListerCache) KDirListerCache::processPendingUpdates: "/data/Downloads"
ark(5629)/kdecore (KConfigSkeleton) KCoreConfigSkeleton::writeConfig:
ark(5629)/kparts KParts::Part::~Part: deleting widget QSplitter(0x1011200) ""
ark(5629)/kdeui (kdelibs) KXMLGUIClient::~KXMLGUIClient: 0x1276768 deleted without having been removed from the factory first. This will leak standalone popupmenus and could lead to crashes. 
ark(5629)/kio (Slave) KIO::Slave::kill: killing slave pid 5633 ( "trash://" )
ark(5629)/kio (Slave) KIO::Slave::kill: killing slave pid 5639 ( "file://" )
ark(5629)/kio (Slave) KIO::Slave::kill: killing slave pid 5634 ( "file://" )

I can see that only one plugin is returned for application/zip mimetype. I checked for Ark 4.14 and only the clizip plugin is registered to handle zip archives. This means that the clizip plugin is always used for zip archives. This plugin uses the unzip binary for extraction, which should be similar to extracting manually using unzip. I can't see why you would get corrupted files.

Is there anything to do with the zip's encoding?

The zip file is from AirDroid and it's using GBK encoding instead of UTF-8.

However it does extract fine with plain 'unzip' or even 'unzip -a'. Can I know the exact command line executed/given by Ark/clizip ?

When unzipping it simply runs: unzip <archive_name>

I'm pretty sure it's not running neither 'unzip' nor 'zip' at all.

I've replaced /usr/bin/unzip and /usr/bin/zip with my own script:

#!/bin/bash
echo "$@"

and Ark is not running that script, Ark can still extract these files in corrupt way.

Hmm, strange. Can you check which kerfuffle_*.desktop files are present in /usr/share/kde4/services/ and maybe attach these files if present:

kerfuffle_clizip.desktop
kerfuffle_libarchive.desktop
kerfuffle_libarchive_readonly.desktop

Created attachment 92902 [details]
kerfuffle files

these are the files (all attached):

ceefour@netadm:/data/Downloads > ll /usr/share/kde4/services/kerf*
-rw-r--r-- 1 root root 2593 Oct 15  2014 /usr/share/kde4/services/kerfuffle_cli7z.desktop
-rw-r--r-- 1 root root 2271 Oct 15  2014 /usr/share/kde4/services/kerfuffle_clilha.desktop
-rw-r--r-- 1 root root 2545 Oct 15  2014 /usr/share/kde4/services/kerfuffle_clirar.desktop
-rw-r--r-- 1 root root 2566 Oct 15  2014 /usr/share/kde4/services/kerfuffle_clizip.desktop
-rw-r--r-- 1 root root 5685 Oct 15  2014 /usr/share/kde4/services/kerfuffle_libarchive.desktop
-rw-r--r-- 1 root root 2843 Oct 15  2014 /usr/share/kde4/services/kerfuffle_libarchive_readonly.desktop
-rw-r--r-- 1 root root 4662 Oct  3  2014 /usr/share/kde4/services/kerfuffle_libbz2.desktop
-rw-r--r-- 1 root root 4676 Oct  3  2014 /usr/share/kde4/services/kerfuffle_libgz.desktop
-rw-r--r-- 1 root root 4579 Oct  3  2014 /usr/share/kde4/services/kerfuffle_libxz.desktop

Aha. The desktop files revealed that libarchive is used for zip archives. This must be a modification that Mint/Ubuntu/Debian does to Ark, probably to remove dependency on unzip. If you add the "application/zip" mimetype to kerfuffle_clizip.desktop, Ark should use unzip/zip for zip archives.

This can be done by adding: "application/zip;" to the last line in the file (e.g. the MimeType field). Run "kbuildsycoca4 --noincremental" after editing the file and try extracting the archive again.

Wow! That works!

So I have to report this bug to libarchive then ?

Well, at least extraction of this zip archive is broken with the libarchive plugin. The error could be in libarchive itself or in Ark's usage of libarchive. It would be good to test extracting the archive with some other application that uses libarchive before reporting a bug in libarchive.

Yup, this is likely to be a libarchive issue. I used bsdcpio:

ceefour@netadm:/data/Downloads/usebsd > bsdcpio -v -i < ../Photos_downloaded_by_AirDroid_corrupted.zip 
Screenshot_2015-05-22-15-57-08.jpg
Screenshot_2015-05-22-15-56-47.jpg
Screenshot_2015-05-22-13-47-02.jpg
Screenshot_2015-05-16-22-10-46.jpg
Screenshot_2015-05-22-15-52-23.jpg
4472 blocks

and get the same corrupted files.

Now should I report to libarchive upstream or libarchive Launchpad.. Or probably both..?

I've reported it to libarchive Launchpad here: https://bugs.launchpad.net/ubuntu/+source/libarchive/+bug/1460038

Great that you confirmed it was a bug in libarchive. It means Ark's libarchive implementation is not at fault. I would probably report it at libarchive upstream, since it's unlikely the issue is due to some modifications done to libarchive by Mint/Ubuntu/Debian.

I found out that Debian applies a patch to Ark that forces it to use libarchive for zip archives (https://packages.debian.org/jessie/ark). You could also file a bug for Ark at Debian and explain the issue.

The issue was in libarchive. Debian modifies Ark to use the libarchive-plugin for zip archives instead of the recommended cli-plugin.

*** Bug 348944 has been marked as a duplicate of this bug. ***