Bug 436919 - Cannot compress large directory as 7zip
Summary: Cannot compress large directory as 7zip
Status: RESOLVED WORKSFORME
Alias: None
Product: ark
Classification: Applications
Component: general (show other bugs)
Version: 21.04.0
Platform: Neon Linux
: NOR normal
Target Milestone: ---
Assignee: Elvis Angelaccio
URL:
Keywords:
: 447935 (view as bug list)
Depends on:
Blocks:
 
Reported: 2021-05-11 09:56 UTC by Michał Zubkowicz
Modified: 2024-05-29 03:45 UTC (History)
6 users (show)

See Also:
Latest Commit:
Version Fixed In:
Sentry Crash Report:


Attachments
Notification about finished compression and broken compressed file (193.12 KB, image/png)
2021-06-23 11:23 UTC, Michał Zubkowicz
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Michał Zubkowicz 2021-05-11 09:56:45 UTC
SUMMARY
When compressing directory with many files (> 2 000 000) ark crashes silently and shows only success message. I'm not sure it is compressing issue or something different, like file listing because there is no any message. 
Compressing as tar.gz works fine. Using command line 7z also works fine 

STEPS TO REPRODUCE
1. Try to compress large directory as 7 zip (you can try clone some node repositories from github and clone it). 
2. It fails without message

OBSERVED RESULT
No file is compressed

EXPECTED RESULT
Directory and all files should be compressed. 

SOFTWARE/OS VERSIONS
Windows: 
macOS: 
Linux/KDE Plasma: 
(available in About System)
KDE Plasma Version: 
KDE Frameworks Version: 
Qt Version: 

ADDITIONAL INFORMATION
Comment 1 Justin Zobel 2021-06-21 03:12:40 UTC
Thanks for the report, can you provide a sample repo that we can clone for testing?
Comment 2 Michał Zubkowicz 2021-06-23 11:23:30 UTC
Created attachment 139611 [details]
Notification about finished compression and broken compressed file
Comment 3 Michał Zubkowicz 2021-06-23 11:26:03 UTC
Just create many files like this:
dd if=/dev/zero of=masterfile bs=1 count=1000000
split -b 10 -a 10 masterfile

Next right click on directory with these files, select "compress to" and choose 7zip. 

In attachment you can see broken 7z file and finished compression message.

Worst thing is that compression errors are not showed to user. Compression is used for archiving old data purposes, and someone can not know that have broken archive.
Comment 4 Bug Janitor Service 2021-07-08 04:33:48 UTC
Dear Bug Submitter,

This bug has been in NEEDSINFO status with no change for at least
15 days. Please provide the requested information as soon as
possible and set the bug status as REPORTED. Due to regular bug
tracker maintenance, if the bug is still in NEEDSINFO status with
no change in 30 days the bug will be closed as RESOLVED > WORKSFORME
due to lack of needed information.

For more information about our bug triaging procedures please read the
wiki located here:
https://community.kde.org/Guidelines_and_HOWTOs/Bug_triaging

If you have already provided the requested information, please
mark the bug as REPORTED so that the KDE team knows that the bug is
ready to be confirmed.

Thank you for helping us make KDE software even better for everyone!
Comment 5 Max 2022-02-20 11:59:07 UTC
Can't reproduce this with ark-21.12. Although opening such archives with ark takes much more time than listing/extracting archive with 7z. Probably because of slow parsing 7z output.
Comment 6 Max 2022-02-21 12:48:07 UTC
*** Bug 447935 has been marked as a duplicate of this bug. ***
Comment 7 Pedro V 2024-04-29 13:37:10 UTC
Was the supposed reproducer method in comment 3 actually work to recreate the problem, or was it just assumed to be good enough?

Suspected the "> 2 000 000" condition as a potential time waster, not recommending targeting that, but for the sake of completeness I went for it.
Optimally there would be multiple directories used for best performance, but went for just 2 in tmpfs for simplicity, running the following in /tmp/test/test2/:
seq 0 2000000 | xargs -I% -P $(nproc) touch %

"Double bagged" the files as I was using Krusader for compressing, and it would just "freeze" when right clicking on the directory containing the ton of files, and figured it's easier to just add an extra layer instead of waiting for not sure how long.

Neither Krusader nor Ark will confirm in reasonable time whether the package is any good, but `7z l test.7z | wc -l` shows 2000024 which is close enough to what's expected without getting into the details of cutting out extra verbosity.

Generally I feel like this will be a duplicate of Bug #453452 , not having to do anything with the count of files.
If that's the case, then suspected issues are:
- 7z being rather unfriendly, not handling all files properly, symbolic links being one common example
- Ark swallowing the errors of the 7z tool instead of properly propagating them which is the more serious problem as there's a fake success
Comment 8 Bug Janitor Service 2024-05-14 03:45:21 UTC
Dear Bug Submitter,

This bug has been in NEEDSINFO status with no change for at least
15 days. Please provide the requested information as soon as
possible and set the bug status as REPORTED. Due to regular bug
tracker maintenance, if the bug is still in NEEDSINFO status with
no change in 30 days the bug will be closed as RESOLVED > WORKSFORME
due to lack of needed information.

For more information about our bug triaging procedures please read the
wiki located here:
https://community.kde.org/Guidelines_and_HOWTOs/Bug_triaging

If you have already provided the requested information, please
mark the bug as REPORTED so that the KDE team knows that the bug is
ready to be confirmed.

Thank you for helping us make KDE software even better for everyone!
Comment 9 Bug Janitor Service 2024-05-29 03:45:38 UTC
This bug has been in NEEDSINFO status with no change for at least
30 days. The bug is now closed as RESOLVED > WORKSFORME
due to lack of needed information.

For more information about our bug triaging procedures please read the
wiki located here:
https://community.kde.org/Guidelines_and_HOWTOs/Bug_triaging

Thank you for helping us make KDE software even better for everyone!