Bug 405886 - opening/comparing large text files is slow (80MB)
Summary: opening/comparing large text files is slow (80MB)
Status: CONFIRMED
Alias: None
Product: kdiff3
Classification: Applications
Component: application (show other bugs)
Version: unspecified
Platform: Compiled Sources Linux
: NOR wishlist
Target Milestone: ---
Assignee: michael
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2019-03-26 11:34 UTC by Josef
Modified: 2023-12-08 09:07 UTC (History)
1 user (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Josef 2019-03-26 11:34:42 UTC
SUMMARY
opening/comparing large text files is slow (~80 MB) - more then 2 min. Older version kdiff3 0.9.98, Qt4 takes about 20 sec.

line example: 'INFO  /      145677      -855.988888    -888.457853      23.058455'


STEPS TO REPRODUCE
1. Text file with 1327000 lines, line not same
2. 
3. 

OBSERVED RESULT
> 2 min

EXPECTED RESULT
 < 20 sec

SOFTWARE/OS VERSIONS
Windows: 
macOS: 
Linux/KDE Plasma: 
(available in About System)
KDE Plasma Version:  5.15.5
KDE Frameworks Version: 
Qt Version: 5.11.2

ADDITIONAL INFORMATION
Comment 1 michael 2019-05-12 17:32:17 UTC
Working the speed issues now. This will require some major changes to the file comparison routine. Currently kdiff3 runs three loops through the entire file and copies three complete copies in memory. The changes will be master only do to the extant of the work required.
Comment 2 michael 2020-01-15 00:21:05 UTC
The processing loops are now combined into one on master. That should help a lot with speed. The triple load of the data is still an issue however.
Comment 3 Dan 2023-12-08 09:07:50 UTC
As for the version kdiff3 1.10.5 (64 bit), the comparison of large files is still far too slow.
Whak diff can make in seconds, kdiff makes in minutes (or more).

I understand that it has low priority (kdiff3 works nicely after all), but improvement in this area would be highly appreciated.

Thank you very much.