Version: 0.9.88 (using KDE KDE 3.5.1) Installed from: Gentoo Packages Compiler: gcc version 3.4.4 (Gentoo 3.4.4-r1, ssp-3.4.4-1.0, pie-8.7.8) OS: Linux trying to compare two dvd iso images (of 2.8 GiB size) leads to a crash right after clicking "OK" after A and B are selected the commandline output: $ kdiff3 terminate called after throwing an instance of 'std::bad_alloc' what(): St9bad_alloc KCrash: Application 'kdiff3' crashing... the crash handler output: Using host libthread_db library "/lib/tls/libthread_db.so.1". [Thread debugging using libthread_db enabled] [New Thread 46912563325440 (LWP 11371)] 0x00002aaaae926262 in nanosleep () from /lib/tls/libc.so.6 #0 0x00002aaaae926262 in nanosleep () from /lib/tls/libc.so.6 #1 0x00002aaaae926100 in sleep () from /lib/tls/libc.so.6 #2 0x00002aaaab9f2bea in KCrash::startDrKonqi () from /usr/kde/3.5/lib64/libkdecore.so.4 #3 0x00002aaaaba1509e in KCrash::defaultCrashHandler () from /usr/kde/3.5/lib64/libkdecore.so.4 #4 0x00002aaaae8c8890 in killpg () from /lib/tls/libc.so.6 #5 0x0000000000000000 in ?? ()
Hi, You are trying to compare two huge files. "two dvd iso images (of 2.8 GiB size)" Because of the way KDiff3 works, KDiff3 tries to load them into memory for comparison and converts them to your codec, which means that twice or even thrice that memory is needed for each file. 2.8 GB * 2 * 3 = 17 GB is what you would need for this operation. No wonder that "std::bad_alloc" crashes. That's where the memory is allocated. KDiff3 should present a messagebox before exiting, giving the user an idea, what's going wrong, but you should prefer command-line diff or xdelta for these huge files. Cheers, Joachim
I have no success with diff either ... however, I have found that cmp is able to tell the positions of the differing bytes it is a pity that kdiff3 is unable to work without loading the whole files; I have noticed this flaw with more KDE applications (e.g. khexedit) ... but, in any case, some warning should be issued instead of crash (the application need not necessarily exit as you write) so, if this is a common case, please do something about it ;-) if you cannot reproduce this, tell me what do you need from me to trace the problem - but keep in mind that I am not programmer
This crash report is at least 4 years old and there were no further comments or status updates since then. Therefore we believe that this crash is already fixed in KDE 4 or the backtrace is no longer applicable to KDE 4. If the crash still happens with a recent KDE version (4.7.4 or 4.8), please add an updated backtrace or provide steps to reproduce. For more information, see http://techbase.kde.org/Development/Tutorials/Debugging/How_to_create_useful_crash_reports (To prevent automatic closing of this bug in the future, please set the bug status to ASSIGNED or NEW)
At least with version 4.6.5, the bug is still valid. I can't test a newer version right now
the same with 0.9.95 - KDE 4.7.3 kavol@kavol ~/virtual $ kdiff3 F13.qcow2 F13.qcow2 kdiff3(9624) KXMLGUIClient::setXMLFile: cannot find .rc file "kdiff3_part.rc" for component "kdiff3part" terminate called after throwing an instance of 'std::bad_alloc' what(): std::bad_alloc KCrash: Application 'kdiff3' crashing... KCrash: Attempting to start /usr/lib64/kde4/libexec/drkonqi from kdeinit sock_file=/home/kavol/.kde4/socket-kavol/kdeinit4_localhost_12 Warning: connect() failed: : Adresář nebo soubor neexistuje KCrash: Attempting to start /usr/lib64/kde4/libexec/drkonqi directly [1]+ Pozastavena kdiff3 F13.qcow2 F13.qcow2 Application: kdiff3 (kdiff3), signal: Aborted [KCrash Handler] #4 0x00007fd240442b05 in *__GI_raise (sig=<optimized out>) at ../nptl/sysdeps/unix/sysv/linux/raise.c:64 #5 0x00007fd240443f86 in *__GI_abort () at abort.c:92 #6 0x00007fd240adaadd in __gnu_cxx::__verbose_terminate_handler () at /var/tmp/portage/sys-devel/gcc-4.5.3-r1/work/gcc-4.5.3/libstdc++-v3/libsupc++/vterminate.cc:93 #7 0x00007fd240ad8d16 in __cxxabiv1::__terminate (handler=<optimized out>) at /var/tmp/portage/sys-devel/gcc-4.5.3-r1/work/gcc-4.5.3/libstdc++-v3/libsupc++/eh_terminate.cc:39 #8 0x00007fd240ad8d43 in std::terminate () at /var/tmp/portage/sys-devel/gcc-4.5.3-r1/work/gcc-4.5.3/libstdc++-v3/libsupc++/eh_terminate.cc:49 #9 0x00007fd240ad8e4e in __cxxabiv1::__cxa_throw (obj=<optimized out>, tinfo=<optimized out>, dest=<optimized out>) at /var/tmp/portage/sys-devel/gcc-4.5.3-r1/work/gcc-4.5.3/libstdc++-v3/libsupc++/eh_throw.cc:83 #10 0x00007fd240ad924d in operator new (sz=18446744072002142308) at /var/tmp/portage/sys-devel/gcc-4.5.3-r1/work/gcc-4.5.3/libstdc++-v3/libsupc++/new_op.cc:58 #11 0x00007fd240ad9309 in operator new[] (sz=<optimized out>) at /var/tmp/portage/sys-devel/gcc-4.5.3-r1/work/gcc-4.5.3/libstdc++-v3/libsupc++/new_opv.cc:32 #12 0x0000000000463eb0 in SourceData::FileData::readFile (this=0x13d5ae8, filename=...) at /var/tmp/portage/kde-misc/kdiff3-0.9.95-r1/work/kdiff3-0.9.95/src-QT4/diff.cpp:324 #13 0x000000000046655b in SourceData::readAndPreprocess (this=0x13d5a48, pEncoding=<optimized out>, bAutoDetectUnicode=<optimized out>) at /var/tmp/portage/kde-misc/kdiff3-0.9.95-r1/work/kdiff3-0.9.95/src-QT4/diff.cpp:476 #14 0x0000000000454d9f in KDiff3App::init (this=0x13d5770, bAuto=<optimized out>, pTotalDiffStatus=0x13d5a30, bLoadFiles=<optimized out>) at /var/tmp/portage/kde-misc/kdiff3-0.9.95-r1/work/kdiff3-0.9.95/src-QT4/pdiff.cpp:301 #15 0x000000000042db12 in KDiff3App::completeInit (this=0x13d5770, fn1=<optimized out>, fn2=..., fn3=...) at /var/tmp/portage/kde-misc/kdiff3-0.9.95-r1/work/kdiff3-0.9.95/src-QT4/kdiff3.cpp:376 #16 0x00000000004297d8 in KDiff3Shell::KDiff3Shell (this=0x130d7f0, bCompleteInit=<optimized out>, __in_chrg=<optimized out>, __vtt_parm=<optimized out>) at /var/tmp/portage/kde-misc/kdiff3-0.9.95-r1/work/kdiff3-0.9.95/src-QT4/kdiff3_shell.cpp:69 #17 0x0000000000428fc9 in main (argc=<optimized out>, argv=<optimized out>) at /var/tmp/portage/kde-misc/kdiff3-0.9.95-r1/work/kdiff3-0.9.95/src-QT4/main.cpp:243
(In reply to comment #1) > Hi, > You are trying to compare two huge files. > "two dvd iso images (of 2.8 GiB size)" > Because of the way KDiff3 works, KDiff3 tries to load them into memory for > comparison and converts them to your codec, which means that twice or even > thrice that memory is needed for each file. > 2.8 GB * 2 * 3 = 17 GB is what you would need for this operation. > No wonder that "std::bad_alloc" crashes. That's where the memory is > allocated. > KDiff3 should present a messagebox before exiting, giving the user an idea, > what's going wrong, but you should prefer command-line diff or xdelta for > these huge files. > > Cheers, > Joachim I would think the real solution would be to use a caching mechanism for very large files, so that we can handle this (plenty of other visual diff tools can). I might grab the source and give it a try one of these weekends. Do you forsee any other problems arising in the UI once the giant files are actually diffed?
*** Bug 335779 has been marked as a duplicate of this bug. ***
wow, this is getting attention after all those years ... :-) to answer the question in bug #335779, I really do not remember my original intent, but I guess I was trying to compare whether the difference is just in signatures or in real data (and I expected the output to look like any other binary file comparison) ... can't tell what was expected for the virtualbox images, aren't they compressed?
Never mind, just want to see red lines :). I don't actually know about compression, I just know that they can extend their size. But for backup it can be interesting to see tar files with added directory I suppose. Thu, 05 Jun 2014 12:18:22 +0000 от kavol <kavol@seznam.cz>: >https://bugs.kde.org/show_bug.cgi?id=123481 > >--- Comment #8 from kavol < kavol@seznam.cz > --- >wow, this is getting attention after all those years ... :-) > >to answer the question in bug #335779, I really do not remember my original >intent, but I guess I was trying to compare whether the difference is just in >signatures or in real data (and I expected the output to look like any other >binary file comparison) ... can't tell what was expected for the virtualbox >images, aren't they compressed? > >-- >You are receiving this mail because: >You are on the CC list for the bug.
I do encounter this same crash with diff'ing 2 large files csv files of sizes 1,158,506,947 and 1,158,448,716 bytes. The machine has 32GB RAM most of which is available. It looks like it reads files into memory using QString::append and that crashes on the resize. A better strategy may be to map the input files into memory using mmap. Here follows automatically generated crash report: Application: kdiff3 (0.9.98 (64 bit)) KDE Platform Version: 4.14.17 Qt Version: 4.8.7 Operating System: Linux 4.4.6-201.fc22.x86_64 x86_64 Distribution: "Fedora release 22 (Twenty Two)" -- Information about the crash: The crash can be reproduced every time. -- Backtrace: Application: kdiff3 (kdiff3), signal: Aborted Using host libthread_db library "/lib64/libthread_db.so.1". [Current thread is 1 (Thread 0x7f4af0cd08c0 (LWP 18243))] Thread 2 (Thread 0x7f4ad2b2a700 (LWP 18246)): #0 0x00007f4aec10e57d in poll () at /lib64/libc.so.6 #1 0x00007f4ae7fefdbc in g_main_context_iterate.isra () at /lib64/libglib-2.0.so.0 #2 0x00007f4ae7fefecc in g_main_context_iteration () at /lib64/libglib-2.0.so.0 #3 0x00007f4aed5d3e7e in QEventDispatcherGlib::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) () at /lib64/libQtCore.so.4 #4 0x00007f4aed5a2131 in QEventLoop::processEvents(QFlags<QEventLoop::ProcessEventsFlag>) () at /lib64/libQtCore.so.4 #5 0x00007f4aed5a24a5 in QEventLoop::exec(QFlags<QEventLoop::ProcessEventsFlag>) () at /lib64/libQtCore.so.4 #6 0x00007f4aed491a99 in QThread::exec() () at /lib64/libQtCore.so.4 #7 0x00007f4aed582203 in QInotifyFileSystemWatcherEngine::run() () at /lib64/libQtCore.so.4 #8 0x00007f4aed49438c in QThreadPrivate::start(void*) () at /lib64/libQtCore.so.4 #9 0x00007f4aed1f2555 in start_thread () at /lib64/libpthread.so.0 #10 0x00007f4aec119ded in clone () at /lib64/libc.so.6 Thread 1 (Thread 0x7f4af0cd08c0 (LWP 18243)): [KCrash Handler] #6 0x00007f4aec04b9c8 in raise () at /lib64/libc.so.6 #7 0x00007f4aec04d65a in abort () at /lib64/libc.so.6 #8 0x00007f4aec986b4d in __gnu_cxx::__verbose_terminate_handler() () at /lib64/libstdc++.so.6 #9 0x00007f4aec9849a6 in () at /lib64/libstdc++.so.6 #10 0x00007f4aec9849f1 in () at /lib64/libstdc++.so.6 #11 0x00007f4aec984c09 in () at /lib64/libstdc++.so.6 #12 0x00007f4aed489402 in () at /lib64/libQtCore.so.4 #13 0x00007f4aed4e423a in QString::realloc(int) () at /lib64/libQtCore.so.4 #14 0x00007f4aed4e7a13 in QString::append(QString const&) () at /lib64/libQtCore.so.4 #15 0x00007f4aed539462 in QTextStreamPrivate::fillReadBuffer(long long) () at /lib64/libQtCore.so.4 #16 0x00007f4aed539b1f in QTextStreamPrivate::read(int) () at /lib64/libQtCore.so.4 #17 0x00007f4aed53aad0 in QTextStream::readAll() () at /lib64/libQtCore.so.4 #18 0x0000000000462401 in SourceData::FileData::preprocess(bool, QTextCodec*) () #19 0x0000000000462ad1 in SourceData::readAndPreprocess(QTextCodec*, bool) () #20 0x000000000044e12f in KDiff3App::mainInit(TotalDiffStatus*, bool, bool) () #21 0x0000000000450db3 in KDiff3App::slotFileOpen() () #22 0x0000000000429f1d in KDiff3App::completeInit(QString const&, QString const&, QString const&) () #23 0x000000000042682d in KDiff3Shell::KDiff3Shell(bool) () #24 0x000000000042221b in main ()
Just looking more at this stack trace, particularly at QString::realloc(int). On one of the reallocations for my large files that int is going to overflow and become negative, which then gets converted to size_t, which becomes a huge positive number that gets passed to operator new, which throws std::bad_alloc.
Dear Bug Submitter, This bug has been stagnant for a long time. Could you help us out and re-test if the bug is valid in the latest version? I am setting the status to NEEDSINFO pending your response, please change the Status back to REPORTED when you respond. Thank you for helping us make KDE software even better for everyone!
trying a big file, I got: Errors occurred during pre-processing of file A. File ... too large to process. Skipping. then Errors occurred during pre-processing of file B. File ... too large to process. Skipping. then a crash: (gdb) bt full #0 0x00005555555d2175 in runDiff(LineData const*, LineRef, LineData const*, LineRef, DiffList&, Options*, Options*) (p1=0x0, size1=size1@entry=0, p2=0x0, size2=size2@entry=0, diffList=empty std::__cxx11::list, pOptions=<optimized out>, pOptions=<optimized out>) at /usr/include/c++/8/bits/stl_list.h:512 pp = {<QObject> = {<No data fields>}, static staticMetaObject = {d = {superdata = 0x7ffff67060a0 <QObject::staticMetaObject>, stringdata = 0x5555556179c0 <qt_meta_stringdata_ProgressProxy>, data = 0x555555617980 <qt_meta_data_ProgressProxy>, static_metacall = 0x555555608aa0 <ProgressProxy::qt_static_metacall(QObject*, QMetaObject::Call, int, void**)>, relatedMetaObjects = 0x0, extradata = 0x0}}} gnuDiff = {no_diff_means_no_output = false, context = 0, text = false, ignore_white_space = GnuDiff::IGNORE_NO_WHITE_SPACE, ignore_blank_lines = false, bIgnoreNumbers = false, bIgnoreWhiteSpace = false, files_can_be_treated_as_binary = false, ignore_case = false, ignore_file_name_case = false, brief = false, expand_tabs = false, initial_tab = false, starting_file = 0x0, paginate = false, group_format = {0x0, 0x0, 0x0, 0x0}, line_format = {0x0, 0x0, 0x0}, sdiff_merge_assist = false, left_column = false, suppress_common_lines = false, sdiff_half_width = 0, sdiff_column2_offset = 0, speed_large_files = false, excluded = 0x0, minimal = false, files = {{buffer = 0x0, bufsize = 0, buffered = 0, linbuf = 0x0, linbuf_base = 0, buffered_lines = 0, valid_lines = 0, alloc_lines = 0, prefix_end = 0x0, prefix_lines = 0, suffix_begin = 0x0, equivs = 0x0, undiscarded = 0x0, realindexes = 0x0, nondiscarded_lines = 0, changed = 0x0, eof = false, equiv_max = 0}, {buffer = 0x0, bufsize = 0, buffered = 0, linbuf = 0x0, linbuf_base = 0, buffered_lines = 0, valid_lines = 0, alloc_lines = 0, prefix_end = 0x0, prefix_lines = 0, suffix_begin = 0x0, equivs = 0x0, undiscarded = 0x0, realindexes = 0x0, nondiscarded_lines = 0, changed = 0x0, eof = false, equiv_max = 0}}, outfile = 0x0} #1 0x00005555555d271d in runDiff(LineData const*, int, LineData const*, int, std::__cxx11::list<Diff, std::allocator<Diff> >&, int, int, std::__cxx11::list<ManualDiffHelpEntry, std::allocator<ManualDiffHelpEntry> >*, Options*) (p1=0x0, size1=size1@entry=0, p2=p2@entry=0x0, size2=size2@entry=0, diffList=empty std::__cxx11::list, winIdx1=winIdx1@entry=1, winIdx2=<optimized out>, pManualDiffHelpList=0x5555558510a8, pOptions=0x5555557691a8) at /usr/src/debug/kdiff3-1.7.90-1.fc29.x86_64/src/diff.cpp:1696 diffList2 = empty std::__cxx11::list l1begin = <optimized out> l2begin = <optimized out> i = {lineA1 = 0, lineA2 = 0, lineB1 = 1851878202, lineB2 = 1953439841, lineC1 = 1862270976, lineC2 = 1815770229} #2 0x00005555555b9cc5 in KDiff3App::mainInit(TotalDiffStatus*, bool, bool) (this=this@entry=0x555555850a30, pTotalDiffStatus=0x555555850cf8, pTotalDiffStatus@entry=0x0, bLoadFiles=bLoadFiles@entry=true, bUseCurrentEncoding=bUseCurrentEncoding@entry=false) at /usr/src/debug/kdiff3-1.7.90-1.fc29.x86_64/src/pdiff.cpp:207 pp = {<QObject> = {<No data fields>}, static staticMetaObject = {d = {superdata = 0x7ffff67060a0 <QObject::staticMetaObject>, stringdata = 0x5555556179c0 <qt_meta_stringdata_ProgressProxy>, data = 0x555555617980 <qt_meta_data_ProgressProxy>, static_metacall = 0x555555608aa0 <ProgressProxy::qt_static_metacall(QObject*, QMetaObject::Call, int, void**)>, relatedMetaObjects = 0x0, extradata = 0x0}}} errors = {<QList<QString>> = {<QListSpecialMethods<QString>> = {<No data fields>}, {p = {static shared_null = {ref = {atomic = {_q_value = {<std::__atomic_base<int>> = {static _S_alignment = 4, _M_i = -1}, <No data fields>}}}, alloc = 0, begin = 0, end = 0, array = {0x0}}, d = 0x555555ddd720}, d = 0x555555ddd720}}, <No data fields>} bGUI = true bVisibleMergeResultWindow = <optimized out> oldHeights = {<QListSpecialMethods<int>> = {<No data fields>}, {p = {static shared_null = {ref = {atomic = {_q_value = {<std::__atomic_base<int>> = {static _S_alignment = 4, _M_i = -1}, <No data fields>}}}, alloc = 0, begin = 0, end = 0, array = {0x0}}, d = 0x555555b72010}, d = 0x555555b72010}} #3 0x00005555555923a2 in KDiff3App::completeInit(QString const&, QString const&, QString const&) (this=this@entry=0x555555850a30, fn1=..., fn2=..., fn3=...) at /usr/src/debug/kdiff3-1.7.90-1.fc29.x86_64/src/kdiff3.cpp:411 bSuccess = true #4 0x000055555558dce6 in KDiff3Shell::KDiff3Shell(bool) (this=0x5555557cc1c0, bCompleteInit=<optimized out>, __in_chrg=<optimized out>, __vtt_parm=<optimized out>) at /usr/include/qt5/QtCore/qstring.h:681 #5 0x000055555558c373 in main(int, char**) (argc=<optimized out>, argv=<optimized out>) at /usr/src/debug/kdiff3-1.7.90-1.fc29.x86_64/src/main.cpp:198 appName = {m_size = 6, m_data = 0x55555560e056 "kdiff3"} app = <incomplete type> i18nName = {static null = {<No data fields>}, d = 0x5555557c5a20} appVersion = {static null = {<No data fields>}, d = 0x5555557cf070} description = {static null = {<No data fields>}, d = 0x555555776c20} copyright = {static null = {<No data fields>}, d = 0x555555776b70} homePage = {static null = {<No data fields>}, d = 0x55555560e5c0 <main::{lambda()#1}::operator()() const::qstring_literal>} bugsAddress = {static null = {<No data fields>}, d = 0x55555560e580 <main::{lambda()#2}::operator()() const::qstring_literal>} aboutData = {d = 0x5555557cebd0} cmdLineParser = 0x5555557765c0 p = <optimized out> retVal = <optimized out>
Thanks for the update it really should be trying to go on after flagging a file as too large. I'll have a look at this.
As of a6d614bc8ae4 kdiff3 should now skip oversized files.