Bug 197313 - hangs the system consuming all memory
Summary: hangs the system consuming all memory
Alias: None
Product: kate
Classification: Applications
Component: general (show other bugs)
Version: unspecified
Platform: unspecified Linux
: NOR crash (vote)
Target Milestone: ---
Assignee: KWrite Developers
: 225345 (view as bug list)
Depends on:
Reported: 2009-06-20 21:29 UTC by Facundo Aguilera
Modified: 2010-02-04 13:02 UTC (History)
4 users (show)

See Also:
Latest Commit:
Version Fixed In:

This is the VHDL file (194.71 KB, application/octet-stream)
2009-06-21 00:02 UTC, Facundo Aguilera
New syntax highlighting file (20.26 KB, text/xml)
2009-09-28 01:17 UTC, Jan Michel

Note You need to log in before you can comment on or make changes to this bug.
Description Facundo Aguilera 2009-06-20 21:29:38 UTC
Version:           3.2.4 (using 4.2.4 (KDE 4.2.4), Debian packages)
Compiler:          cc
OS:                Linux (x86_64) release 2.6.29-amd64-lb0

When I open a .vhd file with 3800 lines in a few minutes the systems hangs up, kate consumes all memory and it takes almost 10 min to be closed.

- Open that vhd file with kate (or kwrite)
- Edit it for a few minutes (2 minutes)
- Open another file or click New file.
- Full memory
Comment 1 Andreas Pakulat 2009-06-20 22:15:54 UTC
can you attach that file here? Without a way for a developer to reproduce the bug its impossible to fix it (unless you want to do it).
Comment 2 Facundo Aguilera 2009-06-21 00:02:57 UTC
Created attachment 34698 [details]
This is the VHDL file
Comment 3 Andreas Pakulat 2009-06-21 00:45:36 UTC
Here's a backtrace I get while apparently kate tries to syntax-highlight the file. It doesn't help forcing kate to open the file in iso-8859-1 encoding (which seems to be what the file is encoded as):

Application: Kate (kate), signal: Segmentation fault
[KCrash Handler]
#5  0xf634a1bf in memcpy () from /lib/libc.so.6
#6  0xf6636cd3 in QVectorData::malloc (sizeofTypedData=20, size=-193830542, sizeofT=4, init=0xf6805e10) at /home/andreas/src/qt-copy/src/corelib/tools/qvector.cpp:53
#7  0xf3973377 in QVector<KateHlContext*>::realloc (this=0x0, asize=1953653104, aalloc=-193830542) at /usr/local/include/QtCore/qvector.h:455
#8  0xf3962008 in KateHighlighting::dropDynamicContexts (this=0x9c590c8) at /usr/local/include/QtCore/qvector.h:321
#9  0xf395b575 in KateHlManager::resetDynamicCtxs (this=0x9c0f880) at /home/andreas/src/kdelibs/kate/syntax/katesyntaxmanager.cpp:375
#10 0xf392561d in KateBuffer::doHighlight (this=0x9cc1370, startLine=128, endLine=192, invalidate=false) at /home/andreas/src/kdelibs/kate/document/katebuffer.cpp:1031
#11 0xf3925f69 in KateBuffer::ensureHighlighted (this=0x9cc1370, line=128) at /home/andreas/src/kdelibs/kate/document/katebuffer.cpp:775
#12 0xf390f61c in KateDocument::kateTextLine (this=0x9c7d338, i=128) at /home/andreas/src/kdelibs/kate/document/katedocument.cpp:6008
#13 0xf394f9fd in KateLineLayout::textLine (this=0xa25e4e8, reloadForce=false) at /home/andreas/src/kdelibs/kate/render/katelinelayout.cpp:67
#14 0xf3944c93 in KateRenderer::layoutLine (this=0x9e30308, lineLayout=..., maxwidth=-1, cacheLayout=true) at /home/andreas/src/kdelibs/kate/render/katerenderer.cpp:756
#15 0xf394ba24 in KateLayoutCache::line (this=0x9e8ad00, realLine=128, virtualLine=128) at /home/andreas/src/kdelibs/kate/render/katelayoutcache.cpp:309
#16 0xf394c968 in KateLayoutCache::updateViewCache (this=0x9e8ad00, startPos=..., newViewLineCount=34, viewLinesScrolled=3) at /home/andreas/src/kdelibs/kate/render/katelayoutcache.cpp:262
#17 0xf39a47ae in KateViewInternal::doUpdateView (this=0x9e8bd20, changed=false, viewLinesScrolled=3) at /home/andreas/src/kdelibs/kate/view/kateviewinternal.cpp:592
#18 0xf39a5065 in KateViewInternal::updateView (this=0x9e8bd20, changed=<value optimized out>, viewLinesScrolled=3) at /home/andreas/src/kdelibs/kate/view/kateviewinternal.cpp:568
#19 0xf39a6f31 in KateViewInternal::scrollPos (this=0x9e8bd20, c=..., force=false, calledExternally=false) at /home/andreas/src/kdelibs/kate/view/kateviewinternal.cpp:514
#20 0xf39a7557 in KateViewInternal::scrollViewLines (this=0x9e8bd20, offset=3) at /home/andreas/src/kdelibs/kate/view/kateviewinternal.cpp:390
#21 0xf39a7815 in KateViewInternal::wheelEvent (this=0x9e8bd20, e=0xffa84b50) at /home/andreas/src/kdelibs/kate/view/kateviewinternal.cpp:3297
#22 0xf6ca1251 in QWidget::event (this=0x9e8bd20, event=0xffa84b50) at /home/andreas/src/qt-copy/src/gui/kernel/qwidget.cpp:7529
#23 0xf6c2fe07 in QApplicationPrivate::notify_helper (this=0x9ac4c18, receiver=0x9e8bd20, e=0xffa84b50) at /home/andreas/src/qt-copy/src/gui/kernel/qapplication.cpp:4057
#24 0xf6c311e5 in QApplication::notify (this=0xffa86708, receiver=0x9e8bd20, e=0xffa84b50) at /home/andreas/src/qt-copy/src/gui/kernel/qapplication.cpp:3822
#25 0xf7a72b6d in KApplication::notify (this=0xffa86708, receiver=0x9e8bd20, event=0xffa84b50) at /home/andreas/src/kdelibs/kdeui/kernel/kapplication.cpp:302
#26 0xf66e96ea in QCoreApplication::notifyInternal (this=0xffa86708, receiver=0x9e8bd20, event=0xffa84b50) at /home/andreas/src/qt-copy/src/corelib/kernel/qcoreapplication.cpp:610
#27 0xf6c3e039 in QCoreApplication::sendSpontaneousEvent (receiver=0x9e8bd20, event=0xffa84b50) at ../../include/QtCore/../../../../qt-copy/src/corelib/kernel/qcoreapplication.h:216
#28 0xf6cb486b in QETWidget::translateWheelEvent (this=0x9cc81c0, global_x=1271, global_y=446, delta=-120, buttons=..., modifiers=..., orient=Qt::Vertical)
    at /home/andreas/src/qt-copy/src/gui/kernel/qapplication_x11.cpp:4486
#29 0xf6cca7a8 in QETWidget::translateMouseEvent (this=0x9cc81c0, event=0xffa862e0) at /home/andreas/src/qt-copy/src/gui/kernel/qapplication_x11.cpp:4263
#30 0xf6ccc277 in QApplication::x11ProcessEvent (this=0xffa86708, event=0xffa862e0) at /home/andreas/src/qt-copy/src/gui/kernel/qapplication_x11.cpp:3444
#31 0xf6cfbe93 in x11EventSourceDispatch (s=0x9ac77f8, callback=0, user_data=0x0) at /home/andreas/src/qt-copy/src/gui/kernel/qguieventdispatcher_glib.cpp:146
#32 0xf599b848 in g_main_context_dispatch () from /usr/lib/libglib-2.0.so.0
#33 0xf599edab in ?? () from /usr/lib/libglib-2.0.so.0
#34 0x09ac6e98 in ?? ()
#35 0x00000000 in ?? ()
Comment 4 Dario Andres 2009-06-21 00:46:38 UTC
I can reproduce here using:

Qt: 4.5.1 (qt-copy  971295)
KDE: 4.2.92 (KDE 4.2.92 (KDE 4.3 >= 20090617))
kdelibs svn rev. 984425 / kdebase svn rev. 984427
on ArchLinux i686 - Kernel

- After scrolling on the testcase document, going to Open to select a new document will start consuming all the memory, hanging up the system. (the application quited/crashed itself after some time, and the memory was freed
Comment 5 Facundo Aguilera 2009-06-21 03:14:33 UTC
Some news: 
- the same file with another syntax-highlight (c++) seems to work fine. 
- I am working with another smaller .vhd files without problems.
- that file was generated automaticaly by a tool. 
- I'm always using iso-8859-1

Comment 6 Facundo Aguilera 2009-08-07 17:15:42 UTC
It still in 4.3.0, kate 3.3.0, vhdl syntax highlighting 1.08, with similar files.
Comment 7 Dominik Haumann 2009-09-17 23:17:59 UTC
My guess is that it's because of the dynamic contexts: vhdl.xml uses dynamic=true in a lot of rules. Are they really necessary?

Another idea is that the contexts are not correctly popped such that the context stack eats more and more memory. I haven't found anything while quickly looking at the xml file, so it's just a thought.

Other ideas: The hl uses beginRegion and lookAhead together. There is no other file doing this that I know of. Maybe this doesn't play together? (it was not tested). Try removing all begin/endRegions.
Comment 8 Jan Michel 2009-09-18 02:04:15 UTC
I will try to find some time this or next week to check the whole vhdl highlighting file again. With the remarks by Dominik this seems to be a doable task.

I'm using the dynamic rules to check that the start and end of a region share the same label. Maybe there is a better solution?
In case you don't know vhdl: 

label_1 : process 
end process label_1;

This and alike are a common though optional construct I want to check using dynamic rules. 

During development I did some "infinite popping" several times by mistake - always the syntax highlighting didn't crash but ran for a while, then stopped, seemingly reaching a limit in the depth of the stack.
Comment 9 Jan Michel 2009-09-28 01:17:05 UTC
Created attachment 37206 [details]
New syntax highlighting file
Comment 10 Jan Michel 2009-09-28 01:22:23 UTC
I made a rework of most of the syntax highlight for vhdl (see attachment below). 
The good news: With "normal" files it has a better performance and covers some more special cases.
The bad news: With the special vhdl file posted above, Kate still crashes when scrolling fast - fortunately without eating all system memory before. The error message says: 
"terminate called after throwing an instance of 'std::bad_alloc' 
what(): std::bad_alloc"
The backtrace is completely empty.

Nevertheless, I would appreciate somebody testing and/or reviewing the syntax highlighting and reporting possible bugs.
Comment 11 Facundo Aguilera 2009-10-01 23:17:54 UTC
Hi! Thanks!
But... I can't load your syntax highlighting file... apparently there is a problem in the context section. Can you check it, please?
Comment 12 Facundo Aguilera 2009-10-02 00:00:02 UTC
Forget it... 
It was another problem...
I'll test the syntax highlight for a while and tell you. It looks better.

In this bug sistem, I can view the attached file with syntax highlight, I can edit the file, I can even comment the file, but I can't download it properly....
Comment 13 Milian Wolff 2010-01-09 18:41:48 UTC
SVN commit 1072238 by mwolff:

update vhdl syntax file, thanks to Chris Higgs, Jan MIchel and Florent Ouchet
CCMAIL: kwrite-devel@kde.org
BUG: 194977
BUG: 197313

 M  +292 -146  vhdl.xml  

WebSVN link: http://websvn.kde.org/?view=rev&revision=1072238
Comment 14 Facundo Aguilera 2010-01-09 19:29:49 UTC
With this syntax highlight file kate doesn't crash when loading/scrolling the test file, and the highlighting is better, covering more special cases. It still consuming about 1GB, hanging my sistem with 2 GB for a while when loading, but then I can work with the file.
I'm using kde 4.3.4, maybe with the new kate it works better.
Comment 15 Dario Andres 2010-02-04 13:02:51 UTC
*** Bug 225345 has been marked as a duplicate of this bug. ***