Bug 263050 - Memory consumption goes crazy with many output statments
Summary: Memory consumption goes crazy with many output statments
Status: RESOLVED FIXED
Alias: None
Product: kdevelop
Classification: Applications
Component: general (show other bugs)
Version: git master
Platform: Compiled Sources Linux
: HI grave
Target Milestone: 4.2.0
Assignee: kdevelop-bugs-null
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-01-13 18:55 UTC by David
Modified: 2011-01-13 23:08 UTC (History)
1 user (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description David 2011-01-13 18:55:05 UTC
Version:           SVN (using Devel) 
OS:                Linux

If I call cout in an inner loop and the output goes to the internal KDevelop output window, the memory consumption just goes up and up until KDevelop crashes. If I output to an external terminal, everything works as expected.

Reproducible: Always
Comment 1 Milian Wolff 2011-01-13 21:36:36 UTC
we should implement some sort of scrollback limit as a quick fix, 1000loc or something like that. those who want the full output should use an external konsole.
Comment 2 Aleix Pol 2011-01-13 22:33:16 UTC
Konsole _doesn't_ provide full output either.
I guess it's just wrong to use QStandardItemModel there.
Comment 3 Milian Wolff 2011-01-13 22:43:16 UTC
you can disable scrollback limit in konsole and it won't make the mem consumption skyrocket, only your hdd gets spammed since it uses a mmapped file in /tmp.

and imo we all agree that getting rid of qstandarditemmodel won't help much, the "real" solution to all/most problems would be a proper embedded kpart with all the functionality we require exposed by an API...

since noone did that sofar, I'd say: lets do the scrollback limit for now and be done with it.
Comment 4 Milian Wolff 2011-01-13 23:08:37 UTC
commit 160642ced68b45e53b4fab2feb230aab020875d5
branch 1.2
Author: Milian Wolff <mail@milianw.de>
Date:   Thu Jan 13 23:06:56 2011 +0100

    don't add infinite number of rows of output, instead limit to 50k lines and only keep the most recent ones
    
    BUG: 263050

diff --git a/outputview/outputmodel.cpp b/outputview/outputmodel.cpp
index d7c9040..317e3ee 100644
--- a/outputview/outputmodel.cpp
+++ b/outputview/outputmodel.cpp
@@ -34,13 +34,16 @@ struct OutputModelPrivate
     QTimer* timer;
     QVector<QStandardItem*> pending;
     
-    static const int MAX_SIZE;
+    static const int MAX_SIZE_PENDING;
     static const int INTERVAL_MS;
+    static const int MAX_SIZE;
 };
 
-const int OutputModelPrivate::MAX_SIZE=10000;
+const int OutputModelPrivate::MAX_SIZE_PENDING=10000;
 const int OutputModelPrivate::INTERVAL_MS=50;
 
+const int OutputModelPrivate::MAX_SIZE=50000;
+
 OutputModel::OutputModel( QObject* parent )
     : QStandardItemModel( parent ), d(new OutputModelPrivate)
 {
@@ -48,7 +51,7 @@ OutputModel::OutputModel( QObject* parent )
     d->timer->setInterval(OutputModelPrivate::INTERVAL_MS);
     d->timer->setSingleShot(true);
     
-    d->pending.reserve(OutputModelPrivate::MAX_SIZE);
+    d->pending.reserve(OutputModelPrivate::MAX_SIZE_PENDING);
     
     connect(d->timer, SIGNAL(timeout()), SLOT(addPending()));
 }
@@ -65,7 +68,7 @@ void OutputModel::appendLine( const QString& line )
     item->setFont( KGlobalSettings::fixedFont() );
     
     d->pending.append(item);
-    if(d->pending.size()<OutputModelPrivate::MAX_SIZE)
+    if(d->pending.size()<OutputModelPrivate::MAX_SIZE_PENDING)
         d->timer->start();
     else
         addPending();
@@ -81,9 +84,16 @@ void OutputModel::appendLines( const QStringList& lines)
 
 void OutputModel::addPending()
 {
-    if(!d->pending.isEmpty())
+    if(!d->pending.isEmpty()) {
+        const int aboutToAdd = d->pending.size();
+        if (aboutToAdd + invisibleRootItem()->rowCount() > OutputModelPrivate::MAX_SIZE) {
+            // https://bugs.kde.org/show_bug.cgi?id=263050
+            // make sure we don't add too many items
+            invisibleRootItem()->removeRows(0, aboutToAdd + invisibleRootItem()->rowCount() - OutputModelPrivate::MAX_SIZE);
+        }
         invisibleRootItem()->appendRows(d->pending.toList());
-    
+    }
+
     d->pending.clear();
 }