Bug 380733

Summary: FIXED IN SVN: Hang on big names arrays
Product: [Applications] rkward Reporter: RKWard Team <rkward-devel>
Component: generalAssignee: RKWard Team <rkward-devel>
Status: RESOLVED FIXED    
Severity: normal    
Priority: NOR    
Version: unspecified   
Target Milestone: ---   
Platform: unspecified   
OS: All   
Latest Commit: Version Fixed In:
Sentry Crash Report:

Description RKWard Team 2011-10-04 13:32:51 UTC 
-- Originally posted by (AT sourceforge.net): nalimilan --

-- This ticket was imported from http://sourceforge.net/p/rkward/bugs/101 on 2017-05-30 15:26:46 +0100 --
Today I added a numeric variable to a big data frame, and rkward.frontend started eating GBs of memory and using all CPU, never finishing. I'm able to reproduce the hang with the attached .RData file, which contains that variable as an array. Loading this file always makes my RKWard hang. This is quite weird, since either converting the array to a vector, or setting dimnames to NULL is enough to fix the problem. But changing the names to 1:length\(a\) for example has no effect, nor changing the values of a.

You can download the .RData file from: http://ubuntuone.com/24J68sdk9kd6PUCtO6Hd1f

No need to say I'm very interested in debugging this, since I need to work with this data. ;-\)

I'm using RKWard 0.5.6z+0.5.7+devel1 on Fedora 15 64-bits.

&gt; str\(a\)
num \[1:1985324\(1d\)\] 0.2619 0.7744 -0.0511 -0.0113 0.256 ...
\- attr\(\*, "dimnames"\)=List of 1
..$ : chr \[1:1985324\] "1969" "1969" "1969" "1969" ...


Backtrace of rkward while hanging on load:
\(gdb\) ba
\#0  0x000000000048e990 in QList&lt;RObject\*&gt;::indexOf \(this=&lt;optimized out&gt;, t=
@0x7fff99b712a8, from=&lt;optimized out&gt;\) at /usr/include/QtCore/qlist.h:833
\#1  0x00000000004e84a2 in RContainerObject::updateChildren \(this=0x2b22d90, 
new\_children=&lt;optimized out&gt;\)
at /home/milan/Dev/rkward/rkward/rkward/core/rcontainerobject.cpp:180
\#2  0x00000000004e8911 in RContainerObject::updateStructure \(this=0x2b22d90, 
new\_data=&lt;optimized out&gt;\)
at /home/milan/Dev/rkward/rkward/rkward/core/rcontainerobject.cpp:93
\#3  0x00000000004e7e11 in RContainerObject::updateChildStructure \(this=
0x2565260, child=0x2b22d90, new\_data=0x2c83710, just\_created=true\)
at /home/milan/Dev/rkward/rkward/rkward/core/rcontainerobject.cpp:55
\#4  0x00000000004e7b8f in RContainerObject::createChildFromStructure \(this=
0x2565260, child\_data=0x2c83710, child\_name=..., position=0\)
at /home/milan/Dev/rkward/rkward/rkward/core/rcontainerobject.cpp:128
\#5  0x00000000004e8014 in RContainerObject::updateChildStructure \(this=
0x2565260, child=0x2c85970, new\_data=0x2c83710, 
just\_created=&lt;optimized out&gt;\)
at /home/milan/Dev/rkward/rkward/rkward/core/rcontainerobject.cpp:72
\#6  0x00000000004e0477 in rCommandDone \(command=0x2c83710, this=0x2c85970\)
at /home/milan/Dev/rkward/rkward/rkward/core/robject.cpp:263
\#7  RObject::rCommandDone \(this=0x2c85970, command=0x2c83710\)
at /home/milan/Dev/rkward/rkward/rkward/core/robject.cpp:253
\#8  0x00000000004db7fb in RKVariable::rCommandDone \(this=0x2c85970, command=
\---Type &lt;return&gt; to continue, or q &lt;return&gt; to quit---
0x2c83710\) at /home/milan/Dev/rkward/rkward/rkward/core/rkvariable.cpp:120
\#9  0x00000000004fdd81 in RCommand::finished \(this=0x2c83710\)
at /home/milan/Dev/rkward/rkward/rkward/rbackend/rcommand.cpp:113
\#10 0x00000000004f57a0 in RInterface::handleCommandOut \(this=&lt;optimized out&gt;, 
command=0x2c83710\)
at /home/milan/Dev/rkward/rkward/rkward/rbackend/rinterface.cpp:209
\#11 0x00000000004fc80d in RInterface::handleRequest \(this=0x24c6ea0, request=
0x7fad6c07f2e0\)
at /home/milan/Dev/rkward/rkward/rkward/rbackend/rinterface.cpp:324
\#12 0x00000035e4f70c0c in QObject::event \(this=0x24c5cf0, e=&lt;optimized out&gt;\)
at kernel/qobject.cpp:1248
\#13 0x00000035f1bb9324 in notify\_helper \(e=0x7fad6c10a500, receiver=0x24c5cf0, 
this=0x21a7df0\) at kernel/qapplication.cpp:4481
\#14 QApplicationPrivate::notify\_helper \(this=0x21a7df0, receiver=0x24c5cf0, e=
0x7fad6c10a500\) at kernel/qapplication.cpp:4453
\#15 0x00000035f1bbe1b1 in QApplication::notify \(this=0x7fff99b721c0, receiver=
0x24c5cf0, e=0x7fad6c10a500\) at kernel/qapplication.cpp:4360
\#16 0x00000035eb841d56 in KApplication::notify\(QObject\*, QEvent\*\) \(\)
from /usr/lib64/libkdeui.so.5
\#17 0x00000035e4f5a20c in QCoreApplication::notifyInternal \(this=
0x7fff99b721c0, receiver=0x24c5cf0, event=0x7fad6c10a500\)
at kernel/qcoreapplication.cpp:787
\#18 0x00000035e4f5d7d4 in sendEvent \(event=0x7fad6c10a500, receiver=0x24c5cf0\)
\---Type &lt;return&gt; to continue, or q &lt;return&gt; to quit---
at kernel/qcoreapplication.h:215
\#19 QCoreApplicationPrivate::sendPostedEvents \(receiver=0x0, event\_type=0, 
data=0x2180070\) at kernel/qcoreapplication.cpp:1428
\#20 0x00000035e4f84973 in sendPostedEvents \(\) at kernel/qcoreapplication.h:220
\#21 postEventSourceDispatch \(s=0x21ab8c0\)
at kernel/qeventdispatcher\_glib.cpp:277
\#22 0x00000035df6427ed in g\_main\_dispatch \(context=0x21ab4e0\) at gmain.c:2441
\#23 g\_main\_context\_dispatch \(context=0x21ab4e0\) at gmain.c:3014
\#24 0x00000035df642fc8 in g\_main\_context\_iterate \(context=0x21ab4e0, 
block=&lt;optimized out&gt;, dispatch=1, self=&lt;optimized out&gt;\) at gmain.c:3092
\#25 0x00000035df64325c in g\_main\_context\_iteration \(context=0x21ab4e0, 
may\_block=1\) at gmain.c:3155
\#26 0x00000035e4f84dcf in QEventDispatcherGlib::processEvents \(this=0x21a7490, 
flags=&lt;optimized out&gt;\) at kernel/qeventdispatcher\_glib.cpp:422
\#27 0x00000035f1c5c12e in QGuiEventDispatcherGlib::processEvents \(
this=&lt;optimized out&gt;, flags=&lt;optimized out&gt;\)
at kernel/qguieventdispatcher\_glib.cpp:207
\#28 0x00000035e4f59722 in QEventLoop::processEvents \(this=&lt;optimized out&gt;, 
flags=...\) at kernel/qeventloop.cpp:149
\#29 0x00000035e4f5991f in QEventLoop::exec \(this=0x7fff99b72110, flags=...\)
at kernel/qeventloop.cpp:201
\#30 0x00000035e4f5da67 in QCoreApplication::exec \(\)
at kernel/qcoreapplication.cpp:1064
\---Type &lt;return&gt; to continue, or q &lt;return&gt; to quit---
\#31 0x0000000000432634 in main \(argc=&lt;optimized out&gt;, argv=&lt;optimized out&gt;\)
at /home/milan/Dev/rkward/rkward/rkward/main.cpp:177
\(gdb\) list
828	Q\_OUTOFLINE\_TEMPLATE int QList&lt;T&gt;::indexOf\(const T &t, int from\) const
829	\{
830	    if \(from &lt; 0\)
831	        from = qMax\(from + p.size\(\), 0\);
832	    if \(from &lt; p.size\(\)\) \{
833	        Node \*n = reinterpret\_cast&lt;Node \*&gt;\(p.at\(from -1\)\);
834	        Node \*e = reinterpret\_cast&lt;Node \*&gt;\(p.end\(\)\);
835	        while \(++n \!= e\)
836	            if \(n-&gt;t\(\) == t\)
837	                return int\(n - reinterpret\_cast&lt;Node \*&gt;\(p.begin\(\)\)\);
Comment 1 Thomas Friedrichsmeier 2011-10-04 14:52:19 UTC 
Hi\!

The core problem with that object is that it looked like an object containing 1985324 sub-objects to RKWard. I have addressed this in the development version \(http://p.sf.net/rkward/svn\) by making sure that RKWard will never try to look at more than 100000 children of any object, and specifically will not treat arrays as hierarchical named objects \(which it never did quite right, anyway\). That should solve the hang.

That said, that object of yours really is a bit unusual. To me it looks more like you wanted to have two separate variables, as in:
x &lt;- data.frame \(year=names \(a\), as.vector \(a\)\)
or, alternatively:
x &lt;- sapply \(unique \(names \(a\)\), function \(x\) as.vector \(a\[x == names \(a\)\]\)\)

While names\(\) can contain duplicates for most R objects, this is rarely ever useful, and typically it indicates a problem. E.g. what's the use of
a\["2009"\]
?

Regards
Thomas
Comment 2 Thomas Friedrichsmeier 2011-10-04 14:52:19 UTC 
- **assigned_to**: nobody --> tfry
- **summary**: Hang when creating or loading a big array/data frame column --> FIXED IN SVN: Hang on big names arrays
- **status**: open --> open-fixed
Comment 3 RKWard Team 2011-10-04 15:03:09 UTC 
-- Originally posted by (AT sourceforge.net): nalimilan --
Ah, cool, I'll try with a more recent SVN then. I would have updated before reporting, but my setup is a little messed up ATM, so that's not as easy as it sounds. ;-\)

That 'a' object doesn't make any sense, it's just that I extracted the problematic variable from a big data frame using a&lt;-df$a. Since it was the smallest unit that triggered the bug, I kept it, but it isn't useful by itself. The names probably correspond to the first column of the data frame, with contains the survey year, but is of course used in relation with many more variables.
Comment 4 Thomas Friedrichsmeier 2011-10-24 07:13:20 UTC 
- **status**: open-fixed --> closed-fixed