Bug 261509

Summary: meinproc4: crash with segfault on Mac OS X
Product: [Websites] docs.kde.org Reporter: Jeremy Lavergne <snc>
Component: ksgmltoolsAssignee: Documentation Editorial Team <kde-doc-english>
Status: CONFIRMED ---    
Severity: normal CC: foolishewe, iandw.au, kde, lueck, luigi.toscano, mk-lists, nicos, rjvbertin
Priority: NOR    
Version: 4.12.3   
Target Milestone: ---   
Platform: Compiled Sources   
OS: macOS   
Latest Commit: Version Fixed In:
Sentry Crash Report:
Attachments: protect against potential conversion of a NULL CFString

Description Jeremy Lavergne 2010-12-29 03:27:45 UTC
Version:           unspecified (using KDE 4.5.90) 
OS:                OS X

Build phases on segfault

[100%] Generating index.cache.bz2
cd /opt/local/var/macports/build/_Users_aeetes_kde_kdepim4/work/kdepim-4.6beta3/doc/akregator && /opt/local/bin/meinproc4 --check --cache /opt/local/var/macports/build/_Users_aeetes_kde_kdepim4/work/build/doc/akregator/index.cache.bz2 /opt/local/var/macports/build/_Users_aeetes_kde_kdepim4/work/kdepim-4.6beta3/doc/akregator/index.docbook
/bin/sh: line 1: 85478 Segmentation fault      /opt/local/bin/meinproc4 --check --cache /opt/local/var/macports/build/_Users_aeetes_kde_kdepim4/work/build/doc/akregator/index.cache.bz2 /opt/local/var/macports/build/_Users_aeetes_kde_kdepim4/work/kdepim-4.6beta3/doc/akregator/index.docbook
make[2]: *** [doc/akregator/index.cache.bz2] Error 139


Reproducible: Always

Steps to Reproduce:
Build in MacPorts

Actual Results:  
Build segfaults

Expected Results:  
Successful build.

This is kdepim4 @4.5.93/4.6beta3
Comment 1 Jekyll Wu 2013-03-01 00:44:42 UTC
Is that still a problem in recent versions ?
Comment 2 Marko Käning 2013-03-07 21:55:08 UTC
I'll check this on a recent version.
Comment 3 Marko Käning 2013-08-03 18:15:41 UTC
Just tested this with kdepim 4.10.3 and it installed for a change... But I guess it only succeeded since the doc subdir in CMakeFiles.txt was deliberately commented out:
---
--- CMakeLists.txt.orig 2013-03-01 15:57:28.000000000 +0900
+++ CMakeLists.txt  2013-03-09 20:05:57.000000000 +0900
@@ -389,7 +389,7 @@
 endif()

 # doc must be a subdir of kdepim or packagers will kill us
-macro_optional_add_subdirectory(doc)
+#macro_optional_add_subdirectory(doc)


 # We really want to encourage users to enable/install QGpgME from kdepimlibs
---

THIS ALL is actually not a problem of kdepim, but rather of meinproc4 which (already since years) CRASHES FAR TOO OFTEN on MacOSX when trying to build various KDE applications (see e.g. [1,2])!!!

:-(

I wonder what I can do to track this down, since it seems to be unforeseeable which file will be the one meinproc4 will crash on...


[1] https://trac.macports.org/ticket/30162
[2] https://trac.macports.org/ticket/37620
Comment 4 Nicolas Pavillon 2013-12-13 13:23:04 UTC
This issue still repeatedly (but apparently rather randomly) occurs with meinproc4 crashing with a segfault. For example in [1], after we tried to reinstate the installation of documentation in kdelibs, or in [2], where poxml uses meinproc4 in checkXML during the test phase. 

[1] https://trac.macports.org/ticket/41326
[2] https://trac.macports.org/ticket/41576

Considering the issue, I would suggest to rename the bug title to something more accurate, such as for example "meinproc4: crash with segfault on Mac OS X"
Comment 5 kde 2013-12-13 18:43:06 UTC
Since nobody has provided a crash log yet, here's one:

https://trac.macports.org/attachment/ticket/41576/meinproc4_2013-12-11-183146_mindgroup-lm.crash

Here's what it says:

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0   com.apple.CoreFoundation            0x00007fff8a32ae4f CFStringGetLength + 15
1   libkdecore.5.dylib                  0x0000000100e49011 convert_CFString_to_QString(__CFString const*) + 49
2   libkdecore.5.dylib                  0x0000000100e4a4da KLocaleMacPrivate::systemCountry() const + 42
3   libkdecore.5.dylib                  0x0000000100dfa7dc KLocalePrivate::initCountry(QString const&, QString const&) + 268
4   libkdecore.5.dylib                  0x0000000100df97ff KLocalePrivate::init(QString const&, QString const&, QString const&, KSharedPtr<KSharedConfig>, KConfig*) + 719
5   libkdecore.5.dylib                  0x0000000100e4a1e0 KLocaleMacPrivate::KLocaleMacPrivate(KLocale*, QString const&, KSharedPtr<KSharedConfig>) + 128
6   libkdecore.5.dylib                  0x0000000100df63f6 KLocale::KLocale(QString const&, KSharedPtr<KSharedConfig>) + 86
7   libkdecore.5.dylib                  0x0000000100d7b3c0 KGlobal::locale() + 208
8   meinproc4                           0x0000000100cc97d3 main + 2387
9   libdyld.dylib                       0x00007fff8c2d35fd start + 1
Comment 6 Marko Käning 2013-12-16 20:05:20 UTC
Wow, this is the first true trace I see for this bug. I wonder whether you're actually able to reproduce this crash.
Comment 7 foolishewe 2014-01-02 18:54:28 UTC
He could not but I could, I reported it to macports.  This crash occurred consistently and prevented the build of the poxml package on this particular machine, and started happening after I updated the system to Mavericks.  I think our odds on replaying this scenario and getting more full diagnostic information may be better on my configuration.

Recently bug fix patch was provided to me by the macports team, who may have applied it to the tree. Although we have a work around, I'd be willing to run a reasonable number of diagnostics to track this down, and the Macports team may also be motivated to help, shall I ask the Macports maintainers to provide me an inverse patch to back it out so that this can be diagnosed, understood and fixed. 

Let'
Comment 8 foolishewe 2014-01-02 19:20:48 UTC
Sorry about the previous message being garbled.  I did a port selfupdate (update my local ports tree) and then uninstalled and reinstalled poxml with success, so I believe the workaround/fix was propagated to the main branch.  Do you want me to go to the macports team and ask for help in rolling back this fix locally on my machine to see if we can track this down?
Comment 9 Marko Käning 2014-05-03 18:54:07 UTC
An update: This issue is discussed lately on KDE-DEVEL and (more up-to-date) on KDE-MAC.

KDE-DEVEL: http://lists.kde.org/?t=139514981600001&r=1&w=2
KDE-MAC: http://lists.kde.org/?l=kde-mac&m=139900328213525&w=2
Comment 10 RJVB 2014-09-03 00:18:37 UTC
(In reply to kde from comment #5)
> Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
> 0   com.apple.CoreFoundation            0x00007fff8a32ae4f CFStringGetLength
> + 15
> 1   libkdecore.5.dylib                  0x0000000100e49011
> convert_CFString_to_QString(__CFString const*) + 49


Whether or not the bug is indeed in convert_CFString_to_QString(), it might be a good idea to protect that function against null CFStringRefs:

    QString convert_CFString_to_QString(CFStringRef str) {
            if( str ){
            CFIndex length = CFStringGetLength(str);
            const UniChar *chars = CFStringGetCharactersPtr(str);
            if (chars)
                return QString(reinterpret_cast<const QChar *>(chars), length);
    
            QVarLengthArray<UniChar> buffer(length);
            CFStringGetCharacters(str, CFRangeMake(0, length), buffer.data());
            return QString(reinterpret_cast<const QChar *>(buffer.constData()), length);
        }
        else{
            return QString();
        }
    }
Comment 11 RJVB 2014-09-03 13:34:34 UTC
(In reply to kde from comment #5)
> Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
> 0   com.apple.CoreFoundation            0x00007fff8a32ae4f CFStringGetLength
> + 15
> 1   libkdecore.5.dylib                  0x0000000100e49011
> convert_CFString_to_QString(__CFString const*) + 49


Whether or not the bug is indeed in convert_CFString_to_QString(), it might be a good idea to protect that function against null CFStringRefs:

    QString convert_CFString_to_QString(CFStringRef str) {
            if( str ){
            CFIndex length = CFStringGetLength(str);
            const UniChar *chars = CFStringGetCharactersPtr(str);
            if (chars)
                return QString(reinterpret_cast<const QChar *>(chars), length);
    
            QVarLengthArray<UniChar> buffer(length);
            CFStringGetCharacters(str, CFRangeMake(0, length), buffer.data());
            return QString(reinterpret_cast<const QChar *>(buffer.constData()), length);
        }
        else{
            return QString();
        }
    }
Comment 12 RJVB 2014-09-06 10:49:27 UTC
Created attachment 88587 [details]
protect against potential conversion of a NULL CFString

This is a "can't hurt if it ain't fix anything" patch that should prevent the kind of crash we saw in the 1 backtrace of meinproc4 crashing that we have.
If indeed it was due to a NULL CFStringRef, and not a corrupt one ...
Comment 13 Luigi Toscano 2014-09-06 21:20:18 UTC
Thanks for the patch: I would say that:
- the patch, as we know, hides some other issue deeper in the code related to KGlobal or so;
- on the other side, this is about the kdelibs4 world and I suspect you want to move to Qt5, where that specific code changed, so a specific fix should be most likely redone;
- I would suggest then to submit the patch through reviewboard, producs kdelibs, branch KDE/4.14, and see if it's worth to forward-port it to the relevant part of Frameworks (which I have no idea if it works or not).
If kdelibs4 MacOSX applications works with this patch, and given that it affects only Mac codepaths, I suspect it won't be difficult to have it in.

What do you think?
Comment 14 RJVB 2014-09-06 21:36:16 UTC
I think that the replacement function ( QCFString::toQString) already does the check my patch adds...
Comment 15 Luigi Toscano 2017-04-02 13:34:16 UTC
Is this still an issue? The question applies both to meinproc4 and meinproc5.
Comment 16 RJVB 2017-04-02 16:58:58 UTC
I've never yet seen any indication of meinproc5 crashing on Mac, and I don't have any particular patches for it either.

That said, I wouldn't actually know when it's called. Each time documentation is built?
Comment 17 Burkhard Lück 2017-04-02 17:42:39 UTC
Yes, see https://cgit.kde.org/kdoctools.git/tree/KF5DocToolsMacros.cmake
Comment 18 Ian Wadham 2017-04-04 06:23:24 UTC
See Jeremy Lavergne's original report of this bug on 2010-12-29.

AFAIK it has only ever occurred during large MacPorts builds, possibly/probably on MacPorts servers that supply binaries and certainly on servers that are used to test ports of FOSS software in general onto Apple OS X. The symptoms were death of the entire build, with the loss of builds of many KDE modules and many FOSS modules other than KDE. It would strike at random in most builds that required meinproc4 to run --- but at varying points in the build.

At the time (2010) DrKonqi was not working at all in Apple OS X (beginning with a crash in KCrash...), so no KDE crash report was available.

The solution adopted by the MacPorts developers was to make the default variant for KDE builds to be "no documentation" --- a sad state of affairs, especially for potential users of KDE software on Apple OS X, but better than having major builds crashing.

DrKonqi, KCrash and successors in KF5 have been fixed in Apple OS X now. Stress tests of meinproc4 on Apple OS X, by me and others, in our home machines, have never experienced a failure.

Conversely, AFAIK, nobody has ever managed to persuade the MacPorts developers to make "KDE with documentation" the default build, not even with versions of KDE4, meinproc4 and DrKonqi more recent than 2010. So, again AFAIK, it is not known if the original problem, reported in 2010 still exists.
Comment 19 RJVB 2017-04-04 08:00:13 UTC
Hi Ian!

>Conversely, AFAIK, nobody has ever managed to persuade the MacPorts developers
>to make "KDE with documentation" the default build, not even with versions of

Off-topic, but:
It's not uncommon at all for MacPorts ports to provide documentation through an optional variant, for package size considerations.

>KDE4, meinproc4 and DrKonqi more recent than 2010. So, again AFAIK, it is not
>known if the original problem, reported in 2010 still exists.

There are 2 questions here:
- does meinproc4 still crash on Mac
- does meinproc5 crash on Mac

It seems the answer to both questions is "apparently not" so I think this ticket can be closed. It can always be reopened if needed, and problem with meinproc5 would probably need to be addressed via a separate ticket (or a patch review if I catch and fix them myself).
Comment 20 Ian Wadham 2017-04-07 05:44:38 UTC
(In reply to RJVB from comment #19)
>> It's not uncommon at all for MacPorts ports to provide documentation through
> an optional variant, for package size considerations.

Meinproc4 provides only the end-user documentation (the handbooks and guides). It converts XML docbook format into a compressed HTML file, which is de-compressed only if and when a user requests to see that doco via KDE application-help menu. But yes, the MacPorts +docs variant does iterate through the entire dependencies list, providing doco of all types for the entire list, in a huge range of formats used by FOSS in general.

> There are 2 questions here:
> - does meinproc4 still crash on Mac
> - does meinproc5 crash on Mac
> 
> It seems the answer to both questions is "apparently not" so I think this
> ticket can be closed.

I agree that this bug report can be closed, perhaps with reason WORKSFORME.

However, that still leaves new users of KDE apps and utilities with the message "There is no documentation available for /<appname>/index.html", if they try to get the handbook for the Help menu, but that is a real turnoff for newbie users, besides making it difficult to get acquainted with some of the more sophisticated KDE4 and KF5packages. And the message  is not actually true for most KDE4 and KF5 packages, unless there has been a build failure.

May I suggest some possible workarounds in MacPorts:-

    a. Change the above message to refer the user to "https:docs.kde.org",
       where online copies of all documentation can be found nowadays.
    b. Add a similar message to the MacPorts Notes for KDE4 and KF5 builds.
    c. Introduce a variant for KDE4 and KF5 ports (eg. +userdoc) that will
       just provide end-user doco (ie. run meinproc for one package only and
       not all the dependencies).

I believe you had idea c. a while back, René, but someone knocked it on the head.
Comment 21 Burkhard Lück 2017-04-07 09:04:58 UTC
(In reply to Ian Wadham from comment #20)
> However, that still leaves new users of KDE apps and utilities with the
> message "There is no documentation available for /<appname>/index.html", if
> they try to get the handbook for the Help menu, but that is a real turnoff
> for newbie users, besides making it difficult to get acquainted with some of
> the more sophisticated KDE4 and KF5packages. And the message  is not
> actually true for most KDE4 and KF5 packages, unless there has been a build
> failure.
> 
> May I suggest some possible workarounds in MacPorts:-
> 
>     a. Change the above message to refer the user to "https:docs.kde.org",
>        where online copies of all documentation can be found nowadays.

If you build frameworks/kio with documentation you will not see the
message "There is no documentation available for /<appname>/index.html", 
but this page in khelpcenter:

https://docs.kde.org/trunk5/en/frameworks/kioslave5/help/documentationnotfound/index.html
Comment 22 RJVB 2017-04-07 11:49:36 UTC
It should be possible to perform an automatic search of the online documentation first, no? In fact, it should be possible to provide all existing handbooks online at least for projects that are part of KF5 Applications and Plasma5. Would be considerably more user-friendly!
Comment 23 Luigi Toscano 2017-04-07 11:56:03 UTC
If KHelpCenter is not installed, the "foobar Handbook" link from the Help menu of Frameworks applications redirects to docs.kde.org.

If KHelpCenter is installed, the issue is on KHelpCenter (and I think it's already reported), but anyway out of scope for this bug...