*** Bug 210875 has been marked as a duplicate of this bug. ***

This could be an issue with libc, and/or caused by some update. Did you had some package update lately?
It would be useful if you could recognize the moment this crash is happening; it is related to the time dataengine (the one that the Plasma clocks use..) so it may be related to some specific time.

A valgrind log could also be useful, but that is more advance task to achieve:
http://techbase.kde.org/Development/Tutorials/Debugging/How_to_create_useful_crash_reports#Retrieving_a_backtrace_with_Valgrind

Thanks.

Waiting for feedback.

*** Bug 228085 has been marked as a duplicate of this bug. ***

*** Bug 241122 has been marked as a duplicate of this bug. ***

I have a valgrind run, it shows the problem as being under ::tzset() in glibc.  See my comments on 241122.  I believe this bug can be closed.  Here's the relevant portion of the valgrind when running on glibc-2.7:

==15150== Invalid free() / delete / delete[]
==15150==    at 0x4B1FB2E: free (vg_replace_malloc.c:323)
==15150==    by 0xAFD57BA: (within /lib64/libc-2.7.so)
==15150==    by 0xAFD5388: (within /lib64/libc-2.7.so)
==15150==    by 0x491C508: _vgnU_freeres (vg_preloaded.c:60)
==15150==    by 0xAF47EBD: _IO_file_read (fileops.c:1199)
==15150==    by 0xAF48E47: _IO_file_underflow@@GLIBC_2.2.5 (fileops.c:596)
==15150==    by 0xAF479F7: _IO_file_xsgetn (fileops.c:1435)
==15150==    by 0xAF46FDD: fread_unlocked (iofread_u.c:45)
==15150==    by 0xAF66AC6: __tzfile_read (tzfile.c:205)
==15150==    by 0xAF65D59: tzset_internal (tzset.c:422)
==15150==    by 0xAF6607F: tzset (tzset.c:578)
==15150==    by 0x8CBEE2F: KSystemTimeZoneBackend::offset(KTimeZone const*, long
) const (ksystemtimezone.cpp:642)
==15150==  Address 0x4050800 is not stack'd, malloc'd or (recently) free'd
==15150==

*** Bug 241296 has been marked as a duplicate of this bug. ***

*** Bug 241308 has been marked as a duplicate of this bug. ***

Happens in glibc 2.12 as well.

Just some more information. Does not happen in 4.4, only in 4.5 beta 2 with me. Both using the same glibc 2.12. 

Original bug post 241308

@David: it probably happens for you in 4.5 but not 4.4 because kholidays has been significantly changed between those two releases (for the better :) and is now probably tripping the same bug that the time engine was in this report.

@Aaron Has it been upstreamed? Or should I post a bug on the glibc bugzilla?

@David: i really have no idea. sorry :( no time to track upstreams these days.. too many of them and too few of me :)

I'm building a KDE checkout now, I'll valgrind the plasma-desktop binary again once it's ready, against my fresh new glibc-2.11.  If there's still badness going on underneath there, even if it no longer results in a crash at logout, I'll do some debugging, check the glibc-git tree, and submit a glibc bug with my findings if it's not yet addressed.

Well, I can't reproduce the crash with revision 1136714.  Valgrind on plasma-desktop does not now show a bad free underneath tzset().  That doesn't mean that glibc doesn't still have the bug, but I can't reproduce the crashes anymore, using that revision of KDE and glibc-2.11.

Yes, it seems not to be related to the old one. 

I valgrinded it and got the following attachment. I also commented out ~HolidayParserDriverPlan() in holidayparserdriverplan.cpp to test if something was being destroyed that should not be and everything was fine. 

Seemed a bit odd so uncommented it and made a new account and tried it. New account seems fine so must be a config/data file that is effecting it in .kde4.

I shall run tests tomorrow to find out what is messing it around and then find out why!

Created attachment 47872 [details]
Valgrind log

*** Bug 241319 has been marked as a duplicate of this bug. ***

Just stumbled across this bug by chance, not sure how it got connected back
from the KHolidays bugs and crashes?  Is it something deeper that KHolidays is
triggering?  Can someone elaborate?

I just started to see these random crashes in KHolidays today and wondered what
I'd messed up.  There's several reports getting logged.  What I'm doing to
trigger it is in the Calendar Plasmoid I can change the Holiday Region as much
as I like and it's happy, but as soon as I change the Calendar System then it
crashes in the scannerReset() call, which is just a call to glibc rewind().

I'm running glibc 2.10.1 on openSuse 11.2 x64.

Note that the lpp/ypp files are run through bison and flex to generate code.

Anything I can do to help further?

If we need to, I can disable the new holiday parser and fall back to the old
parser and hope it doesn't trigger the bug.  We loose the alternative calendar
systems and a few new calculation features like that, but we still keep the new
api and metadata so no apps would be affected.

@John probably better to keep it in so we can find out where this problem lies. I am going to go on a mass debug session tomorrow with this as I can duplicate it by moving my old .kde4 folder over and logging in.

OK, thanks.  But at least we have a fall-back if we can't figure it out or can't otherwise work-around it.  I'll go over the holidays code anyway just to make sure 

Good luck!

I've found if I don't cache the HolidayRegion object in the plasma calendar DataEngine but instead delete it immediately after using it then my crashes go away.  I'd guess this is because the file isn't kept open for ages waiting for something else to mess it up.  It's not a fix, but it's a better work-around than reverting to the old parser.

Created attachment 47913 [details]
New crash information added by DrKonqi

plasma-desktop (0.3) on KDE Platform 4.4.85 (KDE 4.4.85 (KDE 4.5 Beta2)) using Qt 4.7.0

- What I was doing when the application crashed:

As in the bug report, plasma crashed during logout.

-- Backtrace (Reduced):
#10 0xb601d2dd in __libc_message (do_abort=2, fmt=0xb60dc298 "*** glibc detected *** %s: %s: 0x%s ***\n") at ../sysdeps/unix/sysv/linux/libc_fatal.c:170
#11 0xb6023141 in malloc_printerr (action=<value optimized out>, str=0x6 <Address 0x6 out of bounds>, ptr=0xa538b80) at malloc.c:6197
[...]
#13 0xb6013252 in _IO_new_fclose (fp=0xa538b80) at iofclose.c:88
#14 0xaa055e41 in KHolidays::HolidayParserDriverPlan::scannerTerminate (this=0xa538b10) at holidayscannerplan.lpp:311
#15 0xaa0468b7 in ~HolidayParserDriverPlan (this=0xa538b10, __in_chrg=<value optimized out>)
    at /home/porttmp/portage/kde-base/kdepimlibs-4.4.85/work/kdepimlibs-4.4.85/kholidays/parsers/plan2/holidayparserdriverplan.cpp:61

NEW'ing and re-assigning to kdepimlibs then. thanks for all the follow ups!

Seems sometimes the file does not open and when you do rewind or terminate it crashes. Attached file solved the issue for me, at least I think it did.. was a nightmare to reproduce!

Created attachment 47916 [details]
KHols Patch file

How this works by the way I have no idea, as if you delete a null pointer nothing happens ( should not crash ) and rewind should work with a null pointer. 

So I would take this with a "Dunno, probably not fixed". For the debugging I did it seems that the deconstructor of HolidayParserDriverPlan is being called twice. 

How that is possible I don't know. I am unable to replicate the crash at the moment so if it pops it head again I shall give it another go!

Thanks for the efforts David.  I'll apply the patch to the lpp and regenerate the code to see if it fixes the problem here.

I'm still a little mystified as to how the KHolidays bug has been linked back
to this bug, could someone explain?  The traces don't look anything alike to
me.  Are we sure the KHoliday crash is caused by the possible glibc bug in
tzset()?  I need to understand so I can try figure out what underlying issue in KHolidays triggers it.  I get the crash when changing calendar system in the plasmoid, which could conceivably be causing a call to tzset somewhere, but I'm sceptical that it or the other 2 crash scenario's I've seen reported would ever need to touch timezones.  Are we sure it's not just KHolidays loosing track of open/closes?

We will need to set a deadline for solving this before falling back to one of
the work-arounds.  RC1 certainly won't go out with this still in it, and I'd
prefer it earlier to confirm the work-around doesn't just move the problem
elsewhere.

Thanks!

As far as I can see from the initial valgrind log and the valgrind log of this issue I attached is that they are separate. It looks as the initial one was fixed by upgrading glibc, the latter issue happens on glibc2.12 which is verified to solve the initial problem. 

The latter issue looks like something deleting something twice or deleting something that is not referenced. It happens from ~HolidayParserDriverPlan() as far as I can tell.

Addressing the question in comment 28 "Are we sure the KHoliday crash is caused by the possible glibc bug in tzset()?"

The nasty thing about the tzset() bug was that it corrupted the malloc() freespace by free-ing something that had never come from malloc.  This does not necessarily result in an immediate crash.  Instead, things hum along nicely, mallocing and freeing, until some time later one of those operations causes things to explode because the memory allocator can't resolve its innards.  In the specific case I was watching, the eventual crash occurred under fclose() when the library was tearing down the FILE* object, but the damage had been done long before that.

My valgrinding with glibc-2.11 did not show this invalid free() call under the same conditions, with the same binaries.  This suggests that either the bug under tzset() was fixed, or that something in the new glibc setup modified the way tzset() runs behind the scenes in such a way as to avoid triggering the bug.

So, in my specific case (refer to my comments in bug 241122), upgrading the glibc made this problem go away.

I get the feeling David, Christopher and I are seeing 2 or 3 different bugs.

I've tried David's patch, but it doesn't resolve my instance of the crash as the rewind is happening on a yyin that has a value, just one that is now invalid as closed.  However it did lead me to put tracing on the yyin value to see what happens to it, and it is now clear that some kind of yyin confusion is happening.  Creating the plasmoid I'm seeing the original holiday file opened and rewound OK with the original yyin, then when the config dialog is displayed each file is read for its metadata resulting in a series of fopen/rewind/fclose on a new yyin value each time.  However once the config is closed the plasmoid tries to request new holidays from it's original HolidayRegion instance it doesn't try rewind the original yyin as it should, instead the last yyin from the metadata scan is used which has been closed.  The original yyin is never reported as closed.

So this has nothing directly to do with the calendar system change, it just forces a request for updated holidays, not changing anything and simply navigating to a new month triggers a new request and thus a crash as well.  Changing the holiday region doesn't trigger the crash as a new holiday region is created with a new valid yyin.  This also explains why my work-around of not caching the region solved the problem, as it forced a new yyin.

I'll press on in the morning trying to find why the wrong yyin is used.

John:  as a general safety rule, and something that might show you where things are going wrong, I'd suggest setting yyin to NULL after it is closed.  Anybody who tries to read, rewind, or fclose on a NULL pointer should segfault immediately, and give you debugging clues in the backtrace.

Note that, contrary to a point raised in comment 27, there's no guarantee that rewind(NULL) will work.  In glibc-2.11, based on a quick glance at the code, it'll segfault.

I managed to replicate the bug and squash it.. it comes down to FILE descriptors ( in fact the whole C fopen,fclose ) system not being c++ safe. 

Attached is a patch that fixes the problem

Created attachment 47921 [details]
True KHols fix

OK, I didn't like the first patch, which looked to be obscuring symptoms rather than fixing a bug.  The second patch looks OK to me.  Best would be if there were some way to ensure that yyin was invalid before scannerInitialise() and after scannerTerminate(), so that we're sure the lexer isn't churning away on nonsense.  I'd have a method that sets yyin to NULL, one that is called in the HolidayParserDriverPlan constructor, and also after fclose() in scannerTerminate().  Just my opinion, you're free to ignore it.

More info in-case anyone is interested. 

What happens is that when you have a file open and are keeping it open, lets say the region you have set the calender to and then look for all the calenders available.. 

looking for all the calenders you open and close every file, closing the file the other task is using.. it does not like this. 

@Christopher Its better to have a controlled situation than a toggle check IMO :)

Thanks guys!  I was beginning to suspect the flex code was the problem.  Opening/closing the file each time the app requests holidays does seem a little inefficient, but at least the metadata is cached so it's not entirely pointless to keep the HolidayRegion cached.  I'll try work up a patch that allows us to keep the file open by storing the file pointer in a variable we control in a c++ safe way, but if that doesn't work I'll use David's with some validity checking baked in.

Root cause: while Bison is generating proper C++ code using the lalr1.cc template, Flex is only faking it, it's only C and not reentrant.  It seems it is possible to force flex to generate proper C++ that properly encapsulates the lexer state and is reentrant so I'll look at doing that.

Pleasure, it was nice code to work with. :) Looking forward to using this in 4.5

John:  I've written flex code for C++ with real objects.  You can create multiple classes each with its own scanner, and multiple instances of a class, each with its own input, unlike vanilla flex that puts its state into globals.

It's a bit involved to paste into a bugzilla entry.  I've got sample code if you'd like to go over it in email.  I've added your address to the whitelist of addresses allowed to send to my kdebugs email, which normally rejects as spam anything not coming from @kde.org.

Christopher:  Expert help would be most welcome :-)  I'm currently working off http://idlebox.net/2007/flex-bison-cpp-example/ which uses the FlexLexer.h class and std classes and streams, but if you have a more Qt-friendly example wrapping it that would be better.  We'll take this offline now while we work things out.

SVN commit 1137866 by jlayt:

Fix crasher in KHolidays where file descriptors were getting confused.

Done using two main changes:
1) Use Flex in C++ mode which makes the code fully re-entrant
2) Load the holiday file using QFile into a QByteArray so the file is not kept
   open.  This will increase memory usage, but save many file reads.

Note most of this is generated code, and also includes another standard header
from Flex.

Some improvements still to come, but this is a good baseline to tweak from.

Thanks to David and Christopher for their help in sorting this one out.

CCBUG: 210769



 A             FlexLexer.h   [License: GENERATED FILE]
 M  +20 -7     holidayparserdriverplan.cpp  
 M  +12 -17    holidayparserdriverplan_p.h  
 M  +104 -96   holidayparserplan.cpp  
 M  +7 -10     holidayparserplan.hpp  
 M  +49 -20    holidayparserplan.ypp  
 M  +522 -662  holidayscannerplan.cpp  
 M  +73 -19    holidayscannerplan.lpp  
 A             holidayscannerplan_p.h   [License: Public Domain LGPL (v2+)]


WebSVN link: http://websvn.kde.org/?view=rev&revision=1137866