Bug 405422 - ksudoku live-git (commit 09814312d) small-game generation fails with "Unable to generate a puzzle of the chosen variant"
Summary: ksudoku live-git (commit 09814312d) small-game generation fails with "Unable ...
Status: RESOLVED FIXED
Alias: None
Product: ksudoku
Classification: Applications
Component: general (show other bugs)
Version: unspecified
Platform: Gentoo Packages Linux
: NOR normal
Target Milestone: ---
Assignee: Ian Wadham
URL:
Keywords:
: 407714 (view as bug list)
Depends on:
Blocks:
 
Reported: 2019-03-13 09:01 UTC by Duncan
Modified: 2019-05-19 10:26 UTC (History)
3 users (show)

See Also:
Latest Commit:
Version Fixed In:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Duncan 2019-03-13 09:01:49 UTC
This just started happening a few days ago.  I like a quick 6x6 game so that's my normal default, but games smaller than standard 9x9 seem to fail with "Unable to generate a puzzle of the chosen variant", now.

However, given that ksudoku has had only a single GIT_SILENT commit since Jan 11, and it worked well after Jan 11, I suspect the trigger to be an update of some dependency.

So what might trigger the above error on particularly smaller games?

(Note that I'm running live-git of nearly everything kde related, including all frameworks and plasma packages, so if it's a kde-related dep it's likely live-git, while other deps are reasonably recent, being gentoo/~amd64, aka testing.  For toolchain, glibc-2.28-r5 updated on Mar 10, I /think/ after the problem already appeared, and gcc-3.3.0 updated on Feb 24, I believe before the problem altho I wouldn't have rebuilt kde packages until they had a commit, and qt-5.12.1 on Feb 14 as parts of live-git plasma require qt-5.12 now...)

Also, out-of-routine I recently tried a big 25x25 which took me a couple of days  to solve (BTW, the timer apparently wrapped at some point as it said only a few hours, but that'd be a different bug I've not actually filed yet, is it known or should I file a bug on it).  AFAIK the 6x6 games were working before that, and broken very soon after, but I tried deleting (well, renaming) the ksudokurc file and it didn't solve the problem, so if it's corrupted state it must be elsewhere.
Comment 1 Ian Wadham 2019-03-14 05:06:33 UTC
I used to be maintainer/developer for KSudoku, but am out of touch with it now. However I will try to help narrow this problem down.

The message you are getting seems to be from the "welcome" screen (where you choose a puzzle type). It was added to the code in April 2017, see https://phabricator.kde.org/D5671.

Do you stay on the welcome screen after you see the message?  Please confirm.

Please can you give me a precise list of the names of the puzzle types for which you get the message. For example, which "6x6" are you using, the one with rectangular blocks or the Mathdoku - Settable Size with size set to 6x6? Or maybe both?

You say that games smaller than 9x9 seem to fail, but what about variant (non-Classic) games of size 9x9, such as Killer Sudoku, Aztec, Jigsaw or even Samurai?

Your comments about compiler and Qt changes may also be a clue. There is some very old code in KSudoku (2007-08 vintage) that I have never dared touch. Perhaps some of that has been rendered invalid or works differently now.

I regret that I cannot actually build and test the latest KSudoku code, because I now work on an Apple Macbook and am unable to build KF5 and Qt5.

BTW, it does not surprise me that the clock wraps around, but I think you might find Samurai (five 9x9 grids) more fun than a 25x25... :-)
Comment 2 Duncan 2019-03-14 06:40:32 UTC
(In reply to Ian Wadham from comment #1)
> I used to be maintainer/developer for KSudoku, but am out of touch with it
> now. However I will try to help narrow this problem down.

Thanks.

> The message you are getting seems to be from the "welcome" screen (where you
> choose a puzzle type). It was added to the code in April 2017, see
> https://phabricator.kde.org/D5671.

I did grep the error in the source, trying to see if I as a non-coder but reasonably experienced gentoo user/admin who routinely runs live-git of selected packages, reading git logs and sometimes generating patches on my own (I had some basic and pascal classes back in the day and back before I left MS behind did intermediate visual basic, but only really know bash on Linux), could figure out what was triggering it, but no real luck beyond finding out it was in the welcome-screen code, as you mention.  I'm not familiar with the (from memory) color-coding generation as mentioned in the comments, so saw the numbers lists associated with some of the games but don't understand the generation mechanism and was too tired to reason it further, so left it at that.

> Do you stay on the welcome screen after you see the message?  Please confirm.

Yes.  I stay on the welcome (aka game chooser) screen.  The message is a popup asking me to choose another game, so staying on the chooser screen is to be expected.

> Please can you give me a precise list of the names of the puzzle types for
> which you get the message. For example, which "6x6" are you using, the one
> with rectangular blocks or the Mathdoku - Settable Size with size set to
> 6x6? Or maybe both?
> 
> You say that games smaller than 9x9 seem to fail, but what about variant
> (non-Classic) games of size 9x9, such as Killer Sudoku, Aztec, Jigsaw or
> even Samurai?

Looks like it's not just size; almost everything fails.  These are the only ones that don't generate the error:

Standard-9x9, Roxdoku-9-3x3x3, Sudoku-16x16, Sudoku-25x25, Roxdoku-16-4x4x4, Roxdoku-25-5x5x5.

I don't believe I had tried Mathdoku-settable-size so didn't know how the size was set, but it appears to be an option in the ksudoku config.  Neither the existing size of 6, nor the maximum 9 nor minimum 3, would give me a game, only the now familiar error.

> Your comments about compiler and Qt changes may also be a clue. There is
> some very old code in KSudoku (2007-08 vintage) that I have never dared
> touch. Perhaps some of that has been rendered invalid or works differently
> now.

Before investigation I suspected it might be something like an xml-parser update breaking parsing of whatever generator rules files were used, but the files don't seem to be XML-based so that's out.  But whatever it was that changed, is obviously now treating perfectly valid "rules files" (for lack of a better term at this point) as invalid.  I checked for file corruption, etc, too, and came up empty.

> I regret that I cannot actually build and test the latest KSudoku code,
> because I now work on an Apple Macbook and am unable to build KF5 and Qt5.
> 
> BTW, it does not surprise me that the clock wraps around, but I think you
> might find Samurai (five 9x9 grids) more fun than a 25x25... :-)

The 25x25 was an accidental click (touchpad...), that I decided to actually try once it was generated and displayed.  Fine to do once to say I did it but I doubt I'll do it again, especially now that it's getting the negative association of this bug, tho it most likely had nothing to do with it and that's only chance.  But it's a negative association, none-the-less, and those don't have to be logical to be powerful...

Anyway, For that sort of longer puzzle I'd prefer a larger size Palapeli/jigsaw, and have done ~900 pieces on it before, as my family used to do jigsaws and it takes me back.  My main 65-inch 4K TV-as-monitor is about the size of the boards we'd put the puzzles on, too. =:^)

I do I believe it's ksudoku windmill (unfortunately can't confirm ATM...) occasionally tho and like it.  Samurai looks to be similar.  I'll check it (and Sohei) out when I get things working again.  Thanks.  =:^)
Comment 3 Ian Wadham 2019-03-15 02:59:38 UTC
Thank you very much for helping narrow this down, Duncan.

The problem is that the KSudoku code is not able to create the data-structures used by ANY of the "Custom" puzzles (those that are not vanilla Sudoku squares or Roxdoku cubes). Consequently it is unable to go on and fill the structures with data for a puzzle to be solved and then display the puzzle. So it remains in the Welcome screen code (src/gui/welcomescreen.*), where the user can choose another type of puzzle.

The failure is detected in Game CustomGame::createGame(...) and bool CustomGame::createSKGraphObject() (both in src/gui/gamevariants.cpp). I strongly suspect that the error occurs somewhere in the serializer code (src/gui/serializer.*), as called by the line:
    m_graph = ksudoku::Serializer::loadCustomShape(m_url, 0, errorMsg);
in CustomGame::createSKGraphObject().

In other words, any puzzle type that is constructed by one of the XML files in src/shapes fails to be created. These puzzle types are a majority of those available in KSudoku, so the problem is catastrophic.

The relevant KSudoku code has not changed recently, so it looks as though some recent change (in the last month or two) in XML handling, in Qt5 or in the KF5 environment has brought on this problem.
Comment 4 Albert Astals Cid 2019-03-15 23:32:36 UTC
I can generate any game just fine using the current git version of ksudoku + Qt 5.12.1 + KF5 5.56
Comment 5 Ian Wadham 2019-03-17 02:14:52 UTC
Thanks very much, Albert, for taking the time to check up on this.

It looks as if the ball is back in your court, Duncan. Maybe something has gone wrong with your build and compile of KDE software. Looks like you have the same version of Qt5 as current KDE development, but do you have the right KF5?

Albert is an authority on all this. He is a leading member of the kde-games-devel mailing list and also a KDE Release Team member and there is a release of KDE Applications 19.04 (including KSudoku) coming up on 18th April. The branch that is due for release is here:
      https://cgit.kde.org/ksudoku.git/tree//?h=Applications%2F19.04

Instructions for cloning the repository are at https://kde.org/applications/games/ksudoku/development.

If anything at all is wrong and there is no problem in your environment, we would want to know. Please feel free to re-open this bug report if necessary.
Comment 6 Jeremy Whiting 2019-03-20 02:26:13 UTC
Git commit 40e80d73866634c954dce212f2da43cd0fdce8d6 by Jeremy Whiting.
Committed on 20/03/2019 at 02:25.
Pushed by whiting into branch '405422'.

Don't use KIO copy and QTemporaryFile to load xml definition files.

Since files are usually (maybe always?) local anyway, there's not
much reason to copy the file to a temporary file.
This also fixes QDomDocument::setContent failing on an empty file
when the QTemporaryFile is somehow empty after the copy has finished.

Reviewed by: Ian Wadham <iandw.au@gmail.com>

M  +7    -12   src/gui/serializer.cpp

https://commits.kde.org/ksudoku/40e80d73866634c954dce212f2da43cd0fdce8d6
Comment 7 Duncan 2019-03-20 09:56:02 UTC
Just a couple notes for the moment, pending testing of the fix, since the bug has been traced to qt5's tmpfile handling:

When I first noticed the bug and when reported, I was on qt 5.12.1.  Since then I upgraded to 5.12.2.  Unfortunately that didn't solve the problem.

On my system, both system and user tmp are tmpfs, so the copy from installed to a tmpfile is cross-filesystem.  It's possible that may be the complicating factor that explains why I have the bug while others on qt 5.12.x may not.  (Apparently one tester had a single filesystem containing both the installation and tmp, so it wasn't a cross-filesystem copy for him.)

It's now my weekend and I should be doing routine system and git-build updates later today or tomorrow.  If the potential fix isn't on master before that I plan on switching branches and verifying the fix one way or the other, so we should have confirmation then.
Comment 8 Christoph Feck 2019-03-20 10:40:09 UTC
Not fixed unless someone merges the branch to any branch that ends up in releases.
Comment 9 Jeremy Whiting 2019-03-20 16:04:59 UTC
Git commit 3ecebabf44eb1ace3859eafafe93534ab0d3f986 by Jeremy Whiting.
Committed on 20/03/2019 at 16:04.
Pushed by whiting into branch 'Applications/19.04'.

Don't use KIO copy and QTemporaryFile to load xml definition files.

Since files are usually (maybe always?) local anyway, there's not
much reason to copy the file to a temporary file.
This also fixes QDomDocument::setContent failing on an empty file
when the QTemporaryFile is somehow empty after the copy has finished.

Reviewed by: Ian Wadham <iandw.au@gmail.com>

(cherry picked from commit 40e80d73866634c954dce212f2da43cd0fdce8d6)

M  +7    -12   src/gui/serializer.cpp

https://commits.kde.org/ksudoku/3ecebabf44eb1ace3859eafafe93534ab0d3f986
Comment 10 Duncan 2019-03-22 08:55:45 UTC
(In reply to Jeremy Whiting from comment #6)
> Git commit 40e80d73866634c954dce212f2da43cd0fdce8d6 by Jeremy Whiting.
> 
> Don't use KIO copy and QTemporaryFile to load xml definition files.

Confirmed: With this on master now, all games seem to work again.

Thanks. =:^)
Comment 11 Albert Astals Cid 2019-05-19 10:26:03 UTC
*** Bug 407714 has been marked as a duplicate of this bug. ***