Bug 443845 - Database won't load after update to Kubuntu 21.10/Tellico 3.4.1
Summary: Database won't load after update to Kubuntu 21.10/Tellico 3.4.1
Status: RESOLVED FIXED
Alias: None
Product: tellico
Classification: Applications
Component: general (show other bugs)
Version: 3.4.1
Platform: Kubuntu Linux
: NOR normal
Target Milestone: ---
Assignee: Robby Stephenson
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2021-10-16 15:16 UTC by Jim Sanderson
Modified: 2021-10-21 19:30 UTC (History)
0 users

See Also:
Latest Commit:
Version Fixed In: 3.4.2
Sentry Crash Report:


Attachments
attachment-2911-0.html (3.09 KB, text/html)
2021-10-21 19:30 UTC, Jim Sanderson
Details

Note You need to log in before you can comment on or make changes to this bug.
Description Jim Sanderson 2021-10-16 15:16:07 UTC
SUMMARY
Tellico file will not load after update to new version of Tellico (3.4.1).

STEPS TO REPRODUCE
1. Click on database (.tc) file or load database from File menu.
2. 
3. 

OBSERVED RESULT
Tellico is unable to load the file - Collection.tc(I18N_ARGUMENT_MISSING). Unexpected ".

EXPECTED RESULT
Load file

SOFTWARE/OS VERSIONS
Windows: 
macOS: 
Linux/KDE Plasma: Kubuntu 21.10
(available in About System)
KDE Plasma Version: 5.22.5
KDE Frameworks Version: 5.86.0
Qt Version: 5.15.2

ADDITIONAL INFORMATION
Tried on two machines.  One is a new installation, one is an update from Kubuntu 21.04.  Both with same result.  Tried loading and saving the file on a Kutuntu 21.04 machine with Tellico 3.3.5 and transferring file to 21.10 machine.  Same result.  File loads and saves fine under Tellico 3.3.5.
Comment 1 Jim Sanderson 2021-10-18 00:32:22 UTC
I got it to load under Tellico 3.4.1.  The culprit was a mal-formed XML file.
Steps to fix:

1. Extract the XML file.
2. Loaded XML file into XML editor (I used XML Copy Editor) from the Ubuntu repos.
3. Validated the file.
4. Removed offending information between tags.
5. Saved XML file.
6. Compressed as Zip file and changed .zip suffix back to .tc.

The offending information was a long paragraph copied and pasted into a paragraph field.  The underlying HTML formatting was bad.

I'm not sure why version 3.3.5 could load it and 3.4.1 could not but the file was fixable and that's what really matters.
Comment 2 Robby Stephenson 2021-10-18 00:57:52 UTC
Could you email me the data file privately (or attach it here if you're ok with that)? I'd like to figure out why the difference myself and fix it.
Comment 3 Robby Stephenson 2021-10-18 00:58:57 UTC
> 6. Compressed as Zip file and changed .zip suffix back to .tc.

Also, as an FYI, Tellico can load the XML file directly without compressing and without ax extension change.
Comment 4 Jim Sanderson 2021-10-18 21:22:26 UTC
(In reply to Robby Stephenson from comment #2)
> Could you email me the data file privately (or attach it here if you're ok
> with that)? I'd like to figure out why the difference myself and fix it.

The file is 9.6 MB (compressed).  Will your e-mail accept that?  Or would you just like the offending record?
Comment 5 Jim Sanderson 2021-10-18 21:26:05 UTC
(In reply to Robby Stephenson from comment #3)
> > 6. Compressed as Zip file and changed .zip suffix back to .tc.
> 
> Also, as an FYI, Tellico can load the XML file directly without compressing
> and without ax extension change.

Did not know that.  I'll have to try it out.
Comment 6 Robby Stephenson 2021-10-18 22:38:32 UTC
(In reply to Jim Sanderson from comment #4)
> (In reply to Robby Stephenson from comment #2)
> > Could you email me the data file privately (or attach it here if you're ok
> > with that)? I'd like to figure out why the difference myself and fix it.
> 
> The file is 9.6 MB (compressed).  Will your e-mail accept that?  Or would
> you just like the offending record?

I guess the images are included in the file? The offending record would be enough, I think. I can copy it into a Tellico 3.3 file and then see what happens in the 3.4 load.
Comment 7 Jim Sanderson 2021-10-20 04:42:44 UTC
(In reply to Robby Stephenson from comment #6)
> (In reply to Jim Sanderson from comment #4)
> > (In reply to Robby Stephenson from comment #2)
> > > Could you email me the data file privately (or attach it here if you're ok
> > > with that)? I'd like to figure out why the difference myself and fix it.
> > 
> > The file is 9.6 MB (compressed).  Will your e-mail accept that?  Or would
> > you just like the offending record?
> 
> I guess the images are included in the file? The offending record would be
> enough, I think. I can copy it into a Tellico 3.3 file and then see what
> happens in the 3.4 load.

Full file sent.
Comment 8 Robby Stephenson 2021-10-21 02:58:04 UTC
Git commit 3824db1baa75a2b6fdb1806198bdf7582ea4bf2c by Robby Stephenson.
Committed on 21/10/2021 at 02:54.
Pushed by rstephenson into branch '3.4'.

Avoid saving invalid XML characters while stripping them during loading

Using QXmlDefaultHandler in Tellico 3.3 silently allowed loading of
invalid XML characters, while the switch to QXmlStreamReader in 3.4
resulted in a data file failing to load. In the event of a parsing
error, strip invalid characters and attempt to reload.

In addition, change the DOM implementation policy to NOT save the
invalid characters in the fil to begin with.
FIXED-IN: 3.4.2

A  +14   -0    src/tests/data/bug443845.xml
M  +26   -0    src/tests/tellicoreadtest.cpp
M  +1    -0    src/tests/tellicoreadtest.h
M  +15   -0    src/translators/tellico_xml.cpp
M  +1    -0    src/translators/tellico_xml.h
M  +5    -1    src/translators/tellicoimporter.cpp
M  +5    -0    src/translators/tellicoxmlexporter.cpp

https://invent.kde.org/office/tellico/commit/3824db1baa75a2b6fdb1806198bdf7582ea4bf2c
Comment 9 Jim Sanderson 2021-10-21 19:30:17 UTC
Created attachment 142743 [details]
attachment-2911-0.html

Robby,

Thank you!  Look forward to the new version.

Jim

On Wed, Oct 20, 2021 at 7:58 PM Robby Stephenson <bugzilla_noreply@kde.org>
wrote:

> https://bugs.kde.org/show_bug.cgi?id=443845
>
> Robby Stephenson <robby@periapsis.org> changed:
>
>            What    |Removed                     |Added
>
> ----------------------------------------------------------------------------
>    Version Fixed In|                            |3.4.2
>       Latest Commit|                            |
> https://invent.kde.org/offi
>                    |
> |ce/tellico/commit/3824db1ba
>                    |
> |a75a2b6fdb1806198bdf7582ea4
>                    |                            |bf2c
>          Resolution|---                         |FIXED
>              Status|REPORTED                    |RESOLVED
>
> --- Comment #8 from Robby Stephenson <robby@periapsis.org> ---
> Git commit 3824db1baa75a2b6fdb1806198bdf7582ea4bf2c by Robby Stephenson.
> Committed on 21/10/2021 at 02:54.
> Pushed by rstephenson into branch '3.4'.
>
> Avoid saving invalid XML characters while stripping them during loading
>
> Using QXmlDefaultHandler in Tellico 3.3 silently allowed loading of
> invalid XML characters, while the switch to QXmlStreamReader in 3.4
> resulted in a data file failing to load. In the event of a parsing
> error, strip invalid characters and attempt to reload.
>
> In addition, change the DOM implementation policy to NOT save the
> invalid characters in the fil to begin with.
> FIXED-IN: 3.4.2
>
> A  +14   -0    src/tests/data/bug443845.xml
> M  +26   -0    src/tests/tellicoreadtest.cpp
> M  +1    -0    src/tests/tellicoreadtest.h
> M  +15   -0    src/translators/tellico_xml.cpp
> M  +1    -0    src/translators/tellico_xml.h
> M  +5    -1    src/translators/tellicoimporter.cpp
> M  +5    -0    src/translators/tellicoxmlexporter.cpp
>
>
> https://invent.kde.org/office/tellico/commit/3824db1baa75a2b6fdb1806198bdf7582ea4bf2c
>
> --
> You are receiving this mail because:
> You reported the bug.