I have been using Tellico as an address database for years. Current stock: almost 4500 addresses. As part of a system upgrade from Ubuntu Mate 20.04 to 22.04 Tellico thoroughly corrupted the database file. Special letters of different languages (e.g. German: ä, ü, ö / Spanish: ñ / French and Portuguese: ç / etc.) were eliminated. If corresponding places are corrected manually - i.e. the characters are inserted manually - they are preserved. If the corrupt tc-file is exchanged for the original one (backup), it remains undamaged when used with the graphical user interface, but the characters are still not displayed there. An attempt with a live Ubuntu system leads to the same results. A serious error, because the database is no longer usable and and thus Tellico not.
Can you let me know what versions of Tellico you're coming from and going to? Ubuntu 22.04 is Tellico 3.4.4? And to be clear, if you simply open the file with the newer version of Tellico and then save it again, the characters are corrupted? How big is the backup file? Is it small enough to email to me privately, if you didn't mind? That would let me debug directly.
Yes, it is version 3.4.4, previously it was 3.2.3 (Ubuntu 20.04). The database was irreversibly damaged during the migration to 22.04 and the undamaged (backup) can be opened with 3.4.4. It remains undamaged even when you exit Tellico, but the graphical interface shows none of the not accepted characters. Special feature: After inserting such characters (for example, new records or correction of existing ones) they remain and are also displayed. The problem also affects the field names. The backup file is small, but please understand that I do not want to send a file with about 4500 sensitive data. I even suspect that such a thing is not legally permissible for data protection reasons.
(In reply to spots4as from comment #2) > Yes, it is version 3.4.4, previously it was 3.2.3 (Ubuntu 20.04). > > The database was irreversibly damaged during the migration to 22.04 and the > undamaged (backup) can be opened with 3.4.4. It remains undamaged even when > you exit Tellico, but the graphical interface shows none of the not accepted > characters. Special feature: After inserting such characters (for example, > new records or correction of existing ones) they remain and are also > displayed. The problem also affects the field names. Sorry, I can't quite follow. So the file/database is not corrupted or damaged when you open it with Tellico 3.4.4 and then save/exit. It is only a display problem in the view? I'm not sure if the underlying migration to a newer version of Ubuntu matters or not, its hard to tell. Tellico 3.4.4 did fix a problem that had been introduced in 3.4.1 with some characters in the XML, like emoji, but from what I can tell, that shouldn't have affected the characters you mention. I haven't had any other bug reports similar to yours and I know accented characters have wide usage, so I'm struggling to figure out the problem. If you save a copy of the file, open and delete all the entries except one that shows the problem, would that be possible to share just with me by email? Maybe choose an entry that wouldn't have privacy concerns?
The database is definitely corrupted during the upgrade (all affected characters are removed/deleted). If I replace it with a backup, it can be opened. The affected characters are not displayed. When closing Tellico in this cas without or after previous editing, it remains undamaged. So it is also - but not only - a display problem. I will send you some files by mail and explain what it is about. The fact that there is no other, related bug report so far, does not necessarily allow conclusions: My scenario is special, because firstly I use Tellico (KDE application) with the Ubuntu Mate desktop, and secondly as address database not in the standard language English. Here, anyway, I have the problem both on 2 PCs and a notebook, and in a test with an Ubuntu Mate live system. So in the last case it is not even an upgrade. It is tantamount to a new installation.
Hi Robby, I sent you an email with material last Saturday (to the address robby@periapsis.org). Did you get it? Am Fri, 11 Nov 2022 21:41:03 +0000 schrieb "Robby Stephenson" <bugzilla_noreply@kde.org>: > https://bugs.kde.org/show_bug.cgi?id=461700 > > --- Comment #3 from Robby Stephenson <robby@periapsis.org> --- > (In reply to spots4as from comment #2) > > Yes, it is version 3.4.4, previously it was 3.2.3 (Ubuntu 20.04). > > > > The database was irreversibly damaged during the migration to 22.04 > > and the undamaged (backup) can be opened with 3.4.4. It remains > > undamaged even when you exit Tellico, but the graphical interface > > shows none of the not accepted characters. Special feature: After > > inserting such characters (for example, new records or correction > > of existing ones) they remain and are also displayed. The problem > > also affects the field names. > > Sorry, I can't quite follow. So the file/database is not corrupted or > damaged when you open it with Tellico 3.4.4 and then save/exit. It is > only a display problem in the view? I'm not sure if the underlying > migration to a newer version of Ubuntu matters or not, its hard to > tell. > > Tellico 3.4.4 did fix a problem that had been introduced in 3.4.1 > with some characters in the XML, like emoji, but from what I can > tell, that shouldn't have affected the characters you mention. I > haven't had any other bug reports similar to yours and I know > accented characters have wide usage, so I'm struggling to figure out > the problem. > > If you save a copy of the file, open and delete all the entries > except one that shows the problem, would that be possible to share > just with me by email? Maybe choose an entry that wouldn't have > privacy concerns? >
(In reply to spots4as from comment #5) > I sent you an email with material last Saturday. Did you get it? Yes, thank you for sending the screenshots. I'm still trying to reproduce the issue, no luck so far.
An additional, perhaps important piece of information: If the xml file is no longer than 90449 lines (equivalent to 4060 records), everything works properly. Both the database and the GUI remain undamaged.From 4061 records - from 90450 lines - the case of damage occurs.
Another update: 1. An attempt to reduce the data records to less than 4061 via the user interface was unsuccessful: The database (i.e. Adressen.tc or tellico.xml) with 4040 data records was then damaged in the way originally shown. A test showed that a single edit (e.g. adding an entry) was enough to generate the total loss. 2. Another attempt: reduction of the number of datasets with Focal and Tellico 3.2.3 to less than 4061 and use with Jammy / Tellico 3.4.4. Result as with a much larger number: Corresponding data records are displayed garbled on the user interface. The database itself (*.tc or *.xml) is retained if it is not edited on the user interface before exiting. The effect of a correct display described in my previous information could only be achieved by editing the original tellico.xml and reducing it to the lines and data sets mentioned. So it shouldn't just be a bug that becomes apparent when a maximum number of records is exceeded, but something more serious.
Last additional information for the time being: I have now - after adding Focal to the sources - forced the installation of Tellico 3.2.3. It's not a nice solution in terms of future package updates, but at least I can continue to use the database this way for now. I'm now waiting to see if - and if so: when - the bug will be fixed. If not, I'll have to switch to a different database solution.
(In reply to spots4as from comment #9) > Last additional information for the time being: I have now - after adding > Focal to the sources - forced the installation of Tellico 3.2.3. Since you've got a working version, are you able to use Tellico 3.2.3 to remove all the entries except the example one you sent me before and save to a separate file for me to test with? Since you only sent me a snippet before, it doesn't include the XML information about the character encoding which could be helpful to me if you sent me a full .tc file. So far, I'm still unable to reproduce this issue when using 3.2.3 and 3.4.4 on my own. I've used 3.2.3 to create a new database with the example you sent, then I open it with 3.4.4 and everything looks fine. SO I'm still trying hard to figure it out, just running out of things to try. Also to confirm, are there any differences in the languages settings between your previous installation and your new one? I'm trying to think of any other info you could provide since what I have is not enough to identify the issue yet. > The fact that there is no other, related bug report so far, does not necessarily allow conclusions: My scenario is special, > because firstly I use Tellico (KDE application) with the Ubuntu Mate desktop, and secondly as address database not in the standard language English. I assure you, there are plenty of people who use Tellico in a language other than English. I'm not sure about Ubuntu Mate, though.
The current state of affairs here is that I have cleaned up the database. Instead of 4334 records, it currently contains 2000. When opening it with version 3.4.4, exactly the same error occurs: it can be opened, but the "special" characters are not displayed. The file "Adressen.tc" or "tellico.xml" remains undamaged. Opening it again with the graphical user interface and then making a new entry completely damages the file. My original impression that the file would remain undamaged up to 4060 data records could only be achieved by reducing the number of lines in the tellico.xml - and only temporarily: It only remained in this state as long as no changes were made on the graphical user interface. Now I proceeded as requested. I reduced the data sets to one on the graphical user interface with version 3.2.3 (to the already known example) and opened the file "Adressen.tc" under 3.4.4. Result: The record is displayed correctly and the file "tellico.xml" remains undamaged - even after adding a new entry. So the error cannot be reproduced with only one data set. It could well be that the absolute number of records is relevant, but how should I find out? To do this, the entries on the graphical user interface would have to be successively reduced, because the limit is somewhere between 2 and 2000. The effort required to do this, however, is considerably greater than editing the tellico.xml file with an editor. When comparing the headers of the 4 tellico.xml files (3.2.3/2000x, 3.4.4/2000x, 3.4.4./1x, 3.3.4/1+1), however, some clear differences become apparent. I will send you the comparison separately by e-mail - as well as the Adressen.tc with only one data set. There are no differences in the language settings. It was a system upgrade, and the settings from 3.2.3 were taken over. In addition, I tried to switch to English once and then back to German. Didn't change anything. A connection with the Mate desktop of Ubuntu 22.04 can now be ruled out: An attempt with Kubuntu 22.04 (live) resulted in the same error. With the Mate and KDE desktop under Debian 11.05 (live), however, the problem does not occur. However, Tellico 3.3.5 is active here. So it is probably really a faulty version 3.4.4.
For almost 2 years, I have come to terms with using an older Tellico version (3.2.3). After upgrading from Ubuntu Mate 22.04 to 24.04 and another from Debian Mate 11 to 12, I have now noticed that the problem persists even with Tellico versions 3.5.3 (Ubuntu) and 3.4.6 (Debian). When importing data, all letters with special characters are eliminated; when writing new data records or correcting existing ones, they are accepted.
(In reply to spots4as from comment #12) > For almost 2 years, I have come to terms with using an older Tellico version > (3.2.3). After upgrading from Ubuntu Mate 22.04 to 24.04 and another from > Debian Mate 11 to 12, I have now noticed that the problem persists even with > Tellico versions 3.5.3 (Ubuntu) and 3.4.6 (Debian). When importing data, all > letters with special characters are eliminated; when writing new data > records or correcting existing ones, they are accepted. I'm very sorry that I've been unable to determine what is causing this bug and loss of data. I've been unable to recreate the issue which has made it difficult to investigate. I appreciate the information you have provided. The only workaround that I can think to suggest is to use your old version, 3.2.3, to export to a CSV file. Then with the newer version, open the data file with just the fields, or just a single entry, since you noted that it doesn't have the encoding problem. Import the CSV file into the new file and see if that retains all the correct characters. Since new entries seem to not have the issue, when the data file is saved, perhaps the newly imported CSV data could also be saved correctly. I'll continue to investigate, but again I'm sorry that I can't get this resolved for you.
This was one of many attempts I made to solve the problem. It was just as unsuccessful as the others. With Debian Bookworm I was not really surprised because version 3.4.6 is available in the package management. So it is quite close in time to the one I reported on 2 years ago: 3.4.4 (Ubuntu Mate Jammy). But even the forced Bullseye version 3.3.5 causes problems when accessing the address database: The special characters show up here, but the program freezes and crashes after 1 minute. Only the way via the terminal works reasonably well: With the command "tellico -qwindowtitle %c %u" Tellico opens with an empty interface (strangely, the same command as a starter leads to the described crash). If "open" is clicked there, I can select, open and edit the tc file. It also contains the special characters. However, the graphical user interface is damaged. Conclusion: With Debian Bookworm, the use of Tellico no longer makes sense, because version 3.2.3 is not even available as oldstable. With Ubuntu 24.04 it would still work for the time being, because "Focal" can be inserted as a source and the version can thus be forced. For me at any rate (who in principle would like to switch to Debian Bookworm), Tellico is no longer a serious option in view of these serious problems and I will have to look for another program.
A final update: I should have found the cause of the problem and solved it. A large part of the data records were created a long time ago with MS Access. When I switched to Linux, I exported them as a csv file and later imported them with Tellico. The Windows coding was apparently preserved and processed correctly up to Tellico version 3.2.3. Since then, only UTF-8 has been accepted. After solving the encoding problem, the data records are also displayed correctly including all special characters - at least so far.
(In reply to spots4as from comment #15) > A final update: I should have found the cause of the problem and solved it. > > A large part of the data records were created a long time ago with MS > Access. When I switched to Linux, I exported them as a csv file and later > imported them with Tellico. The Windows coding was apparently preserved and > processed correctly up to Tellico version 3.2.3. Since then, only UTF-8 has > been accepted. After solving the encoding problem, the data records are also > displayed correctly including all special characters - at least so far. I'm glad you've been able to resolve the issue. Frankly, I'm not sure how the older version was able to correctly handle a file marked as utf-8 with embedded windows encoding. I hope the newer versions of Tellico work well for you.