Bug 399240

Summary:	Exception thrown when fetching po for GCompris
Product:	[Developer tools] releaseme	Reporter:	Jazeix Johnny <jazeix>
Component:	general	Assignee:	Harald Sitter <sitter>
Status:	RESOLVED INTENTIONAL
Severity:	normal
Priority:	NOR
Version First Reported In:	unspecified
Target Milestone:	---
Platform:	Other
OS:	Linux
Latest Commit:		Version Fixed/Implemented In:
Sentry Crash Report:
Attachments:	patch to catch the exception

Description Jazeix Johnny 2018-09-30 11:36:06 UTC

Created attachment 115327 [details]
patch to catch the exception

SUMMARY
GCompris po files are not retrieved when using fetchpo.rb from releaseme.git.

STEPS TO REPRODUCE
1. git clone --depth 1 https://anongit.kde.org/releaseme.git
2. ruby ./releaseme/fetchpo.rb --origin trunk --project gcompris --output-dir po --output-poqm-dir poqm gcompris/

OBSERVED RESULT
Logging colors are not available. Install logger-colors gem if desired
INFO -- ReleaseMe::L10n: Downloading translations for gcompris
#<Thread:0x00005557f3d9bb38@gcompris/build/releaseme/lib/releaseme/translationunit.rb:129 run> terminated with exception (report_on_exception is true):
gcompris/build/releaseme/lib/releaseme/l10n/asset.rb:59:in `gsub!': invalid byte sequence in US-ASCII (ArgumentError)
	from gcompris/build/releaseme/lib/releaseme/l10n/asset.rb:59:in `strip_comments!'
	from gcompris/build/releaseme/lib/releaseme/l10n/asset.rb:33:in `strip!'
	from gcompris/build/releaseme/lib/releaseme/l10n.rb:99:in `block (2 levels) in download'
	from gcompris/build/releaseme/lib/releaseme/l10n.rb:97:in `each'
	from gcompris/build/releaseme/lib/releaseme/l10n.rb:97:in `block in download'
	from gcompris/build/releaseme/lib/releaseme/translationunit.rb:148:in `block (2 levels) in each_language_with_tmpdir'
	from /usr/lib/ruby/2.5.0/tmpdir.rb:89:in `mktmpdir'
	from gcompris/build/releaseme/lib/releaseme/translationunit.rb:148:in `block in each_language_with_tmpdir'
	from gcompris/build/releaseme/lib/releaseme/translationunit.rb:131:in `block (2 levels) in blocking_thread_pool'
gcompris/build/releaseme/lib/releaseme/l10n/asset.rb:59:in `gsub!': invalid byte sequence in US-ASCII (ArgumentError)
	from gcompris/build/releaseme/lib/releaseme/l10n/asset.rb:59:in `strip_comments!'
	from gcompris/build/releaseme/lib/releaseme/l10n/asset.rb:33:in `strip!'
	from gcompris/build/releaseme/lib/releaseme/l10n.rb:99:in `block (2 levels) in download'
	from gcompris/build/releaseme/lib/releaseme/l10n.rb:97:in `each'
	from gcompris/build/releaseme/lib/releaseme/l10n.rb:97:in `block in download'
	from gcompris/build/releaseme/lib/releaseme/translationunit.rb:148:in `block (2 levels) in each_language_with_tmpdir'
	from /usr/lib/ruby/2.5.0/tmpdir.rb:89:in `mktmpdir'
	from gcompris/build/releaseme/lib/releaseme/translationunit.rb:148:in `block in each_language_with_tmpdir'
	from gcompris/build/releaseme/lib/releaseme/translationunit.rb:131:in `block (2 levels) in blocking_thread_pool'


EXPECTED RESULT
po retrieved at the good place for gcompris

SOFTWARE VERSIONS
releaseme from master

ADDITIONAL INFOS
After applying the patch, the process works but the result is not the one expected too. In the po/ folder, I have po for gcompris_voices project and in poqm/, I have po for gcompris. I would expect the po from gcompris to be in po/ folder and in the poqm/ folder, some qm files

Comment 1 Jazeix Johnny 2018-10-02 06:33:02 UTC

After discussing with Aleix and some investigation:
the issue appears because I run the command from emacs directly, not a terminal. When I run it from a terminal, with either LANG to C or fr_FR, it runs well.
The issue is probably due to a bad configuration of emacs on my side.

Now the po files are generated in the source folder, still having gcompris-voices.po in the $SOURCE/po/ folder, and gcompris-qt.po in the $SOURCE/poqm/ folder.

Is it due that there are 2 po files in https://websvn.kde.org/trunk/l10n-kf5/fr/messages/extragear-edu/ for GCompris?

Comment 2 Harald Sitter 2019-01-08 14:13:52 UTC

Sorry I had bug mails end up in spam somehow :S

The original issue is indeed a bogus setup as it's basically complaining that it cannot read unicode, which happens when LANG is not set or set incorrectly.

As for the directory line up: if I recall correctly *_qt.po always goes into poqm/ and everything else always goes into po/. The former is meant to be used via https://api.kde.org/ecm/module/ECMPoQmTools.html to generate qm files for use with qtranslator, the latter via ki18n_install to generate mo files for use with ki18n/klocalizedstring.

If you expect both translations to be used as qm translations via qtranslator, they both need to not have a _qt suffix and will then end up in poqm/.
If they both should be used as gettext translations they both need to not have the suffix and will then end up in po/.
If they in fact are split between qtranslator and ki18n, then you'll simply want to use ECMPoQmTools and ki18n_install respectively.

Comment 3 Jazeix Johnny 2019-01-22 20:33:43 UTC

Thank you for the explanation.
We can't rename the gcompris_qt.po for now, so I'll workaround it and when we'll have time, we'll fix the name.