Bug 472605

Summary:	lokalize creates .po in branch without a .pot in templates when editing files in trunk
Product:	[Applications] lokalize	Reporter:	Freek de Kruijf <freekdekruijf>
Component:	general	Assignee:	wojnilowicz <lukasz.wojnilowicz>
Status:	RESOLVED FIXED
Severity:	normal	CC:	aacid, lukasz.wojnilowicz, shafff
Priority:	NOR	Keywords:	testcase, triaged
Version First Reported In:	23.04.3
Target Milestone:	---
Platform:	Other
OS:	Linux
Latest Commit:	https://invent.kde.org/sdk/lokalize/-/commit/743d9baf58a334c1785122ecc8509e63a40cb4c3	Version Fixed/Implemented In:
Sentry Crash Report:
Attachments:	test-case-bug-472605 Patch to debug bug 472605

Description Freek de Kruijf 2023-07-25 08:50:38 UTC

SUMMARY
***
lokalize creates .po in branch without a .pot in templates
***

STEPS TO REPRODUCE
1.  Start lokalize with a project file containing PoBaseDir, PotBaseDir and BranchDir
2. Edit a .po file
3. 

OBSERVED RESULT
lokalize creates a .po with the same name in the BranchDir with a corresponding .pot file in that directory

EXPECTED RESULT
lokalize should not create that .po file

SOFTWARE/OS VERSIONS
Windows: 
macOS: 
Linux/KDE Plasma: 6.4.3/5.27.6
(available in About System)
KDE Plasma Version: 5.27.6
KDE Frameworks Version: 5.108.0
Qt Version: 5.15.10

ADDITIONAL INFORMATION
Questions in email:
I need more information about your setup:
1) Is there any bug report opened already? If not, it would be good to
do it.
2) Could you post your project file(s) from Lokalize? The one that you
open through the menu Project->Open project.
3) Do you use PO summit?
https://techbase.kde.org/Localization/Workflows/PO_Summit
4) Do those strange .po files in KF5 branch keeps reappearing to you, or
was it only a single occurrence?
5) Are your .po and .pot files stored on a network storage?
6) Is there any symlink directory anywhere in the path to your your
.po and .pot files?

In general, my change uses BranchDir and PotBaseDir to figure out if
respective .pot file in a branch exists and has no .po file, before it
creates the .po file. Code before me only looked for the .po file, and
if it didn't exist, it did nothing.

The code must have returned positive for the .pot
file because otherwise I pass an empty path to the .po creating function
which already accepted that path earlier, and haven't done anything
spurious.

I have a sub-folder vertalingen with a subfolder svn and in that sub-folder a sub-folder trunkkf5 in which sub-folders nl and templates with respectively one level lower the .po and .pot files.
In sub-folder vertalingen I have the project file KF5trunk.lokalize
Content:
[General]
BranchDir=svn/branchkf5/nl
GlossaryTbx=/home/freek/vertalingen/svn/trunkkf5/nl/woordenlijst.tbx
LangCode=nl
LanguageSource=Project
MailingList=kde-i18n-nl@kde.org
PoBaseDir=/home/freek/vertalingen/svn/trunkkf5/nl
PotBaseDir=svn/trunkkf5/templates
ProjectID=kde4-nl
TargetLangCode=nl

[RegExps]
WordWrap=74

Currently I have commented the line with BranchDir

In the email discussion I mentioned editing nl/messages/kstars/kstars.po and nl/messages/kphotoalbum/
kphotoalbum.po in subfolder svn/trunkkf5/ which resulted in the same files in svn/branchkf5, but these files do not have corresponding .pot files. I did not pay much attention when committing these files, but the next day I inspected the branchkf5 and noticed these strange files. That day I edited other .po files and saw these files appear in branchkf5, but now I deleted these files before adding and committing and commented out the line BranchDir in the project file.

Comment 1 wojnilowicz 2023-07-29 10:10:59 UTC

Created attachment 160600 [details]
test-case-bug-472605

I cannot reproduce your issue. Could you:
1) unpack the attached test case in "/home/freek/vertalingen" without creating any sub directory
2) open test-case-trunk.lokalize
3) open kstars.pot in the "Project Overview" tab
4) translate an entry
5) save
6) see if you have any file under "/home/freek/vertalingen/svn2/branchkf5/nl/messages/kstars"

The above works as expected (i.e. not creating any strange file) on my machine, and I modeled it after yours KF5trunk.lokalize.

BTW, what does following command return?
"ls /home/freek/vertalingen/svn/branchkf5/templates/messages/kstars"

Would you be able to compile Lokalize with a patch yourself, if we wouldn't manage to sort out this bug any other way?

Comment 2 Freek de Kruijf 2023-07-29 13:15:31 UTC

(In reply to NSLW from comment #1)
> Created attachment 160600 [details]
> test-case-bug-472605
> 
> I cannot reproduce your issue. Could you:
> 1) unpack the attached test case in "/home/freek/vertalingen" without
> creating any sub directory
> 2) open test-case-trunk.lokalize
> 3) open kstars.pot in the "Project Overview" tab
> 4) translate an entry
> 5) save

I started lokalize in ~/vertalingen with "lokalize --project test-case-trunk.lokalize", translated the two first items and closed lokalize. I got the following output:

freek@ltfdtum:~/vertalingen> lokalize --project test-case-trunk.lokalize
org.kde.lokalize: db "default139962976560832" opened 2 "nl"
org.kde.lokalize: db "standaard139962976560832" opened 83 "nl"
QStatusBar::insertWidget: Index out of range (5), appending widget
org.kde.lokalize: db "kde4-nl139962976560832" opened 0 "nl"
org.kde.lokalize: db "standaard139962976560832" opened 0 "nl"
org.kde.lokalize: indexed "/home/freek/vertalingen/svn2/trunkkf5/nl/messages/kstars/kstars.po" for word completion in 3 msecs
org.kde.lokalize: "Your names" "Freek de Kruijf" "Your names"
org.kde.lokalize: string: "Freek de Kruijf" searching for placeables in "Your names"
org.kde.lokalize: 1 "Freek de Kruijf"
org.kde.lokalize: "Your emails" "freekdekruijf@kde.nl" "Your emails"
org.kde.lokalize: string: "freekdekruijf@kde.nl" searching for placeables in "Your emails"
org.kde.lokalize: 1 "freekdekruijf@kde.nl"
QFile::remove: Empty or null file name
QFile::remove: Empty or null file name
org.kde.lokalize: MainWindow destroyed
freek@ltfdtum:~/vertalingen> ls "/home/freek/vertalingen/svn2/branchkf5/nl/messages/kstars"
kstars.po
freek@ltfdtum:~/vertalingen> ls "/home/freek/vertalingen/svn2/trunkkf5/nl/messages/kstars"
kstars.po
freek@ltfdtum:~/vertalingen>

As you can see I do get a file in .../svn2/branchkf5/nl/messages/kstars

Could the content of ~/.config/lokalizerc play a role in this problem?


> 6) see if you have any file under
> "/home/freek/vertalingen/svn2/branchkf5/nl/messages/kstars"

So, yes!

> The above works as expected (i.e. not creating any strange file) on my
> machine, and I modeled it after yours KF5trunk.lokalize.
> 
> BTW, what does following command return?
> "ls /home/freek/vertalingen/svn/branchkf5/templates/messages/kstars"

freek@ltfdtum:~/vertalingen> LC_ALL=C ls /home/freek/vertalingen/svn/branchkf5/templates/messages/kstars
ls: cannot access '/home/freek/vertalingen/svn/branchkf5/templates/messages/kstars': No such file or directory

This is because I have the following content in KF5trunk.lokalize:

[General]
#BranchDir=svn/branchkf5/nl
GlossaryTbx=/home/freek/vertalingen/svn/trunkkf5/nl/woordenlijst.tbx
LangCode=nl
LanguageSource=Project
MailingList=kde-i18n-nl@kde.org
PoBaseDir=/home/freek/vertalingen/svn/trunkkf5/nl
PotBaseDir=svn/trunkkf5/templates
ProjectID=kde4-nl
TargetLangCode=nl

[RegExps]
WordWrap=76


> Would you be able to compile Lokalize with a patch yourself, if we wouldn't
> manage to sort out this bug any other way?

I can try.

Comment 3 Freek de Kruijf 2023-07-30 07:25:56 UTC

I managed to generate a new rpm of lokalize with openSUSE Build Service on my own system.

Comment 4 wojnilowicz 2023-07-30 09:27:11 UTC

Created attachment 160623 [details]
Patch to debug bug 472605

(In reply to Freek de Kruijf from comment #2)
> freek@ltfdtum:~/vertalingen> ls
> "/home/freek/vertalingen/svn2/branchkf5/nl/messages/kstars"
> kstars.po
> freek@ltfdtum:~/vertalingen> ls
> "/home/freek/vertalingen/svn2/trunkkf5/nl/messages/kstars"
> kstars.po
> freek@ltfdtum:~/vertalingen>
> 
> As you can see I do get a file in .../svn2/branchkf5/nl/messages/kstars

Strange. I have no clue what's causing it. 

@Albert:
If you want to help, and have some spare time, you could test the attached test case on your system, and see if you have this issue as well. Just don't forget to adjust PoBaseDir in test-case-trunk.lokalize.

> Could the content of ~/.config/lokalizerc play a role in this problem?

Looking at the entries that are there, I don't think so.

> > Would you be able to compile Lokalize with a patch yourself, if we wouldn't
> > manage to sort out this bug any other way?
> 
> I can try.

Ok, then please:
1) download the attached patch
2) put it in the root (there where org.kde.lokalize.appdata.xml file is) of the lokalize source code 
3) invoke "patch -p1 < 0001-Debug-bug-472605.patch" in the terminal
4) compile, and run

If the patching went correctly, you should see entries starting with "[bug 472605]" in your terminal while running the test case. Please run the test case once again and attach whole terminal output.

Comment 5 Freek de Kruijf 2023-07-30 11:51:52 UTC

(In reply to NSLW from comment #4)
> Created attachment 160623 [details]
> Patch to debug bug 472605
> 
> Ok, then please:
> 1) download the attached patch
> 2) put it in the root (there where org.kde.lokalize.appdata.xml file is) of
> the lokalize source code 
> 3) invoke "patch -p1 < 0001-Debug-bug-472605.patch" in the terminal
> 4) compile, and run
> 
> If the patching went correctly, you should see entries starting with "[bug
> 472605]" in your terminal while running the test case. Please run the test
> case once again and attach whole terminal output.

I could not use the command "patch -p1 < 0001-Debug-bug-472605.patch" in the unpacked tar.xz file. However it did the changes by hand and found that apparently your sources are newer. The line numbers in the patch file are a few lines off from the lines in my source file. I also found the text '/' and 't' in my source, where in your source QLatin1Char('/') and QLatin1Char('t') are used.
Anyway I will continue with what I have now without the use of QLatin1Char().

Comment 6 Freek de Kruijf 2023-07-30 12:35:55 UTC

Below is the otput after starting lokalize in the terminal
freek@ltfdtum:~> cd vertalingen ; lokalize --project test-case-trunk.lokalize
org.kde.lokalize: db "default139988083668672" opened 2 "nl"
org.kde.lokalize: db "standaard139988083668672" opened 22 "nl"
QStatusBar::insertWidget: Index out of range (5), appending widget
org.kde.lokalize: db "kde4-nl139988083668672" opened 0 "nl"
[bug 472605] mergeFilePath: "/hbasis/freek/vertalingen/svn2/branchkf5/nl/messages/kstars/kstars.po"
[bug 472605] Project::instance()->branchDir(): "/hbasis/freek/vertalingen/svn2/branchkf5/nl"
[bug 472605] poDir: "/home/freek/vertalingen/svn2/trunkkf5/nl"
[bug 472605] "/home/freek/vertalingen/svn2/trunkkf5/nl" is a symlink: false
[bug 472605] "/home/freek/vertalingen/svn2/trunkkf5" is a symlink: false
[bug 472605] "/home/freek/vertalingen/svn2" is a symlink: false
[bug 472605] "/home/freek/vertalingen" is a symlink: true
[bug 472605] "/home/freek" is a symlink: false
[bug 472605] "/home" is a symlink: false
[bug 472605] potDir: "/hbasis/freek/vertalingen/svn2/trunkkf5/templates"
[bug 472605] potDirRelativelyToPoDir: "../../../../../../hbasis/freek/vertalingen/svn2/trunkkf5/templates"
[bug 472605] branchDir: "/hbasis/freek/vertalingen/svn2/branchkf5/nl"
[bug 472605] branchPotDir: QDir( "/hbasis/freek/vertalingen/svn2/branchkf5/nl/../../../../../../hbasis/freek/vertalingen/svn2/trunkkf5/templates" , nameFilters = { "*" },  QDir::SortFlags( Name | IgnoreCase ) , QDir::Filters( Dirs|Files|Drives|AllEntries ) )
[bug 472605] Project::instance()->branchPotDir(): "/hbasis/freek/vertalingen/svn2/trunkkf5/templates"
[bug 472605] poDir: "/home/freek/vertalingen/svn2/trunkkf5/nl"
[bug 472605] "/home/freek/vertalingen/svn2/trunkkf5/nl" is a symlink: false
[bug 472605] "/home/freek/vertalingen/svn2/trunkkf5" is a symlink: false
[bug 472605] "/home/freek/vertalingen/svn2" is a symlink: false
[bug 472605] "/home/freek/vertalingen" is a symlink: true
[bug 472605] "/home/freek" is a symlink: false
[bug 472605] "/home" is a symlink: false
[bug 472605] potDir: "/hbasis/freek/vertalingen/svn2/trunkkf5/templates"
[bug 472605] potDirRelativelyToPoDir: "../../../../../../hbasis/freek/vertalingen/svn2/trunkkf5/templates"
[bug 472605] branchDir: "/hbasis/freek/vertalingen/svn2/branchkf5/nl"
[bug 472605] branchPotDir: QDir( "/hbasis/freek/vertalingen/svn2/branchkf5/nl/../../../../../../hbasis/freek/vertalingen/svn2/trunkkf5/templates" , nameFilters = { "*" },  QDir::SortFlags( Name | IgnoreCase ) , QDir::Filters( Dirs|Files|Drives|AllEntries ) )
[bug 472605] saidMergeFilePath: "/hbasis/freek/vertalingen/svn2/trunkkf5/templates/messages/kstars/kstars.pot"
[bug 472605] saidMergeFilePath exists
[bug 472605] mergeFilePath passed to loadFromUrl: "/hbasis/freek/vertalingen/svn2/trunkkf5/templates/messages/kstars/kstars.pot"
[bug 472605] saidMergeFilePath passed to loadFromUrl: "/hbasis/freek/vertalingen/svn2/branchkf5/nl/messages/kstars/kstars.po"
org.kde.lokalize: indexed "/home/freek/vertalingen/svn2/trunkkf5/nl/messages/kstars/kstars.po" for word completion in 4 msecs
org.kde.lokalize: "Your names" "Freek de Kruijf" "Your names"
org.kde.lokalize: string: "Freek de Kruijf" searching for placeables in "Your names"
org.kde.lokalize: 1 "Freek de Kruijf"
org.kde.lokalize: "Your emails" "freekdekruijf@kde.nl" "Your emails"
org.kde.lokalize: string: "freekdekruijf@kde.nl" searching for placeables in "Your emails"
org.kde.lokalize: 1 "freekdekruijf@kde.nl"
QFile::remove: Empty or null file name
QFile::remove: Empty or null file name
org.kde.lokalize: MainWindow destroyed
freek@ltfdtum:~/vertalingen>

and

freek@ltfdtum:~/vertalingen> ls svn2/branchkf5/nl/messages/kstars/kstars.po
svn2/branchkf5/nl/messages/kstars/kstars.po
freek@ltfdtum:~/vertalingen> ls svn2/branchkf5/templates/messages/
freek@ltfdtum:~/vertalingen>

Comment 7 wojnilowicz 2023-07-31 15:08:53 UTC

This is interesting
[bug 472605] potDirRelativelyToPoDir: "../../../../../../hbasis/freek/vertalingen/svn2/trunkkf5/templates"
It should be
[bug 472605] potDirRelativelyToPoDir: "../templates"
It zeroes your branchDir and puts your potDir in place. Maybe it's so because you have a symlink in your path. I'll try to post the solution soon. Good job. Thanks.

BTW What is "hbasis"? I expected to see "home" instead.

Comment 8 Freek de Kruijf 2023-07-31 15:24:32 UTC

(In reply to NSLW from comment #7)
> BTW What is "hbasis"? I expected to see "home" instead.

I have other OSes on my disk in separate partitions and the main content of the home folder is in /hbasis/freek (basic home folder). /home/freek contains only the files which differ in the different OSes and have soft links to folders in /hbasis/freek

Comment 9 Bug Janitor Service 2023-08-05 06:48:00 UTC

A possibly relevant merge request was started @ https://invent.kde.org/sdk/lokalize/-/merge_requests/68

Comment 10 wojnilowicz 2023-08-07 13:34:17 UTC

Git commit 85a4dad60bc39d42ff6c35a8fa21d96e1a78d937 by Łukasz Wojniłowicz.
Committed on 07/08/2023 at 15:34.
Pushed by wojnilowicz into branch 'master'.

Fix constructing pot branch dirs for symlinked dirs

M  +1    -1    src/mergemode/mergeview.cpp
M  +4    -0    src/prefs/prefs.cpp
M  +36   -1    src/project/prefs_project_advanced.ui
M  +0    -8    src/project/project.cpp
M  +4    -1    src/project/project.h
M  +1    -0    src/project/projectbase.kcfg

https://invent.kde.org/sdk/lokalize/-/commit/85a4dad60bc39d42ff6c35a8fa21d96e1a78d937

Comment 11 Albert Astals Cid 2023-08-07 19:59:55 UTC

Git commit 743d9baf58a334c1785122ecc8509e63a40cb4c3 by Albert Astals Cid, on behalf of Łukasz Wojniłowicz.
Committed on 07/08/2023 at 21:58.
Pushed by aacid into branch 'release/23.08'.

Fix constructing pot branch dirs for symlinked dirs
(cherry picked from commit 85a4dad60bc39d42ff6c35a8fa21d96e1a78d937)

M  +1    -1    src/mergemode/mergeview.cpp
M  +4    -0    src/prefs/prefs.cpp
M  +36   -1    src/project/prefs_project_advanced.ui
M  +0    -8    src/project/project.cpp
M  +4    -1    src/project/project.h
M  +1    -0    src/project/projectbase.kcfg

https://invent.kde.org/sdk/lokalize/-/commit/743d9baf58a334c1785122ecc8509e63a40cb4c3