| Summary: | Chinese manual search term segmentation issues round 2 | ||
|---|---|---|---|
| Product: | [Applications] krita | Reporter: | Alvin Wong <alvin> |
| Component: | Documentation | Assignee: | Alvin Wong <alvin> |
| Status: | RESOLVED FIXED | ||
| Severity: | normal | CC: | tysontanx |
| Priority: | NOR | ||
| Version: | git master (please specify the git hash!) | ||
| Target Milestone: | --- | ||
| Platform: | Other | ||
| OS: | Other | ||
| Latest Commit: | https://invent.kde.org/documentation/docs-krita-org/commit/9fe299135461d57964547b52ad2b27a093f83d45 | Version Fixed In: | |
A possibly relevant merge request was started @ https://invent.kde.org/documentation/docs-krita-org/-/merge_requests/254 A possibly relevant merge request was started @ https://invent.kde.org/documentation/docs-krita-org/-/merge_requests/254 Git commit 84c73ecb341e3c765ddb58b74642dc2453b52df4 by Alvin Wong. Committed on 14/09/2021 at 14:35. Pushed by alvinwong into branch 'master'. Add Jieba user dict for zh_CN and zh_TW This makes the term for "tablet" in Chinese searchable. M +10 -0 conf.py A +1 -0 jieba-dict-zh_CN.txt A +1 -0 jieba-dict-zh_TW.txt https://invent.kde.org/documentation/docs-krita-org/commit/84c73ecb341e3c765ddb58b74642dc2453b52df4 Git commit 9fe299135461d57964547b52ad2b27a093f83d45 by Alvin Wong. Committed on 15/09/2021 at 16:08. Pushed by alvinwong into branch 'krita/5.0'. Add Jieba user dict for zh_CN and zh_TW This makes the term for "tablet" in Chinese searchable. (cherry picked from commit 84c73ecb341e3c765ddb58b74642dc2453b52df4) M +10 -0 conf.py A +1 -0 jieba-dict-zh_CN.txt A +1 -0 jieba-dict-zh_TW.txt https://invent.kde.org/documentation/docs-krita-org/commit/9fe299135461d57964547b52ad2b27a093f83d45 Git commit 7a805c65d8333c394f705924bdc06aa012edcfdc by Alvin Wong. Committed on 16/09/2021 at 07:07. Pushed by alvinwong into branch 'master'. Use absolute path to Jieba user dict This should fix Jieba dict usage on binary-factory. M +6 -2 conf.py https://invent.kde.org/documentation/docs-krita-org/commit/7a805c65d8333c394f705924bdc06aa012edcfdc Git commit bc03e95d3c9bbf910318bffa9d7a7ed0849ceab4 by Alvin Wong. Committed on 16/09/2021 at 07:08. Pushed by alvinwong into branch 'krita/5.0'. Use absolute path to Jieba user dict This should fix Jieba dict usage on binary-factory. (cherry picked from commit 7a805c65d8333c394f705924bdc06aa012edcfdc) M +6 -2 conf.py https://invent.kde.org/documentation/docs-krita-org/commit/bc03e95d3c9bbf910318bffa9d7a7ed0849ceab4 I have a question: Currently the Chinese translation for Opacity (不透明度) and transparency (透明度)，only 透明度 is being recognized. How do we handle this? If I added 不透明度 to the dictionary, will it make 透明度 to fail? (In reply to Tyson Tan from comment #7) > I have a question: > Currently the Chinese translation for Opacity (不透明度) and transparency > (透明度)，only 透明度 is being recognized. How do we handle this? If I added 不透明度 > to the dictionary, will it make 透明度 to fail? It depends, but the best way to tell is to build and test it locally. In theory, since Sphinx uses the `cut_for_search` method, it can produce multiple overlapping terms from a phrase. That is, if both "透明度" and "不透明度" exists in the dictionary, it should produce both terms for indexing. You may explicitly add both terms to the user dictionary to be on the safe side. Thanks for the explaination! Soooo... is there no downside for being excessive on the terms? And is there a way to add comments to the dictionary? (In reply to Tyson Tan from comment #9) > Thanks for the explaination! > Soooo... is there no downside for being excessive on the terms? I can't answer that. The algorithm of Jieba is mostly unknown to me. > And is there a way to add comments to the dictionary? I don't think there is any. |
I noticed that searching does not work with the tern "数位板" in zh_CN ("數位板" in zh_TW), likely because the dictionary that Jieba uses does not contain this term, which causes "数位" to be indexed as a term and "板" being dropped. We need to somehow be able to add custom terms into the Jieba dictionary for the manual build.