Bug 420195 - no support for UTF16 characters under Windows
Summary: no support for UTF16 characters under Windows
Status: RESOLVED FIXED
Alias: None
Product: digikam
Classification: Applications
Component: Plugin-DImg-HEIF (show other bugs)
Version: 7.0.0
Platform: Microsoft Windows Microsoft Windows
: NOR normal
Target Milestone: ---
Assignee: Digikam Developers
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2020-04-17 11:25 UTC by serge.kvieto
Modified: 2020-08-30 11:19 UTC (History)
4 users (show)

See Also:
Latest Commit:
Version Fixed In: 7.1.0


Attachments
screenshot (57.14 KB, image/gif)
2020-04-17 11:25 UTC, serge.kvieto
Details

Note You need to log in before you can comment on or make changes to this bug.
Description serge.kvieto 2020-04-17 11:25:23 UTC
Created attachment 127609 [details]
screenshot

SUMMARY
no thumbnails and previews of heic files.


STEPS TO REPRODUCE
1. add to albums folder with heic files
2. wait for scan
3. 

OBSERVED RESULT
no thumbnails and previews of heic files.
"failed to load image" on preview screen.
it shows that file is "heif" type, but no more information.

EXPECTED RESULT
thumbnails and previews of heic files.

SOFTWARE/OS VERSIONS
Windows: 7sp1
Qt Version: 5.13.1

ADDITIONAL INFORMATION
Comment 1 Maik Qualmann 2020-04-17 11:32:15 UTC
Please test it with the current digiKam-7.0.0-Beta3 if the problem still exists. Here the problem cannot be reproduced under Windows with HEIC files. You can find the current Beta3 here:

https://files.kde.org/digikam/

Don't forget to create new thumbnails with "F5".

Maik
Comment 2 serge.kvieto 2020-04-17 11:45:42 UTC
(In reply to Maik Qualmann from comment #1)
> Please test it with the current digiKam-7.0.0-Beta3 if the problem still
> exists. Here the problem cannot be reproduced under Windows with HEIC files.
> You can find the current Beta3 here:
> 
> https://files.kde.org/digikam/
> 
> Don't forget to create new thumbnails with "F5".
> 
> Maik

Maik, thank you for your reply, I tried digiKam-7.0.0-Beta3, got the same result: no heic thumbnails and previews.
Comment 3 Maik Qualmann 2020-04-17 11:48:27 UTC
Can you provide a sample HEIC image? There are no problems with our test image from the IPhone.

Maik
Comment 4 caulier.gilles 2020-04-17 11:49:55 UTC
Can you share your HEIF files to test, please. Use Cloud Web Service to host files.

Thanks in advance

Gilles Caulier
Comment 5 serge.kvieto 2020-04-17 13:19:50 UTC
hmmmm.
Seems to be a problem with folder name.

Photo app (MacOS) added some strange symbol to the end of each folder name while exporting library (windows explorer shows it as "centered dot" in folder name).
In folders with such names digiKam can read all formats normally except heic format.  

I was preparing altenative folder with example files to share with you, and tested it... and everything became ok! digiKam could process the same files in folder with another name.  

Than I simply removed "centered dot" in folder name at old location, and it also works now.

Thank you!
Comment 6 Maik Qualmann 2020-04-17 13:53:19 UTC
Can you please try to create a ZIP file with such a folder with the special character. A HEIC image does not have to be included.

Maik
Comment 7 serge.kvieto 2020-04-19 14:17:38 UTC
https://drive.google.com/open?id=1bgDzSBFMXr_r5P19pM51v3qDgtuUEk2U

here it is.

folder names are cyrillic, it's ok, but the last symbol is not valid for heic files in this folder.
Comment 8 Maik Qualmann 2020-04-19 19:03:49 UTC
Thank you, with the directory structure the problem can be reproduced well under Windows. The problem is clear, but can hardly be solved. JPG files are only loaded by the QImage loader, Exiv2 cannot read metadata. The QImage loader uses the Windows API directly and uses the correct UTF16 file name. We use QFile::encodeName() this function can convert all characters within the current code page, but not if characters are outside. We would have to use a wrapper to replace fopen() that under Windows only uses code page with _wfopen(). However, a test shows that even Exiv2 does not handle it correctly internally when we pass UTF8 file names.

Maik
Comment 9 caulier.gilles 2020-04-19 19:10:30 UTC
Maik,

Exiv2 has a compilation for windows to handle UTF16 path strings. It' normally enabled now since few weeks has this option compile now with MinGW.

https://invent.kde.org/kde/digikam/-/blob/master/project/bundles/3rdparty/ext_exiv2/CMakeLists.txt#L34

Gilles
Comment 10 Maik Qualmann 2020-04-19 19:31:16 UTC
Hmm, interesting. I had exchanged all QFile::encodeName() calls in the metadataengine directory to FILENAME.toUtf8().constData(). However, Exiv2 was unable to read metadata from images in these special directories. I test it again the days.

Maik
Comment 11 caulier.gilles 2020-04-20 07:26:10 UTC
Maik,

Another possibility is a bug in last stable Exiv2 0.27.2. 0.27.3 is on the way after one year of project stand bye. The bug fix is very long, so when the first 0.27.3-RC1 will out, i will switch all bundles on this tag.

Gilles
Comment 12 Maik Qualmann 2020-04-26 06:46:01 UTC
*** Bug 420601 has been marked as a duplicate of this bug. ***
Comment 13 caulier.gilles 2020-04-27 16:59:15 UTC
Maik,

Next Exiv2 0.27.3 is near to release :

https://github.com/Exiv2/exiv2/issues/1186

What do you think to include this RC in all bundles ?

Gilles
Comment 14 Maik Qualmann 2020-04-28 06:35:20 UTC
For me that's fine to use the Exiv2-RC.

Maik
Comment 15 caulier.gilles 2020-04-28 08:20:26 UTC
Ok, well i will switch all bundles from Exiv2 0.27.2 to 0.27-maintance git branch. Like this we will use next 0.27.3 and future small improvement automatically. I will recompile automatically Exiv2 when digiKam is compiled (aka Lensfun and QtAv).

I see the big changes are operated in master (next 0.28)

Gilles
Comment 16 caulier.gilles 2020-04-28 09:24:22 UTC
Note : bugfixes list from Exiv2 0.27.3 RC1 :

https://github.com/Exiv2/exiv2/issues/1018#issuecomment-604539346

Gilles
Comment 17 caulier.gilles 2020-04-28 11:22:30 UTC
Git commit e0360b3812396769a0ce2fd58124e9dc4feac555 by Gilles Caulier.
Committed on 28/04/2020 at 11:21.
Pushed by cgilles into branch 'master'.

Switch to Exiv2 0.27-maintenance branch instead 0.27.2 older release.
compile and install this Exiv2 version at all digiKam update for all bundles

M  +12   -10   project/bundles/3rdparty/ext_exiv2/CMakeLists.txt
M  +0    -1    project/bundles/appimage/01-build-host.sh
M  +1    -0    project/bundles/appimage/03-build-digikam.sh
M  +0    -1    project/bundles/macports/01-build-macports.sh
M  +1    -0    project/bundles/macports/03-build-digikam.sh
M  +0    -1    project/bundles/mxe/01-build-mxe.sh
M  +1    -0    project/bundles/mxe/03-build-digikam.sh

https://invent.kde.org/kde/digikam/commit/e0360b3812396769a0ce2fd58124e9dc4feac555
Comment 18 caulier.gilles 2020-04-28 11:38:29 UTC
MAik,

See the configuration of Exiv2 from 0.27-maintenance branch under Windows :

-- ------------------------------------------------------------------
-- Building shared library:            YES
-- Building PNG support:               YES
-- XMP metadata support:               YES
-- Native language support:            YES
-- Conversion of Windows XP tags:      YES
-- Nikon lens database:                YES
-- Building video support:             NO
-- Building webready support:          NO
-- Dynamic runtime override:           NO
-- Unicode paths (wstring):            YES
-- Building exiv2 command:             NO
-- Building samples:                   NO
-- Building PO files:                  NO
-- Building unit tests:                NO
-- Building doc:                       NO
-- Building with coverage flags:       NO
-- Using ccache:                       NO
-- ------------------------------------------------------------------

As you can see Unicode paths feature is enabled...

Gilles
Comment 19 Maik Qualmann 2020-04-28 21:02:59 UTC
I will patch the days and pass Exiv2 UTF8 strings to see if it is used correctly in Windows.

Maik
Comment 20 Tobias 2020-05-04 16:12:51 UTC
Before I report this as another bug, I wanted to check if this is related:

SUMMARY

Files of any type (not just HEIF) with a non-latin Unicode character in their file name/path will not show their metadata in the metadata side panel or the metadata editor. The "properties" side panel is not affected and continues to show basic data.

REPRODUCE
1. Take existing file, verify that metadata is showing up.
2. Rename file or parent folder to include a Unicode character (e.g. "δΈ‰").
3. After refresh, verify that metadata panel is blank and metadata editor will be "greyed out".

OS: Windows 10 1910
Happens in 6.4.0 up to 7.0.0-beta3 (Win64)
Comment 21 Maik Qualmann 2020-05-04 20:18:25 UTC
Git commit cc61ea9abd278960d8a5caaa1bc66d93d074ba90 by Maik Qualmann.
Committed on 04/05/2020 at 20:16.
Pushed by mqualmann into branch 'master'.

first attempt to use unicode file names with Exiv2

M  +1    -2    core/libs/metadataengine/engine/metaengine_comments.cpp
M  +1    -2    core/libs/metadataengine/engine/metaengine_exif.cpp
M  +2    -2    core/libs/metadataengine/engine/metaengine_fileio.cpp
M  +1    -2    core/libs/metadataengine/engine/metaengine_iptc.cpp
M  +4    -4    core/libs/metadataengine/engine/metaengine_p.cpp
M  +1    -1    core/libs/metadataengine/engine/metaengine_previews.cpp
M  +1    -2    core/libs/metadataengine/engine/metaengine_xmp.cpp

https://invent.kde.org/kde/digikam/commit/cc61ea9abd278960d8a5caaa1bc66d93d074ba90
Comment 22 Maik Qualmann 2020-05-05 21:00:39 UTC
Git commit a28e8c9512c53a2ee68e60446c9e57ef6f988214 by Maik Qualmann.
Committed on 05/05/2020 at 20:58.
Pushed by mqualmann into branch 'master'.

try to give Exiv2 under Windows a UTF16 string

M  +5    -0    core/libs/metadataengine/engine/metaengine_comments.cpp
M  +5    -0    core/libs/metadataengine/engine/metaengine_exif.cpp
M  +9    -1    core/libs/metadataengine/engine/metaengine_fileio.cpp
M  +5    -0    core/libs/metadataengine/engine/metaengine_iptc.cpp
M  +10   -0    core/libs/metadataengine/engine/metaengine_p.cpp
M  +5    -0    core/libs/metadataengine/engine/metaengine_previews.cpp
M  +5    -0    core/libs/metadataengine/engine/metaengine_xmp.cpp

https://invent.kde.org/kde/digikam/commit/a28e8c9512c53a2ee68e60446c9e57ef6f988214
Comment 23 Maik Qualmann 2020-05-06 05:52:16 UTC
Git commit cf8aa4a0a9475b0e9ea35ebeda421303c6a783f0 by Maik Qualmann.
Committed on 06/05/2020 at 05:50.
Pushed by mqualmann into branch 'master'.

revert all unicode changes at Exiv2

M  +1    -6    core/libs/metadataengine/engine/metaengine_comments.cpp
M  +1    -6    core/libs/metadataengine/engine/metaengine_exif.cpp
M  +2    -12   core/libs/metadataengine/engine/metaengine_fileio.cpp
M  +1    -6    core/libs/metadataengine/engine/metaengine_iptc.cpp
M  +2    -12   core/libs/metadataengine/engine/metaengine_p.cpp
M  +1    -6    core/libs/metadataengine/engine/metaengine_previews.cpp
M  +1    -6    core/libs/metadataengine/engine/metaengine_xmp.cpp

https://invent.kde.org/kde/digikam/commit/cf8aa4a0a9475b0e9ea35ebeda421303c6a783f0
Comment 24 Maik Qualmann 2020-05-06 18:32:55 UTC
Git commit f2f35d19fb56e2e5e9a8f398afcec4e93bb4ca2b by Maik Qualmann.
Committed on 06/05/2020 at 18:30.
Pushed by mqualmann into branch 'master'.

now Exiv2 can read unicode paths under Windows

M  +6    -1    core/libs/metadataengine/engine/metaengine_comments.cpp
M  +6    -1    core/libs/metadataengine/engine/metaengine_exif.cpp
M  +12   -2    core/libs/metadataengine/engine/metaengine_fileio.cpp
M  +6    -1    core/libs/metadataengine/engine/metaengine_iptc.cpp
M  +12   -2    core/libs/metadataengine/engine/metaengine_p.cpp
M  +6    -1    core/libs/metadataengine/engine/metaengine_previews.cpp
M  +6    -1    core/libs/metadataengine/engine/metaengine_xmp.cpp

https://invent.kde.org/kde/digikam/commit/f2f35d19fb56e2e5e9a8f398afcec4e93bb4ca2b
Comment 25 Maik Qualmann 2020-05-06 20:17:52 UTC
Git commit 183933ce5c6e52bb0e4f307d85a0fa041cb84e7b by Maik Qualmann.
Committed on 06/05/2020 at 20:16.
Pushed by mqualmann into branch 'master'.

first DImg loaders support unicode under Windows

M  +6    -2    core/dplugins/dimg/jpeg/dimgjpegloader_load.cpp
M  +6    -2    core/dplugins/dimg/jpeg/dimgjpegloader_save.cpp
M  +6    -1    core/dplugins/dimg/jpeg/dimgjpegplugin.cpp
M  +6    -2    core/dplugins/dimg/jpeg2000/dimgjpeg2000loader_load.cpp
M  +6    -2    core/dplugins/dimg/jpeg2000/dimgjpeg2000loader_save.cpp
M  +6    -1    core/dplugins/dimg/jpeg2000/dimgjpeg2000plugin.cpp
M  +13   -12   core/dplugins/dimg/pgf/dimgpgfloader_load.cpp
M  +9    -12   core/dplugins/dimg/pgf/dimgpgfloader_save.cpp
M  +6    -1    core/dplugins/dimg/pgf/dimgpgfplugin.cpp
M  +5    -1    core/dplugins/dimg/png/dimgpngloader_load.cpp
M  +5    -1    core/dplugins/dimg/png/dimgpngloader_save.cpp
M  +6    -1    core/dplugins/dimg/png/dimgpngplugin.cpp
M  +1    -1    core/dplugins/dimg/tiff/dimgtiffloader_load.cpp
M  +6    -1    core/dplugins/dimg/tiff/dimgtiffplugin.cpp

https://invent.kde.org/kde/digikam/commit/183933ce5c6e52bb0e4f307d85a0fa041cb84e7b
Comment 26 caulier.gilles 2020-05-06 20:29:10 UTC
Maik,

Windows CI build do not pass because It do not use last stable release or last code from git 0.27 maintenance branch. I suspect that the Exiv2 API to support Windows Unicode path is not too old.

https://build.kde.org/job/Extragear/job/digikam/job/kf5-qt5%20WindowsMSVCQt5.14/472/

Gilles
Comment 27 caulier.gilles 2020-05-06 20:37:27 UTC
Maik,

To know which Exiv2 features are enabled, an exported header must be available :
exv_conf.h

... with this kind of definition:

// Define to enable the Windows unicode path support.
/* #undef EXV_UNICODE_PATH */

Gilles
Comment 28 Maik Qualmann 2020-05-06 21:09:03 UTC
The MSVC may be more precise and also requires the std::wstring instead of wchar_t*. Let's see, you can not see from the error message whether the function is available.

Maik
Comment 29 Maik Qualmann 2020-05-06 21:12:01 UTC
I just see the build is done, Exiv2 is without unicode support.

Maik
Comment 30 caulier.gilles 2020-05-07 05:04:56 UTC
Maik, 

MetaEngine private container has the Exiv2 configuration header included :

https://invent.kde.org/kde/digikam/-/blob/master/core/libs/metadataengine/engine/metaengine_p.h#L69

You can just check if the UNICODE feature is enabled of not, as we do already with XMP support here :

https://invent.kde.org/kde/digikam/-/blob/master/core/libs/metadataengine/engine/metaengine_p.h#L84

And just decide which API call to use with Exiv2 under Windows here :

https://invent.kde.org/kde/digikam/-/blob/master/core/libs/metadataengine/engine/metaengine_fileio.cpp#L107

... and at other places. This will work for the bundles and the CI.

Gilles
Comment 31 Maik Qualmann 2020-05-07 05:41:22 UTC
Git commit df2144e3f3f88ca4be660d80d306d216d3f4fdd2 by Maik Qualmann.
Committed on 07/05/2020 at 05:39.
Pushed by mqualmann into branch 'master'.

first try to support unicode with utime()

M  +20   -4    core/libs/metadataengine/engine/metaengine_p.cpp

https://invent.kde.org/kde/digikam/commit/df2144e3f3f88ca4be660d80d306d216d3f4fdd2
Comment 32 Maik Qualmann 2020-05-07 05:54:01 UTC
Git commit 17124d5f042778c9ac0038a63caf4c822e863b04 by Maik Qualmann.
Committed on 07/05/2020 at 05:51.
Pushed by mqualmann into branch 'master'.

libtiff has a wchar open function

M  +9    -1    core/dplugins/dimg/tiff/dimgtiffloader_load.cpp
M  +10   -2    core/dplugins/dimg/tiff/dimgtiffloader_save.cpp

https://invent.kde.org/kde/digikam/commit/17124d5f042778c9ac0038a63caf4c822e863b04
Comment 33 Maik Qualmann 2020-05-07 06:04:34 UTC
to Comment 30: Ok, I'll do it tonight.

Maik
Comment 34 Maik Qualmann 2020-05-08 05:34:26 UTC
Git commit e057a771cecd3f155d5857be507a8dd05ef9a649 by Maik Qualmann.
Committed on 08/05/2020 at 05:32.
Pushed by mqualmann into branch 'master'.

try to use unicode under Windows for the RAW decoder

M  +29   -5    core/libs/rawengine/drawdecoder.cpp
M  +6    -1    core/libs/rawengine/drawdecoder_p.cpp

https://invent.kde.org/kde/digikam/commit/e057a771cecd3f155d5857be507a8dd05ef9a649
Comment 35 Maik Qualmann 2020-05-09 06:47:59 UTC
Git commit 3b1c95ab04573f3b892d72a95278e2a756f419d7 by Maik Qualmann.
Committed on 09/05/2020 at 06:45.
Pushed by mqualmann into branch 'master'.

add unicode support for the JPEG2000 loader

M  +16   -10   core/dplugins/dimg/jpeg2000/dimgjpeg2000loader_load.cpp
M  +12   -5    core/dplugins/dimg/jpeg2000/dimgjpeg2000loader_save.cpp

https://invent.kde.org/kde/digikam/commit/3b1c95ab04573f3b892d72a95278e2a756f419d7
Comment 36 caulier.gilles 2020-05-12 07:45:51 UTC
Maik,

Look well on the libraw 0.20.-beat1 announcement. We have now a compilation flag to enable to support UTF16 paths under Windows.

https://www.libraw.org/news/libraw-0-20-beta1

So i will backport this new libraw version in digiKam core and enable the flag under Windows.

Gilles
Comment 37 Maik Qualmann 2020-05-12 08:19:24 UTC
Ok, fine. Libraw already works in digiKam with UTF16 paths.

https://invent.kde.org/kde/digikam/-/commit/e057a771cecd3f155d5857be507a8dd05ef9a649

We still have the problem with the HEIF Loader. This does not support Windows UTF16 paths. I think we could add it.

Maik
Comment 38 caulier.gilles 2020-05-12 11:12:45 UTC
Git commit 65fd05a84cb463edc976e2120ac66e09c9cb87ea by Gilles Caulier.
Committed on 12/05/2020 at 11:10.
Pushed by cgilles into branch 'master'.

Update internal libraw to last 0.20.0-beta1 (https://www.libraw.org/news/libraw-0-20-beta1)
Enable Unicode paths support under Windows

M  +16   -1    NEWS
M  +5    -0    core/libs/rawengine/CMakeLists.txt
M  +1    -1    core/libs/rawengine/libraw/COPYRIGHT
M  +94   -35   core/libs/rawengine/libraw/Changelog.txt
M  +1    -10   core/libs/rawengine/libraw/README.md
M  +1    -1    core/libs/rawengine/libraw/internal/dcraw_common.cpp
M  +5    -1    core/libs/rawengine/libraw/internal/dcraw_defs.h
M  +1    -1    core/libs/rawengine/libraw/internal/dcraw_fileio.cpp
M  +1    -1    core/libs/rawengine/libraw/internal/dcraw_fileio_defs.h
M  +8    -1    core/libs/rawengine/libraw/internal/defines.h
M  +1    -1    core/libs/rawengine/libraw/internal/dmp_include.h
A  +300  -0    core/libs/rawengine/libraw/internal/libraw_cameraids.h     [License: UNKNOWN]  *
D  +0    -85   core/libs/rawengine/libraw/internal/libraw_const.h
M  +24   -4    core/libs/rawengine/libraw/internal/libraw_cxx_defs.h
M  +41   -4    core/libs/rawengine/libraw/internal/libraw_internal_funcs.h
M  +2    -1    core/libs/rawengine/libraw/internal/var_defines.h
M  +1    -1    core/libs/rawengine/libraw/internal/x3f_tools.h
M  +4    -2    core/libs/rawengine/libraw/libraw/libraw.h
M  +1    -1    core/libs/rawengine/libraw/libraw/libraw_alloc.h
M  +183  -75   core/libs/rawengine/libraw/libraw/libraw_const.h
M  +4    -4    core/libs/rawengine/libraw/libraw/libraw_datastream.h
M  +4    -2    core/libs/rawengine/libraw/libraw/libraw_internal.h
M  +12   -5    core/libs/rawengine/libraw/libraw/libraw_types.h
M  +3    -3    core/libs/rawengine/libraw/libraw/libraw_version.h
M  +1    -1    core/libs/rawengine/libraw/samples/4channels.cpp
M  +1    -1    core/libs/rawengine/libraw/samples/dcraw_emu.cpp
M  +1    -1    core/libs/rawengine/libraw/samples/dcraw_half.c
M  +1    -1    core/libs/rawengine/libraw/samples/half_mt.c
M  +1    -1    core/libs/rawengine/libraw/samples/half_mt_win32.c
M  +1    -1    core/libs/rawengine/libraw/samples/mem_image_sample.cpp
M  +1    -1    core/libs/rawengine/libraw/samples/multirender_test.cpp
M  +2    -2    core/libs/rawengine/libraw/samples/openbayer_sample.cpp
M  +1    -1    core/libs/rawengine/libraw/samples/postprocessing_benchmark.cpp
M  +648  -252  core/libs/rawengine/libraw/samples/raw-identify.cpp
M  +2    -2    core/libs/rawengine/libraw/samples/rawtextdump.cpp
M  +1    -1    core/libs/rawengine/libraw/samples/simple_dcraw.cpp
M  +3    -3    core/libs/rawengine/libraw/samples/unprocessed_raw.cpp
M  +1    -1    core/libs/rawengine/libraw/src/decoders/canon_600.cpp
M  +4    -1    core/libs/rawengine/libraw/src/decoders/crx.cpp
M  +21   -7    core/libs/rawengine/libraw/src/decoders/decoders_dcraw.cpp
M  +5    -1    core/libs/rawengine/libraw/src/decoders/decoders_libraw.cpp
M  +140  -4    core/libs/rawengine/libraw/src/decoders/decoders_libraw_dcrdefs.cpp
M  +2    -2    core/libs/rawengine/libraw/src/decoders/dng.cpp
M  +1    -1    core/libs/rawengine/libraw/src/decoders/fp_dng.cpp
M  +1    -1    core/libs/rawengine/libraw/src/decoders/generic.cpp
M  +1    -1    core/libs/rawengine/libraw/src/decoders/kodak_decoders.cpp
M  +9    -9    core/libs/rawengine/libraw/src/decoders/load_mfbacks.cpp
M  +1    -1    core/libs/rawengine/libraw/src/decoders/smal.cpp
M  +19   -14   core/libs/rawengine/libraw/src/decoders/unpack.cpp
M  +9    -3    core/libs/rawengine/libraw/src/decoders/unpack_thumb.cpp
M  +1    -1    core/libs/rawengine/libraw/src/demosaic/ahd_demosaic.cpp
M  +1    -1    core/libs/rawengine/libraw/src/demosaic/misc_demosaic.cpp
M  +1    -1    core/libs/rawengine/libraw/src/demosaic/xtrans_demosaic.cpp
M  +154  -43   core/libs/rawengine/libraw/src/integration/dngsdk_glue.cpp
M  +1    -1    core/libs/rawengine/libraw/src/integration/rawspeed_glue.cpp
M  +1    -1    core/libs/rawengine/libraw/src/libraw_c_api.cpp
M  +1    -1    core/libs/rawengine/libraw/src/libraw_cxx.cpp
M  +112  -44   core/libs/rawengine/libraw/src/libraw_datastream.cpp
M  +3    -7    core/libs/rawengine/libraw/src/metadata/adobepano.cpp
M  +200  -152  core/libs/rawengine/libraw/src/metadata/canon.cpp
M  +341  -55   core/libs/rawengine/libraw/src/metadata/ciff.cpp
M  +1    -1    core/libs/rawengine/libraw/src/metadata/cr3_parser.cpp
M  +6    -6    core/libs/rawengine/libraw/src/metadata/epson.cpp
M  +58   -12   core/libs/rawengine/libraw/src/metadata/exif_gps.cpp
M  +379  -216  core/libs/rawengine/libraw/src/metadata/fuji.cpp
M  +5    -3    core/libs/rawengine/libraw/src/metadata/hasselblad_model.cpp
M  +2092 -1986 core/libs/rawengine/libraw/src/metadata/identify.cpp
M  +1    -1    core/libs/rawengine/libraw/src/metadata/identify_tools.cpp
M  +17   -17   core/libs/rawengine/libraw/src/metadata/kodak.cpp
M  +12   -11   core/libs/rawengine/libraw/src/metadata/leica.cpp
M  +136  -46   core/libs/rawengine/libraw/src/metadata/makernotes.cpp
M  +13   -13   core/libs/rawengine/libraw/src/metadata/mediumformat.cpp
M  +1    -3    core/libs/rawengine/libraw/src/metadata/minolta.cpp
M  +113  -5    core/libs/rawengine/libraw/src/metadata/misc_parsers.cpp
M  +82   -68   core/libs/rawengine/libraw/src/metadata/nikon.cpp
M  +455  -260  core/libs/rawengine/libraw/src/metadata/normalize_model.cpp
M  +84   -57   core/libs/rawengine/libraw/src/metadata/olympus.cpp
M  +1    -1    core/libs/rawengine/libraw/src/metadata/p1.cpp
M  +84   -73   core/libs/rawengine/libraw/src/metadata/pentax.cpp
M  +40   -34   core/libs/rawengine/libraw/src/metadata/samsung.cpp
M  +472  -493  core/libs/rawengine/libraw/src/metadata/sony.cpp
M  +305  -242  core/libs/rawengine/libraw/src/metadata/tiff.cpp
M  +1    -1    core/libs/rawengine/libraw/src/postprocessing/aspect_ratio.cpp
M  +1    -1    core/libs/rawengine/libraw/src/postprocessing/dcraw_process.cpp
M  +1    -1    core/libs/rawengine/libraw/src/postprocessing/mem_image.cpp
M  +1    -1    core/libs/rawengine/libraw/src/postprocessing/postprocessing_aux.cpp
M  +1    -1    core/libs/rawengine/libraw/src/postprocessing/postprocessing_ph.cpp
M  +1    -1    core/libs/rawengine/libraw/src/postprocessing/postprocessing_utils.cpp
M  +11   -4    core/libs/rawengine/libraw/src/postprocessing/postprocessing_utils_dcrdefs.cpp
M  +1    -1    core/libs/rawengine/libraw/src/preprocessing/ext_preprocess.cpp
M  +1    -1    core/libs/rawengine/libraw/src/preprocessing/preprocessing_ph.cpp
M  +1    -1    core/libs/rawengine/libraw/src/preprocessing/raw2image.cpp
M  +1    -1    core/libs/rawengine/libraw/src/preprocessing/subtract_black.cpp
M  +84   -55   core/libs/rawengine/libraw/src/tables/cameralist.cpp
M  +1    -1    core/libs/rawengine/libraw/src/tables/colorconst.cpp
M  +51   -22   core/libs/rawengine/libraw/src/tables/colordata.cpp
M  +157  -45   core/libs/rawengine/libraw/src/tables/wblists.cpp
M  +1    -1    core/libs/rawengine/libraw/src/utils/curves.cpp
M  +19   -1    core/libs/rawengine/libraw/src/utils/decoder_info.cpp
M  +51   -51   core/libs/rawengine/libraw/src/utils/init_close_utils.cpp
M  +319  -257  core/libs/rawengine/libraw/src/utils/open.cpp
M  +1    -1    core/libs/rawengine/libraw/src/utils/phaseone_processing.cpp
M  +49   -10   core/libs/rawengine/libraw/src/utils/read_utils.cpp
M  +3    -1    core/libs/rawengine/libraw/src/utils/thumb_utils.cpp
M  +2    -2    core/libs/rawengine/libraw/src/utils/utils_dcraw.cpp
M  +35   -1    core/libs/rawengine/libraw/src/utils/utils_libraw.cpp
M  +1    -1    core/libs/rawengine/libraw/src/write/apply_profile.cpp
M  +4    -4    core/libs/rawengine/libraw/src/write/file_write.cpp
M  +1    -1    core/libs/rawengine/libraw/src/write/tiff_writer.cpp
M  +1    -1    core/libs/rawengine/libraw/src/write/write_ph.cpp
M  +5    -2    core/libs/rawengine/libraw/src/x3f/x3f_parse_process.cpp
M  +4    -0    core/libs/rawengine/libraw/src/x3f/x3f_utils_patched.cpp

The files marked with a * at the end have a non valid license. Please read: https://community.kde.org/Policies/Licensing_Policy and use the headers which are listed at that page.


https://invent.kde.org/kde/digikam/commit/65fd05a84cb463edc976e2120ac66e09c9cb87ea
Comment 39 Maik Qualmann 2020-05-19 19:37:07 UTC
Git commit c9d0d10f1c38607bc888b4b484106533e13be906 by Maik Qualmann.
Committed on 19/05/2020 at 19:35.
Pushed by mqualmann into branch 'master'.

load HEIF file into memory if loading from Unicode path detected

M  +46   -4    core/dplugins/dimg/heif/dimgheifloader_load.cpp

https://invent.kde.org/graphics/digikam/commit/c9d0d10f1c38607bc888b4b484106533e13be906
Comment 40 Maik Qualmann 2020-05-20 11:29:09 UTC
Ok, loading HEIF images from Unicode paths now works. It remains to save HEIF images. I think we have to patch the libheif.

Maik
Comment 41 caulier.gilles 2020-05-20 12:23:39 UTC
This is the advantage to use libheif internally until the library has all main bug fixed.

Typically, we can already use it as external dependency, since the cmake compilation and install rules from the library are really better since is reported as upstream to have a clean cmake support (aka modern cmake config shared file).

This must be also true for libde265, but i don't yet tested.

So the rules is really to patch in DK heif plugin, and if it work, make a UPSTREAM PR on github for backport integration. I already do it for IPTC support in heif. The liheif team (which is the same for libde265) is well responding...

Gilles
Comment 42 Maik Qualmann 2020-05-20 13:47:52 UTC
I like the solution in FFmpeg. FFmpeg uses char* as UTF8 file name. Under Windows, a UTF8ToUTF16WChar() function is used to convert and the corresponding Windows wchar* functions are called. Therefore FFmpeg works without any change.

Maik
Comment 43 caulier.gilles 2020-05-20 13:54:16 UTC
ffmpeg rock (:=)))...
Gilles
Comment 44 Maik Qualmann 2020-05-22 14:55:10 UTC
*** Bug 421919 has been marked as a duplicate of this bug. ***
Comment 45 caulier.gilles 2020-08-01 07:49:45 UTC
Maik,

It still a patch to apply to libheif to close this file ?

Gilles
Comment 46 Maik Qualmann 2020-08-01 08:36:26 UTC
Hi Gilles,

Saving HEIF images does not currently support UTF-16. There is a patch for KImageFormats, a function of the libheif is overloaded and saved using QIODevice. If I see it correctly, libheif also has a C++ part and we use only the C part?

Maik
Comment 47 caulier.gilles 2020-08-01 13:10:29 UTC
yes we use C API, even if C++ compiler is used even if all libheif implementation use .cc everywhere.

I think it's better to patch C API. As libheif is currently included in DK core, it's easy do to, apply and test. Later this patch can be backport officially in libheif with a github PR.

Gilles
Comment 48 caulier.gilles 2020-08-28 05:13:06 UTC
Maik, 

I move this file in HEIF plugin as it still only libheif problem with UTF16 support under Windows to fix.

Gilles
Comment 49 Maik Qualmann 2020-08-30 11:19:49 UTC
Git commit c851104f6b65ef0cb463c4769af907534ad1283c by Maik Qualmann.
Committed on 30/08/2020 at 11:17.
Pushed by mqualmann into branch 'master'.

add the QIODevice writer to save HEIF files
FIXED-IN: 7.1.0

M  +3    -4    NEWS
M  +38   -1    core/dplugins/dimg/heif/dimgheifloader_save.cpp

https://invent.kde.org/graphics/digikam/commit/c851104f6b65ef0cb463c4769af907534ad1283c