Bug 205776

Well, maybe current behavior isn't optimal but in this aspect digiKam behaves as it should. Option means that in preview will be used *real* image. In this case RAW is real image and digiKam loads it.

IMO proper way to handle this would be adding sub-option: "In case of RAW files still use embedded preview" (default on).

What I meant is that the GUI text is "Embedded preview loads full image size".
It doesn't say anything about "real" or "high-quality", only that full *size*
of image will be used for preview.

I think the wording of the user preference should clearly say how each setting
chooses preview size and whether it uses embedded preview or not. And that it
should be possible to use embedded preview jpg in its full resolution.

I agree with Mikolaj about a sub-option, since users of raw-based previews
probably want full resolution previews anyway (actually half-size, if dcraw -h
equivalent is used to create the quick preview).

Matti

I just had a new look at the source, and in fact Mikolaj's sub-option idea is the easiest to implement:

- in digikam/imagepreviewview.cpp ImagePreviewView::setLoadFullImageSize, set d->previewSize to some huge maximum value, if full-size image is requested. Otherwise recalculate d->previewSize based on display size like in constructor.

- in digikam/imagepreviewview.cpp ImagePreviewView::setImagePath use the new raw-preview sub-option instead of d->loadFullImageSize to choose whether to call d->previewThread->loadHighQuality or d->previewThread->load

Matti,

Patch against code from trunk is welcome...

Gilles Caulier

Gilles,

I'm ready to try to make a patch (I'm not very familiar with Digikam source,
but after a quick look I think I can learn enough as I go). However, there's a couple of design decision that should be made and I'm not the right person to make them:

What kind of GUI should the new setting have? There are at least three choices I can think of:
- A new checkbox for selecting raw-decoding vs. embedded preview for raw files ("use embedded preview for raw files")
  + Easiest to implement
  + Most backward compatible
  - Adds yet another user preference
  - Combination not-full-size & no-embedded probably not useful (and difficult to implement in current source)
- Similar to above, but raw-decoding selection is grayed out if full-size is not selected (reduced-size preview always uses embedded preview if available)
  + Solves the combination problem above
  - I'm not experienced enough with KDE GUI programming to write the GUI logic
- A single radio button / dropdown list for selecting fullsize-raw, fullsize-embedded and reduced-size-embedded
  + Changes the user preference from a boolean to a three-state value
  - Above makes it less backward compatible? (Old 0.10.0 preference files?)

In any of the cases above, on the source level there will be changes to the interface of several classes, since current preview classes explicitly name the member functions "...LoadFullImageSize...", so using existing member functions to select raw/embedded-jpg is not appropriate in my opinion. Either new methods "...LoadEmbeddedPreview..." should be added or the current methods should be renamed "...PreviewQuality..." or something similar.

Any opinions before I try to mess up the source? :-)

2nd version is the best. 3rd also isn't bad (I think dropdown list better to save size of dialog).

My vote for 2) as well.

Is the embedded preview in raw files always full size, or only for some formats? Can we know in advance?

I see what you mean. That affects how the settings should be described in GUI.

Currently Digikam offers two choices for raw files:
1) full-size preview off: use embedded preview (if exists), limit preview size to display size
2) full-size preview on: raw-decode preview, use full image size in preview

So if a raw file from some camera only contains a small preview, the user simple switches full-size preview on (using raw decoding as a side effect). However, if the raw file contains a full-size preview, there's no way to use it full-sized.

With my suggestion, there would be three choices (these choices are a little bit different than what I suggested before):
1) embedded preview off: raw-decode preview, use full image size in preview
2) embedded preview on & limit preview size off: use embedded preview (if exists) using its native size
3) embedded preview on & limit preview size on: use embedded preview (if exists), limit preview size to display size

Now raw files with full-sized embedded preview can be viewed properly. If some raw file only contains tiny embedded preview, the user just switches embedded preview off to use raw decoded preview.

What do you think?

I was mainly asking because of implementation details involved. I think the wording of the options should be plain and intuitive.

Let's think about the rationale behind this: This is mainly about JPEGs and RAW files. It is faster to load a JPEG to a smaller size; it is much faster to load a RAW embedded preview than loading a full RAW file. So users should choose between speed and quality. Imagine two radio boxes
(x) Load previews fast
( ) Load previews in high quality
Now comes RAW; for RAWs, "high quality" is not clearly defined. For choice 2, we can
a) load a full-quality embedded preview
b) load a reduced quality embedded preview
c) load with RAW loader (automatic colors and white balance)

For the "fast" choice, a) + b) + c) are acceptable (in this order); for the high quality choice, I would default to b) iff we can verify that the preview is really full-quality - that was why I was asking if we can verify this.
For me it's all right to add a dependent non-default option "Always load RAW files from actual RAW data" or a default option "Use full quality embedded preview in RAW files".

We need to decide on the intended behavior; secondly we need to look at the implementation and tweak it a bit.

I only have first-hand experience of Nikon raw files which to my knowledge all store full-resolution jpg previews. But I think I have read that there are other cameras where the preview is a thumbnail only. (Nikon have designed NEF format to contain non-destructive editing information, so full-size preview is a must-have thing).

But even for Nikon raw files the "quality" of embedded full-size preview varies. Low-end models use much higher compression than high-end models (to save space). And raw files edited with Capture NX2 have an extremely high-end jpg preview (its size can be over 40 % of the raw file size in extreme cases).

To some people the size of the preview may be most important. But if the compression is high, the quality may be too low to some people in some cases. So I don't think there's any automatic way to know whether embedded jpg preview is of acceptable quality to the user.

Marcel,

With last Exiv2, you can extract a specific preview that you want from RAW file, if more than one is available. I don't know if libraw can do it too...

Gilles Caulier

> With last Exiv2, you can extract a specific preview that you want from RAW
> file, if more than one is available. 

Here is some info on this: http://dev.exiv2.org/wiki/exiv2/Accessing_preview_images

Andreas

Gilles,

I'm not familiar with Digikam's internals, but I got the impression that raw files' embedded previews are stored in exif data (at least in the raw files I've worked with). So is libraw should not be needed for that?

Marcel,

In your second list, are choices a) and b) switched? I.e. reduced-size embedded preview is better than fulls-size for fast preview)?

And yes, I think also that "high-quality" is a difficult concept with raw previews. 

The embedded preview is produced by camera or raw editing program, so it's conversion parameters are probably better than Digikam's. Also, if dcraw -h -style preview raw decoding is used by Digikam (is it?), raw-decoded preview is only half-size (makes it worse for checking focus). If the raw file has been edited (Capture NX2 etc.), the embedded preview is the only right option, since there's no way to tell how much the image has been edited (without being able to decode the editing metadata stored by the raw editor). So people using Digikam only for photo management but not raw photo editing (that's me) need embedded previews.

On the other hand, if the embedded preview is very small or of low quality, then Digikam's raw decoding produces a better result. The same probably applies if the user is going to edit the raw file in Digikam anyway (preview shows the starting point).

My opinion is that the choice between embedded raw previews and raw decoding should be user's. There's no way to know which is "high quality" for a particular user's needs.

Matti,

Yes, RAW preview image is stored Exif, but in proprietary makernote tags. This is not a standardized place.

2 way can be done to extrcat preview :

1/ libraw, as you can see with dcraw command line tool
2/ exiv2, but you need to parse file type to get right makernote tag.

Way 1/ is the most simple to do because libraw will analyze Raw format for you automatically. This is the current way used in digiKam ,thrue libkdcraw interface :

http://websvn.kde.org/trunk/KDE/kdegraphics/libs/libkdcraw/

You will find last libraw code here... 

Currently, libkdcraw extract preview to emulate dcraw -e option :

http://lxr.kde.org/source/KDE/kdegraphics/libs/libkdcraw/libkdcraw/kdcraw.cpp#99

But i'm not sure if libraw API is able to select a specific preview image if more that one is available. Perhaps Alex can guide us here...

Gilles Caulier

Gilles,

> 2 way can be done to extrcat preview :
>
> 1/ [...]
> 2/ exiv2, but you need to parse file type to get right makernote tag.

No, it's transparent for the application. See the link I posted above.

Andreas

Andreas

Great. Thanks !

Gilles

LibRaw is another story. LibRaw extracts only one preview (usually the largest one) from RAW. There is no near plans to change this behavior in 0.8.x (current) version.

I like Marcel's "speed / quality" idea for user preference.

How about a checkbox "Fast previews (limit preview size)"? When checked, Digikam limits preview size (to display size or something) and tries to load embedded preview for raw files (in future, maybe smallest preview that is at least of required size) and scales it to preview size (just like now).

When that box unchecked, Digikam loads full-size preview for jpegs. Unchecking the box also enables a sub-option check box "prefer embedded preview for raw files". Checking it uses embedded preview (largest that can be found) without scaling. If the option is unchecked, raw decoding preview is used.

Would this be clear enough? Or is it illogical that the sub-option becomes available only you the main option is *not* checked? From my own experience I hope that the main option text would mention both speed and the fact the preview size is limited.

Of course the main option could be close to current "preview loads full image size", but I think the wording could be improved somewhat.

Gilles: It seems to me that exiv2 provides the superior mechanism here. It's just what we need to decide in advance if we can use the preview for our purpose.

Matti: For the sake of clarity, maybe to radio buttons and the second one with the check box:
( ) Fast preview loading (limiting preview size)
(X) Load previews full size
    [X] Always prefer embedded preview for RAW files

Marcel, radio button is great and makes the preferences visually clear! Just what I was looking for but was too stupid to realise. :-)

With those user preferences, changes to the source should not be too difficult. The old ...LoadFullImageSize... functions can remain there, they are controlled by the radio buttons. New functions ...PreferEmbeddedPreview... should be added.
With current source, loading full-size preview is easiest to implement by setting preview size to a large value (65535 for example) in ImagePreviewView. That should cause loading of embedded preview, but since the preview is smaller than preview size, it is not scaled. Would that be ok?

If what I wrote above is ok, I'm willing to attempt writing a patch (no promises though). But I'll wait for an OK from you before starting. (And if you have different opinions about how this should be implemented, just say so.)

Some remarks: 
You can start with a patch if you want. Setup, AlbumSettings field, ImagePreviewView (and the very similar utilities/lighttable/lighttablepreview.cpp, utilities/slideshow/slideshow.cpp)
I will tweak the loading code to make this work, I need to think about the right solution there. You could add a third parameter to PreviewLoadThread's loadHighQuality() to prefer embedded previews.

Marcel,

I have no strong feelings about doing the patch. The feature itself (using embedded raw previews in full size) is important for me, so I'm willing to try to help, if help is needed. However, I have no prior experience in Digikam's source, so I'm sure it takes me much more time to do the patch than if one of you regular developers has time to invest in this matter. My goal is just to get the needed feature as quickly as possible. :-)

I have some questions about the design:
- There are now two separate user preferences for preview quality, one for previews and one for lighttable. Should the "use embedded preview for raw, if available" setting duplicated to both places, so that the user can change behaviour of album preview and lighttable independently?
- Earlier there has been only one bool option and one set of functions (...LoadFullImageSize...) for preview quality. The most straightforward solution for embedded previews would be to add another bool and set of functions (...UseEmbeddedRawPreview...) to control this feature. However, I'm no longer sure if that is good for future development. What if more control over preview quality is needed later? Would it be wise to make a struct (or enum) PreviewQuality, and replace current setLoadFullImageSize() functions with setPreviewQuality() etc.? Then new preview quality options could be added more easily without having to modify the intermediate classes which now only pass through changes in preview quality. Currently PreviewQuality could contain just two bools (full-size & embedded-preview), but later it could be expanded to include user-defined maximum size etc, if need arises.

What do you think?

After some contemplation pause I'd like to ask:

RAW users are more advanced users. If they want full version they want real RAWs. If they want preview they probably want embedded full size preview. I any case they don't want to use some crippled preview created from embedded preview. Am I right or not? If I am right additional option isn't required, just change mechanism, if I am mistaken... am returning under my bridge :)

Mikolaj,

I think you have a good point. Shooting raw requires much more CPU power and memory than jpeg. Which means that you probably do not have an old low-end machine. Which makes the combination "use reduced-size embedded previews" useless for raw shooters.

I see the following user groups:

1) Jpeg-only shooter
- Does not care how raw previews work
- May have a low-end machine, so reduced-size previews are useful

2) Raw-shooter whose camera stores full-size embedded previews
- Probably wants to use those full-size previews since they show how the picture looked like on camera's lcd. Full-size is better for focus checking.
- Using embedded previews is also faster
- Wants jpegs full-size, since CPU and memory are not a problem

3) Raw-shooter whose camera only stores small embedded previews (are there such cameras?)
- Small embedded preview shows how picture was on camera's lcd, but is not big enough for checking focus etc.
- Needs full-size raw-decoded previews for checking focus
- Jpegs can be full-size
- I'll have to check which cameras belong to this group

For group 1 everything works now. For group 2 (me, and I think most raw-shooters), the problem is that full-size preview does not use embedded preview, even if it's available. For group 3 (if such group exists) things are more complicated: for overall preview, small embedded preview is better. For focus checking, full-size raw-decoded preview is better.

So what if: "Load full-size previews" uses embedded preview, if it's full-size. Otherwise it uses raw-decoded preview. Would this be enough for every user group?

Folks,

I think, that for most previews (_including_ focus-checking) one-half of camera resolution is enough.

So, for 3-rd user group you may consider LibRaw/dcraw 'half mode'. This mode is way faster than usual AHD interpolation and 1/2-resolution (1/4 in pixels) is enough for most preview-related tasks.

Raw full preview use already half mode currenttly. Only editor load full raw image...

Gilles Caulier

Gilles,

Does Digikam still use half-size previews? That's what I thought, but when I tested with current trunk, 100 % full-size preview seems... well... full-size for me.

Looking at the source, it seems that enabling full-size previews causes previews to be loaded as normal images, not previews (but I'm not familiar enough with the source to be 100 % sure).

yes this is must be done :

- Raw  : preview full sized on ==> RAW half size image.
- JPEG : preview full sized on ==> real image size.

- Raw  : preview full sized off ==> RAW embedded JPEG.
- JPEG : preview full sized off ==> reduced version loaded using libjpeg.

Gilles Caulier

Gilles,

I'm not sure I understand. That is how it is done now. That's exactly the reason I filed this bug report.

That behaviour is ok for group 3 in my earlier comment. For group 2 (people whose cameras store full-size embedded jpegs), current Digikam does not allow showing the full-size high-quality jpeg at full size.

I had time to do some quick tests:

The preview list mechanism in libexiv2 seems to work nicely. I wrote a small program to list sizes of all embedded previews in raw files.

After trying that on raw files from several different cameras, I can say that my "user group 3" exists, i.e. there are still lots of cameras whose raw file does not contain a full-size embedded jpeg preview (or at least exiv2 could not find it). The maximum size of embedded preview differs a lot, for example Sony A380 seems to store an 1.5 mpix preview even though it's a 18 mpix camera. Old cameras often stored only the 160x120 exif thumbnail.

So clearly it is not enough to use the embedded preview always when it's available. In some cameras raw-decoding is the only way to get a full-size (or even half-size) preview.

@24, Matti

I am suggesting leaving interface as is and mechanics:

Use previews:
- reduced JPEGs in case of JPEGs; full-size JPEGs from RAWs whenever possible, half-size previews for the rest

Full images:
- full JPEGs in case of JPEGs; in case of RAWs I have doubts, running full blown RAW converter each time in Preview mode, or in LightTable seems... problematic but as Matti wrote in #30 something more complicated have to be done; I'd vote for simplified "import" of half-size image or even full-size but not real RAWs processing.

Mikolaj,

What about the following, which is evolved from you suggestion?

[ ] Embedded preview loads full-sized images (off)
- reduced JPEGs in case of JPEGs (limit to display size, as is done now)
- For raw, use smallest embedded preview that >= required size (display size). If no embedded preview is available or largest embedded preview is less than half of required size, use half-size raw-decoded preview (this makes sure 160x120 thumbnail previews etc. are not selected).
- Or this could be as you suggested, I was mainly thinking about libexiv2's ability to list all available previews, so Digikam could pick one based on its size

[X] Embedded preview loads full-sized images
- full JPEGs in case of JPEGs
- For raw, use largest preview available (including half-size raw-decoded preview). This means that full-size embedded previews are chosen instead of half-size raw-decoded preview, but that is better than small embedded previews.

Sound fine for me. 

Anyway, never load full RAW image in preview mode. It will take a while. Only editor will use this way.

Gilles Caulier

Sounds fine for me.

Ok I will change the logic accordingly in the previewthread and previewtask.

Support for Exiv2::PreviewManager need to be added to libkexiv2 directly.
For now I will try to work with the available libkdcraw methods.

SVN commit 1019872 by mwiesweg:

Implement suggested behavior from b.k.o. for loading fast + high quality previews.

CCBUG: 205776

 M  +2 -8      previewloadthread.cpp  
 M  +2 -8      previewloadthread.h  
 M  +96 -33    previewtask.cpp  


WebSVN link: http://websvn.kde.org/?view=rev&revision=1019872

This is now an attempt to implement according to comment #32. As said above, we want to use PreviewManager in the future but we can't at this point.
Please report if this is now as intended.
Color management for embedded previews (bug #195050) is not implemented yet.

Marcel,

I had time to do some limited testing yesterday. To me this fix seems to work as it should, but of course I could only test it with Nikon files and just a couple of raw files from other sources. The behaviour seemed reasonable, but in case of reduced-size previews I didn't try to estimate if the sizes of previews were as suggested. :-)

Thanks for the fix!

I just discovered a flaw in the new preview algorithm when full-size previews are requested:

When deciding whether the embedded image is large enough, the width of the preview is compared to the width of the raw image itself. However, at least in the case of Nikon raw files edited with NX2, if the image was taken in vertical orientation, NX2 has already reset the exif orientation flag and the embedded preview has been rotated. This causes Digikam to compare the rotated width of the preview and the unrotated width of the raw data, which often causes it to discard the preview as "too small".

Marcel, I'll send you the raw files so you can see for yourself.

I think this clever new algorithm may not be so clever after all. It's very confusing to browse photos, and some of the previews are embedded, some are raw-decoded by Digikam.

Maybe we need a user preference after all, since one user probably always wants either to use embedded previews or raw previews decoded by Digikam.

We use image area rather than only height or width. Or qMax(width, height) for both.

Marcel,

From previewtask.cpp, line 219 (this code is used when size == 0):

   // Discard if too small
   if (qimage.width() < dcrawIdentify.imageSize.width() / 2)

But this is still beside the point. Since Nikon NEF format can contain edit step, the embedded result of editing can be cropped or resized. For that reason comparing its size to the size of the raw image doesn't tell which one is "better".

Are there any more "raw" image formats which can contain edited images, or is Nikon NEF the only one?

Matti,

DNG has JPEG preview editable.

All preview in raw file are stored in a dedicated makenotes. In theory, if you can change makernote tags, you can edit preview image (has in PEF format for ex.)

Currently, digiKam use Exiv2 library which provide a limited list of editable RAW metadata.

Exiftool has a bigger list. So i let's you imagine...

In proprietary world, each maker provide software in this way. I'm sure that Canon do it...

But the best format for that is DNG, because it's standardized and based on TIFF/EP.

Gilles Caulier

Thanks for the info Gilles,

That makes embedded jpegs even more important. But since some raw formats only contain small-size embedded jpegs by default, users should be able to choose Digikam generated raw-decoded previews also.

I that light I suggest that a new user preference "use embedded jpeg in raw images when available" should be added. But I don't know where that preference should be in the GUI. There are now two places where the user can choose between full-size preview and reduced-size (for album preview and for light table). However, I don't think there's any point in letting the user to choose raw-decoding vs. embedded jpeg separately for album preview and light table. So there should be one check box "use embedded jpeg in raw images when available" which is used for both.

SVN commit 1022779 by mwiesweg:

For now, use qMax(width, height) to detect if a preview is too small.
Doesn't solve the issue of cropped images in NEFs.

CCBUG: 205776

 M  +1 -1      previewtask.cpp  


WebSVN link: http://websvn.kde.org/?view=rev&revision=1022779

I read this thread and I got a bit lost as to what your latest code looks like now. Whatever it is, I hope that you name the options clearly:
"Embedded preview" is much more clear and tells me more than "large preview"
"Full size preview" tells me nothing, I'd prefer an "embedded preview / generate preview" option, with a "full size / half size" option.

Also, I'd like an option to force a preview that shows me the real RAW file, meaning that if the RAW file is underexposed, I want the preview and thumbnail to be underexposed too. I can't do that in digiKam-0.10.0.
So if you implement an option to have digiKam generate the preview, I'd like an option to let digiKam use auto-exporure correction, or to not touch the exposure. Some users will prefer one, some the other. It should be up to them in the end. That's what I like about the KDE way :]

*** Bug 208293 has been marked as a duplicate of this bug. ***

Perhaps we should ask raw shooters ... So, is there any raw shooter who  at least once has reviewed, deleted some and tag the rest of newly imported 500 (only ...;) pictures using the mode with decoding raw? ;)

I shoot in cannon raws. I guess that each raw format decoding takes a bit of time since demosaicing algorithms are common. Therefore I have set this option a long time ago and never changed. I it was the reason I chose digikam. If I want to see raw decoded I use raw developer (currently just pressing F4 ;).

With 1.1.0 This is no longer issue though it seem s it developed some misconception in the text which is displayed in right top corner of image when viewed now. 

It seems there is some mismatch:

-When I have checked option "Embedded preview loads full image size" and view RAW image I can see the text in right top corner: "Half Size Raw Preview" while actually displayed image is "Full size Embedded Preview"

-When I uncheck option "Embedded preview loads full image size" I can se this text: "Embedded JPEG preview". While acutally this "Half Size Embedded Preview"

So it's quite apparently switched. I experienced this behavior with both .PEF raw and .DNG.

Anyway i would argue using text "Embedded JPEG preview" with raw images. I'm not aware of any current case though in future definitely we will have raw previews which won't be JPEG but i.e. PNG or other. So I suggest using "Embedded Preview" term only without format specification.

Also another rather unimportant though not very logical issue is displaying of normal (not RAW) images:

While the full size image is viewed I can see the text "Full Size Preview". Though in this case using word preview is misleading. This is not the preview, this actually Image. So I would suggest to use "Full Size Image" instead.

The same for "Half Size Preview" which actually is "Half Size Image".

Hi,

Is there any progress on this "bug"?

I agree with DrSlony (#46) on that "embedded preview / generate preview" or "embedded preview / decoded preview" with raw files would be clearer than current functionality.

Especially because in current Digikam there is still heuristics which choose raw decoded preview instead of embedded if the embedded preview image is less than half the size of the raw image. So if edit a Nikon raw file and crop it, the embedded preview image is not shown because Digikam thinks it's "too small".

This is especially confusing because Digikam still uses embedded preview for all other raw files, so we get a confusing mix of embedded previews and raw-decoded previews (and without any indication about which is which).

If Digikam gets non-destructive raw editing some day, this problem will probably raise its head more and more.

Any chance of getting rid of hard-coded heuristics and simply letting users to choose whether they want embedded previews or raw-decoded ones?

Matti Rintala

Matti,

This file still valid using digiKam 2.x serie ?

Gilles Caulier

For me it's fixed in Digikam 2.3 (and I think it's fixed for quite some already).

No, unfortunately the bug is not fixed.

Digikam still uses heuristics in addition to user settings to decide whether to use embedded jpg preview or raw decoding for showing the picture. I still can reproduce the following scenario with 2.4.1:

- I have "Embedded preview loads full-sized images" set
- Normally Digikam uses embedded jpg preview for Nikon NEF files.
- If I edit a Nikon NEF file with an external editor (Nikon Capture NX2) and crop it heavily, Digikam starts using its own raw decoding for its previews (apparently because the size of the embedded preview has dropped below a hard-coded threshold).

As I have said earlier, it's unlikely that anyone wants Digikam to automatically switch between embedded preview and raw decoded previews.

I suggest that current heuristics for disabling embedded previews are removed and that the UI text for "Embedded preview loads full-sized images" is changed to "Use embedded preview for raw files, if available" or something similar.

Matti

> I suggest that current heuristics for disabling embedded previews are removed
> and that the UI text for "Embedded preview loads full-sized images" is changed
> to "Use embedded preview for raw files, if available" or something similar.

This is appropriate for your special use case (using the NEF format's capabilities to the extent that the embedded JPG is an edited version of the RAW) but does not help to make it work with as many RAW formats as possible

Marcel,

I totally agree that Digikam should work with most (if not all) raw formats, not just NEFs, unedited or not. If I have given the impression that I just want to change Digikam to suit my personal tastes, I'm sorry.

In my opinion, the two raw preview approaches have the following pros and cons:

1) Using embedded jpeg preview produced by the camera (or a raw editing software)
- Its resolution depends on the raw format, some have full resolution previews, other formats only offer reduced resolution
- If the embedded preview is a low resolution image, it may be too small to evaluate sharpness of the photo
- Its appearance is equal to what the user saw at the camera's LCD, i.e. all camera's settings have been applied to the preview
- If the user has used camera settings to get close to what he wants, this is what he wants to see (he can then later try to recreate the look in raw editing, if the raw editor does not understand all camera settings)

2) Using Digikam's raw decoding to render a prevew
- It is always a full resolution preview, regardless of the raw format, so it's better for sharpness evaluation than a small resolution embedded preview
- Its appearance depends on the parameters of Digikam's raw decoding algorithm, which (to my knowledge) means that many/most camera settings are ignored.
- This in turn is ok if the user does not care about camera settings and wants the preview to represent the "starting point" for later raw editing.

My point is that regardless of the raw format, there are good and bad sides with both approaches. Only the user can know which he likes better. And that's why I think the user should be able to make the ultimate choice without being overridden by a hard-coded heuristics based on embedded preview size. And that's why I think the user preference UI text should clearly say which approach is chosen.

Of course, if the raw image containes no embedded preview, then raw-decoded previews could be used no matter what the user preference says.

Matti

I simply want to be able to make digiKam either use the embedded JPEG when I click on a raw photo (fast, but misrepresents the real raw data), or to generate a preview from the raw data (slow, but true to the raw data). In 2.5.0 I still can't do that - at least that's what I'm lead to believe, based on the 2-3 seconds it takes digiKam to show me a preview of a raw file after I click on it despite having "Thumbnail click action > show embedded preview" selected. I wrote about this specifically in bug 296651.

Some raw files have more than one embedded JPEG - at different resolutions. Which one you choose to display is of less importance to me than the speed with which I should be able to preview raw files, because as it is now, the 2-3 seconds is really annoying when sorting through 500 raw files when other software like Geeqie manage to display that embedded JPEG in a fraction of a second.

Hello,

I have been suffering a bit of from this issue as, since the heuristicis went in, I have been forced to preview all the raw files using the half size raw preview instead of an embedded jpg.  (I use Canon 350D, 40D and G9).  I thought this was "the way it is" until I discovered that my laptop viewed the same files with the embedded jpg.  This was strange since the laptop has a higher resolution screen than my primary system.  After poking around, I found that the heuristics were using the entire size of my desktop, which on a dual screen system is quite wide.  I submitted a patch to bug 296651 which uses the actual screen size instead, which I think is the intent.  Hopefully, it makes it into future releases as I think it may help others.

As to the above requests, would the following two dropdowns work?

Click on RAW Files
1. Load Embedded Preview  (always, unless fails)
2. Decode Half Size Raw
3. Decode Raw into Editor

Click on Non-RAW Files  <- Is there a better way to describe non-raw files?
1. Load Embedded Preview  (unless fails)
2. Load Full size Image
3. Start Image Editor

Best regards,

- Gerry

Matti,

What's about this file using last digiKam 4.2.0 ?

Gilles Caulier

Tested Digikam 4.4.0:

Preview: 
depending on Album View -> Preview Settings -> Embedded preview loads full size image 
either loads embedded JPEG preview (off) or half size RAW (on)

Light Table:
Copies Preview settings, ignores its own setting

Full Size RAW is never loaded.

Test file: several Canon CR2, PowerShot S95 and EOS 350D

Following up on the discussion in the mailing list and using Digikam 4.4.0 on Fedora 20:

Looking at the code, the choice of using the embedded JPG or Half Size Raw Preview depends (between other things) on the screen
resolution ...

It turns out my home 27" monitor is 2560x1440 and the Embedded JPG in the Canon S100 RAW files are 1600x1200.
Digikam therefore displays the Half Size Raw Preview no matter the setting in the preferences.
Similar behaviour for the Canon 400D RAW files whose embedded JPG are 2.5Mpx :-(
If I change my screen resolution to, say, 1920x1080, it displays the Embedded JPG !

However, the Embedded JPG of my Canon 60D CR2 are 17Mpx so for these RAW files it will indeed display the embedded JPG ...

The thing is that if I manually extract the 1600x1200 JPG embedded in my RAW file and display it in digikam, it's just fine ! on my screen, it's zoomed at 96% so it's completely acceptable to view images of this size as previews !

I'm therefore thinking that the heuristics are not good as is ... Maybe choice should be left to the end user by the addition of a second checkbox in the settings (as suggested in other comments) ?

Vincent,

I think i found where is the problem.

The option from digiKam/Config/AlbumView/Preview Load Full Image is well used by Preview widget. Setting is passed into these methods :

https://projects.kde.org/projects/extragear/graphics/digikam/repository/revisions/master/entry/libs/widgets/graphicsview/dimgpreviewitem.cpp#L301

https://projects.kde.org/projects/extragear/graphics/digikam/repository/revisions/master/entry/libs/widgets/graphicsview/dimgpreviewitem.cpp#L146

In both case d->loadFullImageSize is settings from Config dialog. Depending of bool value, we will call PreviewLoadThread::loadHighQuality() or  PreviewLoadThread::load().

 The first one is not a problem. As expected it load full image.

The second one use an argument which is the minimum image size to load (d->previewSize). Look how it set this integer value :

https://projects.kde.org/projects/extragear/graphics/digikam/repository/revisions/master/entry/libs/widgets/graphicsview/dimgpreviewitem.cpp#L82

Typically, size is bounded between VGA to WQXGA resolutions.

We need to be sure if this minimum image size is correctly set with you high resolution displays.

Gilles Caulier

Vincent,

The second point to check is code from PreviewTask is well adapted to all situation.
I add comment and i polished source code. Look here :

https://projects.kde.org/projects/extragear/graphics/digikam/repository/revisions/master/entry/libs/threadimageio/previewtask.cpp

Code relevant to load preview is given from line 211 to line 460. The rest is cache management and post processing. You can see the different between RAW and non RAW image.

Note that in your case, the "dcraw -e " to extract manually embedded JPEG is performed by KDcrawIface::KDcraw::loadEmbeddedPreview().

Gilles Caulier

Ok, I did a little gdb session and here's what I I saw:

First of all, my screen  resolution is 2560x1440
RAW file being displayed is 4000x3000
JPG preview embedded in the RAW file is 1600x1200

in dimgpreviewitem.cpp, the previewSize is set to 2560

Then, when stepping through previewtask.cpp

line 213: size = 2560 (which is the previewSize set before and = width of my screen)
line 241: bestSize = 4000 (this is the width of my RAW image)
line 247: bestSize = 2000 (4000 / 2)
line 259: aBitSmallerThanSize = 2048 (2560 * 0.8)
line 260: sizeLimit = 2000
line 293: KDCRaw loads the preview : width = 1600 and height = 1200 (correct)

The test takes us to line 305 (loadHalfPreview) since 1600 < 2000

So the behaviour is correct, all the sizes (screen , image, embedded jpg) are properly discovered.

The "problem" is therefore the heuristics because as I explained in the forum, displaying a 1600x1200 image on my screen is fine ! zoom level is 92%

If we're going to keep the heuristics to take the decision, then maybe we should look at the size of the preview area instead of looking at the entire screen resolution ?
On my system, where digikam is always maximized, that area is roughly 1700x1100 ...

It also makes sense if you are not using digikam in maximized mode, what's the point in looking at the entire screen size if we don't have access to it :/

Looking forward to hearing your thoughts !

Created attachment 89490 [details]
Use preview area size instead of screen resolution

Use the preview area size instead of the entire screen resolution when making the decision to load the embedded JPG preview or the half size preview.
This reflects the reality of the space available for display ! (window maximized or not, sidebars opened or not, etc)

Here's the behavior now on my WQXGA screen (2560x1440) with a 4000x3000 RAW file containing a 1600x1200 JPG preview:

Window maximized + Left sidebar open + Right Sidebar open:
digikam(1592)/digikam (core) Digikam::PreviewLoadingTask::execute: Preview Loading - size =  1688
digikam(1592)/digikam (core) Digikam::PreviewLoadingTask::execute: Preview Loading - bestSize =  4000
digikam(1592)/digikam (core) Digikam::PreviewLoadingTask::execute: Preview Loading - bestSize =  2000
digikam(1592)/digikam (core) Digikam::PreviewLoadingTask::execute: Preview Loading - aBitSmallerThanSize =  1350
digikam(1592)/digikam (core) Digikam::PreviewLoadingTask::execute: Preview Loading - sizeLimit =  1350
digikam(1592)/digikam (core) Digikam::PreviewLoadingTask::execute: Preview Loading - exiv2 Preview largest dimension >= sizeLimit (  1600  >=  1350  )
digikam(1592)/digikam (core) Digikam::PreviewLoadingTask::execute: Preview Loading - Loading embedded preview


Window maximized + Left sidebar open + Right Sidebar closed:
digikam(1592)/digikam (core) Digikam::PreviewLoadingTask::execute: Preview Loading - size =  1925
digikam(1592)/digikam (core) Digikam::PreviewLoadingTask::execute: Preview Loading - bestSize =  4000
digikam(1592)/digikam (core) Digikam::PreviewLoadingTask::execute: Preview Loading - bestSize =  2000
digikam(1592)/digikam (core) Digikam::PreviewLoadingTask::execute: Preview Loading - aBitSmallerThanSize =  1540
digikam(1592)/digikam (core) Digikam::PreviewLoadingTask::execute: Preview Loading - sizeLimit =  1540
digikam(1592)/digikam (core) Digikam::PreviewLoadingTask::execute: Preview Loading - exiv2 Preview largest dimension >= sizeLimit (  1600  >=  1540  )
digikam(1592)/digikam (core) Digikam::PreviewLoadingTask::execute: Preview Loading - Loading embedded preview


Window maximized + Left sidebar closed + Right Sidebar closed:
digikam(1592)/digikam (core) Digikam::PreviewLoadingTask::execute: Preview Loading - size =  2494
digikam(1592)/digikam (core) Digikam::PreviewLoadingTask::execute: Preview Loading - bestSize =  4000
digikam(1592)/digikam (core) Digikam::PreviewLoadingTask::execute: Preview Loading - bestSize =  2000
digikam(1592)/digikam (core) Digikam::PreviewLoadingTask::execute: Preview Loading - aBitSmallerThanSize =  1995
digikam(1592)/digikam (core) Digikam::PreviewLoadingTask::execute: Preview Loading - sizeLimit =  1995
digikam(1592)/digikam (core) Digikam::PreviewLoadingTask::execute: Preview Loading - kdcrawPreview largest dimension < sizeLimit (  1600  <  1995  )
digikam(1592)/digikam (core) Digikam::PreviewLoadingTask::execute: Preview Loading - Loading half preview

That's the behavior I expected.

Could someone please summarize what the final behavior is?

For my purposes, any preview which does not show me what I explicitly desire to see and is not intuitive and predictable is fundamentally flawed. I explained my suggested behavior in Bug 319241.

Current behavior is the same as before so for RAW files:

option checked in the settings:
 Always display Half Size preview

option unchecked in the settings:
 If embedded JPG is large enough, display it
 If not, display Half Size preview

The only difference is that with the patch I submitted, the decision whether the embedded JPG is large enough is based on the size of the preview area and not the entire screen resolution so behavior will change if the window is maximized or not, if the sidebars are opened or not, etc ...

In the end, I guess the idea is to support all possible RAW/EmbeddedJPG combination ... so if a RAW file's largest embedded JPG is much smaller than the preview area, there's no point in displaying it ! not good enough as a preview so instead let's display a fast processing of the RAW ...

I was originally in line with your requirements but then, thinking about it, I guess it could be problematic to FORCE loading the embedded preview because if you have RAW files with too small embedded JPGs, you would have to switch the option to view these files correctly, and then switch again for others, it would be unpractical.
I myself have 3 different types of RAW files and am not facing the situation I'm describing so I'm not serving my own purpose here ;) but I'm just guessing this is the reason these "heuristics" have been implemented in the first place ...

Thank you for explaining that. For people with screens 4MP and larger (or cameras that write small JPEG previews) the heuristics become messy, impossible to predict.

Taking this into account, I would propose the following choice for raw files (again empowering the user):
● Automatic preview (this is using your heuristics).
○ Fast demosaiced image (full resolution fast demosaic).
○ Fast demosaiced half-resolution image (optional choice mayb useful for those with 36MP cameras).
○ High quality demosaiced image (AMaZE or whatever).
○ Largest embedded JPEG preview.
○ Largest embedded JPEG preview, high quality demosaiced image on keyboard shortcut.

Explanation for the last option: the JPEG preview is almost never representative of the raw data. When checking hundreds of raw shots for sharp focus, the JPEG is enough. When checking for over/under-exposure, demosaicing is the safe choice. While quickly browsing through shots, would be nice to be able to hit a keyboard shortcut to quickly render a demosaiced preview to screen. Yes this last option is a separate feature request, but I thought I'd mention it here as it involved the other changes.

The heuristics were indeed implemented with regards to some RAW images (older formats in my test collection) which only had very small embedded previews.

I am open for a refined proposal here.

I the fast demosaiced algorithm readily available for us? What we need should find its way into libraw for us to use it.

No problem.

amsterdam.pef from http://rawtherapee.com/shared/test_images/amsterdam.pef
YA-DSC_1900.NEF from http://www.imaging-resource.com/PRODS/nikon-d810/nikon-d810GALLERY.HTM

Demosaicing times (median of 5 runs):
Pentax K10D PEF 10.1MP using "fast" took 74ms.
Pentax K10D PEF 10.1MP using "AMaZE" took 368ms.
Nikon D810 NEF 36.3MP using "fast" took 251ms.
Nikon D810 NEF 36.3MP using "AMaZE" took 1292ms.

Gentoo Linux 3.17.2 x86_64 Intel(R) Core(TM) i7 CPU Q 820 @ 1.73GHz
Used RawTherapee:
for n in {1..5}; do ~/rt_default_Release_stopwatch/rawtherapee -o /dev/null/foo.tif -p /tmp/fast.pp3 -b8 -t -c /tmp/YA-DSC_1900.NEF; done

"Fast"
https://code.google.com/p/rawtherapee/source/browse/rtengine/fast_demo.cc

"AMaZE"
https://code.google.com/p/rawtherapee/source/browse/rtengine/amaze_demosaic_RT.cc

Although I'm not currently using Digikam in my workflow (this may change again in the future), I'll add a small comment as I originally filed this "bug".

The embedded preview and previews created from raw data have really different processing done to them (by the camera for embedded previews, by the raw decoder in the other case). The resulting image may look very different. That's why I think it's important that what is shown to the user is *consistent*, otherwise it becomes difficult to preview images if they sometimes have camera's processing done to them, sometimes not.

Making the choice based on screen resolution is problematic if the user does changes between monitors (laptop user sometimes using external display, sometimes laptop's screen). Using the preview are size is even more problematic if Digikam is not used fullscreen, then just changing the window size may change the preview method (if I understood correctly).

I personally vote for letting the user choose the method and not using heuristics. But since the heuristics may be good for people with multiple cameras, what about letting the user choose the minimum embedded preview size? If the embedded preview is larger than this size, it is show. Otherwise a preview is generated from raw data?

Git commit b56fc72ca455da8844011e78880e0cd1dbf4bd81 by Marcel Wiesweg.
Committed on 14/11/2014 at 17:57.
Pushed by mwiesweg into branch 'master'.

Adapt image preview loading to use the new settings.
PreviewTask is restructured and much cleaner now.

M  +3    -1    libs/threadimageio/loadingdescription.cpp
M  +4    -10   libs/threadimageio/loadingdescription.h
M  +37   -14   libs/threadimageio/previewloadthread.cpp
M  +22   -5    libs/threadimageio/previewloadthread.h
M  +204  -228  libs/threadimageio/previewtask.cpp
M  +13   -3    libs/threadimageio/previewtask.h
M  +2    -2    libs/threadimageio/thumbnailloadthread.cpp

http://commits.kde.org/digikam/b56fc72ca455da8844011e78880e0cd1dbf4bd81

Git commit 9ddedf90d4de2a504a0feb71de5064b209207422 by Marcel Wiesweg.
Committed on 14/11/2014 at 17:58.
Pushed by mwiesweg into branch 'master'.

Apply settings UI for previews:
() Fast preview
() High quality preview
	RAW: - Automatic
	     - Always embedded preview
	     - Always half-size demosaic

M  +43   -2    app/settings/applicationsettings.cpp
M  +3    -2    app/settings/applicationsettings.h
M  +4    -4    app/settings/applicationsettings_iconview.cpp
M  +2    -2    app/settings/applicationsettings_p.cpp
M  +5    -1    app/settings/applicationsettings_p.h
M  +1    -1    app/utils/slideshowbuilder.cpp
M  +2    -1    app/views/imagepreviewview.cpp
M  +48   -14   utilities/setup/setupalbumview.cpp

http://commits.kde.org/digikam/9ddedf90d4de2a504a0feb71de5064b209207422

Git commit 7edc98d1efc9da7104bdd13379295047029d9fc5 by Marcel Wiesweg.
Committed on 14/11/2014 at 18:00.
Pushed by mwiesweg into branch 'master'.

Adapt to PreviewSettings everywhere.
Import tool has its own, simpler settings.
LightTable reuses main settings.

M  +1    -0    CMakeLists.txt
M  +2    -1    libs/widgets/graphicsview/dimgitemspriv.h
M  +16   -19   libs/widgets/graphicsview/dimgpreviewitem.cpp
M  +2    -1    libs/widgets/graphicsview/dimgpreviewitem.h
M  +4    -1    utilities/importui/views/importpreviewview.cpp
M  +3    -3    utilities/lighttable/lighttableview.cpp
M  +3    -1    utilities/lighttable/lighttableview.h
M  +6    -4    utilities/lighttable/lighttablewindow.cpp
M  +1    -1    utilities/lighttable/lighttablewindow.h
M  +1    -6    utilities/maintenance/fingerprintstask.cpp
M  +1    -6    utilities/maintenance/imagequalitytask.cpp
M  +5    -5    utilities/setup/setuplighttable.cpp
M  +10   -24   utilities/slideshow/slideimage.cpp
M  +3    -1    utilities/slideshow/slideimage.h
M  +1    -1    utilities/slideshow/slideshow.cpp
M  +0    -1    utilities/slideshow/slideshowsettings.cpp
M  +2    -1    utilities/slideshow/slideshowsettings.h

http://commits.kde.org/digikam/7edc98d1efc9da7104bdd13379295047029d9fc5

I close this file as resolved for 4.6.0, following last commits from Marcel in comments #71 #72 and #73

Re-open if necessary.

Gilles Caulier