Bug 209000

Summary: More options when converting raws
Product: [Applications] digikam Reporter: Me <linuxguy123>
Component: Plugin-DImg-RAWAssignee: Digikam Developers <digikam-bugs-null>
Status: RESOLVED FIXED    
Severity: wishlist CC: caulier.gilles, languitar, linuxguy123
Priority: NOR    
Version: 1.0.0   
Target Milestone: ---   
Platform: Fedora RPMs   
OS: Linux   
Latest Commit: Version Fixed In: 5.1.0
Sentry Crash Report:
Attachments: Editor screenshot.
setup editor page
Raw Import tool in action (1)
Raw Import tool in action (2)
Raw Import tool in action (3)
digiKam First Run : Raw File Handling Settings
digiKam Batch Queue Manager Converting RAW file to PNG plus other actions

Description Me 2009-09-30 17:11:30 UTC
Version:            (using KDE 4.3.1)
OS:                Linux
Installed from:    Fedora RPMs

Unless I am shooting action situations, I shoot everything in raw.  I am presently shooting with a 40D, but I'll be getting a 7D in the near future. 

Prior to digiKam, I converted all my images with ufraw batch from the command line.  I really like how digikam allows one to preview the raw files prior to converting them and then convert them in a batch using ufraw.  Nicely done.  However, it would be really, really nice if the raw conversion process allowed a lot more of the ufraw options to be utilized.  

For me, it would be really nice if the following ufraw options were accessible from digikam: 

1) Either --shrink or --size.  Maybe both ?  (Leave it up to users to always want both when there is an option between two things !) 

Although digiKam allows me to view my raw files without converting them, I almost always convert all my raws to jpgs.  However, with the number of images I capture a year and the size of the files, I often want to create small jpgs, not full sized ones, for general use (emailing, posting to the web, use in documents, etc.) and viewing.  I only generate large jpgs (and tiffs) for images that I want to either print or manipulate.  Thus I would like the raw conversion process in digikam to allow me to use --shrink or --size.

2) --base-curve

It would be nice to be able to tell digikam what type of curve to use.

3) --curve-file

It would be really nice to be able to go into a folder of raws, open up the first one with ufraw, make a curve and save it and then convert the rest via digikam using that curve.  Without this option, I am back to running ufraw-batch manually from the command line OR manually applying curves to jpgs after the fact. 

Another thing: it would be really nice if digikam had a single place for storing curves or if it went through the hierarchy and found all the curves so that one could use a curve from one directory in another directory. 

4) --exposure, -- restore and --clip

5) --wb

6) --create-id

This is probably an advanced feature, but I've found it to be really useful.  Lets say we go into a directory and convert some raws to jpgs and do a great job.   A month later we look at those jpgs and want to convert some other files from the directory OR we want to make them a different size.  Without the ufraw-id file, we have no idea how we created the jpg.  With the ufraw-id file, all we have to do it open it up and look.  And then copy it and use it for the new images.  Ideally, eventually the raw conversion process would allow the user to specify a ufraw-id file to use as a default template for another conversion.

I think these options would handle almost everything that I would need to control the raw conversion process.

I understand that this list might be a little daunting to implement all at once.  A fairly simple way to allow advanced users access to this functionality without too much development work would be to allow the user to pass ufraw options directly to the command line.  Maybe have a text box that the user could put --shrink=2 and   such in.  

I see a few problems to this approach:  

1) users could put in options that contradict what digikam already uses.  Solution: put the digikam options in the box and let the user edit them too. 

2) users could put in options that generate errors.  Like --base-curve-file=<file that doesn't exist>.  Solution: show the user the error that occurs if one does.

3) users could mess up the options to the point where they can't get ufraw to work at all.  Solution: have a "Restore defaults" that puts things back to just what digikam uses.

If I had to pick any of the command line options that I would like to see put into a gui, it would be the use of the curve file, ie give the user a file select dialog to select the curve file.  I say this because I might generate 3 or more curves for raw files in a directory depending upon the lighting situation when the images were shot and thus I am always specifying a different curve for various files.  Doing it on the command line with --base-curve-file="somefilename" requires me to "see" the filenames that I used (ie have konsole or Dolphin open) and I frequently mess them up. 

Thanks for listening.  I hope this has been helpful and I look forward to hearing your comments on it.
Comment 1 Me 2009-09-30 17:14:41 UTC
This should have been tagged as a wish, not a bug.  Sorry. 

I should also add that I develop software and I might be willing to implement this functionality myself if I got a little guidance from the development team in terms of how they would like it done.  It would be really helpful if someone drew laid out the gui how they would like it.
Comment 2 caulier.gilles 2009-09-30 17:23:04 UTC
digiKam don't use ufraw to process RAW files, in single or batch mode.

We use libraw with libkdcraw...

If something must be improved with digiKam, it must be done in libraw or/and libkdcraw...

More info : www.libraw.org

Gilles Caulier
Comment 3 Me 2009-09-30 22:34:03 UTC
Sorry. For some reason I though digikam was using ufraw as its converter.  I didn't mean any slight to digikam by my mistake.

Having said that, most of the features I am asking for, if not all, map directly to components of the dcraw like API that libraw provides and implementing them should be fairly straightforward except for the fact that a GUI must be designed so that the user can access the features. 

If digikam were using ufraw as the converter, these features could have been made usable to the end user via a command line options text box.  The ease of which this could have been done is what prompted me to ask for this enhancement like I did.

So... could "we" implement a GUI that would allow digikam to make libraw to do some of the things I requested ?  Ie do you see the workflow case for what I am asking for ?

Thanks
Comment 4 caulier.gilles 2009-09-30 22:54:00 UTC
No, we don't use ufraw. It's not a library.

libraw is fine for digiKam. A lots of new features are available in current code, but not yet used in libkdcraw.

Note libraw code is in libkdcraw core, for simplicity...

Look there : http://websvn.kde.org/trunk/KDE/kdegraphics/libs/libkdcraw/

digiKam use libkdcraw interface which provide QT/KDE access to libraw and some widgets.

libkdcraw is used everywhere in digiKam and kipiplugins. Mst relevant code are there :

Raw image loader for digiKam core (DImg class) :

http://websvn.kde.org/trunk/extragear/graphics/digikam/libs/dimg/loaders/rawloader.cpp?view=markup

Raw decoding settings for digiKam :

http://websvn.kde.org/trunk/extragear/graphics/digikam/libs/dimg/drawdecoding.h?view=markup

digiKam editor raw import tool :

http://websvn.kde.org/trunk/extragear/graphics/digikam/utilities/imageeditor/rawimport/

kipi-plugins raw converter :

http://websvn.kde.org/trunk/extragear/graphics/kipi-plugins/rawconverter/

Gilles
Comment 5 Me 2009-10-01 17:58:32 UTC
I just reread my original post and did some thinking.   I feel that the original feature requests are reasonable, ie they are what an advanced user would want to do.  Including:

1) the ability to use a raw converter utility of some sort (ufraw, raw studio or something else) to create a curve file and then apply it to the batch conversion that will be done.

2) having the conversion process create conversion id file like ufraw does so that one can see what was done to generate an image and repeat it if desired.

So the question becomes how hard is this to do with the libraw libraries.  

In order to do item #1, digikam would need to build in a raw generator/editor tool similar to ufraw or raw studio.  Or it could use the output of a tool like those as the input to its raw conversion process. 

I think item #2 is even more work to do.

Basically we are talking about building a raw studio or ufraw application into digikam.  Do we want to re-invent the wheel ?

What if digikam has a preferences option that allowed the user to a) use the built in libraw tools as they are now or b) allowed the user to use ufraw (ufraw-batch).  This is what the grip application does for specifying the ripper and enocders.  See Grip->Config->Rip->Ripper and Grip->Config->Encode.

I know its a bit of work to implement but it would allow the user a lot more functionality.

Thinking ahead, the Windows users are really going to want this sort of stuff because they have access to the OEM raw conversion tool (ie DPP from Canon) as well as tools from Adobe and others.

To limit digikam to just its built in raw converter without being able to use and manage curve files and generate conversion id files will severely limit its desirability for advanced raw users. 

The more I think about this the more I think the end user needs to be able to specify which raw converter they want to use and be able to set it up to use more or all of its features. 

Comments ?
Comment 6 caulier.gilles 2009-10-01 18:08:05 UTC
... Do you hae tested RAW Import tool for digiKam image editor ? Go to setup/editor and set on right option...

After that, look what is missing in import tool settings.

For batch conversion, kipi tool will become obsolete, because in digiKam a new Batch Queue MAnager is implemented, but RAW Import tool is not yet added (RAW conversion is done using default RAW conversion settings as well.

I recommend to check out code from trunk and test it, instead to use 0.10.0

Gilles Caulier
Comment 7 Me 2009-10-01 22:33:08 UTC
"... Do you hae tested RAW Import tool for digiKam image editor ? Go to
setup/editor and set on right option..."

Where do I find this ?

$ rpm -q digikam
digikam-0.10.0-2.fc11.i586

"After that, look what is missing in import tool settings."


The only things I see related to raw settings are:

Gallery View -> Settings-> Raw Decoding (libraw0.7.2)
Gallery View-> Settings -> Kipi Plugins - shows RawConverter is enabled.

I don't see anything in the image editor.

"For batch conversion, kipi tool will become obsolete, because in digiKam a new
Batch Queue MAnager is implemented, but RAW Import tool is not yet added (RAW
conversion is done using default RAW conversion settings as well."

Maybe I could write a ufraw kipi tool ?

"I recommend to check out code from trunk and test it, instead to use 0.10.0"

I might do that.  Is that where this new functionality is ?
Comment 8 caulier.gilles 2009-10-01 23:10:23 UTC
>"... Do you hae tested RAW Import tool for digiKam image editor ? Go to
>setup/editor and set on right option..."

>Where do I find this ?

Setup, editor page, "Use Raw import tool" option...

>"After that, look what is missing in import tool settings."

>The only things I see related to raw settings are:

>Gallery View -> Settings-> Raw Decoding (libraw0.7.2)
>Gallery View-> Settings -> Kipi Plugins - shows RawConverter is enabled.

>I don't see anything in the image editor.

really. check again...

>"For batch conversion, kipi tool will become obsolete, because in digiKam a new
>Batch Queue MAnager is implemented, but RAW Import tool is not yet added (RAW
>conversion is done using default RAW conversion settings as well."

>Maybe I could write a ufraw kipi tool ?

Definitively... NO

>"I recommend to check out code from trunk and test it, instead to use 0.10.0"

I might do that.  Is that where this new functionality is ?

http://websvn.kde.org/trunk/extragear/graphics/digikam/NEWS?revision=1030132&view=markup

Gilles
Comment 9 Johannes Wienke 2009-10-01 23:17:15 UTC
(In reply to comment #8)
> >"... Do you hae tested RAW Import tool for digiKam image editor ? Go to
> >setup/editor and set on right option..."
> 
> >Where do I find this ?
> 
> Setup, editor page, "Use Raw import tool" option...

I also had to ask to find this option. Maybe we should think about making this option more prominent in the gui...?
Comment 10 caulier.gilles 2009-10-01 23:23:47 UTC
In 1.0.0, there is a First Run Assistant where the question is clearly asked to enable or not this option...

Gilles Caulier
Comment 11 Me 2009-10-02 00:52:40 UTC
Created attachment 37310 [details]
Editor screenshot.

I don't see a "Setup" menu item anywhere.
Comment 12 caulier.gilles 2009-10-02 06:22:09 UTC
Created attachment 37313 [details]
setup editor page

use "Settings/Configure digiKam" menu entry
Comment 13 caulier.gilles 2009-10-02 06:23:31 UTC
Created attachment 37314 [details]
Raw Import tool in action (1)
Comment 14 caulier.gilles 2009-10-02 06:24:23 UTC
Created attachment 37315 [details]
Raw Import tool in action (2)
Comment 15 caulier.gilles 2009-10-02 06:25:11 UTC
Created attachment 37316 [details]
Raw Import tool in action (3)
Comment 16 Me 2009-10-02 06:57:28 UTC
OK, found it.   

For future reference the way I would have told someone to find that is digikam-> Settings-> configure Digikam -> Image Editor -> Raw... 

I'll have a look at what is there in depth tomorrow and compare it to my list above.  It looks like a good start.  It looks really good actually.

I suggest that this functionality option should be added/moved to the Raw area of the configuration list rather than remain in the editor area.  While its true that the functionality is used in the editor, its also used for batch conversions from the album view area and I think users would more likely associate it with a raw attribute than an editor attribute. 

Just my $0.02.
Comment 17 caulier.gilles 2009-10-02 07:49:36 UTC
>I suggest that this functionality option should be added/moved to the Raw area
>of the configuration list rather than remain in the editor area.

Raw Import tool is for Editor and editor only... This si why it's in Editor section

>While its true that the functionality is used in the editor, its also used for >batch conversions from the album view area and

RAW section is all default demosaicing settings used when this option is not enabled in editor AND for batch RAW conversion in Batch Queue Manager.

In other word, In BQM, we need to find RAW Import setting as base tool... It still TODO.

Gilles Caulier
Comment 18 caulier.gilles 2009-10-02 07:53:06 UTC
Created attachment 37318 [details]
digiKam First Run : Raw File Handling Settings
Comment 19 caulier.gilles 2009-10-02 07:58:34 UTC
Created attachment 37319 [details]
digiKam Batch Queue Manager Converting RAW file to PNG plus other actions
Comment 20 Andi Clemens 2009-10-02 11:46:20 UTC
The first start wizard is only shown to people that do a fresh install of digiKam, all the others will never see this screen.
Is there a reason why we don't enable it by default?
Sure people with an existing config will still not have the feature enabled and have to search for this option.
I suppose the wizard uses the raw import tool option as a default, right?

But still my question is: Why not make it default and completely remove this option? What is the benefit of NOT using the tool?
Comment 21 caulier.gilles 2009-10-02 12:25:29 UTC
The first approach of digiKam is to be simple.

This is the goal of LightZone. You click on RAW file and all is done automatically. RAW handling is the same than JPEG.

In fact, the world photography must be done like this. A Lots of photograph don't care, don't know, don't want to understand what's RAW files, but know that RAW is better than JPEG from color depth viewpoints.

Users only know advantages, not disadvantage.

So, the default option is to open and let's digiKam/Libkdcraw/libraw handle raw  as well, without to ask any questions about demosaicing.

This is not my case of course, and for pro users sure... I only edit RAW files with in-deep settings adjustments.

Of course, auto-gamma/WB with raw must be improved. This is the job of libraw, not to digiKam libkdcraw. This is why, i update frequently libraw source code in libkdcraw...

The current way of RAW in photography is really a shame. I hope in the future that JPEG-XR will change that. Yes, i know it's M$ stuff, but all is ready to be included in camera firmware. 

http://en.wikipedia.org/wiki/JPEG_XR

After that, there is no reason to support RAW format in camera anymore.

Gilles Caulier
Comment 22 Me 2009-10-06 18:00:11 UTC
I'll provide further comment after I have done some work in 1.0.0 beta 5.  I am building it now.
Comment 23 caulier.gilles 2012-09-23 08:10:47 UTC
Any fresh comments since comment #22 ???

Gilles Caulier
Comment 24 caulier.gilles 2014-08-29 22:55:07 UTC
We need a fresh feedback here using last digiKam 4.2.0.

Gilles Caulier
Comment 25 caulier.gilles 2015-06-28 09:40:51 UTC
New digiKam 4.11.0 is available :

https://www.digikam.org/node/740

Can you reproduce the problem with this release ?

Gilles Caulier
Comment 26 caulier.gilles 2016-07-15 18:54:08 UTC
With digiKam 5.0.0, this problem is not reproducible.
I close this file now. Don't hesitate to re-open if necessary.
Gilles Caulier