Unexplainable results for Highight/Shadow Warning when using DPP4

[aajax]

As a bit of a novice when it comes to photo processing I've been developing my Canon raw files using both Digital Photo Professional (DPP4) & Rawtherapee (RT). While they are quite different in the way they work the ultimate goal is the same. In that, produce the best possible picture from a given raw file. One of the things that appears to be fairly similar, in purpose anyway, is what DPP4 calls highlight/shadow warning. A big problem that I'm having is that photos which I develop in DPP4 that show NO clipping show very large amounts of clipping when processed by RT. For example, a .tif file created by DPP4 with no clipping contains massive clipping when opened in RT.

First problem has to do with setting the values for Threshold. The values (0-64 for shadows, 192-255 for highlights) appear to pertain to a color space (gamut ?) with 8 bits of color depth. It seems to me that raw files are always (aren't they?) using more than 8 bits of color depth. Is it possible that these values are substitutes for what would be equivalent for the applicable color depth which I believe varies from camera to camera?

Next, I assume that the clipping also pertains to the applicable color profile. In the settings tab there is a selection called "Work Color Space". The values that can be selected have very general sounding names such as sRGB, AdobeRGB, Wide Gamut RGB, etc. Ultimately, these have to correspond to color profiles (i.e., files that specify the color space specifics). Don't they?

I'm thinking that the difference between what RT & DPP4 indicate, when it comes to clipping, may be attributed to the difference in working profiles. In the case of RT I can locate the files that specify these working profiles. However, it remains unclear what actual (icc/icm type) profile files are being used by DPP4. I'm thinking I should be able to experiment by using the same profile files in both RT & DPP4. Does this make sense in the case of DPP4? Can I supply my own work color space profiles for DPP4? If so how?

I should point out that I do use a calibrated monitor with the same ICC profile file being used with both DPP4 & RT. I've also learned that viewing pictures on display devices can very quite a lot. As a result, my goal when developing photos is to produce a print that I like. What the subject digital image file will look like on a variety of display devices is something I cannot control. Therefore, I don't try.

In summary, from what I see now it looks like RT provides meaningful feedback when it comes to clipping but NOT so with DPP4.

[Peter]

A raw sample (and the exported tif file) would be nice. Use a filehost like Google Drive.

[aajax]

Here is a link to an archive in zip format which contains a collection of relative files.  Included is a ReadMe.txt file which tries to explain the contents.  Sorry but it is pretty large.  I thought I should send what I'd actually experienced.

[Peter]

Ok, tried myself to export a tif file from DPP and import it into RawTherapee. The highlight clipping warning shows a bigger area in RawTherapee than in DPP4. Discuss.pixls.us may be a better place for the question.

Any reason you want to import a tif file into RawTherapee instead of working with the raw file?

[aajax]

No!  The idea of importing the .tif to RT was simply to try and create a situation where both applications were dealing with the same thing.  In that, same data, nothing done but inspect the clipping indicator.

My normal workflow involves using both applications to develop the same raw file, separately, purely to see which gets a preferred result.  In practice, each application sometimes produces the preferred result.

The reason for identifying clippings is to guide the editing process.  DPP4 allows producing what is expected to be an unclipped image file that actually is severely clipped at least according to RT.  If there is some reason to believe which one is better (i.e., more correct) I don't know what it is.  However, I'm pretty sure they cannot both be right.

[Mark35mmF2]

Hi there,

I am not 100% sure, but I suspect this has to do with the RAW processor in each program treating the raw data differently. Since RawTherapee is going to have their own proprietary algorithm for treating your Canon RAW files which will be different from Canon's RAW processing algorithm the translation of those initial values will be different. This can also be exacerbated by the in-camera settings like the PictureStyle, noise reduction, and other settings which are still applied to the RAW image file by the camera via metadata which can be read and applied by DPP (while still not baked in, so they can be adjusted at any time) but not by other programs. Since the processing algorithm is different and these other settings will affect the file in DPP, applying the exact same edit in two different programs will produce varied results. Since you point out the highlight and shadow clipping specifically I suspect this may also be impacted by highlight optimization and dynamic range settings in your camera as well.

[aajax]

I don't dispute any of what you (Mark35mmF2) say but this experiment tried to eliminate everything but clipping. That is why it is based on a .tif format file rather than a raw file. Yes the image came from a raw file which was used to produce the .tif file. However, the comparison between RT and DPP is based on a single .tif file.

As to the "in camera" complications that you mention, I think this explains why I don't do it (i.e., picture style). I'm both a novice and an old timer. When it comes to camera operation I understand aperture & shutter priority which is just about the only thing that I use. I'm aware that I can change the color profile from sRGB to Adobe RGB in the camera and have thought this might be something I'd like to try but for the moment I'm focused on learning how to develop raw files using software.

I'd like to try my experiment using the same working profile (i.e., what DPP appears to call "work color space" in Settings). However, I could use some help figuring out how to interchange those profiles in the case of DPP.

[Peter]

I checked your files now. I can get similar highlight warning in RawTherapee as in DPP4 when I use 253 for highlights and the luminosity preview. From RawPedia about Clipping indication:

"The clipped highlight indicator will highlight areas where at least one channel lies at or above the specified highlight threshold. If you want to see only where all channels are clipped, then enable the luminosity preview mode in addition to the clipped highlight indicator."

How does DPP4 work? Showing just one clipped channel or where all channels are clipped?

And for your raw file you have 0,5-0,7 stop before clipping the highlights (if you don't count the 0,000249654% of the highlights clipped in some reflections), but I suppose you already knew about that and that wasn't your question.

[aajax]

---

@Peter wrote:

...

"The clipped highlight indicator will highlight areas where at least one channel lies at or above the specified highlight threshold. If you want to see only where all channels are clipped, then enable the luminosity preview mode in addition to the clipped highlight indicator."

---

I am familiar with that description from Rawpedia.  Are you saying that DPP does NOT work that way?  I suppose in my examples the result I received might be explained if DPP expects all channels to be above rather than just one.

While I've never changed the preview modes to which you refer I do see what you are saying but it is NOT clear to me how this helps.  If DPP can do such I haven't been able to figure that out.  My thinking has been that I'm wanting to know how each application is able to do similar things.

---

@Peter wrote:
..

And for your raw file you have 0,5-0,7 stop before clipping the highlights (if you don't count the 0,000249654% of the highlights clipped in some reflections), but I suppose you already knew about that and that wasn't your question.

---

You've lost me here but I think you are discussing properties of the raw file which is something that I know next to nothing about.  I suppose it would be useful to know if the raw file is hopelessly clipped.

Can you elaborate on what it means when you say "your raw file you have 0,5-0,7 stop".  I think you are probably correct when saying "that wasn't your question".  However, I don't get the point you are making and would be grateful for some help doing so if only to improve my overall understanding of this subject.

Finally is the last of your 3 screenshots from some other software used to analyze raw files?

[Peter]

I see. I can check your sample later. Meanwhile, if you just want to check the highlight clipping in the raw file you will need for example Hraw or darktable (white balance module turned off).