Incorrect white level for Canon 5D Mark II at ISO 320

[dhoulder]

Observed in darktable 5.4.1, rawspeed at commit 4c511d6

CR2 file from a Canon 5D Mark II shot at ISO 320. Available here

The automatically determined white level is 15312. The sky is clipped, showing the expected magenta hue, but the raw clipping indicator and highlight reconstruction indicator show no clipping.

Manually setting the white level to 12965 activates highlight reconstruction and the raw clipping indicator.

That value is very close to what used to be in cameras.xml for ISO 320

git show 6a54251d
…
        <Camera make="Canon" model="Canon EOS 5D Mark II" decoder_version="1">
                <ID make="Canon" model="EOS 5D Mark II">Canon EOS 5D Mark II</ID>
                <CFA width="2" height="2">
                        <Color x="0" y="0">GREEN</Color>
                        <Color x="1" y="0">BLUE</Color>
                        <Color x="0" y="1">RED</Color>
                        <Color x="1" y="1">GREEN</Color>
                </CFA>
                <Crop x="158" y="51" width="5634" height="3753"/>
                <Sensor black="1024" white="12995" iso_list="160 320 640 1250"/>
                <Sensor black="1024" white="15950"/>

I have been through my entire collection of images form this camera. ISO 160, 640 and 1250 also seem to be affected, as the cameras.xml snippet suggests. Other ISO values seem to have the correct white level, in that I can find images where the automatically determined white level does trigger the raw clipping indicator. I'm happy to provide more samples upon request.

[LebedevRI]

If the "right" white level was 14800, then that would mean we are using the wrong tag:

$ EXIFTOOL img_2578.cr2 | grep -i "white"
[EXIF]          0xa403 WhiteBalance                    : Manual
[MakerNotes]    0x0007 WhiteBalance                    : Daylight
[MakerNotes]    0x006f WhiteBalance                    : Daylight
[MakerNotes]    0x0006 WhiteBalanceRed                 : 0
[MakerNotes]    0x0007 WhiteBalanceBlue                : 0
[MakerNotes]    0x02cf NormalWhiteLevel                : 14800
[MakerNotes]    0x02d0 SpecularWhiteLevel              : 15312

but given ~12k, it would mean there is some other way to pick up the white level rather than a single makernotes field.

[LebedevRI]

(or, that it is simply not encoded in metadata at all, and is supposed to be somehow picked up from the image itself)

[dhoulder]

Maybe NormalWhiteLevel and SpecularWhiteLevel are just not reliable for this camera at certain ISOs. Looking at https://exiftool.org/TagNames/Canon.html#ColorData4 it looks like ColorData4 was the first to include these two fields. Maybe Canon just failed to implement this properly in some of their first consumer-grade cameras. I see that the <Sensor> tags in cameras.xml take precedence over the MakerNotes levels, so I added

<Sensor black="1024" white="12995" iso_list="160 320 640 1250"/>

back into cameras.xml and that seems to work. I had to use white="12810" to get the raw clipping indicator to work in this case, but maybe that's because there's no specular white in this image. I have other images at ISO 160, 320, 640 and 1250 where white="12995" seems to be about right though, so maybe just putting that <Sensor> tag back might be a reasonable approach. The white level from the MakerNotes for the other ISOs seems to be correct.

[dhoulder]

Just adding that <Sensor> tag causes a crash near

rawspeed/src/librawspeed/decoders/RawDecoder.cpp

Line 270 in 7cf3dc3

if (const CameraSensorInfo* sensor = cam->getSensorInfo(iso_speed)) {

for images not in iso_list, so this needs more work. I'll keep investigating.

[dhoulder]

I've analysed all the CR2 files that I have (from the 5D Mark II and 5D Mark IV) using exiftool to extract the relevant metadata. Results below.

For the Mark II, NormalWhiteLevel is mostly stuck around 14800, and it sometimes varies within a given ISO value for reasons that aren't clear to me. However, NormalWhiteLevel is never around 12995, which is where it needs to be for some images. For what it's worth, SpecularWhiteLevel varies a lot. I also looked at the firmware version and the highlight tone priority setting but couldn't see any obvious correlation with the results.

For the Mark IV, the NormalWhiteLevels seem reasonable because highlight reconstruction seems to get activated appropriately in darktable for all images from this camera. The NormalWhiteLevel is also deterministic for a given ISO — for the Mark II it's not.

My tentative conclusion is that NormalWhiteLevel for the Mark II is not reliable for detecting clipped highlights because there are many cases where it isn't low enough.

Looking at

rawspeed/src/librawspeed/metadata/Camera.cpp

Line 418 in d775714

const CameraSensorInfo* Camera::getSensorInfo(int iso) const {

it looks like it wants <Sensor> elements to cover the entire ISO range not just the iso_list cases, which sounds reasonable. In that case, putting something like

<Sensor black="1024" white="12995" iso_list="160 320 640 1250"/>
<Sensor black="1024" white="15950"/>

back into cameras.xml will probably do the trick.

I'll test this and generate a pull request if it works as intended.

Canon EOS 5D Mark II

ISO	NormalWhiteLevel	image count	LinearityUpperMargin	SpecularWhiteLevel
100	14800	1257	10000	15312, 15499, 16383, 15398, 16164, 15890, 16208, 15629, 15340, 16049, 15485, 15904, 15788, 15991, 15413, 16352, 16092, 15369, 15427, 15803, 16034, 16020, 15326, 15919, 15355, 15875, 15528, 16265, 15557, 15442, 15658, 15861, 15846, 16237, 15817, 16309, 15644
125	14800	84	10000	15312
160	14800	133	10000	15312
200	14800	891	10000	15312, 15658, 15861, 15355, 15629, 15326, 15601, 15890, 15543, 15514, 15716, 16383, 16323, 16005, 16078, 16092, 16034, 16237, 15528, 15427, 15760, 15456, 15557, 15369, 15413
200	15120	26	10000	15632, 15661
250	14800	135	10000	15312, 15658, 15745, 16383, 15369, 15398, 15384, 15615, 15673, 15572, 15514, 15340
320	14800	49	10000	15312, 16383, 16251, 16034
400	14800	529	10000	15312, 15355, 15384, 16078, 16383, 15760, 15413, 15514, 15470, 16381, 15817
400	15120	17	10000	15632
500	14800	76	10000	15312, 16383, 15470, 15890, 15760
640	14800	63	10000	15312, 15499, 15485, 15326, 15875, 15442, 15716
800	14800	129	10000	15312, 15976, 16383, 15485, 15340, 15427, 15846, 15919
800	15120	6	10000	15794, 15632, 15956
800	14899	3	10067	15411
1000	14800	70	10000	15312, 16383, 15384, 15340, 15658, 15832, 15904, 15890
1250	14800	40	10000	15312, 15890
1250	14854	11	10037	15366
1600	14800	39	10000	15312, 15745, 15673, 15629, 16383, 15514, 15499, 15774, 15384, 15832
1600	14899	2	10067	15411
1600	14854	1	10037	15366
2000	14800	17	10000	15543, 15312, 15832, 15644
2000	14854	10	10037	16178, 15366
2500	14800	20	10000	15312, 16383, 15947, 15629, 15615, 15702, 15745, 15875, 15832
3200	14800	25	10000	15312, 16383, 15615, 15904, 16121, 16034, 16237
3200	14854	1	10037	15366
6400	16383	2	10000	16383
6400	14800	5	10000	15312

Canon EOS 5D Mark IV

ISO	NormalWhiteLevel	image count	LinearityUpperMargin	SpecularWhiteLevel
100	12735	1956	10000, 10087, 10154, 10419, 10273	14008
125	12735	128	10000, 10154, 10087	14008
160	10422	179	10000	10838
200	13135	159	10000, 10154	14448
250	13135	90	10000	14448
320	10422	114	10000	10838
400	13135	239	10000, 10087	14448
500	13135	89	10000	14448
640	10422	122	10000	10838
800	13135	135	10000	14448
1000	13135	97	10000	14448
1250	10422	62	10000	10838
1600	13135	241	10000	14448
2000	13135	55	10000	14448
2500	10422	37	10000	10838
3200	13135	178	10000	14448
4000	13135	4	10000	14448
6400	13135	28	10000	14448
12800	13135	5	10000	14448

[dhoulder]

I've checked images for every ISO from this camera by hand and can confirm that the original <Sensor> values are about right, certainly way closer than NormalWhiteLevel. My particular camera seemed to have a faction of a percent more headroom than the <Sensor> values, but that might well be just unit-to-unit variation in the analog part of the system. I don't have any sraws from this camera so I have left those to be detected by NormalWhiteLevel. See PR #977

[LebedevRI]

Hm, so are only the -1/3EVISO levels affected, but not -2/3EVISO levels?

The automatically determined white level is 15312. The sky is clipped, showing the expected magenta hue, but the raw clipping indicator and highlight reconstruction indicator show no clipping.
Manually setting the white level to 12965 activates highlight reconstruction and the raw clipping indicator.

$$15312 \times 2^{-1/3} = 12153 \lesssim 12965 $$
I wonder if we should/could just adjust white level for these ISO levels...

[dhoulder]

Hm, so are only the -1/3EVISO levels affected, but not -2/3EVISO levels?

Yes it seems so, up to ISO 1600. The NormalWhiteLevel of 14800 used for whole-stop and $\frac{-2}{3}$ EV ISOs, (with some exceptions) is about 0.1 stop lower than the default <Sensor> value of 15950, so that's probably usable for those cases.

$$15312 \times 2^{-1/3} = 12153 \lesssim 12965 $$
I wonder if we should/could just adjust white level for these ISO levels...

I guess that would work. I don't know if it's a better solution than reinstating the <Sensor> values though. Hard-coding these special cases into Cr2Decoder.cpp based on observations seems just as ugly as putting them into cameras.xml, which is also derived from observations. Note that
$$14800 \times 2^{-1/3} = 11747 $$
and 11747 is about 0.15 stops lower than the special-case <Sensor> value of 12995, so maybe that's too much of a discrepancy.