Crowd-sourcing Sony’s ARW 2.3.1 embedded lens CA corrections

Update: With some more trying, I have (partly) figured this out and have documented my solution in another post. There are still open questions, so would still appreciate any help.

Some cameras (especially mirrorless ones) embed lens-correction data in their RAW files. This can be seen for example in Adobe’s Lightroom or Camera Raw by the little info balloon in the lens-correction panel that says “Built-in Lens Profile applied.” I believe it can separately apply distortion, chromatic aberration, and vignetting corrections. In the case of my Sony A7, Lightroom says “This raw file contains a built-in lens profile for chromatic aberration. The profile has already been applied automatically to this image.”

I am trying to reverse-engineer the lens-correction information out of Sony’s raw files (from an A7 / ILCE-7 in my case). However, the encoding methodology is not publicly available (although Adobe clearly knows it) and vendor-dependent. Other people are trying similar things for other cameras.

I’m using Adobe’s DNG-converter as a reference, since it reads the ARWs and converts them into a standardised, publicly available format. Specifically, it writes 3 items of lens-specific information into the DNG-files (apart from the documented LensId/Model/etc Exif data) – I’m currently interested in the last one:

• “CameraCalibration1/2” matrices: These are colour calibration matrices in addition to the colour matrices the converter sources from its camera profile database. These are identical and completely fixed per lens (i.e., independent of any other settings or measurements), e.g., my Sony FE 55m f/1.8 gives

  $ dngvalidate -v photo.dng | grep -A3 CameraCalibration
  CameraCalibration1:
    1.0150 0.0000 0.0000
    0.0000 1.0000 0.0000
    0.0000 0.0000 0.9707
  CameraCalibration2:
    1.0150 0.0000 0.0000
    0.0000 1.0000 0.0000
    0.0000 0.0000 0.9707

  I am currently not sure how the DNG-converter derives these values, if they’re in the ARW-file or Adobe’s camera profile (unlikely).

• “LensDistortInfo”: This is written in the XMP (not in the DNG-spec) and (I think) determines the distortion correction. However, I believe it’s not applied but just kept for information for Lightroom, etc. It’s 4 double-precision floating values, e.g.,

  $ exiftool -LensDistortInfo photo.dng
  Lens Distort Info : 1066182087/1073741824 22027007/1073741824 
  -13160186/1073741824 -4519714/1073741824

  Again, I haven’t yet tried to figure out from which tag this information is derived but it seems to follow the same format as the next item (just not for multiple colour planes).

• “WarpRectilinear” Opcode: This is an official DNG-tag and determines a distorting correction for each colour plane (RGB) to avoid chromatic aberrations based on 4 Radial and 2 Tangential parameters per plane. The exact details can be found on page 84 of Adobe’s DNG specification.

  $ dngvalidate -v photo.dng | grep -A 13 WarpRectilinear
  Opcode: WarpRectilinear, minVersion = 1.3.0.0, flags = 0
  Planes: 3
    Optical center:
      h = 0.500000
      v = 0.500000
    Plane 0:
      Radial params: 1.000055, -0.000417, 0.000571, -0.000394
      Tangential params: 0.000000, 0.000000
    Plane 1:
      Radial params: 1.000000, 0.000000, 0.000000, 0.000000
      Tangential params: 0.000000, 0.000000
    Plane 2:
      Radial params: 0.999942, 0.000387, -0.000355, 0.000206
      Tangential params: 0.000000, 0.000000

  For the CA-correction that the converter is applying here, the optical center (0.5/0.5), Tangential params (0) and plane 1 correction (green=identity) are always constant, so we have effectively 8 floating point values (4 radial params for 2 colour planes) coming out of the DNG-conversion.

I am currently only interested in the last item, since that’s what’s being applied automatically in Lightroom. After a lot of trial and error, I have found that the relevant information is encoded in the ARW’s MakerNotes under tag 0x7980 in the SR2-SubIFD as an list of 33 signed integers (16 bit).

$ exiftool -config user_config -u -SR27980 -G1 photo.ARW
[SR2SubIFD] SR27980 : 32 256 128 128 128 0 0 0 0 0 -128 -128 -128 -128 
-256 -256 -384 -128 -128 -128 -128 0 0 0 0 0 128 128 128 256 256 256 384

The first int determines the number of following data elements (always 32), the next 16 ints determine the radial parameters for plane 0, the last 16 for plane 2. Some observations:

• The 2 planes are encoded in the same way. Effectively we have 16 signed ints determining 4 floats.

  Plane 0: 256 128 128 128 0 0 0 0 0 -128 -128 -128 -128 -256 -256 -384
  ==> Radial params: 1.000055, -0.000417, 0.000571, -0.000394
  
  Plane 2: -128 -128 -128 -128 0 0 0 0 0 128 128 128 256 256 256 384
  ==> Radial params: 0.999942, 0.000387, -0.000355, 0.000206
  

• The data elements in the ARW are all multiples of 128 (i.e., they’re not using the least significant 7 bits), highest absolute value in my sample set is 2048 of a possible 32767 (but that might be based on my sample pictures).
• My sample set mostly combines a declining set of values for plane 0 with an increasing set for plane 1 (as in above example), however with some exceptions

  Plane 0: 256>128>128>128>0=0=0=0=0>-128=-128=-128=-128>-256>-256>-384
  Plane 2: -128=-128=-128=-128<0=0=0=0=0<128=128=128<256=256=256<384
  

• However, the DNG-converter will happily take any values in this tag (i.e., also outside above observations) and calculate Radial parameters from it.

  Tag: 0 0 0 45 0 -984 22222 0 -9999 7777 0 6666 0 15827 0 3867
  ==> Radial params: 1.003609, -0.024452, 0.061264, -0.039174

• Setting all 16 elements to 0 results in Radial parameters 1, 0, 0, 0, i.e., the identify function.

  Tag: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
  ==> Radial params: 1.000000, 0.000000, 0.000000, 0.000000

I now have a large data-set of tag-values and the corresponding Radial parameters created by Adobe’s DNG-converter. What I’m struggling with is figuring out the transfer-function. The Radial parameters p0, p1, p2, p3 determine a scale value f for each colour plane (as below – r being a pixel’s distance from the center in [0..1]) that determines a pixel shift radially from the center of the picture (see DNG-specification, page 84f).

f = p0 + p1*r^2 + p2*r^4 + p3*r^6

The tag seems to do something similar but either with more coefficients or by dividing the radial distance into 16 elements or something else. Could also be that I (and exiftool) have interpreted the byte encoding wrongly (e.g., endianness) but that seems unlikely, since at least the sign seems to be correct.

I assume that the algorithm uses some kind of approximation/correlation (i.e., not direct conversion), since even seriously weird tag values seem to result in reasonable looking Radial params – but not sure.

If somebody wants to play with the data and help figure this out – below are 3 sets of data, each with 16 tag int16 values and the corresponding 4 double Radial Params created by Adobe’s DNG-converter

• 116 data sets from real pictures taken on an Sony A7 with a Sony/Zeiss FE 55mm f/1.8 lens.
• 35 data sets with ‘fake’ (i.e., invented) tag values to help figure out the transfer function.
• Another 32 data sets with just one (positive/negative) value per tag-field.

Any help is appreciated – please post any thoughts/hints/solutions in the comments.