Here’s a summary of what I’ve found on regarding the lens-correction data that Sony embeds in their ARWs. This is tested with a Sony A7 (ILCE-7) and the Sony/Zeiss 35mm F/2.8 and 55mm F1/1.8 lenses. I would believe that this also holds for other cameras using v2.3.1 of the ARW format (e.g., A7r, A7s, A7m2) but haven’t tested it.

The A7 allows to separately set 3 possible corrections in the menu (‘Auto’ vs ‘off’).

  • Vignetting correction
  • Chromatic Aberration correction
  • Distortion correction

Not every lens seems to support all modes, my 55mm has the distortion correction greyed out to ‘Auto’, so it can’t be switched off. However, Sony is embedding the data into the ARW in any case anyway, no matter the setting (at least for CA/dist, probably also Vignetting). It’s up to the software to decide if to use it.

Also interesting is Adobe’s take on this:

  • Lightroom automatically applies CA correction and the DNG-converter writes CA-correction opcodes into the DNG-file.
  • Distortion correction is not applied in Lightroom. However, the DNG-converter writes an non-standard XMP-tag into the DNG-file that saves the decoded correction data. It does not write distortion-correction opcodes (although they’re part of the DNG-spec), so other DNG-readers won’t apply lens-correction either (unless they interpret Sony’s MakerNotes on their own).
  • As far as I can tell, Lightroom and the DNG-converter completely ignore any Vignetting correction information.
  • The DNG-converter also inserts 2 colour calibration matrices depending on the lens (fix per lens type and in addition to the colour matrices specified in Adobe’s camera profile). I’m not quite sure why (different lenses should not influence hue shifts?) and where it gets the data for this.

I do not know where Sony stores the Vignetting correction information but the CA and Distortion corrections are here:

  • Distortion correction: SR2SubIFD tag 0x7982 and mirrored in Exif SubIFD tag 0x7037. 17 signed integers (16 bit), first one is always 16 (denoting the number of numbers to follow), the remaining ones describe one radial transformation function.
  • CA correction: SR2SubIFD tag 0x7980 and mirrored in Exif SubIFD tag 0x7035. 33 signed integers (16 bit), first one is always 32, followed by 2 times 16 numbers, describing 2 radial transformation functions (for colour planes red and blue, denoting their shift vs green).

The logic and format behind the 3 different types of transformation function encodings (dist, CA-red, CA-green) seems to be the same although the precision is much different. CA-correction numbers are always multiples of 128, while distortion are multiples of 1. Plotting the tag values of a set of 116 photos makes for some pretty graphs:

CA-red:
CA-red

CA-green:
CA blue

Distortion:
distortion

The exact format of these 16 numbers is not known but based on how the DNG-converter interprets them, they are scaling factors to 16 base transformation functions (polygonal of 6th degree), whose sum gives the final radial transformation function (see my previous post). It’s probably a bit more complicated than that, I’m not sure where the base functions come from – they are likely computed themselves based on some fixed parameter.

Decoding the base functions from Adobe’s DNG-conversion is ok for CA-correction (see previous post) but difficult (i.e., imprecise) for the distortion correction, most likely due to the much higher range of the possible values. By running a least square fitting algorithm, I have deducted a set of functions that (at least for my test set of photos) should have a maximum deviation from Adobe’s results of 3-4 pixels in the picture corner at the strongest distortion in my set. For most pictures the deviation should be less than 1 pixel shift, which is good enough for me.

More things left to figure out (I will stop here though…):

  • What’s the exact meaning of the set of 16 numbers, i.e., what are the original base functions and how are they derived?
  • Why does Adobe’s DNG-converter add the camera calibration matrices and where does the data come from?
  • Where is the vignetting information stored in the ARW?
  • What do the tags 0x0064/0x189c denote? It’s another 16 value transformation function, seems to be 1:1 correlated to the distortion correction function and has very similar values (just a slight delta). Adobe does not seem to interpret them for DNG-creation.
  • And what do the other ~380 currently undocumented tags in the Sony-data mean…? 🙂

btw: Huge thanks to the developers of Exiftool. Amazingly useful, couldn’t even have started on looking into this without it!

Advertisements

In my last post, I documented my findings on where Sony embeds CA-corrections in their ARW-files but couldn’t figure out the encoding. With some more try and error I have now partially solved this and can reproduce the Adobe’s DNG-converter’s radial transformation functions parameters (to a certain level of precision), which is what I wanted. I still don’t fully understand the exact meaning of the values – so if somebody wants to dig in further, there’s still work to be done…

As I wrote, Sony uses 16 signed 16-bit integers to encode the CA-correction for one colour plane. Adobe’s DNG converter transforms those 16 integers into 4 double-precision floating point numbers (p0-p3), encoding a radial transformation function f(r)=p_{0}+p_{1}\cdot r^{2}+p_{2}\cdot r^{4}+p_{3}\cdot r^{6} .

Turns out that each of the 16 individual tag-number describes one such f_{t}(r) function (with t being tag [0-15]), scaled according to the number given. The final transformation function f(r) is then the sum of the 16 individual functions.

Given that each tag-value always seems to be a multiple of 128, if we can describe 16 “base-functions” f_{t}^{base}(r)=p_{t,0}^{base}+p_{t,1}^{base}\cdot r^{2}+p_{t,2}^{base}\cdot r^{4}+p_{t,3}^{base}\cdot r^{6} with only one tag t set to 128 and everything else to 0, we can easily calculate the final function as follows

f(r)=\sum_{i=0}^{15}\frac{t_{i}}{128}\cdot f_{i}^{base}(r) with p_{[0-3]}=\sum_{i=0}^{15}\frac{t_{i}}{128}\cdot p_{i,[0-3]}^{base}

(The above is not fully correct, since you need to subtract/add 1 from p_{0} before/after the operation).

How do we get the parameters p_{t,[0-3]}^{base} to describe the base functions? Here they are:

 
 t	        p0                 p1                 p2                p3
 0	1.000000000000000 -0.000000000000003  0.000000000000004  0.000000000000000
 1	1.000002185924800 -0.000013742120879  0.000024331123832 -0.000012881183203
 2	1.000008015306730 -0.000049514864695  0.000086817490976 -0.000045679490013
 3	1.000015476524690 -0.000092372857099  0.000158805754548 -0.000082507850262
 4	1.000021771704520 -0.000121922077811  0.000201673293326 -0.000102132667907
 5	1.000024066033500 -0.000118380587996  0.000179036891600 -0.000084963746248
 6	1.000020370946620 -0.000069640192430  0.000070863379852 -0.000021085470984
 7	1.000010395330620  0.000022094319767 -0.000113499274789  0.000082425461175
 8	0.999996151504952  0.000135918385120 -0.000327203799781  0.000197151826958
 9	0.999982055351825  0.000232330029149 -0.000487383601737  0.000274714579025
10	0.999974212580889  0.000260464758777 -0.000491767996422  0.000257137217849
11	0.999978535726942  0.000175513032767 -0.000256821636478  0.000099955930692
12	0.999997289092430 -0.000030222967688  0.000215351050453 -0.000187663058249
13	1.000023611459520 -0.000278769872875  0.000733243740232 -0.000481036136103
14	1.000033519009740 -0.000336150894464  0.000766383874106 -0.000451040811218
15	0.999974843502193  0.000284395910313 -0.000759830289663  0.000557605398484

Now we can easily calculate the final transformation function based on the tag values. With the example from last post, it looks like this:

Where do these base-values come from? No idea. However, it’s highly likely that they themself can be calculated – plot the base parameters across the base functions and what do you get…?

…one polynomial transfer functions with 4 different scalings. So, it seems that these values are derived from another 4 values, maybe specified in some other tag or–more likely–fixed (the base values seem to be constant across the different photos and lenses that I tested).

The above solution works for my purpose but is not ideal, since it loses precision by using the precomputed decimal values. If somebody wants to figure out the parameters for the base-functions to allow for calculation with full precision, be my guest (I tried Excel-solver but gave up quite quickly…).

Update: With some more trying, I have (partly) figured this out and have documented my solution in another post. There are still open questions, so would still appreciate any help.

Some cameras (especially mirrorless ones) embed lens-correction data in their RAW files. This can be seen for example in Adobe’s Lightroom or Camera Raw by the little info balloon in the lens-correction panel that says “Built-in Lens Profile applied.” I believe it can separately apply distortion, chromatic aberration, and vignetting corrections. In the case of my Sony A7, Lightroom says “This raw file contains a built-in lens profile for chromatic aberration. The profile has already been applied automatically to this image.”

I am trying to reverse-engineer the lens-correction information out of Sony’s raw files (from an A7 / ILCE-7 in my case). However, the encoding methodology is not publicly available (although Adobe clearly knows it) and vendor-dependent. Other people are trying similar things for other cameras.

I’m using Adobe’s DNG-converter as a reference, since it reads the ARWs and converts them into a standardised, publicly available format. Specifically, it writes 3 items of lens-specific information into the DNG-files (apart from the documented LensId/Model/etc Exif data) – I’m currently interested in the last one:

  • “CameraCalibration1/2” matrices: These are colour calibration matrices in addition to the colour matrices the converter sources from its camera profile database. These are identical and completely fixed per lens (i.e., independent of any other settings or measurements), e.g., my Sony FE 55m f/1.8 gives
    $ dngvalidate -v photo.dng | grep -A3 CameraCalibration
    CameraCalibration1:
      1.0150 0.0000 0.0000
      0.0000 1.0000 0.0000
      0.0000 0.0000 0.9707
    CameraCalibration2:
      1.0150 0.0000 0.0000
      0.0000 1.0000 0.0000
      0.0000 0.0000 0.9707

    I am currently not sure how the DNG-converter derives these values, if they’re in the ARW-file or Adobe’s camera profile (unlikely).

  • “LensDistortInfo”: This is written in the XMP (not in the DNG-spec) and (I think) determines the distortion correction. However, I believe it’s not applied but just kept for information for Lightroom, etc. It’s 4 double-precision floating values, e.g.,
    $ exiftool -LensDistortInfo photo.dng
    Lens Distort Info : 1066182087/1073741824 22027007/1073741824 
    -13160186/1073741824 -4519714/1073741824

    Again, I haven’t yet tried to figure out from which tag this information is derived but it seems to follow the same format as the next item (just not for multiple colour planes).

  • “WarpRectilinear” Opcode: This is an official DNG-tag and determines a distorting correction for each colour plane (RGB) to avoid chromatic aberrations based on 4 Radial and 2 Tangential parameters per plane. The exact details can be found on page 84 of Adobe’s DNG specification.
    $ dngvalidate -v photo.dng | grep -A 13 WarpRectilinear
    Opcode: WarpRectilinear, minVersion = 1.3.0.0, flags = 0
    Planes: 3
      Optical center:
        h = 0.500000
        v = 0.500000
      Plane 0:
        Radial params: 1.000055, -0.000417, 0.000571, -0.000394
        Tangential params: 0.000000, 0.000000
      Plane 1:
        Radial params: 1.000000, 0.000000, 0.000000, 0.000000
        Tangential params: 0.000000, 0.000000
      Plane 2:
        Radial params: 0.999942, 0.000387, -0.000355, 0.000206
        Tangential params: 0.000000, 0.000000

    For the CA-correction that the converter is applying here, the optical center (0.5/0.5), Tangential params (0) and plane 1 correction (green=identity) are always constant, so we have effectively 8 floating point values (4 radial params for 2 colour planes) coming out of the DNG-conversion.

I am currently only interested in the last item, since that’s what’s being applied automatically in Lightroom. After a lot of trial and error, I have found that the relevant information is encoded in the ARW’s MakerNotes under tag 0x7980 in the SR2-SubIFD as an list of 33 signed integers (16 bit).

$ exiftool -config user_config -u -SR27980 -G1 photo.ARW
[SR2SubIFD] SR27980 : 32 256 128 128 128 0 0 0 0 0 -128 -128 -128 -128 
-256 -256 -384 -128 -128 -128 -128 0 0 0 0 0 128 128 128 256 256 256 384

The first int determines the number of following data elements (always 32), the next 16 ints determine the radial parameters for plane 0, the last 16 for plane 2. Some observations:

  • The 2 planes are encoded in the same way. Effectively we have 16 signed ints determining 4 floats.
    Plane 0: 256 128 128 128 0 0 0 0 0 -128 -128 -128 -128 -256 -256 -384
    ==> Radial params: 1.000055, -0.000417, 0.000571, -0.000394
    
    Plane 2: -128 -128 -128 -128 0 0 0 0 0 128 128 128 256 256 256 384
    ==> Radial params: 0.999942, 0.000387, -0.000355, 0.000206
    
  • The data elements in the ARW are all multiples of 128 (i.e., they’re not using the least significant 7 bits), highest absolute value in my sample set is 2048 of a possible 32767 (but that might be based on my sample pictures).
  • My sample set mostly combines a declining set of values for plane 0 with an increasing set for plane 1 (as in above example), however with some exceptions
    Plane 0: 256>128>128>128>0=0=0=0=0>-128=-128=-128=-128>-256>-256>-384
    Plane 2: -128=-128=-128=-128<0=0=0=0=0<128=128=128<256=256=256<384
    
  • However, the DNG-converter will happily take any values in this tag (i.e., also outside above observations) and calculate Radial parameters from it.
    Tag: 0 0 0 45 0 -984 22222 0 -9999 7777 0 6666 0 15827 0 3867
    ==> Radial params: 1.003609, -0.024452, 0.061264, -0.039174
  • Setting all 16 elements to 0 results in Radial parameters 1, 0, 0, 0, i.e., the identify function.
    Tag: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
    ==> Radial params: 1.000000, 0.000000, 0.000000, 0.000000

I now have a large data-set of tag-values and the corresponding Radial parameters created by Adobe’s DNG-converter. What I’m struggling with is figuring out the transfer-function. The Radial parameters p0, p1, p2, p3 determine a scale value f for each colour plane (as below – r being a pixel’s distance from the center in [0..1]) that determines a pixel shift radially from the center of the picture (see DNG-specification, page 84f).

f = p0 + p1*r^2 + p2*r^4 + p3*r^6

The tag seems to do something similar but either with more coefficients or by dividing the radial distance into 16 elements or something else. Could also be that I (and exiftool) have interpreted the byte encoding wrongly (e.g., endianness) but that seems unlikely, since at least the sign seems to be correct.

I assume that the algorithm uses some kind of approximation/correlation (i.e., not direct conversion), since even seriously weird tag values seem to result in reasonable looking Radial params – but not sure.

If somebody wants to play with the data and help figure this out – below are 3 sets of data, each with 16 tag int16 values and the corresponding 4 double Radial Params created by Adobe’s DNG-converter

  • 116 data sets from real pictures taken on an Sony A7 with a Sony/Zeiss FE 55mm f/1.8 lens.
  • 35 data sets with ‘fake’ (i.e., invented) tag values to help figure out the transfer function.
  • Another 32 data sets with just one (positive/negative) value per tag-field.

Any help is appreciated – please post any thoughts/hints/solutions in the comments.