How to get Depth map image(jpg) of an JPG image in Android?

[Istiakmorsalin]

I have used the libraries following code to get a depth map image:

JPEGTweaker.extractDepthMap()

But the problem is that the only sample pictures are providing outputs. Other images like pictures taken on pixel 2 xl or pixel 3(Even in the portrait mode) are not working.

[dragon66]

Are you sure the pictures even contain depth map? If so, can you provide sample images for me to look at if possible?

[Istiakmorsalin]

@dragon66 Here are the images. I sent you these via email too. Please check.

First one is a pixel 2XL device photo.

The second one is a bokeh image.

[dragon66]

The current implementation for extracting depth map is based on an earlier format of Google depth map documentation. Google has since changed to something called dynamic depth map.

I did some homework regarding the new Google dynamic depth map format and although it specifies the depth map data is saved under one data URI called android/depthmap, I couldn't find it in either the image you provided or a similar image I downloaded from a post talking about similar issue.

The XMP metadata did contain quite a bit information regarding the depth map but not the map itself as in the old format did.

By the way, the bokeh image doesn't contain even XMP information related to depth map.

[thyung]

Pixel phone portrait mode photo is concatenated of 4 JFIF structure https://en.wikipedia.org/wiki/JPEG_File_Interchange_Format . Each JFIF structure is an jpeg image.

A JFIF structure starts with marker 0xFFD8 and ends with marker 0xFFD9. Therefore, we can split a portrait mode image into 4 jpeg files.

The following python code prints the marker positions and splits PXL_20210102_093446326.PORTRAIT.jpg into,

1. pxl_out_0.jpg: display image
2. pxl_out_1.jpg: original image
3. pxl_out_2.jpg: depthmap with 256 grey level
4. pxl_out_3.jpg: dummy image filled with 255

with open('PXL_20210102_093446326.PORTRAIT.jpg', mode='rb') as infile:
    buffer = infile.read()

bufferlen = len(buffer)
pos = 0
pos_d8 = 0
n = 0
i = 0
while i < bufferlen:
    if buffer[i] == 0xff:
        pos = i
        i += 1
        if buffer[i] == 0xd8:
            print('ffd8: {0}'.format(pos))
            pos_d8 = pos
        elif buffer[i] == 0xd9:
            print('ffd9: {0} len: {1}'.format(pos, pos - pos_d8 + 2))
            with open('pxl_out_{0}.jpg'.format(n), mode='wb') as outfile:
                n += 1
                outfile.write(buffer[pos_d8: pos + 2])
    i += 1

Original image: PXL_20210102_093446326.PORTRAIT.jpg

Depthmap: pxl_out_2.jpg

[dragon66]

@thyung thanks for the information. So if I understand it correctly, basically the JPG image actually consists of 4 images conforming to JFIF spec and one of it is the depth map?

Wen

[thyung]

@dragon66 different phones have different ways to store depthmap. My example photo was taken by Pixel 4. I see a website tells iPhone depthmap is store in exif and can be extracted by

exiftool -b -MPImage2 IMG_XXXX.jpg > IMG_XXXX_depth.jpg