Converting 2D images to 3D

Some experiences with conversion of 2D images to 3D are discussed.

If the side-by-side stereo pairs below are too far apart for you to fuse them by parallel viewing, you may be able to zoom out of this web page in your browser. In Chrome, use .

Leiapix Converter

A 2D image can be converted to 3D by uploading to the Leiapix Converter website. The Leia Image Format file downloaded from the Leiapix Converter contains the original image and a depth map. These can be separated by the LIF1.exe utility program and combined into a side-by-side pair:-

The side-by-side pair can be opened in StereoPhoto Maker with File->Open Stereo Image and converted to 3D with Edit->Depth map->Create 3D image from 2D+Depth map:-

For each pixel in the depth map, a 256-bit value d indicates the depth from 0: farthest to 255: nearest. Each depth map value d can be interpreted as a disparity—the horizontal distance between corresponding pixels in the left and right images. In fact, a first-order conversion to 3D can be performed just by shifting each pixel to the right by d/2 in the left image and to the left by d/2 in the right image:-

The result is defective, however, as there are gaps where pixels have been shifted and not replaced by pixels with greater disparity. SPM does a better job, using an improved gap filling algorithm.

Here is another example from the Leiapix Converter:-

The depth of the carriage is well rendered, but the passengers look like cardboard cutouts. It can be seen that there is not much detail in the depth map within the outlines which represent each passenger. This is confirmed by plotting the depth map with matplotlib:-

The shortcoming of the depth map is presumably because the Leiapix Converter works by finding well-defined edges of objects and does not register subtle variations of modelling within the edges. This can be seen more clearly when a better depth map is available for comparison, as shown below.

iPhone Portrait Mode

Later iPhone models can take photos in portrait mode, which uses the dual cameras and AI software to create depth effects. Portrait mode photos saved on the iPhone have this logo in the top left corner. They contain a depth map which can be displayed in SPM with Edit->Depth map->Open Jpeg include depth map... In the depth map from portrait mode, the values are reversed: d indicates the depth from 0: nearest to 255: farthest.

This can be converted to 3D with Edit->Depth map->Create 3D image from 2D+Depth map:-

Processing of this portrait mode photo by the Leiapix Converter and LIF1.exe gives:-

This can be converted to 3D with Edit->Depth map->Create 3D image from 2D+Depth map:-

Now there is less 3D modelling in the face resulting from less detail in the depth map produced by the Leiapix Converter. The plot of the depth map from the iPhone shows plausible modelling of the face and chest:-

The plot of the depth map from the Leiapix Converter, however, shows the face and chest as relatively flat with apparently random excrescences:-

Alternatives

Conclusions

Christopher B. Jones 2021-11-27