This forum contains results from the eponymous seminar from WS2013/2014. Each work from the students contains a summary of their results and the presentations attached (partly in German Language).
Post Reply
mpanhwar
Posts: 1
Joined: 15.10.2014, 10:07

Seminar Report Stereoscopic 3D Visualization

Post by mpanhwar » 16.03.2015, 16:01

Report: Stereo-Film-Quality Analysis

Authors:
Mehmood Ghaffar, 2nd year, Naturwissenschaftlich Informatik, Bielefeld University
Muzaffar Hussain Panhwar, 2nd year, Naturwissenschaftlich Informatik, Bielefeld University


In this study we analyzed some 3D stereoscopic movies (The Avengers, The Chronicles of Narnia: The Voyage of the Dawn Treader) by using a predefined approach developed by a team of Russian researchers at Graphics and Media Lab in Lomonosov Moscow State University Russia. This project is lead by Dr. Dmitry Vatoline (senior researcher at Graphic and Media Lab). They introduced various metrics along with some mathematical formulas that addresses specific parameter while analyzing films. We studied some of the possible visual discomforts while watching stereoscopic movies. These visual discomforts are the results of some pre and post productions. Metrics help us to detect these errors and enable us to improve these errors. Some of them can be covered in pre production state while others may cover in post production. In pre production state we can use accurate geometry of cameras and its movement can avoid some of the problems that can occur later. Similarly using some computer algorithms it is also possible to resolve a little bit according to the efficient algorithm and programs availability.

Much advancement has been made in the field of visualization since the discovery of magnifying nature of glass in the first century till now. The work of Robert Hooke is remarkable; because he was the first person ever who visualized the microscopic image of a piece of cork and used for the first time the word 'cell' in the biological context. And then the race started to dig deep into the microscopic nature of biological objects. Today, we have various microscopes to visualize nano scale objects. Now we can visualize medical entities ranging from mesoscopic level to molecular level. Stereoscopy comes into action at the end where we merge these nano objects together and model a 3d structure and visualize it in a better way. The Stereoscopic techniques is itself self emerging field. It could prove itself as a better visualization technique and also it could enable future scientist to work in this field and enable them to find a treatment of disease that are not so far categorized as curable disease. Such as cancer, and other dangerous diseases whose treatment is yet impossible.

Stereoscopy (also called stereoscopic or 3D imaging) is a technique for creating or enhancing the illusion of depth in an image by means of stereopsis for binocular vision. The present report is the 5th report of the series VQMT3D Project (Video Quality Measurement Tool 3D Project) S3D content with more focus on two types of visual discomforts. These are Cardboard effect and Edge color mismatch. To be more specific this report analyses the conversion of 2D to 3D movies and comparing the results with natively captured stereoscopic movies, i.e. with stereoscopic cameras. So, basically there are three approaches to create S3D content.
  1. Capturing using a Stereo camera.
  2. Conversion from 2D content which means creation of the two views of the source content or picture and
  3. Rendering; a process of generating views from a model by means of a computer program.
Then each film or content is analyzed in two parts.
  1. per frame analysis with charts showing metric values for each frame and
  2. Visualization that demonstrates artifacts in frames.
Image

Fig [1] shows the typical stereoscopic setup with which one can perceive 3d effect when viewing different perspective of a single source image. If we close our left eye we observe a shift in object or scene towards right side and vice versa. These two perceptions of the same source will develop a stereoscopic effect. If the angle of perception observed at projection plan as shown in figure then one perceives a flat object however if perception is observed otherwise will give 3D effect (in far plan or near plan).

Some important visual discomforts are necessary to understand prior to analyze the visuals in frames. Horizontal and vertical disparity is the shift in the left or right eye picture horizontally and vertically respectively. Difference in contrast between the views, difference in blur between the views, Edge sharpness mismatch and finally the cardboard effect which occurs due to lack of depth. Each of these discomforts has their own matrices but our focus in this report is on edge sharpness mismatch and cardboard effect. Matrices analysis as Horizontal disparity (0.5% equals to ~10 pixels in full HD) and for vertical disparity measured in per mil (0/00) of frame width whereas (1 0/00 equals to ~2 pixels in full HD) and Per mil = one-Tenth of a percent.

Edge Sharpness Mismatch (ESM) can be caused by occlusion techniques, background or foreground pixel stretching, rubber sheet (defined as warping the pixels surrounding the occlusion regions to avoid the explicit occlusion-filling step) and lack of proper alpha-channel treatment. Cardboard effect is the result of lack of depth in a scene. That depth relates the foreground object with background according to the depth map matrix which contains the depth information about the whole scene. This could be reduced by shading, texture gradients and motion parallax, stereo geometric analysis and shooting-&-viewing conditions. Causes for ESM under natural conditions are destruction or specific deformation of object edges, objects-edge oscillation along the visual axis of one eye, local optical or retinal defects in one of viewers eye, details near the object edge that are visible only to one of the viewers eye and properties of the medium between the eyes and the object. This could lead the artifacts such as object edge appear to be damaged, shaky, unstable or variable details may arise along object edges and edges may appear to be masked be the veil or viewed through an unsteady, jelly-like medium.

In Avengers (2012) we analyzed the whole movie and we have observed many artifacts at different time lapse in the movie for example at frames like:
  1. Frame at 1:48:5 (Edge sharpness mismatch)
  2. Frame at 2:37 (Cardboard effect)
  3. Frame at 1:46:11(Positive Disparity)
  4. Frame at 18:57 (Negative Disparity)
It can be clearly seen in the movie which makes discomfort for the viewer. In this movie at some places artifacts were more viable which almost give the impression of lack of depth same as watching a 2D movie.

Results are interesting, because they show a clear indication that the more the budget is spent the more accuracy one can get. And it is also shown that the converted movies or S3D content posses low amount of visual discomforts that are mentioned above as compare to natively-captured content

In conclusion many artifacts can be overcome by applying different stereoscopic visualization techniques in the field of biology e.g. when we are watching at molecular level many extremely small structures at a particular level give a blur effect and two objects cannot be resolved as separate structures thus computer algorithms and graphics techniques can be helpful to overcome these deficiencies. Another example for cell visualization would be to simulate the complex cell molecular structures to understand their interactions and functions using the rendering techniques.

Some of the artifacts can occur while visualization in medicine as watching the anatomical structure using Computerized Tomography(CT Scan) or Magnetic Resonance Imaging (MRI) artifacts like edge sharpness mismatch, cardboard effect ,vertical disparity and horizontal disparity can occur due to 2D to 3D conversion, because the analysis of specialist doctors entirely depends on this visual perspective of these images or video so if any of these visualization artifacts occur it will be hard to diagnose any irregularity. Therefore we learn from this research seminar that even in modern era it is difficult to give the perfect result by using all available resources and field of S3D visualization still needs attention in every discipline of life from applications in medicine to gaming and 3D movies.


References
[1] Sommer, B., C. Bender, T. Hoppe, C. Gamroth, and L. Jelonek. “Stereoscopic Cell Visualization: From Microscopic to Molecular Scale.” Journal of Electronic Imaging 23, no. 1 (2014): 011007–1 – 011007–10. doi:10.1117/1.JEI.23.1.011007.
[2] D. Vatolin, A. Voronov, D. Sumin, G. Rozhkova, V. Napadovsky, A., Bokov, V. Yanushkovsky, A. Belous, A. Shalpegin, A. Novikov, A. Borisov, and M. Arsaev, “VQMT3D Project Stereo-Film Quality Analysis Report 5,” 2014. [Online]. Available: http://compression.ru/video/vqmt3d/report5
[3] A. Voronov, D. Vatolin, D. Sumin, V. Napadovsky, and A. Borisov, “Methodology for stereoscopic motion-picture quality assessment,” Proc. SPIE 8648, Stereoscopic Displays and Applications XXIV, vol. 8648, pp. 864810-1–864810-14, March 2013 doi:10.1117/12.2008485.
[4] στερεός Tufts.edu, Henry George Liddell, Robert Scott, A Greek-English Lexicon, on Perseus Digital Library
[5] σκοπέω, Henry George Liddell, Robert Scott, A Greek-English Lexicon, on Perseus Digital Library

Post Reply