MSU Graphics & Media Lab Report #1 on stereo films quality assessment
Graphics & Media Lab CMC of Lomonosov Moscow State University has about 10 years' experience in comprehensive analyzing of compressed video. Recently the Graphics & Media Lab (further: the Lab; in cooperation with IITP RAS) has published its first detailed report on Stereo-Film-Quality Analysis within VQMT3D (Video Quality Measurement Tool for 3D) Project. Meanwhile the Lab is working on the next reports on stereoscopic video quality estimation. They are expected to be made public in the second half of 2013.
Stereo and multiview video entertainment has become a very popular trend. A rapidly growing demand for 3D films after Avatar release inspired the production houses to create a lot of new stereo content. But sometimes doing fast doesn't mean doing it well enough. Insufficient stereo video quality provoked in the audience such 'special effects' as eyestrain and headache after visiting 3D cinemas. The Lab decided to analyze five stereo films, captured with stereoscopic camera systems and to investigate the reasons of the irritating annoyance. Finding just a set of bad frames is rather a simple task, but it is not a useful result for a comparative analysis. For achieving a valuable result it's important to estimate a ratio of bad and well-done frames in a movie. Such results enable finding the worst frames in one movie or comparing different films. For the purpose of revealing stereoscopic errors the Lab developed a unique tool for stereo video quality measurement (VQMT3D). Its algorithms are able to discover the key reasons of visual discomfort from 3D films viewing:
- horizontal disparity
- vertical disparity
- color mismatch
- sharpness mismatch
The free downloadable report consists of 246 pages and comprises of three main parts:
- Thorough description of visual-discomfort potential causes while viewing stereo 3D films. Explanation of human vision physics and the stereo errors impact on human eyes from an ophthalmologist professional point of view.
-
Detailed analysis of five films. Each film description consists of two parts:
- Full per-frame analysis with charts showing metric values for each frame
- Visualizations demonstrating artefacts in frames that earned a poor rating according to their metrics
- Overall film comparison. The charts depict the average metric values and distribution of these values in each film.
- Accommodation-convergence conflict
- Horizontal disparity
- Vertical disparity
- Interocular differences in contrast/luminance
- The effects of blur (symmetric and asymmetric)
Ross Copeland (Post Production Stereographer) described the key artifacts in stereo films and the means of eliminating the errors:
The proposed evaluation methodology provides a value for each frame of an evaluated stereo film. An average result for the film Avatar has been used as a reference value (marked green), because it was the first popular and high-quality stereoscopic film. These reference-level lines are especially useful when dealing with rather obscure color and sharpness mismatch metrics.
Per-frame video evaluation enables detection of the worst frames in a film or detection of the scenes that should be fixed primarily. Color and sharpness mismatch values are presented as dimensionless. The values represent the strength of the color or sharpness difference between two views of a stereoscopic frame. The greater is the color mismatch, the higher is the value for this metric. The sharpness mismatch value depends on the size of the region containing the sharpness mismatch and the strength of this mismatch.
Demonstration of color-mismatch issues detection in Resident Evil: Afrerlife using per-frame charts (Image # 41425):
Image #1
(Image source: http://www.compression.ru/video/vqmt3d/press-releases/report1/images/per-frame-color-charts.gif)
Horizontal disparity values are measured in percent of frame width. Vertical disparity is measured in permil (h) of frame width. A permil is a one-tenth of a percent. A tilt is measured in degrees. Percents and permils are used to get invariant values relative to the actual frame width measured in pixels. For vertical disparity, the lower value means better video quality.
Demonstration of vertical-disparity issues detection in Resident Evil: Afrerlife using per-frame charts (Image #2):
Image #2
(Image source: http://www.compression.ru/video/vqmt3d/press-releases/report1/images/per-frame-chart-vertical-disparity-problem.png)
The horizontal-disparity charts enable visualization of depth-budget distribution in Resident Evil: Afterlife for a given frame (Image #3):
Image #3
(Image source: http://www.compression.ru/video/vqmt3d/press-releases/report1/images/per-frame-disparity-chart.png)
Examples of detected problems:
-
Huge inter-view color and geometry mismatches were detected in Resident Evil: Afrerlife, frame #41350 (Image #4):
Image #4
(Image source: http://www.compression.ru/video/vqmt3d/press-releases/report1/images/res_evil_color_geometry_problem.gif) -
Excessive positive parallax artifact: positive disparity of 5% limits display size to 55" in Galapagos: The Enchanted Voyage, frame #12256. It means that this frame might be painful when viewing on a big TV or a 3D projector because of eyes divergence (Image #5):
Julian Napier (Director, editor, stereographer) commented about an acceptable positive parallax value:
Image #5
(Image source: http://www.compression.ru/video/vqmt3d/press-releases/report1/images/Excessive_parallax_vis.png)"I set out to never exceed a 2% positive parallax, as beyond this level, the eyes must diverge beyond what is comfortable. Generally I aim for 0.5% to 1% positive; thus, the 2% ceiling gives me some room to maneuver if I want or need to shift convergence a little in postproduction." -
Inter-frame sharpness mismatch was revealed in Resident Evil: Afrerlife, frame #87796 (Images #6-7):
Image #6: left frame
(Image source: http://www.compression.ru/video/vqmt3d/press-releases/report1/images/res_evil_color_sharpness_problem_left.png )
Image #7: right frame
(Image source: http://www.compression.ru/video/vqmt3d/press-releases/report1/images/res_evil_color_sharpness_problem_right.png )
To compare films with each other the Lab has used the charts of two types (for each problem-detection metric). The first chart simply shows average metric values for a set of evaluated films. Image #8 shows an example chart for color mismatch metric (comparison of 10 movies):
Image #8
(Image source: http://www.compression.ru/video/vqmt3d/press-releases/report1/images/color_mismatch_average_values.png)
The second chart is an integral histogram that enables visualization of metric values distribution through the film. This histogram requires some explanations for a better understanding: for example, 50% frames in Pirates of the Caribbean: On Stranger Tides have metrics value below 1. The film Step Up 3D has 50% frames with metrics value below 10 and just 3% of frames with a value below 1.
Image #9 depicts an example chart for color mismatch metric (comparison of 10 movies):
Image #9
(Image source: http://www.compression.ru/video/vqmt3d/press-releases/report1/images/color_integral_histogram.png)
Acknowledgements
The Lab is grateful for valuable comments contributed by professional stereographers to this report (in alphabetical order):
- Ross Copeland
- Alfredo Gonzalez
- Srboljub Hetlerovic
- Jon Karafin
- Julian Napier
Analysed films
- Pirates of the Caribbean: On Stranger Tides (2011)
- Resident Evil: Afterlife (2010)
- Step Up 3D (2010)
- Galapagos: The Enchanted Voyage (1999)
- Into the Deep (1994)
A key motive of this report is to improve the stereo quality and therefore to reduce eye- and headache. The Graphics & Media Lab hopes, that a high 3D video quality would contribute to a continually increasing number of people, leaving the modern theaters without annoyance and with good impressions from the image quality and storytelling. The report is available to all who may be concerned and interested in improving the quality of stereoscopic content. The next reports on 3D quality will be published in the near future on the web-site http://compression.ru/video/vqmt3d/.
For press
- Additional info for bloggers and press for Report #1
- Additional info for bloggers and press for Report #2
- Additional info for bloggers and press for Report #3
- Additional info for bloggers and press for Report #5
- Additional info for bloggers and press for Report #6
- Additional info for bloggers and press for Report #7
- Additional info for bloggers and press for Report #8
- Additional info for bloggers and press for Report #9
Reports overview
|
Report 1 (S3D shooting quality analysis of 5 movies) Release date: March 7, 2013 Download (Additional info for bloggers and press) Pages: 246; Figures: 295 |
Film list:
|
||
|
Report 2 (S3D shooting quality analysis of 5 movies) Release date: May 31, 2013 Download (Additional info for bloggers and press) Pages: 342; Figures: 442 |
Film list:
|
||
|
Report 3 (2D-3D conversion quality analysis of 5 movies) Release date: August 28, 2013 Download (Additional info for bloggers and press) Pages: 305; Figures: 336 |
Film list:
|
||
|
Report 4 (S3D shooting quality analysis of 5 movies) Release date: October 1, 2013 Pages: 301; Figures: 402 |
Film list:
|
||
|
Report 5 (2D-3D conversion quality analysis of 5 movies) Release date: April 11, 2014 Download (Additional info for bloggers and press) Pages: 384; Figures: 404 |
Film list:
|
||
|
Report 6 (Stereo Window analysis of 10 movies) Release date: September 10, 2014 Download (Additional info for bloggers and press) Pages: 415; Figures: 455 |
Film list:
|
||
|
Report 7 (Stereo Window analysis of 10 movies) Release date: October 22, 2014 Download (Additional info for bloggers and press) Pages: 333; Figures: 348 |
Film list:
|
||
|
Report 8 (Rotate Analysis, Temporal Shift, Channels Swap, Zoom Mismatch in 25 movies) Release date: June 16, 2015 Download (Additional info for bloggers and press) Pages: 366; Figures: 361 |
Film list:
|
||
|
Report 9 (Temporal Shift, Stuck-to-Background Objects, 2D to S3D conversion in Captured Films) Release date: January 18, 2017 Download (Additional info for bloggers and press) Pages: 467; Figures: 529 |
Over 100 films are in film list, including:
|
||
|
105 films are in film list, including:
|
|||
|
Report 11 (Overall analysis of 10 selected Chinese movies) Approximate release date: 2021 |
The film list includes 10 Chinese movies |
Contacts
| For questions and proposition please contact us |  |
Video resources:
Server size: 8069 files, 1215Mb (Server statistics)
Project updated by
Server Team and
MSU Video Group
Project sponsored by YUVsoft Corp.
Project supported by MSU Graphics & Media Lab