VQMT3D Project: Report 2 on 3D-video Quality Analysis
MSU Graphics & Media Lab (Video Group)
Projects, ideas: Dr. Dmitriy Vatolin
Implementation: Alexander Voronov, Denis Sumin, Marat Arsaev, Vyacheslav Napadovsky, Alexander Belous, Vladimir Yanushkovsky
In cooperation with IITP RAS
About the Report
The report presents detailed (342 pages) evaluation results of 5 S3D films, which were captured with stereoscopic camera systems.
Analysed films |
Examined problems: |
|
|
The report comprises three main parts.
- Thorough description of visual-discomfort potential causes while viewing stereo 3D films (see page from report)
-
Detailed analysis of five films. Each film description consists of two parts:
- Full per-frame analysis with charts showing metric values for each frame; (see page from report)
- Visualizations demonstrating artifacts in frames that earned a poor rating according to our metrics (see page from report).
Please, see example visualizations for Galapagos at the VQMT3D project page.
-
Overall film comparison. Here go charts depicting the average metric values and distribution of these values in each film. (see page from report).
Some charts showing overall comparison results are presented in the Overall evaluation results section on this page.
Professional stereographers contributed to the analysis with valuable comments and feedbacks.
New Sharpness Mismatch Metric
A new metric is presented in the second report for evaluation of the views correspondence in sharpness.
MSU-3Dfocus-2011 metric, which was presented in the first report, was used for detailed analyzing of artifacts in examined stereo films. That metric provided us a valuable assistance while working on the first report. Meanwhile we have developed an improved metric (MSU-3Dsharpness-2012).
The new MSU-3Dsharpness-2012 metric enables the following results:
- More precise values for frames with strong color distortions between views;
- More precise values for frames with large occlusion areas between views;
- More precise values for frames with several regions with sharpness mismatch.
Please, see a detailed description of the new metric in Appendix A of the second report.
Contributed Stereographers
(in alphabetical order)
- Ross Copeland, Online Editor / Stereographer & Colourist, Post Production Consultant
- Clyde DeSouza, Author, Stereographer and Creative Technology Evangelist
- Alfredo Gonzalez, Stereoscopic Consultant / Stereographer
- Srboljub Hetlerovic, Stereographer, VFX Supervisor
- Jon Karafin, Senior Scientist and Director of Production Technology at RealD
- Julian Napierr, Director, Editor, Stereographer
- Celine Tricart, Independent Filmmaker and Stereographer
Overall evaluation results
In this section we present overall comparison charts.
First chart in each subsection (e.g. Depth Budget or Vertical Disparity) shows average score of each film according to the metric.
The second chart is an integral histogram which shows distribution of metric values throughout the film.
Please, read more about the charts and particular films in the report.
Depth Budget
Comparison of average depth budget
Depth budget distribution analysis
Vertical Disparity
Comparison of average vertical disparity
Vertical disparity values distribution
Color Mismatch
Comparison of average color mismatch
Color mismatch values distribution
Sharpness Mismatch
Comparison of average sharpness mismatch
Sharpness mismatch values distribution
Publications
When you want to make a reference to this project, please refer one of the following publications:
-
Mikhail Erofeev, Dmitriy Vatolin, Alexander Voronov, Alexey Fedorov,
“Toward an Objective Stereo-Video Quality Metric: Depth Perception of Textured Areas,” [pdf]
International Conference on 3D Imaging (IC3D),
pp. 1–6, 2012. doi:10.1109/IC3D.2012.6615120 -
D. Akimov, A. Shestov, A. Voronov, D. Vatolin,
“Automatic Left-Right Channel Swap Detection,” [pdf]
International Conference on 3D Imaging (IC3D),
pp. 1–6, 2012. doi:10.1109/IC3D.2012.6615126 -
A. Voronov, A. Borisov, D. Vatolin,
“System for Automatic Detection of Distorted Scenes in Stereo Video,” [pdf]
Proceedings of the Sixth International Workshop on Video Processing and Quality Metrics (VPQM),
2012. -
A. Voronov, D. Vatolin, D. Sumin, V. Napadovsky, A. Borisov,
“Towards Automatic Stereo-video Quality Assessment and Detection of Color and Sharpness Mismatch,” [pdf]
International Conference on 3D Imaging (IC3D),
pp. 1–6, 2012. doi:10.1109/IC3D.2012.6615121 -
A. Voronov, D. Vatolin, D. Sumin, V. Napadovsky, and A. Borisov,
“Methodology for stereoscopic motion-picture quality assessment,” [pdf]
Proc. SPIE 8648, Stereoscopic Displays and Applications XXIV,
vol. 8648, pp. 864810-1–864810-14, March 2013. doi:10.1117/12.2008485 -
A. Bokov, D. Vatolin, A. Zachesov, A. Belous, and M. Erofeev,
“Automatic detection of artifacts in converted S3D video,” [pdf]
Proc. SPIE 9011, Stereoscopic Displays and Applications XXV,
vol. 9011, pp. 901112-1–901112-14, March 2014. doi:10.1117/12.2054330
Reports overview
|
Report 1 (S3D shooting quality analysis of 5 movies) Release date: March 7, 2013 Download (Additional info for bloggers and press) Pages: 246; Figures: 295 |
Film list:
|
||
|
Report 2 (S3D shooting quality analysis of 5 movies) Release date: May 31, 2013 Download (Additional info for bloggers and press) Pages: 342; Figures: 442 |
Film list:
|
||
|
Report 3 (2D-3D conversion quality analysis of 5 movies) Release date: August 28, 2013 Download (Additional info for bloggers and press) Pages: 305; Figures: 336 |
Film list:
|
||
|
Report 4 (S3D shooting quality analysis of 5 movies) Release date: October 1, 2013 Pages: 301; Figures: 402 |
Film list:
|
||
|
Report 5 (2D-3D conversion quality analysis of 5 movies) Release date: April 11, 2014 Download (Additional info for bloggers and press) Pages: 384; Figures: 404 |
Film list:
|
||
|
Report 6 (Stereo Window analysis of 10 movies) Release date: September 10, 2014 Download (Additional info for bloggers and press) Pages: 415; Figures: 455 |
Film list:
|
||
|
Report 7 (Stereo Window analysis of 10 movies) Release date: October 22, 2014 Download (Additional info for bloggers and press) Pages: 333; Figures: 348 |
Film list:
|
||
|
Report 8 (Rotate Analysis, Temporal Shift, Channels Swap, Zoom Mismatch in 25 movies) Release date: June 16, 2015 Download (Additional info for bloggers and press) Pages: 366; Figures: 361 |
Film list:
|
||
|
Report 9 (Temporal Shift, Stuck-to-Background Objects, 2D to S3D conversion in Captured Films) Release date: January 18, 2017 Download (Additional info for bloggers and press) Pages: 467; Figures: 529 |
Over 100 films are in film list, including:
|
||
|
105 films are in film list, including:
|
|||
|
Report 11 (Overall analysis of 10 selected Chinese movies) Approximate release date: 2021 |
The film list includes 10 Chinese movies |
Download
Full report (342 pages) is absolutely free but download is available only for industry professionals (due to publication of real mistakes in the post-production process).
To download the report, please fill-in the request form.
You will get the download link for all reports (Reports #1–5) via e-mail.
All materials in this evaluation are used for research purposes and in accordance with Fair Use conditions. This evaluation is not published widely, but it is available for free under a subscription for stereoscopy and cinema technology professionals.
Contacts
| For questions and proposition please contact us |  |
Server size: 8069 files, 1215Mb (Server statistics)
Project updated by
Server Team and
MSU Video Group
Project sponsored by YUVsoft Corp.
Project supported by MSU Graphics & Media Lab