| RUSSIAN |
MSU Video Quality Measurement Tool
MSU Graphics & Media Lab (Video Group)
Projects, ideas: Dr. Dmitriy Vatolin
Implementation: Alexey Moskvin, Oleg Petrov, Sergey Putilin, Sergey Grishin, Arsaev Marat
- MSU VQMT Information
- Basic Information About VQMT
- Setting & Screenshots (advanced files settings, visualization settings)
- Metrics Info (PSNR, MSE, MSAD, SSIM, VQM, MSU Blurring/Blocking)
- FAQ (Frequently Asked Questions)
- Plugins & Plugin SDK[BI-PSNR, DFM, BFM, NE, SCD]
- MSU VQMT program performance (New!)
- Subjective quality metrics comparison (New!)
- PRO Version Info (with command-line interface) [Full Documentation, PDF, 2.98 MB]
- Download & Purchase [Brochure, PDF, 475kB]
Basic Information About VQMT
- What's new in MSU VQMT 3.0?
- Brief Description
- Main Features
- Plots Examples
- Comparison with Analogs
- Contributors
- Testimonials
- Changelog
What's new in MSU VQMT 3.0?
- Increased metric calculation speed We improved our program performance up to 13 times for PC's with CUDA-enabled devices making it fastest tool for video comparison and quality estimation! More information you can find on MSU VQMT performance page.
- Added metrics with high subjective quality correlation Such metrics as Multi-Scale SSIM and Spatio-Temporal SSIM in MSU VQMT 3.0 show highest subjective quality correlation. Use them for more precisely quality estimation of your videos. You can read description of our correlation measurements on the subjective quality comparison page
- Additional features:
- Added *.y4m format support
- New Spatio-Temporal SSIM metric added
- Added 64-bit version of MSU VQMT (up to 10% speedup)
- Added "Save bad frames" feature to the command-line interface
- Autoupdate feature for the MSU VQMT Free version
Brief Description
Demo Video: Basic Use Case Overview
MSU Video Quality Measurement Tool (VQMT) is a program for objective video quality assessment. It provides functionality for both full-reference (two videos are examined) and single-reference (one video is analyzed) comparisons. Basic scheme of the tool is illustrated on Pic.1 below.
 |
| Pic.1. VQMT input and output |
VQMT inputs:
- Original: original video file (or reference one in case of single-reference test)
- Reference 1 [optional]: first video file the quality estimate is to be calculated for
- Reference 2 [optional]: second video file the quality estimate is to be calculated for
- Video with mask: video file with mask designating the regions of interest. Metric values are calculated in the specified area only.
- Per-frame values: CSV (comma separated values) file, containing values of each selected metric for each frame of each reference file.
- Average values: CSV file, containing average values of each selected metric for each reference file.
- Visualization: video file (*.AVI) indicating (by color and luminance) metric values in each pixel of each frame of each reference video. Visualization video is generated for each metric each color component and each reference file.
Video quality is a characteristic of a video passed through a video transmission/processing system, a formal or informal measure of perceived video degradation (typically, compared to the original video). Video processing systems may introduce some amounts of distortion or artifacts in the video signal, so video quality evaluation is an important problem. MSU VQMT is a program for video quality measurements. This application allows you to create objective comparison of video codecs and perform video processing filters analysis.
Visualization examples for several metrics
Program allows to answer the next questions:
- One codec has more blurring than another. On what frames? Average mark? *
- One codec has more blocking than another. On what frames? Average mark?*
- One codec has lower quality than another. On what frames? Average mark in PSNR, VQM, SSIM?*
Current version: 3.0 [Changelog]
Main Features
Here is the list of VQMT main features, some of them are available in PRO version only:
| Table 1. VQMT Main Features | |||
|
# |
Feature |
FREE version |
PRO version |
|---|---|---|---|
| 1 | Flexible Functionality
he tool provides functionality for calculating:
|
 |
|
| 2 |
20 Objective Metrics Support:
|
|
|
| 3 |
Over 20 Video Formats Support:
|
|
|
| 4 | Results File Format is *.CSV
Results are saved in *.CSV (comma-separated values) file which can be opened everywhere. |
|
|
| 5 | Metrics Visualization Video visualization file is generated for combination [metric, color component, reference]. Color and luminance of each pixel in visuzalization video is defined by metric value in that pixel (very useful for codec and filter debugging). |
|
|
| 6 | ROI Support (via masking) Allows to calculate metrics in Regions of Interest (ROI) only. ROI should be specified by additonal provided video with mask for each frame. |
|
|
| 7 | Comparative Analysis
Two references can be compared against the original simultaneously. |
|
|
| 8 |
Professional Bit Depth Support (over 8 bits per color component for *.YUV)
|
 |
|
| 9 | Batch Processing
VQMT provides GUI and command-line (CL) interfaces. It is possible using CL-tool to calculate several metrics for a given set of color components simultaneously (in parallel), thus achieveing significant speed advantage comparing to case of using GUI tool. |
|
|
| 10 | HD Support
Processing of video with resolution higher than 1280x720. |
|
|
Plots Examples
These plots were built in MATLAB, based on measurement results from MSU VQMT.
Pic.2. Perframe Y-PSNR for several codecs
Pic.3. Y-PSNR dependency vs bitrate and frame number
Pic.4. Average Y-PSNR for several codecs and bitrates
Pic.5. Colored surface of Y-PSNR vs frame number and bitrate
Pic.6. Example of the error in codec, detected with VQMT.
Pic.7. Another examples of plots built based on VQMT output.
Comparison with Analogs
Table 2 presents the comparison of VQMT with analogs:
| Table 2. Comparison with Analogs | |||||
|
MSU VQMT |
Elecard Video QuEst (commercial) |
Video Quality Studio (freeware) |
PSNR Checker (freeware) |
Avisynth plugins (SSIM, VQM) (freeware) |
|
| Supported objective metrics | 16 + 5 (plugins) + 3 (GPU) | 8 | 3 | 1 | 2 |
| Plug-ins interface | |
|
|
|
|
| Comparative analysis (comparison of two codecs in one pass) | |
|
|
|
|
| Supported input formats | *.AVI, AviSynth (*.AVS), *.BMP, *.YUV, *.Y4M and other media formats (*.MOV, *.VOB, *.WMV, *.MP4, *.MPG, *.MKV, *.FLV, etc.) using AviSynth and automatic scripts generation | Supports any video stream if compatible decoder supporting the YV12 or UYVY output is registerd in the system | AVI | AVI | Formats, supported by AviSynth |
| User interface | GUI + command line (PRO) | GUI | GUI | GUI | Called from AviSynth scripts (you need to learn AviSynth scripting) |
| Automatically saving frames with maximum difference in quality | |
|
|
|
|
| Plots with results of the comparison | |
|
|
|
|
| Video with metric value visualization | |
 |
|
|
|
Contributors
Special thanks to following contributors.
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
 |
Testimonials
We have testimonials from the following big companies:
- Analog Devices(analog.com)
- Samsung (samsung.com)
- Intel (intel.com)
- Hewlett-Packard (hp.com)
- Thomson (thomson.net)
- Harmonic (harmonicinc.com)
- Matrox (matrox.com)
- Zoran (zoran.com)
- Kodak (kodak.com)
- Honeywell (honeywell.com)
- AudioCodecs (audiocodes.com)
- Envivio (envivio.fr)
- Ambric Inc (ambric.com)
- Smart phones (smartphones.no)
- TestPro (testpro.co.il)
- Bouygues Telecom (bouyguestelecom.fr)
- CDRinfo (cdrinfo.com)
- Infoteck (infoteck.ru)
- Cubiworld (cubiworld.ch)
- INESC Porto (inescporto.pt)
- University of Technology Sydney (it.uts.edu.au)
- Fraunhofer (igd.fraunhofer.de)
- Brunel University (brunel.ac.uk)
- Alpen-Adria University (edu.uni-klu.ac.at)
- California State University (csudh.edu)
- Universeit Gent (intec.ugent.be)
Changelog
3.0
[+] Added stSSIM metric
[+] Added “.y4m” raw video internal support
[+] Added Autoupdate feature for free version (our PRO customers receive updates automatically)
[+] Added GPU realization for SSIM-based metrics (SSIM, 3-SSIM,
MS-SSIM. Requires CUDA-capable device. See metrics info page for additional info)
[*] Added subjective comparison for the most popular metrics (see metrics info)
[*] Added 64-bit version of MSU VQMT (up to 10% speedup)
[!] Program crashes due memory lack when -metr ALL specified with large (i.e. 1280x720) video frames.
2.7.3
[*] Fixed bug with MSSIM metric causing source frame change
[*] Fixed some metrics inaccuracy causing different metric values by enabling\disabling visualization
[!] Program crashes due memory lack when -metr ALL specified with large (i.e. 1280x720) video frames
2.7.2
[*] Fixed bug causing incorrect metric values, when using 3SSIM and MSSSIM simultaneously
[*] Fixed bug causing incorrect PSNR metric values in CSV files
[*] Fixed bug causing no metric calculation for large (>4gb) files
[*] Not existing directory specified in "-cod" parameter will be created now and processing will not cancel.
[!] Program crashes due memory lack when -metr ALL specified with large (i.e. 1280x720) video frames
2.7.1
[*] Fixed bug in CVS file generation. Sometimes first frame metric value was empty
[*] Fixed bug causing incorrect MSE metric values after calculating SSIM metric
2.7
[+] MSSSIM (fast and precise) metric implemented
[+] 3SSIM metric implemented
[*] Fixed bug in calculation of VQM metric under Windows 7
[*] Fixed bug during program launch on some machines
2.6 (Windows Vista & Windows 7 support)
[*] Fixed bug in Scene Change Detection plugin when working under Windows Vista or Windows 7
[*] Fixed bug in saving visualization video when running on Windows Vista or Windows 7
[*] Fixed dependency with vcomp.dll
2.5
[+] Video with any resolution is now supported by all metrics. Video with resolution which is not appropriate for some metric is now expanded (via data duplication, separately for each metric) to make resolution acceptable
[+] 1.95 times speed up of command line tool multiple metrics calculation on average (PRO version only)
[+] YUV files with size more than 2Gb are supported now
[+] Output directory for *.CSV and visualization files is automatically set to folder of last specified reference file
[*] Fixed bug in processing of *.YUV files with non-standard resolution
[*] Fixed bug in loading the mask from *.YUV files
[*] Fixed bug in masking of L (LUV colorspace) component
[*] Fixed bug in processing of non-standard resolution *.AVS files
[*] Fixed bug in calculation of SSIM (precise) for second reference file
[*] Fixed bug in conversion from RGB32 to YUV color spaces for video with non-standard resolutions (affects calculation of metric for *.AVI files)
2.01 beta
[+] 1.5 times speed up of command line tool multiple metrics calculation on average (PRO version only)
[+] Masking is added
[*] Fixed bug in 4:2:2 raw files with more than 8 bits per component support
2.0 beta
[+] *.MOV, *.VOB, *.WMV, *.MP4, *.MPG, *.MKV, *.FLV formats support simplified
[+] HDTV support (PRO version only)
[+] Raw files with more than 8 bits color depth per component are supported (PRO version only)
[+] Alternative SSIM and PSNR are added for compatibility with other implementations
[+] New version of *.CSV files with average metric values (PRO version only)
[+] Minor acceleration
[+] Preview buttons are added
[+] Options save is improved
[+] All MSU plugins are renamed (names are now more correct in GUI and simpler to call from PRO console)
[*] MSU Noise Estimation plugin bug with incorrect (identical) values for some videos is fixed
[*] MSU Noise Estimation and MSU BI-PSNR plugins provide correct information about their home pages now
[*] MSU BI-PSNR plugin crash during visualizing a metric for video sequences with dimensions less than 255 is fixed
1.52
[*] Error in saving CSV file for comparative analysis is fixed
1.51
[*] Error at the opening YUV-files is fixed
1.5
[*] Set of interface fixes
1.4
[*] Bugfixing in BMP processing (visualization saving, etc)
1.3
[*] Bug with YV12 yuv files fixed
1.2
[+] Now it is possible to compress visualization
[+] Plugin mechanism released
[!] Problem with some DV codecs
1.0
[+] More YUV filetypes are supported, including YV12, YUY2, YUV
[+] Supports unicode
[+] Visualization dialog was extensively reworked
[*] Interface is more user-friendly
0.81
[+] New AVI Reading system (support large AVI Files, VP 70)
[*] Bugfixing in final dialog
0.8
(This version was developed with Nicolay Trunichkin)
[+] New dialog with visualisation of the comparison and for comparison of the selected frames added.
0.75
[+] Improved codecs support (x264)
[+] Now data from YUV-AVI files is extracted without convertion
[*] Bugfixing: #NAN in VQM calculation, calculation of RGB-metrics for YUV-files
[*] MSU Blocking Measure changed
0.74
[+] MSE, MSAD metrics added
[+] Saving of averange value of the metric added
[+] Improved codecs support (DivX3, WMV, mjpeg2000)
[*] Delta metric was changed
[*] Bugfixing (AviSynth - wrong result for comparison of three files)
0.73
[+] All colorspaces from AVISynth are supported
[+] I420 (IYUV) support added
[+] XviD and B-frames support added
[*] Bugfixing (YUV-files, SSIM)
0.72
[+] AVS Support added
[*] Bugfixing
[!] Doesn't work with XVID
0.71
[+] First public beta
MSU Video Quality Measurement Tools
- MSU Video Quality Measurement Tool (program for objective comparison)
- MSU Human Perceptual Quality Metric (several metrics for exact visual tests)
Contacts
| e-mail: |  |
Other Resources
Video resources:
Call for MPEG4-AVC/H.264, HEVC codecs 2013!
First Report on S3D quality assessment (5 films) is released!
Appendix to Eighth MSU Í.264 codec comparison is released
Add to Del.icio.us
Digg It
reddit
| Last updated: 25-August-2011 |
|
|
Project updated by
Server Team and MSU Video Group
Project sponsored by YUVsoft Corp.
Project supported by MSU Graphics & Media Lab
