| RUSSIAN |
MSU Quality Measurement Tool: FAQ
MSU Graphics & Media Lab (Video Group)
MSU Video Quality Measurement Tool FAQ
- What about command-line interface? Does MSU VQMT support it?
- How can I perform batch processing with MSU VQMT?
- Your program reports that it failed to open input file.
- What's about values of the metrics? What values for each metrics means, that quality is better?
- Where can I get more information about your own metrics (MSU Blocking/Blurring metrics and others)?
- I receive unequal results for AviSynth scripts. What's wrong?
- Is it possible, to open video files, which are not supported by MSU VQMT directly?
- Results are unequal (too low) for one or more codecs?
- What about masking? I really need it!
- Your masking is not convenient; I want to draw a mask on a video frame!
- Why MSU VQMT has four variants of PSNR calculation?
- What about OPSNR? Why MSU VQMT does not have it?
- Why PSNR is slightly different from previous version?
- Why MSU VQMT has two variants of SSIM calculation?
- Why neither of your SSIM calculations does not match AviSynth SSIM plugin?
- Why SSIM (fast) visualization seems to be shifted?
- Why MSU Noise Estimation Metric plugin does not match to its previous version?
- Why MSU Noise Estimation Metric plugin does not match to VirtualDub MSU Noise Estimation filter?
- How can I obtain version for Linux?
-
Q: What about command-line interface? Does MSU VQMT support it?
A: Public version doesn't support command line. We have such feature in PRO version, but it's available only with commercial license.
-
Q: How can I perform batch processing with MSU VQMT?
A: you can't perform batch processing with the free version. There is PRO version with command line interface that supports this feature.
-
Q: Your program reports that it failed to open input file.
A: Our tool supports .avi, .yuv, .avs, .bmp files as input. Other file types (.wmv, .vob, .mpeg, etc.) can be opened through AviSynth. Our tool can generate the necessary AviSynth script automatically. For more information read AviSynth Introduction paragraph.
If you experience a problem with .avi, then it is possible that MSU VQMT is not compatible with this codec. Please send us information about this codec, or part of such avi file (1-2 megabytes) by e-mail.
-
Q: What about values of the metrics? What do values for each metrics mean, what quality is better?
A:Metric Full Reference (FR)/No Reference (NR) Interpretation PSNR, PSNR (256),
APSNR, APSNR (256)FR 100 for equal frames, higher values are better SSIM (fast), SSIM (precise), 3SSIM, MS-SSIM (fast), MS-SSIM(precise), stSSIM FR Higher values are better, 1 for equal frames VQM FR 0 for equal frames, lower values are better MSU Blocking metric NR Lower value corresponds to lower blocking. MSU Blurring metric NR (but 2 files are necessary in case of measure visualization) Lower value corresponds to higher blurring Delta FR 0 means equal frames, positive and negative values mean deviation, lower absolute values are better MSAD FR 0 means equal frames, lower values are better MSE FR Lower values are better, 0 for equal frames MSU Brightness Flicking Metric NR The higher the value, the higher the brightness flicking in comparison with the previous frame MSU Brightness Independent PSNR FR 100 for equal frames, higher values are better MSU Drop Frames Metric NR Metric has two values: 0 means that current frame exists, 1 that it is dropped MSU Noise Estimation Metric NR The higher the value, the higher the noise level MSU Scene Change Detector NR Metric has two values: 1 means that current frame is the first frame in a scene, for other frames metric value is 0.
-
Q: Where can I get more information about your own metrics (MSU Blocking/Blurring metrics and others)?
A: We plan to prepare some paper first and next publish these assessment methods on our webpage.
-
Q: I receive unequal results for AviSynth scripts. What's wrong?
A: Try to review resulting video from AviSynth script processing in VirtualDub to find out possible errors in the script.
-
Q: Is it possible, to open video files, which are not supported by MSU VQMT directly?
A: Our program supports AviSynth, it is possible to open such files using that software. Most of files can be opened using autogenerated scripts. If it is not working, try to write AviSynth script on your own. Script examples are provided in the installation folder.
-
Q: Results are unequal (too low) for one or more codecs?
A: Some codecs (for example DivX 6.0) shift video file one or two frames back, or make two first frames equal. Check for this shift in your video files after encoding. You can open two encoded videos in two different instances of Virtual Dub and switch to the 10th frame, for example. And then do switching between them to see if 10th frames in two videos correspond to each other or not. If shift really exists, use AviSynth scripts with DeleteFrame() and DuplicateFrame() to fix it.
-
Q: What about masking? I really need it!
A: Since 2.01 beta MSU VQMT supports masking. A mask file should be a two-color file: one color should mark masked area and another color should mark unmasked area. One of the colors should be black.
User can specify mask color. Masked areas are filled with the specified mask color. User has two choices for mask color definition.- Specify that the black color is a mask – it means that a black color is taken as a mask color
- Specify that not black color is a mask – it means that a non black color is taken as a mask color
- We assume the color with three zero components in RGB and with zero Y-component in YUV as "black"
- For example, if the mask file is provided in RGB and calculations are performed in YUV, then mask color is converted into YUV using provided conversion settings and the masked areas are filled with the color achieved
- When black color is used as a mask, an unmasked area may contain different colors – it is useful when the user wants to mark mask area on an actual video frame
-
Q: Your masking is not convenient; I want to draw a mask on a video frame!
A: Actually, you can. Specify black color as a mask and draw black mask on frames, everything will be fine. -
Q: Why MSU VQMT has four variants of PSNR calculation?
A: PSNR formula is:
"PSNR" and "APSNR" use the correct way of PSNR calculation and take maximum possible absolute value of color difference as MaxErr. But this way of calculation gives an unpleasant effect after color depth conversion. If color depth is simply increased from 8 to 16 bits, the PSNR will change, because MaxErr should change according to maximum possible absolute value of color difference (255 for 8 bit components and 255 + 255/256 for 16 bit components). Thus, "PSNR (256)" and "APSNR (256)" are implemented. They would not change because they use upper boundary of color difference as MaxErr. The upper boundary is 256. This approach is less correct but it is used often because it is fast.
The difference between "PSNR" and "APSNR" is the same as between "PSNR (256)" and "APSNR (256)" and is in the way of average PSNR calculation for a sequence. The correct way to calculate average PSNR for a sequence is to calculate average MSE for all frames (average MSE is arithmetic mean of the MSE values for frames) and after that to calculate PSNR using ordinary equation for PSNR:
The next table summarizes the differences:Metric MaxErr calculation Average PSNR calculation PSNR
correct
correct
PSNR (256)
256 (fast, inexact)
correct
APSNR
correct
averaging
APSNR (256)
256 (fast, inexact)
averaging
"PSNR" metric is recommended for PSNR calculation since it is implemented according to the original PSNR definition.
-
Q: What about OPSNR? Why MSU VQMT does not have it?
A: Actually, MSU VQMT does have OPSNR. OPSNR is a shortening from the "Overall PSNR" which means that average PSNR for the sequence is calculated in the "correct" way. Thus "PSNR" and "PSNR (256)" of the MSU VQMT are two implementations of OPSNR. Please refer to "Why MSU VQMT has four variants of PSNR calculation?" paragraph to find out the difference between these two.
-
Q: Why PSNR is slightly different from previous version?
A: New version of MSU VQMT uses IPP L2 measure to calculate MSE for PSNR, since IPP does not have direct MSE calculation function. This is a cause of slight difference.
-
Q: Why MSU VQMT has two variants of SSIM calculation?
A: SSIM (fast) is identical to the previous version of MSU VQMT SSIM. SSIM (precise) is implemented as a more precise and correct SSIM. It uses Gauss blur instead of box filter, as SSIM (fast) does.
-
Q: Why neither of your SSIM calculations does not match AviSynth SSIM plugin?
A: SSIM (fast) is close to AviSynth SSIM plugin. Nevertheless MSU VQMT does not use luminance masking, because it is not used in the original paper.
-
Q: Why SSIM (fast) visualization seems to be shifted?
A: This effect originates from the sum calculation algorithm for the box filter. The sum is calculated over the block to the bottom-left or up-left of the pixel (depending on if the image is bottom-up or top-down).
-
Q: Why MSU Noise Estimation Metric plugin does not match to its previous version?
A: The previous version of MSU Noise Estimation Metric plugin has a bug: it calculates measure correctly only when MSU VQMT opens file in RGB colorspace. The bug is fixed now, but colorspace conversion sequence has changed. The closest result is achieved when conversion algorithm is set to PC.601. But result is not the same because previous conversions were made using integer data, and now the conversion is made using floating-point data.
-
Q: Why MSU Noise Estimation Metric plugin does not match to VirtualDub MSU Noise Estimation filter?
A: The reason is MSU VQMT and VirtualDub sequences of color space conversions are different. For example, if the file is opened in YV12 colorspace then MSU VQMT plugin gets Y component of the image right away. But VirtualDub plugin gets RGB, converted from YV12 by VirtualDub, and extracts Y component from this RGB. The closest result is achieved when conversion algorithm is set to PC.601. However, result is not the same because VirtualDub conversions are made using integer data, and MSU VQMT conversions are made using floating point data.
-
Q: How can I obtain version for Linux?
A: There is no native windows version now. You can use the application with Wine (version 1.7.53 tested).
| e-mail: |  |
MSU Video Quality Measurement Tools
Other resources
Video resources:
Server size: 8069 files, 1215Mb (Server statistics)
Project updated by
Server Team and
MSU Video Group
Project sponsored by YUVsoft Corp.
Project supported by MSU Graphics & Media Lab