РУССКИЙ

MSU Frame Rate Conversion Method

MSU Graphics & Media Lab (Video Group)

Take a look at this article on the new site! Follow the link
https://videoprocessing.ai/other/frame-rate-conversion.html

Algorithm, ideas: Dr. Dmitriy Vatolin
Algorithm, implementaion: Sergey Grishin

See also:

Algorithm allows to double frame rate (fps) of the video stream. This conversion is performed by calculating and inserting the new frames in the stream (see picture).

fps x2 scheme
Double up-conversion of the frame rate

The main advantages of the method are:

  • high accuracy of motion tracking and compensation
  • constant temporal sharpness level of the frames in converted video stream
This is achieved by construction of compensated frames with quarter pixel accuracy. Such high accuracy is obtained by using of bilinear and bicubic interpolation. Second advantage means that interpolated frames have the same sharpness as the original ones. Note that such the feature is not common for the majority of methods of this kind.


Examples


Here we present results of comparison of described (MSU), Frame Repetition (FR) and Linear Interpolation (LI) methods on "akiyo" sequence. Results include PSNR metric visualization of interpolated frames in comparison to original ones on the right side. Following is the color semantics on these images: red color - high difference region, this difference is visually noticeable; yellow - little less than previous difference; green and blue - almost unnoticeable difference; black - no difference.

akiyo sequence
Performance of MSU, LI and FR methods at "akiyo" sequence

On the pictures below there are the frames from sequences with four times increased frame rate. Such the up-conversion is achieved by repeated using of FRC method in accordance to picture:

akiyo sequence
Four times frame rate up-conversion
On the next picture there is the comparison of MSU and LI FRC methods by the example of conversion of "waterfall" sequence:

Waterfall.avi MSU x4
Waterfall.avi LI x4
Performance of MSU and LI at "waterfall" sequence
Quality comparison of MSU, LI and FR methods is shown at the graphs below. In vertical direction there is PSNR quality level of interpolated frames in comparison to original ones (the higher values of metric means the higher quality of interpolated frames), in horizontal direction interpolated frame numbers are marked.

bbc psnr graph bus psnr graph foreman psnr graph Performance quality of MSU, LI and FR at "BBC", "buc", "foreman" sequences

Below there is a comparison of MSU and UCSD (University of California, San Diego) FRC methods. Sequences with four times increased frame rate were used for this comparison:

Dogs.avi MSU x4
Dogs.avi UCSD x4
Comparison of MSU and UCSD methods by the example of sequence "dogs"
On the pictures below it is easy to see the sharpness difference on frames, which were calculated by increasing FPS four times:

MSU & UCSD sharpness
Frames of MSU and UCSD methods for sequence "dogs"

Two sequences where the quality difference of MSU and UCSD methods can be clearly seen are available for download. In these sequences each frame contains the same part of frame from sequence processed by FR, MSU and LI. This method of sequence generation makes comparison of FRC methods easy and comfortable:

Sample frame
Frame from "crowd" sequence for comparing FR, MSU and LI methods


Download


Filter unavailable due to contract limitation.You can license Advanced FRC Project result.

E-mail: 


Another resources


Video resources:

Last updated: 12-May-2022


Server size: 8069 files, 1215Mb (Server statistics)

Project updated by
Server Team and MSU Video Group

Project sponsored by YUVsoft Corp.

Project supported by MSU Graphics & Media Lab