Second Annual MSU MPEG-4 AVC/H.264 Video Codec Comparison

For professional users and codec authors

MSU Graphics & Media Lab (Video Group)

Return to second H.264/AVC comparison home page!

Part 2. "Best quality" preset

Take a look at this article on the new site! Follow the link
https://videoprocessing.ai/codecs/mpeg4-avc-h264-2005-part2.html

Contents

  • Part1. Introduction
  • Part2. "Best quality" preset
  • Part3. "Best speed" preset
  • Part4. Additional information and conclusions
  • Preset "Best quality"

    Y-PSNR

    In this mode codecs tried to show best quality in single-pass mode. Therefore major importance in this section is paid to graphs of quality of compressed sequences in different metrics. All H.264 codecs used main profile of encoding standard.

    Below are some Y-PSNR/Bitrate and Delta Y-PSNR/Bitrate graphs.

    PSNR (peak-to-peak signal-to-noise ratio) is a classic metric for video quality measurement. For two pictures xi,j and yi,j metric's value is defined by the following formula:

    Despite this metric does not generally reflect human perception of pictures' distortion, during last decades it is commonly used as main criterion for video codecs' performance evaluation.

    Delta Y-PSNR (or Relative Y-PSNR) are comparative PSNR graphs, where x264 codec was chosen as reference. For every measurement of each codec the difference between this measurement and PSNR value of reference codec with the same bitrate is plotted on these graphs. If corresponding reference value is absent then it is linearly interpolated.


    Picture 1.   Y-PSNR. Sequence "foreman"

     

    Picture 2.   Delta Y-PSNR. Sequence "foreman"

     

    Picture 3.   Y-PSNR. Sequence "susi"

     

    Picture 4.   Delta Y-PSNR. Sequence "susi"

     

    Picture 5.   Y-PSNR. Sequence "bbc"

     

    Picture 6.   Delta Y-PSNR. Sequence "bbc"

     

    Picture 7.   Y-PSNR. Sequence "battle"

     

    Picture 8.   Delta Y-PSNR. Sequence "battle"

     

    Picture 9.   Y-PSNR. Sequence "simpsons"

     

    Picture 10. Delta Y-PSNR. Sequence "simpsons"

     

    Picture 11. Y-PSNR. Sequence "matrix"

     

    Picture 12. Delta Y-PSNR. Sequence "matrix"

     

    Picture 13. Y-PSNR. Sequence "concert"

     

    Picture 14. Delta Y-PSNR. Sequence "concert"

     


    Conclusions:

  • Despite significant compression quality improvement in DivX, it is not a leader. And even more, if DivX is not far from leaders on "susi", "battle" and "matrix" sequences, on the rest sequences ("foreman", "bbc", "simpsons") it is most likely to be related to trailing codecs. New version of DivX codec shows worse results on "bbc" sequence than its old version 5.2.1; it was not able compress "concert" sequence at all.

  • There is no obvious leader among H.264 codecs. In most cases leaders are x264 and Ateme codecs.

  • The bad performance of Ateme H.264 codec on low birates is due to a bug, which was fixed by developers after publication of this comparison.

  • Elecard codec could not manage with low bitrates of "concert" sequence. It is clearly seen from Y-PSNR graph for this codec.

  • U-PSNR, V-PSNR

    Besides luminance component (Y-plane) sometimes it is interesting to look at codecs' behavior on color components (U and V planes).

    Picture 15. U-PSNR. Preset "Best quality". Sequence "foreman"

     

    Picture 16. U-PSNR. Preset "Best quality". Sequence "susi"

     

    Picture 17. U-PSNR. Preset "Best quality". Sequence "bbc"

     

    Picture 18. V-PSNR. Preset "Best quality". Sequence "concert"

     

    Conclusions:

  • In color planes x264 codec shows slightly better result than Ateme codec.

  • Elecard codec has some problems with bitrate control on "foreman" sequence. It is shown as "fall" of its curve on UV-PSNR graphs. But this codec shows the best results on low bitrates on this sequence.

  • SSIM, VQM

    Several metrics recently appeared which aspire to be better approximations of human perception. The most popular among them are SSIM and VQM.

    We have used them in our comparison, but they have presented no big differences from PSNR metric.

    Below are some graphs of these metrics for different sequences.

    Picture 19. Y-SSIM measure. Preset "Best quality". Sequence "foreman"

     

    Picture 20. Y-VQM measure. Preset "Best quality". Sequence "foreman"

     

    Picture 21. Y-SSIM measure. Preset "Best quality". Sequence "bbc"

     

    Picture 22. Y-VQM measure. Preset "Best quality". Sequence "concert"

     

    Blurring measure, blocking measure

    During some time our laboratory is developing metrics, which allow measuring blocking and blurring of frames in video sequences. Work on these metrics is in progress, therefore we show only several graphs with them.

    Red horizontal line is a blocking measure (or blurring measure) of source sequence. For blurring measure: the more is its value, the lesser is the blurring of frame; for blocking measure: the more is its value, the bigger is blocking of frame.


    Picture 23. Y-Blurring measure. Preset "Best quality". Sequence "foreman"

     

    Picture 24. Y-Blocking measure. Preset "Best quality". Sequence "foreman"

     

    Graph type

    Total number of graph

    Inserted in this document

    Average metric

    196

    24 (12%)


    Bitrate handling

    Consider graphs showing how accurately every codec kept requested bit rate. Grades on x-axis show target bitrates (point 0 is 100 kbps, point 9 - 2340 kbps). Grades on y-axis show how many times codec exceeded declared bit rate (real bit rate to requested bit rate ratio).


    Picture 25. Bitrate handling. Preset "Best quality". Sequence "foreman"

     


    Picture 26. Bitrate handling. Preset "Best quality". Sequence "susi"

     


    Picture 27. Bitrate handling. Preset "Best quality". Sequence "bbc"

     


    Picture 28. Bitrate handling. Preset "Best quality". Sequence "battle"

     


    Picture 29. Bitrate handling. Preset "Best quality". Sequence "simpsons"

     


    Picture 30. Bitrate handling. Preset "Best quality". Sequence "matrix"

     


    Picture 31. Bitrate handling. Preset "Best quality". Sequence "concert"

     

    Conclusions:

  • DivX codec keeps low bit rates badly. Frequently it generates files which are several times bigger than the worst H.264 codec (by bit rate handling).

  • "bbc" is the hardest sequence for bit rate handling for all codecs.

  • Fraunhofer IIS codec keeps bit rate better than others.

  • Time

    Now let's consider graphs showing encoding time for each sequence using "Best Quality" preset and average normalized time for this preset.

    Average normalized time is obtained as follows. At first, for every sequence codec with largest compression time is found. Compression times of other codecs were divided by compression time of this codec. After this step every codec has value from 0 to 1 for each video sequence.

    On the second step every codec gets evaluation calculated as arithmetic average of obtained values for all sequences. And these evaluations are shown on average normalized time graph.

    Picture 32. Encoding time. Preset "Best quality". Sequence "foreman"

     

    Picture 33. Encoding time. Preset "Best quality". Sequence "susi"

     

    Picture 34. Encoding time. Preset "Best quality". Sequence "bbc"

     

    Picture 35. Encoding time. Preset "Best quality". Sequence "battle"

     

    Picture 36. Encoding time. Preset "Best quality". Sequence "simpsons"

     

    Picture 37. Encoding time. Preset "Best quality". Sequence "matrix"

     

    Picture 38. Encoding time. Preset "Best quality". Sequence "concert"

     

    Picture 39. Normalized average encoding time. Preset "Best quality"

     

    Conclusions:

  • The fastest codec is ArcSoft codec, the slowest - Fraunhofer IIS codec.

  • There is a noticeable work slowdown of Fraunhofer IIS and x264 codecs during bit rate increase.

  • Fraunhofer IIS codec is approximately five times slower than all the rest codecs.

  • Contents

  • Introduction
  • Thanks
  • Overview
  • Goal and testing rules
  • Sequences
  • Codecs
  • "Best quality" preset
  • Y-PSNR
  • U-PSNR, V-PSNR
  • SSIM, VQM
  • Blurring measure, blocking measure
  • Bitrate handling
  • Time
  • "Best speed" preset
  • Y-PSNR
  • Speed/Quality Tradeoff
  • Bitrate handling
  • Time
  • Final part
  • 2-pass mode and High profile
  • Per frame comparison
  • Visual comparison
  • Informal codec comparison
  • General conclusions

  • Download



    Contact Information



    Additional information


    See all MSU Video Codecs Comparisons

    MSU video codecs comparisons resources:


    Other materials


    Video resources:

    Last updated: 12-May-2022


    Server size: 8069 files, 1215Mb (Server statistics)

    Project updated by
    Server Team and MSU Video Group

    Project sponsored by YUVsoft Corp.

    Project supported by MSU Graphics & Media Lab