Video Area Home >> Video Codecs Comparisons >> MSU Video Codecs Comparison 2021

MSU Video Codecs Comparison 2021 Part 1: FullHD, objective

Sixteen Annual Video-Codecs Comparison by MSU

Video group head: Dr. Dmitriy Vatolin
Project head: Dr. Dmitriy Kulikov
Measurements, analysis: Dr. Mikhail Erofeev,
Anastasia Antsiferova,
Egor Sklyarov,
Alexander Yakovenko,
Nickolay Safonov,
Alexander Gushin,
Nikita Alutis
compression.ru Lomonosov
Moscow State University (MSU)
Graphics and Media Lab
Dubna International
State University
Institute for Information
Transmission Problems,
Russian Academy of Science

News

  • 15.11.2021 New metrics added in free report
  • 04.11.2021 Release of the comparison

Navigation


Results


  • The places below are given only for quality scores, not taking encoding speed into account
  • Encoders with scores closer than 1% share one place
Slow (1 fps) Fast (30 fps)
Best quality
(YUV-SSIM 4:1:1)
1st: HW266
2nd: S266_v1
3rd: Tencent266
1st: Tencent V265
2nd: QAV1
3rd: Phoenix265
Best quality
(YUV-SSIM 6:1:1)
1st: HW266
2nd: S266_v1
3rd: Tencent266
1st: Tencent V265
2nd: QAV1
3rd: Phoenix265
Best quality
(YUV-MS-SSIM 4:1:1)
1st: HW266
2nd: S266_v1
3rd: Tencent266
1st: Tencent V265
2nd: QAV1, Phoenix265
3rd: by265
Best quality
(YUV-MS-SSIM 6:1:1)
1st: HW266
2nd: S266_v1
3rd: Tencent266
1st: Tencent V265
2nd: QAV1
3rd: Phoenix265
Best quality
(YUV-PSNR avg.log 4:1:1)
1st: S266_v2
2nd: S266_v1
3rd: Tencent VAV1, Tencent266
1st: Tencent V265
2nd: Phoenix265
3rd: QAV1
Best quality
(YUV-PSNR avg.log 6:1:1)
1st: S266_v2
2nd: S266_v1, Tencent266, Tencent VAV1
3rd: HW266
1st: Tencent V265
2nd: Phoenix265
3rd: QAV1
Best quality
(YUV-PSNR avg.MSE 4:1:1)
1st: S266_v1
2nd: S266_v2
3rd: HW266
1st: Tencent V265
2nd: Phoenix265
3rd: QAV1
Best quality
(YUV-PSNR avg.MSE 6:1:1)
1st: S266_v1
2nd: Tencent VAV1, Tencent266, S266_v2, HW266
3rd: QAV1
1st: Tencent V265
2nd: Phoenix265, QAV1
3rd: by265
Best quality
(Y-VMAF 0.63)
1st: Tencent266
2nd: Tencent VAV1
3rd: S266_v1
1st: Tencent V265
2nd: Phoenix265
3rd: QAV1
Best quality
(Y-VMAF NEG)
1st: Tencent266
2nd: Tencent VAV1
3rd: S266_v1
1st: Tencent V265
2nd: Phoenix265
3rd: QAV1
Best quality
(YUV-VMAF* 0.6.3 4:1:1)
1st: S266_v2
2nd: S266_v1
3rd: HW266
1st: Tencent V265
2nd: Phoenix265
3rd: QAV1
Best quality
(YUV-VMAF* NEG 4:1:1)
1st: S266_v2
2nd: S266_v1
3rd: HW266
1st: Tencent V265
2nd: Phoenix265
3rd: QAV1
Best quality
(YUV-VMAF* 0.6.3 6:1:1)
1st: S266_v2, S266_v1
2nd: Tencent VAV1, Tencent266, HW266
3rd: Tencent V265
1st: Tencent V265
2nd: Phoenix265
3rd: QAV1
Best quality
(YUV-VMAF* NEG 6:1:1)
1st: S266_v2, S266_v1
2nd: Tencent266, Tencent VAV1
3rd: HW266
1st: Tencent V265
2nd: Phoenix265
3rd: QAV1

* - YUV-VMAF was calculated as VMAF for all colour-planes (Y, U, V) following the same methodology as YUV-SSIM, YUV-PSNR and other metrics.

The biggest number of codecs took part in comparison of Slow encoding (1 fps). The winners vary for different objective quality metrics. The participants were rated using BSQ-rate (enhanced BD-rate) scores [1].

[1] A. Zvezdakova, D. Kulikov, S. Zvezdakov, D. Vatolin, "BSQ-rate: a new approach for video-codec performance comparison and drawbacks of current solutions," 2020.

Download and buy report


Free Enterprise
Number of test sequences 3 40+
Test video descriptions
Basic codec info
Objective metrics Only YUV-SSIM 20+ objective metrics
Test videos download
Encoders presets description
PDF report 62 pages 119 pages
HTML report 28 interactive charts 7000+ interacive charts
Price Free 950 USD
Download/Buy
PDF & HTML reports + videos description (ZIP)
You will receive enterprise versions of all 2021 reports (FullHD, Subjective, 4K)

Participated codecs


Codec name Use cases Standard Version
1 Reference x265
MulticoreWare, Inc.
Slow (1 fps),
Fast (30 fps)
H.265/HEVC 3.5+1-f0c1022b6, Windows
2 SVT-HEVC
Open Visual Cloud
Slow (1 fps),
Fast (30 fps)
H.265/HEVC 1.5.1, Windows
3 SVT-VP9
Open Visual Cloud
Slow (1 fps),
Fast (30 fps)
VP9 0.3.0, Windows
4 SVT-AV1
Open Visual Cloud
Slow (1 fps),
Fast (30 fps)
AV1 v0.8.6, Windows
5 xin26x (HEVC)
A Father (xin26x)
Slow (1 fps),
Fast (30 fps)
H.265/HEVC v1.0, Windows
6 xin26x (VVC)
A Father (xin26x)
Slow (1 fps),
Fast (30 fps)
H.266/VVC v1.0, Windows
7 x264
x264 project
Slow (1 fps),
Fast (30 fps)
H.264/AVC r3065-ae03d92, Windows
8 VVenC
Fraunhofer HHI
Slow (1 fps) H.266/VVC v1.0.0, Linux
9 SIF
SIF Codec LLC
Slow (1 fps),
Fast (30 fps)
SIF 1.0, Windows
10 rav1e
The rav1e contributors
Slow (1 fps) AV1 0.5.0-alpha (p20210518), Windows
11 HW266
Huawei Technologies Co., Ltd. & Multimedia Communi- cation Lab, Xidian University
Slow (1 fps) H.266/VVC v1.0.7, Windows
12 Phoenix265
Phoenix Codec Team
Slow (1 fps),
Fast (30 fps)
H.265/HEVC v4.4, Linux
13 x265
MulticoreWare, Inc.
Slow (1 fps),
Fast (30 fps)
H.265/HEVC 3.5+1-ce882936d, Windows
14 aom
AOMedia
Slow (1 fps) AV1 AOMedia Project AV1 Encoder 3.1.0-309-g12287adee, Linux
15 QAV1
iQIYI Inc.
Slow (1 fps) AV1 v2.1, Linux
16 QAVS3
iQIYI Inc.
Slow (1 fps) AVS3 v1.0.0, Linux
17 Tencent VAV1
Tencent
Slow (1 fps) AV1 0.4.0, Linux
18 Tencent V265
Tencent
Slow (1 fps),
Fast (30 fps)
H.265/HEVC 1.5.0, Linux
19 Tencent266
Tencent
Slow (1 fps) H.266/VVC 0.0.2, Windows
20 S266_v1
Alibaba Group
Slow (1 fps) H.266/VVC v1.1, Windows
21 S266_v2
Alibaba Group
Slow (1 fps) H.266/VVC v2.1, Windows
22 by265
Boyan Technology Ltd.
Slow (1 fps) HEVC v1.0.5, Linux

Comparison Rules


FullHD codec testing objectives

The main goal of this report is the presentation of a comparative evaluation of the quality of new and existing codecs using objective measures of assessment. The comparison was done using settings provided by the developers of each codec. Nevertheless, we required all presets to satisfy minimum speed requirement on the particular use case. The main task of the comparison is to analyze different encoders for the task of transcoding video – e.g., compressing video for personal use.

Test Hardware Characteristics

  • CPU: Intel Socket 1151 Core i7 8700K (Coffee Lake) (3.7Ghz, 6C12T, TDP 95W)
  • Mainboard: ASRock Z370M Pro4
  • RAM: Crucial CT16G4DFD824A 2x16GB (totally 32 GB) DIMM DDR4 2400MHz CL15
  • OS: Windows 10 x64, Linux(

For this platform we considered three key use cases with different speed requirements:

  • Online – 1080p@30fps
  • Offline – 1080p@1fps

See more on Call For Codecs 2021 page

Videos

Videos for testing set were chosen from MSU video collection via a voting among comparison participants, organizers and an independend expert.

Number of videos in MSU video collection
Year # FullHD videos # FullHD samples # 4K videos # 4K samples Total # of videos Total # of samples
2016 3 7 882 2902 885 2909
2017 1996 4638 1544 4561 3540 9299
2018 4342 10330 1946 5503 6288 15833
2020 4945 12402 2091 6016 7036 18418

Bitrate distribution of videos in MSU video collection Videos bitrate distribution

Final video set consists of 51 sequences including new videos from Vimeo and media.xiph.org derf's collection.


Video sequences selection

Descriptions of all test videos are presented in a separate PDF provided with the report.


Codec Analysis and Tuning for Codec Developers and Codec Users


Computer Graphics and Multimedia Laboratory of Moscow State University:

  • 17+ years working in the area of video codec analysis and tuning using objective quality metrics and subjective comparisons.
  • 30+ reports of video codec comparisons and analysis (H.265, H.264, AV1, VP9, MPEG-4, MPEG-2, decoders' error recovery).
  • Methods and algorithms for codec comparison and analysis development, separate codec's features and codec's options analysis.

Strong and Weak Points of Your Codec

  • Deep encoder parts analysis (ME, RC on GOP, mode decision, etc).
  • Weak and strong points for your encoder and complete information about encoding quality on different content types.
  • Encoding Quality improvement by the pre and post filtering (including technologies licensing).

Independent Codec Estimation Comparing to Other Codecs for Different Use-cases

  • Comparative analysis of your encoder and other encoders.
  • We have direct contact with many codec developers.
  • You will know place of your encoder between other newest well-known encoders (compare encoding quality, speed, bitrate handling, etc.).

Encoder Features Implementation Optimality Analysis

We perform encoder features effectiveness (speed/quality trade-off) analysis that could lead up to 30% increase in the speed/quality characteristics of your codec. We can help you to tune your codec and find best encoding parameters.

Thanks


Special thanks to the following contributors of our previous comparisons
Apple Google Intel NVidia
Huawei AMD Adobe Tencent
Zoom video communications Facebook Inc. Netflix Alibaba
KDDI R&D labs Dolby Tata Elxsi Octasic
Qualcomm Voceweb Elgato Telecast
ATI MainConcept Vitec dicas

Contact Information

We appreciate any feedback on our comparison


Subscribe to report updates




Other Materials


Video resources:

Last updated: 29-October-2021


Server size: 8069 files, 1215Mb (Server statistics)

Project updated by
Server Team and MSU Video Group

Project sponsored by YUVsoft Corp.

Project supported by MSU Graphics & Media Lab