MSU Video Codecs Comparison 2021 Part 1: FullHD, objective
Sixteen Annual Video-Codecs Comparison by MSU
|
|||||||||
|
|
|
|
||||||
| compression.ru |
Lomonosov Moscow State University (MSU) Graphics and Media Lab |
Dubna International
State University |
Institute for Information Transmission Problems, Russian Academy of Science |
||||||
News
- 15.11.2021 New metrics added in free report
- 04.11.2021 Release of the comparison
Navigation
Results
- The places below are given only for quality scores, not taking encoding speed into account
- Encoders with scores closer than 1% share one place
| Slow (1 fps) | Fast (30 fps) | |||||||||||||
|
Best quality (YUV-SSIM 4:1:1) |
|
|
||||||||||||
|
Best quality (YUV-SSIM 6:1:1) |
|
|
||||||||||||
|
Best quality (YUV-MS-SSIM 4:1:1) |
|
|
||||||||||||
|
Best quality (YUV-MS-SSIM 6:1:1) |
|
|
||||||||||||
|
Best quality (YUV-PSNR avg.log 4:1:1) |
|
|
||||||||||||
|
Best quality (YUV-PSNR avg.log 6:1:1) |
|
|
||||||||||||
|
Best quality (YUV-PSNR avg.MSE 4:1:1) |
|
|
||||||||||||
|
Best quality (YUV-PSNR avg.MSE 6:1:1) |
|
|
||||||||||||
|
Best quality (Y-VMAF 0.63) |
|
|
||||||||||||
|
Best quality (Y-VMAF NEG) |
|
|
||||||||||||
|
Best quality (YUV-VMAF* 0.6.3 4:1:1) |
|
|
||||||||||||
|
Best quality (YUV-VMAF* NEG 4:1:1) |
|
|
||||||||||||
|
Best quality (YUV-VMAF* 0.6.3 6:1:1) |
|
|
||||||||||||
|
Best quality (YUV-VMAF* NEG 6:1:1) |
|
|
* - YUV-VMAF was calculated as VMAF for all colour-planes (Y, U, V) following the same methodology as YUV-SSIM, YUV-PSNR and other metrics.
The biggest number of codecs took part in comparison of Slow encoding (1 fps). The winners vary for different objective quality metrics. The participants were rated using BSQ-rate (enhanced BD-rate) scores [1].
[1] A. Zvezdakova, D. Kulikov, S. Zvezdakov, D. Vatolin, "BSQ-rate: a new approach for video-codec performance comparison and drawbacks of current solutions," 2020.
Download and buy report
| Free | Enterprise | |||
| Number of test sequences | 3 | 40+ | ||
| Test video descriptions | ||||
| Basic codec info | ||||
| Objective metrics | Only YUV-SSIM | 20+ objective metrics | ||
| Test videos download | ||||
| Encoders presets description | ||||
| PDF report | 62 pages | 119 pages | ||
| HTML report | 28 interactive charts | 7000+ interacive charts | ||
| Price | Free | 950 USD | ||
| Download/Buy |
PDF & HTML reports + videos description (ZIP)
|
You will receive enterprise versions of all 2021 reports (FullHD, Subjective, 4K)
|
Participated codecs
| Codec name | Use cases | Standard | Version | |
| 1 |
Reference x265
MulticoreWare, Inc. |
Slow (1 fps), Fast (30 fps) |
H.265/HEVC | 3.5+1-f0c1022b6, Windows |
| 2 |
SVT-HEVC
Open Visual Cloud |
Slow (1 fps), Fast (30 fps) |
H.265/HEVC | 1.5.1, Windows |
| 3 |
SVT-VP9
Open Visual Cloud |
Slow (1 fps), Fast (30 fps) |
VP9 | 0.3.0, Windows |
| 4 |
SVT-AV1
Open Visual Cloud |
Slow (1 fps), Fast (30 fps) |
AV1 | v0.8.6, Windows |
| 5 |
xin26x (HEVC)
A Father (xin26x) |
Slow (1 fps), Fast (30 fps) |
H.265/HEVC | v1.0, Windows |
| 6 |
xin26x (VVC)
A Father (xin26x) |
Slow (1 fps), Fast (30 fps) |
H.266/VVC | v1.0, Windows |
| 7 |
x264
x264 project |
Slow (1 fps), Fast (30 fps) |
H.264/AVC | r3065-ae03d92, Windows |
| 8 |
VVenC
Fraunhofer HHI |
Slow (1 fps) | H.266/VVC | v1.0.0, Linux |
| 9 |
SIF
SIF Codec LLC |
Slow (1 fps), Fast (30 fps) |
SIF | 1.0, Windows |
| 10 |
rav1e
The rav1e contributors |
Slow (1 fps) | AV1 | 0.5.0-alpha (p20210518), Windows |
| 11 |
HW266
Huawei Technologies Co., Ltd. & Multimedia Communi- cation Lab, Xidian University |
Slow (1 fps) | H.266/VVC | v1.0.7, Windows |
| 12 |
Phoenix265
Phoenix Codec Team |
Slow (1 fps), Fast (30 fps) |
H.265/HEVC | v4.4, Linux |
| 13 |
x265
MulticoreWare, Inc. |
Slow (1 fps), Fast (30 fps) |
H.265/HEVC | 3.5+1-ce882936d, Windows |
| 14 |
aom
AOMedia |
Slow (1 fps) | AV1 | AOMedia Project AV1 Encoder 3.1.0-309-g12287adee, Linux |
| 15 |
QAV1
iQIYI Inc. |
Slow (1 fps) | AV1 | v2.1, Linux |
| 16 |
QAVS3
iQIYI Inc. |
Slow (1 fps) | AVS3 | v1.0.0, Linux |
| 17 |
Tencent VAV1
Tencent |
Slow (1 fps) | AV1 | 0.4.0, Linux |
| 18 |
Tencent V265
Tencent |
Slow (1 fps), Fast (30 fps) |
H.265/HEVC | 1.5.0, Linux |
| 19 |
Tencent266
Tencent |
Slow (1 fps) | H.266/VVC | 0.0.2, Windows |
| 20 |
S266_v1
Alibaba Group |
Slow (1 fps) | H.266/VVC | v1.1, Windows |
| 21 |
S266_v2
Alibaba Group |
Slow (1 fps) | H.266/VVC | v2.1, Windows |
| 22 |
by265
Boyan Technology Ltd. |
Slow (1 fps) | HEVC | v1.0.5, Linux |
Comparison Rules
FullHD codec testing objectives
The main goal of this report is the presentation of a comparative evaluation of the quality of new and existing codecs using objective measures of assessment. The comparison was done using settings provided by the developers of each codec. Nevertheless, we required all presets to satisfy minimum speed requirement on the particular use case. The main task of the comparison is to analyze different encoders for the task of transcoding video – e.g., compressing video for personal use.
Test Hardware Characteristics
- CPU: Intel Socket 1151 Core i7 8700K (Coffee Lake) (3.7Ghz, 6C12T, TDP 95W)
- Mainboard: ASRock Z370M Pro4
- RAM: Crucial CT16G4DFD824A 2x16GB (totally 32 GB) DIMM DDR4 2400MHz CL15
- OS: Windows 10 x64, Linux(
For this platform we considered three key use cases with different speed requirements:
- Online – 1080p@30fps
- Offline – 1080p@1fps
See more on Call For Codecs 2021 page
Videos
Videos for testing set were chosen from MSU video collection via a voting among comparison participants, organizers and an independend expert.
| Year | # FullHD videos | # FullHD samples | # 4K videos | # 4K samples | Total # of videos | Total # of samples |
| 2016 | 3 | 7 | 882 | 2902 | 885 | 2909 |
| 2017 | 1996 | 4638 | 1544 | 4561 | 3540 | 9299 |
| 2018 | 4342 | 10330 | 1946 | 5503 | 6288 | 15833 |
| 2020 | 4945 | 12402 | 2091 | 6016 | 7036 | 18418 |
Final video set consists of 51 sequences including new videos from Vimeo and media.xiph.org derf's collection.
Descriptions of all test videos are presented in a separate PDF provided with the report.
Codec Analysis and Tuning for Codec Developers and Codec Users
Computer Graphics and Multimedia Laboratory of Moscow State University:
- 17+ years working in the area of video codec analysis and tuning using objective quality metrics and subjective comparisons.
- 30+ reports of video codec comparisons and analysis (H.265, H.264, AV1, VP9, MPEG-4, MPEG-2, decoders' error recovery).
- Methods and algorithms for codec comparison and analysis development, separate codec's features and codec's options analysis.
Strong and Weak Points of Your Codec
- Deep encoder parts analysis (ME, RC on GOP, mode decision, etc).
- Weak and strong points for your encoder and complete information about encoding quality on different content types.
- Encoding Quality improvement by the pre and post filtering (including technologies licensing).
Independent Codec Estimation Comparing to Other Codecs for Different Use-cases
- Comparative analysis of your encoder and other encoders.
- We have direct contact with many codec developers.
- You will know place of your encoder between other newest well-known encoders (compare encoding quality, speed, bitrate handling, etc.).
Encoder Features Implementation Optimality Analysis
We perform encoder features effectiveness (speed/quality trade-off) analysis that could lead up to 30% increase in the speed/quality characteristics of your codec. We can help you to tune your codec and find best encoding parameters.Thanks
Special thanks to the following contributors of our previous comparisons
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Contact Information
Subscribe to report updates
Other Materials
Video resources:
Server size: 8069 files, 1215Mb (Server statistics)
Project updated by
Server Team and
MSU Video Group
Project sponsored by YUVsoft Corp.
Project supported by MSU Graphics & Media Lab