Video Area Home >> Video Codecs Comparisons >> MSU Video Codecs Comparison 2022

MSU Video Codecs Comparisons 2022:
Call for Codecs

You can apply for participation in comparison of 2023-2024 ->>

#1 codecs comparisons in the world

Video group head:	Dr. Dmitriy Vatolin
Project head:	Dr. Dmitriy Kulikov
Measurements, analysis:	Nickolay Safonov, Nikita Alutis, Ekaterina Shumitskaya, Mikhail Voronin, Dr. Mikhail Erofeev, Anastasia Antsiferova, Egor Sklyarov, Alexander Gushin

compression.ru

Lomonosov
Moscow State University (MSU)
Graphics and Media Lab

Dubna International
State University

Institute for Information
Transmission Problems,
Russian Academy of Science

Important Dates

August, 15	FullHD comparison application deadline
August, 30	Cloud comparison application deadline
October, 30 (recently changed)	Subjective and FPGA/Ultra-fast comparisons deadlines

You can apply for participation using this form:

Or contact us

Navigation

News

About

2022 Comparison parts

New features
Comparison Rules and Methodology
Frequently asked question (FAQ)
Participate
Feedback
Thanks
Contact

News

26.07.2023 - New preset requirements for 4K comparison
26.07.2023 4K comparison application deadline changed to July, 30
26.06.2023 - New preset requirements for 4K comparison
10.10.2022 - New preset requirements for FullHD and FPGA comparisons
18.08.2022 - Subjective comparison application deadline changed
26.07.2022 - FullHD, Subjective and FPGA comparisons application deadlines changed
26.07.2022 - Changed in methodology
22.07.2022 - New preset requirements for FullHD comparison
14.07.2022 - New preset requirements for FullHD comparison
10.07.2022 - FullHD and Subjective comparisons application deadline changed
16.06.2022 - Preset requirements have been updated

About Annual MSU Video Codecs Comparisons

We perform comparative unbiased analysis for

software implementations
hardware (GPU-based) implementations
cloud-based implementations

of different video coding standards (H.265/HEVC, H.264/MPEG-4 AVC, AV1, VP9, VP8, MPEG-4 and other) using objective metrics (SSIM, VMAF, PSNR and other).

With MSU Codecs Comparison developers can verify the performance of their codec. We share test sequences, encoding parameters and codecs versions so all developers can reproduce the results of the comparison. Participation with publishing of all results is for FREE. Private participation is for fee (contact us for the details).

MSU codecs analysis team has been conducting video codec analysis, testing and optimization since 2004. Some facts about previous MSU Video Codecs Comparisons:

There were more than 400.000 downloads of previous H.264/MPEG-4 AVC and H.265/HEVC video codec comparisons reports
Many codecs' bugs were found and reported to developers
210+ encoders were tested
More than 35 private reports for codec developers (description of codec's weak and strong points) after public report versions

Summary report topics:

Objective measurements
Subjective analysis
Encoding time
Speed/Quality trade-off analysis
Averaged objective results analysis
Leaders in different use cases

2022 Comparison parts and requirements

FullHD Comparison

Subjective Comparison

4K Comparison

FPGA/Hardware-accelerated/Ultra-fast Comparison

Cloud Comparison

Use Cases

Real-time: FullHD@30fps
VoD: FullHD@5fps
Offline: FullHD@1fps

Real-time: 4K@30fps
VoD: 4K@5fps
Offline: 4K@1fps

Ultra-fast: FullHD@60fps

No speed limits, offline encoding scenario, several resolutions

Videos

60 FullHD videos

15 FullHD videos

10 4K videos

60 FullHD videos

15 FullHD videos

Objective metrics

Objective metrics: SSIM, PSNR, VMAF
3 color-planes - Y,U,V with different proportions: 4:11, 6:1:1, 8:11, 10:11
Powered by VQMT

Subjective metrics

Yes, powered by: Subjectify.us

Target Bitrate

FullHD H.264: 1, 2, 4, 6, 8, 10, 12, 15 Mbps
FullHD H.265: 0.7, 1, 2, 3, 4, 6, 8, 10 Mbps
FullHD AV1, VVC: 0.25, 0.5, 1, 2, 3, 4, 6 Mbps
4K H.264: 2, 4, 6, 8, 10, 14, 20 Mbps
4K H.265: 2, 3, 4, 6, 8, 12 Mbps
4K AV1, VVC: 0.5, 1, 2, 3, 4, 5, 6, 8 Mbps

Hardware

CPU: Intel Core i7 12700K (Alder Lake)
SSD: 1Tb
RAM: 4x16GB (64GB)
OS: Windows 11 x64, Ubuntu 22.04 LTS

FPGA Xilinx Alveo U250
GPU: NVIDIA Titan RTX

No hardware required

Download requirements

To be anounced soon

New features in MSU Video Codecs Comparison

Since 2022	Updated hardware to Alder Lake Quality-alignment instead of speed-alignment (see details in methodology) Enlarging of video collection
Since 2020	Automatic codecs submission system Open to compare Linux-based solutions Open to compare cloud-based solutions Enlarging of video collection
Since 2019	Increasing the number of test videos up to 100+, adding UGC (user-generated) videos (vlogs, noise videos, etc.) and HDR videos Open to compare cloud-based solutions Enlarging of video collection Going to use new no-reference metrics
Since 2018	Increased target bitrates for comparison on 4K videos Started to use VMAF objective metric Upgraded testing hardware to Coffee Lake Enlarged video collection (15833 videos)

Comparison Rules and Methodology

2022 Comparison methodology in presentation

Please pay attention that we will use multi-core CPU for encoding, so you can use multi-threading

Decoding is performed with reference decoder (H.264/AVC, H.265/HEVC, VP9, AV1). For other standards participants should provide decoder
We don't limit GOP size and intra-period
We don't limit number of passes in the encoding. Total encoding time should fulfill use case speed requirements

Comparison Rules

Applying for participating, you agree with comparison rules.

Codec Requirements

Presets for different speed requirements should be provided by the developers
Codec should allow to set arbitrary bitrate of resulting stream in constant quality mode
Preferable codec interface - console codec version (with batch processing support - bitrate and file names must be possible to assign from the command line)
Encoder should be compatible with reference decoder, for encoders of custom standards a decoder should be provided

Developers Deliverables

The following deliverables should be provided by each developer:

Codec files (CLI executable file is preferable)
Codec's presets

Frequently asked questions (FAQ)

If you have any more questions about rules or methodology please refer to FAQ or contact us at

Take part in 2022 Video Codecs Comparison!

Deepest codecs review: 5 reports, including subjective with 1000+ viewers and 7000+ charts

Our pipeline of codecs testing:

Prepare encoder. After your application, we send you instructions on how to prepare encoding presets for this year testing use cases (there are limitations for speed) and login for a submission system.
Submission. You send us encoder and presets (our hardware characteristics can be found on the comparison page). Testing methodology is described in presentation.
Speed and quality pre-check. We test the speed of your encoder. We let you know if some speedup is needed or any bugs are found. If the requirements are not satisfied, you submit an updated version and we test it again. Finally, when all requirements are satisfied, you confirm the usage of presets and encoder.
Measurements. We perform the main measurements on this year test video set and prepare a draft of the report. This will take about 1-3 months depending on the number of participants.
Draft report and verification. We send you a draft report with your results and the results of other public participants. You can double-check the results. If you participate privately, in this step you will need to decide whether you will publish your results.
Report release. After your confirmation, we prepare and publish the final report on the comparison page. We send you the Enterprise version of the report (with all sequences, charts, etc.).