Video Area Home >> Video Codecs Comparisons >> MSU Video Codecs Comparison 2021

MSU Video Codecs Comparison 2021
Part 5: FPGA/Hardware-accelerated/Ultra-fast encoders

Sixteen Annual Video-Codecs Comparison by MSU

Video group head:	Dr. Dmitriy Vatolin
Project head:	Dr. Dmitriy Kulikov
Measurements, analysis:	Dr. Mikhail Erofeev, Anastasia Antsiferova, Egor Sklyarov, Alexander Yakovenko, Nickolay Safonov, Alexander Gushin, Nikita Alutis

compression.ru

Lomonosov
Moscow State University (MSU)
Graphics and Media Lab

Dubna International
State University

Institute for Information
Transmission Problems,
Russian Academy of Science

News

11.04.2022 Report release

Description

In MSU FPGA/Hardware-accelerated/Ultra-fast Video Codecs Comparison 2021, we analyzed 4 hardware and 3 software codecs. The speed requirement for all platforms was 60 fps. Some of the participants could not reach speed requirement, but we decided to publish all results anyway to show the current landscape. 50 FullHD video sequences were used.

Results

The places below are given by quality scores
Encoders with scores closer than 1% share one place

Ultra-fast (60 fps)

Best encoder
YUV-SSIM, Y-VMAF (v0.6.3)

	Tencent V265 (CPU)
	BVE 1.0 (FPGA)
	QAV1 (CPU)

Best encoder
YUV-PSNR

Tencent V265 (CPU), BVE 1.0 (FPGA)

QAV1 (CPU)

Reference x265 (CPU)

Best CPU/GPU encoder
YUV-SSIM

Tencent V265 (CPU)

Best FPGA encoder
YUV-SSIM

BVE 1.0 (FPGA)

Best encoder, for long video
(neglecting lookahead i/o time)
YUV-SSIM, Y-VMAF (v0.6.3)

	Tencent V265 (CPU), BVE 1.0 (FPGA)
	QAV1 (CPU)
	NVENC H.265 (GPU)

Best encoder, for long video
(neglecting lookahead i/o time)
YUV-PSNR

	BVE 1.0 (FPGA)
	Tencent V265 (CPU)
	QAV1 (CPU)

Speed/quality trade-off "Ultra-fast 60 fps" all sequences, YUV (4:1:1)-SSIM metric

Speed/quality trade-off "Ultra-fast 60 fps (neglecting lookahead i/o time" all sequences, YUV (4:1:1)-SSIM metric

The winners vary for different objective quality metrics. The participants were rated using BSQ-rate (enhanced BD-rate) scores [1].

[1] A. Zvezdakova, D. Kulikov, S. Zvezdakov, D. Vatolin, "BSQ-rate: a new approach for video-codec performance comparison and drawbacks of current solutions," 2020.

Download

FPGA Express Report
Objective comparison of ultra-fast and hardware-accelerated video encoders
Released on April, 11

Full version for free

4 hardware-accelerated and 3 software encoders
BVE 1.0 (FPGA), Fudan (FPGA), NVENC H.265 (GPU), NVENC H.264 (GPU); Tencent V265 (CPU), QAV1 (CPU), x265 (CPU)

50 FullHD videos
60 fps encoding

28+ objective metrics (VMAF, SSIM, MS-SSIM, PSNR of different variants)

HTML report + links to source videos (ZIP)
5400+ interactive charts

Participants

	Codec name	Platform	Standard	Version
1	Reference x265 MulticoreWare, Inc.	CPU	H.265/HEVC	3.5+1-f0c1022b6, Windows
2	Tencent V265 (CPU) Tencent	Ultra-fast (60 fps)	H.265/HEVC	-
3	BVE 1.0 (FPGA) Bytedance Inc.	FPGA (60 fps)	-	-
4	Fudan (FPGA) Fudan University	FPGA (60 fps)	-	-
5	NVENC H.265 (GPU) -	Ultra-fast (60 fps)	H.265/HEVC	-
6	NVENC H.264 (GPU) -	Ultra-fast (60 fps)	H.264/AVC	-
7	QAV1 (CPU) iQIYI Inc.	Ultra-fast (60 fps)	AV1	-

Comparison Rules

Hardware-accelerated and ultra-fast codecs testing objectives

The main goal of this report is the presentation of a comparative evaluation of the quality of new and existing codecs using objective measures of assessment. The comparison was done using settings provided by the developers of each codec. Nevertheless, we required all presets to satisfy minimum speed requirement on the particular use case. The main task of the comparison is to analyze different encoders for the task of transcoding video – e.g., compressing video for personal use.

Test Hardware Characteristics for CPU encoders

CPU: Intel Socket 1151 Core i7 8700K (Coffee Lake) (3.7Ghz, 6C12T, TDP 95W)
Mainboard: ASRock Z370M Pro4
RAM: Crucial CT16G4DFD824A 2x16GB (totally 32 GB) DIMM DDR4 2400MHz CL15
OS: Windows 10 x64, Linux

Test Hardware Characteristics for GPU encoders

CPU: Intel Socket 1151 Core i7 8700K (Coffee Lake) (3.7Ghz, 6C12T, TDP 95W)
GPU: NVIDIA TITAN RTX
Mainboard: ASRock Z370M Pro4
RAM: Crucial CT16G4DFD824A 2x16GB (totally 32 GB) DIMM DDR4 2400MHz CL15
OS: Windows 10 x64, Linux

Test Hardware Characteristics for FPGA encoders

CPU: Intel Socket 1151 Core i7-6700K (Skylake) (4.00GHz, 4C8T, TDP 91 W)
FPGA: Xilinx Alveo U250 Accelerator
Mainboard: ASUSTeK Z170-K
RAM: Crucial CT16G4DFD824A 4x8GB (totally 32 GB) DIMM DDR4 2100MHz CL15
OS: CentOS 7

See more on Call For Codecs 2021 page

Videos

Videos for testing set were chosen from MSU video collection via a voting among comparison participants, organizers and an independend expert.

Number of videos in MSU video collection

Year	# FullHD videos	# FullHD samples	# 4K videos	# 4K samples	Total # of videos	Total # of samples
2016	3	7	882	2902	885	2909
2017	1996	4638	1544	4561	3540	9299
2018	4342	10330	1946	5503	6288	15833
2020	4945	12402	2091	6016	7036	18418

Bitrate distribution of videos in MSU video collection Videos bitrate distribution

Final video set consists of 50 sequences including new videos from Vimeo and media.xiph.org derf's collection.

Descriptions of all test videos are presented in a separate PDF provided with the report.

Codec Analysis and Tuning for Codec Developers and Codec Users

Computer Graphics and Multimedia Laboratory of Moscow State University:

17+ years working in the area of video codec analysis and tuning using objective quality metrics and subjective comparisons.
30+ reports of video codec comparisons and analysis (H.265, H.264, AV1, VP9, MPEG-4, MPEG-2, decoders' error recovery).
Methods and algorithms for codec comparison and analysis development, separate codec's features and codec's options analysis.

Strong and Weak Points of Your Codec

Deep encoder parts analysis (ME, RC on GOP, mode decision, etc).
Weak and strong points for your encoder and complete information about encoding quality on different content types.
Encoding Quality improvement by the pre and post filtering (including technologies licensing).

Independent Codec Estimation Comparing to Other Codecs for Different Use-cases

Comparative analysis of your encoder and other encoders.
We have direct contact with many codec developers.
You will know place of your encoder between other newest well-known encoders (compare encoding quality, speed, bitrate handling, etc.).

Encoder Features Implementation Optimality Analysis

We perform encoder features effectiveness (speed/quality trade-off) analysis that could lead up to 30% increase in the speed/quality characteristics of your codec. We can help you to tune your codec and find best encoding parameters.