[an error occurred while processing this directive]
[an error occurred while processing this directive]
MSU Annual Video Codecs Comparison 2019:
Call for codecs
All industry professionals will know about your codec’s success!
[an error occurred while processing this directive]
Important Dates
[an error occurred while processing this directive]
| March, 10 |
Deadline for receipt of a codec with required presets |
Participants are to provide a codec's name (and the name of encoding standard if not H.265/HEVC), a company name, and list of use cases in which they want to participate
| March, 1 |
Deadline for applications |
February, 25 March, 10 |
Deadline for settling technical problems with codec’s functioning |
| September |
Draft version of report that will be sent to all participants |
| September |
Reception of comments to the draft |
| September-October |
Comparison report release |
[an error occurred while processing this directive]
About Annual MSU Video Codecs Comparisons
[an error occurred while processing this directive]
MSU team has up to 21 years of experience in video codec analysis, testing and optimization.
Some facts about previous MSU Video Codecs Comparisons:
[an error occurred while processing this directive]
New features in MSU Video Codecs Comparison
|
Since 2019
|
- Increasing the number of test videos up to 100+, adding UGC (user-generated) videos (vlogs, noise videos, etc.)
- Open to compare cloud-based solutions
- Enlarging of video collection
- Going to use new no-reference metrics
|
|
Since 2018
|
- Increased target bitrates for comparison on 4K videos
- Started to use VMAF objective metric
- Upgraded testing hardware to Coffee Lake
- Enlarged video collection (15833 videos)
|
|
Since 2017
|
- Started to perform subjective comparisons with hundreds of observers
- Started to compare AV1
- Enlarged video collection (9299 videos)
|
|
Since 2016
|
- New methodology of videos selection
- New video collection (2909 videos)
|
[an error occurred while processing this directive]
Task of the Comparison
[an error occurred while processing this directive]
We perform comparative unbiased analysis for
- software implementations
- hardware (GPU-based) implementations
- cloud-based implementations
of H.265/HEVC video coding standard and compare it to the best implementations of other video coding standards (H.264/MPEG-4 AVC, AV1, VP9, VP8, MPEG-4 and other) using objective metrics (SSIM, VMAF, PSNR and other).
With MSU Codecs Comparison developers can verify the perfomance of their codec. We share test sequences, encoding parameters (presets) and encoder versions so all developers can reproduce the results of the comparison. Participation with publishing of all results is FREE.
[an error occurred while processing this directive]
Scope of Test
[an error occurred while processing this directive]
Summary report topics:
- Objective measurements + Subjective analysis
- Encoding time
- Bitrate keeping for evaluating rate-control mechanism
- Speed/Quality trade-off analysis
- Averaged objective results analysis
- Leaders in different use-cases
Comparison methodology main points:
-
25-30 HD video sequences (main report) + 10-12 4K video sequences (report appendix).
Number of videos may be increased (up to 100+) and depends on the number of participated codecs. This year UGC (user-generated content) videos may be also added to the test set.
- SSIM, PSNR, VMAF objective metrics and subjective evaluation
- 3 color-planes (Y,U,V) and integral metric values
-
3 various use-cases (Fast, Universal and Ripping) differ by speed/quality trade-off + special use-case for comparison on 4K videos. Also plan to use special Ultra-Ripping use-case (for comparison with AV1).
- 8-10 different target bitrates (1-12 Mbps for HD and 2-20 Mbps for 4K)
- Prosumer-level modern hardware
- Fully automatic testing system
- 7000+ result figures, PDF and HTML reports
[an error occurred while processing this directive]
Encoders analysis methodology
[an error occurred while processing this directive]
Full comparison methodology (PDF, 1.1 MB)
Comparison Rules
This year encoder developers send us a bundle of same presets (with different speed/quality characteristics) for all use-cases.
Please pay attention that we will use multi-core CPU for encoding, so you can use multi-threading
- Decoding is performed with reference decoder (for H.264/AVC and H.265/HEVC only). For other standards participants should provide decoder
- We don't limit GOP size and intra-period
- We don’t limit number of passes in the encoding. Total encoding time should fulfill
- VBR mode is used during all tests (other modes are discussed individually)
- Before results' publishing each developer will receive the results of its codec and competitive free open-source codecs. Developers of each codec can validate the results and write a comment (one paragraph) about the comparison results which will be included in the report
- Participation is free with publication of the results
-
You can join comparison for free if you agree that your codec's results will be published
-
Private participation. Compare your codec with world leaders staying incognito! If your company wants to know results of your codec's testing with possibility to exclude them from publication and information disclosure, you should pay for measurements and report preparing before comparison begins
- Enterprise version of comparison report is available for direct participants for free
- All participants will receive following deliverables to verify the results for free:
- video sequences used in comparison
- binaries of all free encoders used in comparison to verify the results
- all raw video quality metric and encoding speed data for its encoder and for all of free encoders used in comparison
- We are willing to completely or partially delete information about some codec in the public version of comparison report only in exceptional cases (e.g. critical errors in the codec)
Next hardware for codecs testing will be used:
- CPU: Intel Socket 1151 Core i7 8700K (Coffee Lake) (3.7Ghz, 6C12T, TDP 95W)
- Mainboard: ASRock Z370M Pro4
- RAM: Crucial CT16G4DFD824A 2x16GB (totally 32 GB) DIMM DDR4 2400MHz CL15
- OS: Windows 10 x64
For encoder alignment selected presets should provide following encoding speed.
All speed requirements are checked at video sequence encoded at 6Mbps:
- Fast/High Density – 1080p@60fps
- Universal/Broadcast VQ – 1080p@25fps
- Ripping/Pristine VQ – 1080p@1fps and SSIM-RD curve better than x264-veryslow
- For subjective comparison: 1080p@1fps
- For comparison on 4K videos: 20fps and 1fps
- Ultra-Ripping 1080p@0.005fps
Following deliverables should be provided by each developer:
[an error occurred while processing this directive]
Take part in MSU codecs comparison!
Deepest codecs review: 5 reports, including subjective with 1000+ viewers and 7000+ charts
If you want to participate with several codecs, please list them and point their standards via checkboxes.
[an error occurred while processing this directive]
What would you like to see in MSU Codecs Comparison reports?
[an error occurred while processing this directive]
Thanks
[an error occurred while processing this directive]
Special thanks to the following contributors of our previous comparisons
[an error occurred while processing this directive]
Contact Information
[an error occurred while processing this directive]
See all MSU Video Codecs Comparisons
MSU video codecs comparisons resources:
[an error occurred while processing this directive]
Other Materials
[an error occurred while processing this directive]
[an error occurred while processing this directive]
[an error occurred while processing this directive]
[an error occurred while processing this directive]
