Compression project >> Video Area Home

Call for MPEG4-AVC/H.264 codecs 2012

Eighth H.264 video codec comparison
For real researchers and developers in field of high-end video compression

MSU Graphics & Media Lab (Video Group)

Important Dates

Due some technical issues all date are shifted. Sorry for possible inconvenience. New dates are following:

~~February, 17~~	~~Deadline for receipt of a H.264 codec with required presets~~
~~February, 28~~	~~Deadline for settling technical problems with codec’s functioning~~
May, 22	Draft version of report that will be sent to all participants
June, 1	Deadline for reception of comments to the draft
June, 12	Comparison report release

Task of the Comparison

To perform comparative unbiased analysis of the current software and hardware (GPU-based) implementations of H.264/MPEG-4 AVC video coding standard using objective metrics

Scope of Test

Summary report topics:

Objective measurements (SSIM, PSNR, Average Advantage and etc.)

Encoding time

Bitrate keeping

Speed/Quality trade-off analysis

Averaged objective results analysis

Leaders in different areas

Three main usage areas for this comparison are:

Videoconference

Movie

HDTV

For Videoconference main use-case is high-quality low compressed (inc. lossy) video encoding.
For Movie and HDTV areas main use-case is video transcoding. See previous comparison for details.

New in This Comparison

This comparison will include some new features that were missed in previous comparisons

GPU codecs will be analyzed: Nvidia, AMD, and Ivy Bridge based encoders.

New video codecs including non H.264 standard

We plan to include new codecs that did not participate in previous comparison by choosing presets for them ourself. For that task we will use option analysis. So we will be glad to have a direct contact with codec developers. The main benefit of direct participation for developers is that developer could help us to choose correct settings for better results and additionally developers receive Enterprise version of comparison free of charge.

Videoconference encoding analysis will be more complex comparing to previous comparison.

Local encoding quality variation analysis

Local bitrate keeping analysis, etc.

New objective quality metrics MS-SSIM and 3-SSIM in addition to classical SSIM and PSNR

New video sequences including professional video sequences for video codec analysis

This year we want to make GPU-encoders analysis more interesting and complex. For this analysis we plan to use top GPUs, e.g. Nvidia and AMD and also to anqalyze Intel Ivy Bridge (Intel HD Graphics 4000) based encoder. You can send us your GPU-accelerated or GPU-based encoder with or without CPU-based encoder!

Comparison Rules

There will be three types of testing sequences and for every type there will be specific set of presets, bitrates:

HDTV streams

resolutions: 1920x1080, 1280x720

bitrates 0.5-18 Mbps

3 presets: "High Speed", "High Quality", "Normal"

speed requirements (at 3 Mbps, 1280x720 sequences):

Minimum 100 fps for "High Speed" preset

Minimum 50 fps for "Normal" preset

Minimum 20 fps for "High Quality" preset

Movie streams

different types of content (movie, animation)

resolutions: PAL, NTSC, 4CIF

bitrates 500-2000 Kbps

3 presets: "High Speed", "High Quality", "Normal"

speed requirements (at 750 Kbps, 4CIF sequence):

Minimum 120 fps for "High Speed" preset

Minimum 80 fps for "Normal" preset

Minimum 40 fps for "High Quality" preset

Videoconferences

Videoconference content ("talking head", static camera)

resolutions: CIF, 4CIF, 720p

bitrates 128-1500 Kbps

speed requirements - minimum 60 fps. One-pass encoding

Technical note for developers:

We plan to change preset choosing method for this comparison. We ask you to send us set of presets (10-20) or description how to change speed/quality for it. Then we test all presets and with your help choose the best presets for comparison.

The main reason of speed limitation is to compare objective quality without regarding speed/quality trade-off, so if preset provided by developer will work much faster than requirements but with low quality this can lead to low total results.

An important restriction on a preset is encoding time for it. A few iterations of compliance testing and preset optimization are possible to meet the requirements set above. Please pay attention that we will use 4-core CPU for encoding, so you can use multi-threading.

All speed measurements will be performed using our testing hardware (see detailed description below).

Decoding is performed with JM decoder.

All measurements will be performed using

YUVsoft Video Codec Scoring System and

MSU Video Quality Measurement Tool (Pro Version).

Before results' publishing each developer will receive the results of its codec and competitive free codecs. Developers of each codec can write a comment (one paragraph) about the comparison results. That comment will be included in the report.

We are willing to completely or partially delete information about some codec in the public version of comparison report only in exceptional cases (e.g. critical errors in a codec).

If your company wants to receive results of your codec testing without publication and information disclosure, you should pay for measurements and report preparing before comparison begins. You can join comparison for free if you agree that your codec's results will be published.

Enterprise version of comparison report is available for direct participants for free.

Test Hardware Characteristics

Sugar Bay platform, 3rd Generation Core i7 3xxx(IVB), 4 Cores CPU @3.4 GHz,

Integrated GPU: Intel HD Graphics 4000

GPU: NVIDIA GeForce GTX 580 or 590

GPU: AMD (will be defined soon) for encoders that use AMD GPU

Total Physical Memory: 2x2 Gb RAM (1600 MHz)

HDD: SSD160G

Operation System: Windows 7

Codec Requirements

Presets for different types of video sequences should be provided by the developers

Codec should allow to set arbitrary bitrate of resulting stream

3 variants of codec interface are possible:

Console codec version (with batch processing support — bitrate and file names must be possible to assign from the command line). This variant is most preferable.

Video for Windows Codec with correct state saving (batch processing support).

Direct Show filter. In this case software for batch processing should be provided by the developers.

Codec should open and save *.yuv or *.avi (YV12 colorspace) files

Encoder should be compatible with JM reference decoder