Deblocking filter

A deblocking filter is a video filter applied to decoded compressed video to improve visual quality and prediction performance by smoothing the sharp edges which can form between macroblocks when block coding techniques are used. The filter aims to improve the appearance of decoded pictures. It is a part of the specification for both the SMPTE VC-1 codec and the ITU H.264 (ISO MPEG-4 AVC) codec.^{[1] [2] [3]}

In contrast with older MPEG-1/2/4 standards, the H.264 deblocking filter is not an optional additional feature in the decoder.^{[1] [2]} It is a feature on both the decoding path and on the encoding path, so that the in-loop effects of the filter are taken into account in reference to macroblocks used for prediction. When a stream is encoded, the filter strength can be selected, or the filter can be switched off entirely. Otherwise, the filter strength is determined by coding modes of adjacent blocks, quantization step size, and the steepness of the luminance gradient between blocks.

The filter operates on the edges of each 4 ×ばつ 4 or 8 ×ばつ 8 transform block in the luma and chroma planes of each picture. Each small block's edge is assigned a boundary strength based on whether it is also a macroblock boundary, the coding (intra/inter) of the blocks, whether references (in motion prediction and reference frame choice) differ, and whether it is a luma or chroma edge. Stronger levels of filtering are assigned by this scheme where there is likely to be more distortion. The filter can modify as many as three samples on either side of a given block edge (in the case where an edge is a luma edge that lies between different macroblocks and at least one of them is intra coded). In most cases it can modify one or two samples on either side of the edge (depending on the quantization step size, the tuning of the filter strength by the encoder, the result of an edge detection test, and other factors).

Although the concept of an "in loop" deblocking filter is best known in relation to H.264, it was actually also standardized earlier (in a somewhat different form) in the H.263v2 Annex J.^[4]

Prior to H.263 Annex J and H.264, the use of a deblocking filter as a post-processing technique to improve the visual quality of decoded pictures was already a well-known technology. This was particularly true in the video conferencing industry, where the low bit rates used tended to produce significant blocking artifacts that could be substantially reduced by such a filter.