Discover the Benefits of our Video Processing Filters

By Sergio Sanz-Rodríguez, PhD — Quality has no Substitute. This is the slogan of the EU-funded Immersify project aiming at developing cutting-edge tools for next-generation of immersive media. Because Spin Digital cares about the final quality of video, the HEVC encoder (Spin Enc) and its high precision pre-processing filters (aka Spin Filter) have been optimized to produce the maximum possible quality for a given bitrate.

Supported Filters and Implementation

Not only the encoder plays a paramount role in the final quality of video, but also the pre-processing filters. Before compression, the original video is usually transformed by means of some well-known operations, such as resolution scaling (8K to 4K, or 4K to FHD) or contribution-to-distribution conversion (4:4:4 12-bit to 4:2:0 10- or 8-bit). Therefore, video filters must be implemented in such a way that the maximum possible quality is preserved.

Video filters must be implemented in such a way that the maximum possible quality is preserved.

Spin Filter operates in floating point and have been performance optimized with multi-threading and SIMD instructions. It includes the following operations:

These filters can be chained together to get the desired conversion pipeline. Such a chain can be specified through the filter options.

Below there is an example of some transformations before encoding:

# Input video: 8Kp60, RGB 8-bit
# Output video: 4Kp60, YUV 4:2:0 10-bit, QP 30
# Filter chain:
#     - "s:w=3840,": resolution scaling from 8K to 4K
#     - "f:pixfmt=420p,": chroma sampling conversion from RGB to YUV 4:2:0
#     - "f:bd=10": bit depth conversion from 8-bit to 10-bit. 

./spinenc -s 7680x4320 -r 60 --fmt rgbp:8 -i raw.rgb --filter s:w=3840,f:pixfmt=420p,f:bd=10 -q 30 -o out.hevc

Before encoding the video, the spinenc application lists all the operations introduced by the user. Below we can see the generated filter chain:

Filter config
-------------
Resolution scaling       (7680x4320)->(3840x2160) imeth=lanc3
Color conversion         rgb->yuv
Chroma conversion        444p->420p
Bit depth conversion     8b->10b

As expected, the execution order of the filters exactly corresponds to that specified in the command line. We can also observe that the default resolution scaling method is lanc3 (Lanczos filter with 3 lobes). Although not stated in the command line, color conversion from RGB to YUV is also applied, since 4:2:0 can only be possible in YUV format.

Automatic Conversion Tool

Given an input format, it is not trivial sometimes to construct a conversion pipeline that generates the desired output format. This happens especially  when the number of required operations is high.

In addition, there are different ways to chain video filters. For example, two filters (filter A, filter B) result in two possible chains (filter A –> filter B, or filter B –> filter A), and three filters in up to six chains. Nevertheless, not all the possible combinations produce the same output video and have the same speed performance. In other words, some filter chains can be better than others in terms of quality and processing speed.

Spin Digital has developed an automatic conversion tool able to simplify the composition of valid filter chains with the best trade-off between video quality and speed performance.

Example 1

Enabled by option “a”, with this tool the resulting command line of the previous example is given next:

./spinenc -s 7680x4320 -r 60 --fmt rgbp:8 -i raw.rgb --filter a:w=3840:cf=420:bd=10 -q 30 -o out.hevc

And the generated conversion pipeline is:

Filter config
-------------
Bit depth conversion     8b->10b
Resolution scaling       (7680x4320)->(3840x2160) imeth=lanc3
Color conversion         rgb->yuv
Chroma conversion        444p->420p

As can be observed, with the automatic filter bit depth upconversion is executed first, resolution downscaling second, and, finally, two-step chroma sampling downscaling (RGB->YUV4:4:4->YUV4:2:0).

Spin Digital has developed an automatic conversion tool able to simplify the composition of valid filter chains with the best trade-off between video quality and speed performance.

Example 2

Let us analyse  another example with a more complicated filter chain:

# Input video: equirectangular (erp), 4Kp60, bt709l (limited range), YUV 4:2:0 10-bit
# Output video: cubemap (cmp), 4Kp60, bt2020l (limited range), YUV 4:2:2 10-bit, QP 30
# Filter chain:
#     - "a:cs=bt2020l": color space conversion from bt709l to bt2020l
#     - ":cf=422": chroma sampling conversion from 4:2:0 to 4:2:2
#     - ":geom=cmp": geometry conversion from erp to cmp

./spinenc -s 4096x2048 -r 60 -f 420p:10 --geom erp -i raw.yuv --filter a:cs=bt2020l:cf=422:geom=cmp -q 30 -o out.hevc

For this example, the resulting conversion pipeline is:

Filter config 
------------- 
Chroma conversion       420p->444p
Color conversion        bt709l->bt2020l lut=33 
Chroma conversion       444p->422p
Geometry projection     erp->cmp (4096x2048)->(3456x2304) imeth=lanc3

The order in which the filters are executed is as follows: 1) chroma sampling conversion from 4:2:0 to 4:4:4, 2) color space conversion from bt709l to bt2020l, 3)  chroma sampling conversion from 4:4:4 to 4:2:2, and 4) spherical format conversion from equirectangular to cubemap.

As we can see, the automatic conversion tool has utilized some filters that were not specified in the command line. More specifically: upscaling from 4:2:0 to 4:4:4 and downscaling from 4:4:4 to 4:2:2 after color conversion. These extra filters are needed as color conversion can only operate on 4:4:4 format. Therefore, by using this tool the user does not need to know this kind of non-trivial operations.

Spin Filter Package

The video processing filters are available in the following products:

• Spin Enc: as a pre-processing module in the standalone HEVC encoder (spinenc) and as a plugin for the FFmpeg transcoder (spinffmpeg)
• Spin SDK: as a component of the Spin Digital HEVC SDK

More Information

For more information about features and usage of Spin Filter, contact Sergio Sanz-Rodriguez: sergio@spin-digital.com