Video Encoding and Decoding Optimization

For AI purposes, video files are processed as raw video frames but for streaming and storing video material, it makes more sense to use compressed video formats such as H.264, H.265, and VP9. This means that converting video - decoding and encoding it - becomes an important part of every AI system involving either live or pre-recorded material.

Since Intel offers best-in-class hardware for encode/decode operations, Open Edge Platform provides great performance for your video workloads, mostly due to optimizations offered with DL Streamer.

It enables you to maximize concurrent streams, decoding as many video streams as possible simultaneously, and use resources efficiently, by minimizing CPU/GPU load, memory bandwidth, and copy operations. What’s important, it also helps to find the right balance between latency and throughput.

	Low latency	High throughput
Best For	Real-time analytics or monitoring.	Batch video ingestion and processing pre-recorded material.
Benefits	Get immediate results for single frames.	Process very large batches of frames at once, fully utilizing GPU bandwidth and multiple streams.
How it’s achieved	Decoding frames one-by-one or in minimal batches, with minimal buffering.	Decoding multiple frames or streams in batches, with pipeline parallelism.
Trade-offs	May underutilize GPU, which works better with larger chunks of data.	Shows results only once they are all processed.

In DL Streamer, running these operations is as easy as adding two lines of code, for example:

decoding:

filesrc location=input.mp4 ! decodebin ! videoconvert ! video/x-raw,format=BGRx ! appsink

encoding:

... ! videoconvert ! vaapih264enc ! mp4mux ! filesink location=output.mp4

You will find a detailed description in the DL Streamer Performance Guide.