VDPAU Playback Issue (Problem Solved).

One of the facts which apply to Linux computing is, that NVIDIA created an API, which allows for certain video-streams to be played back in a way accelerated by the GPU, instead of all the video decoding taking place on the CPU. And, users don’t necessarily need to have an NVIDIA graphics card, in order for certain graphics drivers to offer this feature, which is called ‘VDPAU’, an acronym that stands for “Video Decode and Playback API for Unix”. Simultaneously, what some Linux users can do, is to experiment with shell-scripts that allow us to click on a specific Application Window, in order to perform screen-capture on that Window for a specified number of seconds, ad then to compress the resulting stream into MP4, AVI, or MPG -Files, once the screen-capture has finished. This latter piece of magic can be performed using elaborate ‘ffmpeg’ commands, which would need to be a part of the script in question. And in recent days, I’ve been tweaking such scripts.

But then an odd behaviour crept up. My NVIDIA graphics card supports the real-time playback of MPEG-1, MPEG-2, DIVX and H.264 -encoded streams, with GPU-acceleration. Yet, when I clicked on the resulting animations, depending on which player I chose to play those with, I’d either obtain the video stream, or I’d just obtain a grey rectangle, replacing the captured video stream. And what I do know, is that which of these results I obtain, depends on whether I’m playing back the video stream using a software decoder purely, or whether I’m choosing to have the stream played back with GPU-acceleration.

I’ve run in to the same problem before, but this time, the cause was elsewhere.

Basically, this result will often mean that the player application first asks the graphics card, whether the latter can decode the stream in question, and when the VDPAU API responds ‘Yes’, hands over the relevant processing to the GPU, but for some unknown reason, the GPU fails to decode the stream. This result can sometimes have a different meaning, but I knew I needed to focus my attention on this interpretation.

Linux users will often need to have some sort of file-format, in which they can store arbitrary video-clips, that do not need to conform to strict broadcasting and distribution standards, even when the goal is ‘just to monkey around with video clips’.

I finally found what the culprit was…

(Updated 8/15/2019, 22h15 … )

Continue reading VDPAU Playback Issue (Problem Solved).

Video Compression

A friend of mine once suggested that ‘a good way’ to compress a (2D) video stream would be, to compute the per-pixel difference of each frame with respect to the previous frame, and then to JPEG-Compress the result. And wouldn’t we know it, this is exactly how MJPEG works! However, the up-to-date, state-of-the-art compression schemes go further than to do that, in order to achieve smaller file-sizes, and are often based on Macroblocks.

Also, my friend failed to notice that at some point within 2D video compression, ‘reference frames’ are needed, which are also referred to sometimes as key-frames. These key-frames should not be confused however with the key-frames that are used in video editing software, 2D and 3D, to control animations which the power-user wants to create. Reference frames are needed within 2D video compression, if for no other reason, than the fact that given small amounts of error with which ‘comparison frames’ are decompressed, the actual present frame’s contents will deviate further and further from the intended, original content, beyond what is acceptable given that the stream is to be compressed.

The concept behind Macroblocks can be stated quite easily. Any frame of a video stream can be subdivided into so-called “Transform Blocks”, which are typically 8×8 pixel-groups, and of which the Discrete Cosine Transform can computed, in what would amount into the simple compression of each frame. The DCT coefficients are then quantized, as is familiar. Simply because the video is also encoded as having a Y’UV colour scheme, there are two possible resolutions at which the DCT could be computed, one for the Luminance Values, and the lower resolution, spanning the doubled number of pixels, for the Chroma Values. However, it is in the comparison of each frame with the previous frames, that ‘good’ 2D video compression has an added aspect of complexity, which my friend did not foresee.

The preceding frame is first translated in 2D, by a vector that is encoded with each Macroblock, in an estimation of motion on the screen, and only after this translation of the subdivided image by an integer number of pixels by X and by Y a sub-result forms, with which the per-pixel difference of the present frame is computed, resulting in per-pixel values that may or may not be non-zero, and resulting in the possibility that an entire Transform Block has DCT coefficients which may all be zeroes.

Continue reading Video Compression