A way of visualizing YUV color-representation.

In the past I have linked to This WiKiPedia article, to explain what YUV color-encoding is. But I find that even though the article does a good job of explaining, their ‘UV Chroma Map’ is flawed. This would be the square image which they used, together with an assumed Luminance value of 0.5, just to give a basic impression of how the colors get mapped.

The reason I see the above visual as flawed, has to do with the fact that at any one value for Y’ – i.e. for any one value of Luminance – the full range of UV- chroma values is not in fact available, in a way that leads to full color saturation.

I think that any visual should highlight, that many Y’UV combinations do not lead to possible RGB values, for which reason even a full gamut of RGB colors will experience losses, when encoded into practical, integer-based YUV.

I would propose that a modified method be used to exhibit what the YUV encoding does, specifically the ‘UV’ part, when the Y part is allowed to vary, in an arbitrary way that adapts to the full range of UV-chroma values. And the visual which I obtain would be as follows:

yuv_plot_1-001

http://dirkmittler.homeip.net/YUV_Plot_1.html

I realize that the Math is somewhat fudged, with which I created the visual above, but also know the more-precise Math which gets used, for YUV or Y’UV color-encoding. Here is my worksheet on that:

http://dirkmittler.homeip.net/YCbCr_1.html

The reader would need to allow JavaScript from ‘MathJax.org’ to run on his or her browser, for the Math to display correctly.

It would be correct to infer, that in this one Y’UV profile, the Android developers chose ‘Umax’ and ‘Vmax’ to exceed 0.5 deliberately, which effectively ‘over-modulates’ U and V, and that therefore, Android devices will not be able to encode fully-saturated primary colors Red and Blue, when using this profile. And one reason fw the developers may have done this, would be at least to improve the available resolution somewhat, for ‘natural colors’, that do not correspond to saturated Red or Blue.

(Updated 08/02/2018, 20h05 : )

It’s a basic fact in (U,V)-chroma encoding, that the (U) and (V) components can become negative by the same amount, by which they can become positive, which in today’s syntax is also referred to as ‘Umax’ and ‘Vmax’.

But the fact did occur to me, that when the Green primary color, which I’ve named (cGreen), has as amplitude (1.0), while my (cRed) and (cBlue) each have an amplitude of zero, the negativity of (U) and (V) should be less than those of (-Umax) and (-Vmax), even though my first visual above would show the ranges for both (U) and (V) as (±1.0) …

(Updated 08/05/2018, 14h10 … )

(As Of 07/31/2018, 21h35 : )

This last phenomenon which I described above, can only be graphed properly, as a function of 3 variables: Y, U, and V. Any attempt to graph it using just U and V, such as in an effort to produce a 2D visualization and not a 3D system of equations, will inevitably introduce inaccuracies. Yet, the Mathematical creativity can come to people, ‘Just to plot this in 2D anyway':

yuv_plot_2-001

http://dirkmittler.homeip.net/YUV_Plot_2.html


 

I suppose that why this happens, can also be explained using ‘Horse Sense’.

The way Luminance (Y) is said to form, is as the following, weighted summation:

Y = Kr(cRed) + Kg(cGreen) + Kb(cBlue)

Where (Kr), (Kg) and (Kb), are the amount of visual luminosity – to Human eyes – with which the physical intensity of the additive primary colors (cRed), (cGreen) and (cBlue) are mixed, by this system of representation. But in addition to (Y), this system also defines (U) and (V), and I’m just going to focus on how (U) is encoded for the moment:

U = Umax (cBlue – Y) / (1 – Kb)

If we can examine the case in which the color consists fully of (cBlue), then it follows that (Y) will be equal to (Kb), hence the divisor. Without (Umax) being defined, a value for (U) of (1.0) would follow:

U = Umax (1 – Kb) / (1 – Kb)

But if we assume instead that (cBlue) and (cRed) are both zero, and that the color consists fully of (cGreen), then it will follow that (Y) is equal to (Kg), so that (U) will follow as

U = Umax (-Kg) / (1 – Kb)

There’s a trick to understanding this:

(1 – Kb) == (Kg + Kr)

And, because (Kr > 0),

(1 – Kb) > Kg

Hence,

(-Kg) / (1 – Kb) > -1.0

Tada!

(Edit 08/02/2018, 17h25 :

Actually, the Math above failed to mention what would happen, if both (cRed) and (cGreen) were simultaneously at their maximum. In that case,

U = Umax (0 – Kg – Kr) / (1 – Kb)

= Umax (Kb – 1) / (1 – Kb)

= Umax (-1)

So in fact, It can happen after all that U goes negative, by as much as it goes positive. )

 


 

(Update 08/01/2018, 16h50 : )

As another example, I decided to create a 2D visual, this time with the Android Color Profile which I cited earlier in this posting, but with a value for (Y) exactly equivalent to (Kg). This 2D image is more-similar to the way the WiKiPedia describes ‘YUV’, and represents a slice through the ‘YUV’ gamut, at that (Y) value and parallel to the plane of (U,V). The reason it still reveals a rectangle as possible ‘RGB’ pixel colors, is due to (Y) being a constant, and the (U) as well as (V) multipliers being constants, for which reason the rectangle is mainly defined by Minimum and Maximum (cRed) and (cBlue) values. But as an interesting feature, because (Y) has been chosen to coincide with (Kg), the lower-left-hand corner of this rectangle also represents the point, where (cGreen) is at its theoretical Maximum:

yuv_plot_3-001

The presence of some amount of (cGreen) component has not completely subsided, along the top and the right-hand edges in this case, which is why, everywhere else in the lit rectangle, the colors are not saturated, as the green corner is.

http://dirkmittler.homeip.net/YUV_Plot_3.html

Now, if I wanted to show ‘where’ the points of maximum saturation are, in the same Android color profile, for (cRed) and (cBlue), but using the same old method, I’d need to set (Y) to (Kr) or to (Kb), each time creating yet another slice through the ‘YUV’ gamut. This is what I get with (Y ~ Kb) :

yuv_plot_4-001

http://dirkmittler.homeip.net/YUV_Plot_4.html

The bottom, straight edge results, because it’s not possible for there to be less than zero (cRed), the left edge results, because there cannot be less than zero (cBlue), and the diagonal edge results, because there cannot be less than zero (cGreen).

This is what I’d get, with (Y) arbitrarily set higher than (Kg), namely set to (0.7):

yuv_plot_5-001

http://dirkmittler.homeip.net/YUV_Plot_5.html

This time it’s even harder, to understand what the plot shows. (Y) is brighter than any primary color can be by itself – even (cGreen) – And the left-hand edge results, because (cBlue) cannot go below zero. But at the same time, the right-hand edge results, because (cBlue) cannot go any higher than full. The top edge results, because (cRed) cannot go any higher than full. And the angled edge along the bottom results, because (cGreen) cannot go any higher than full.


 

(Update 08/02/2018, 14h20 : )

So a plausible question which my reader could have would be, ‘Given that Dirk can reproduce the more-conventional slices through the YUV color-space, which lead to plots that are often rectangular, as a function of RGB being within a constrained range, why did Dirk create his own, triangular representation of what YUV does?’ And this would be my answer:

If the 3rd dimension of the conventional visuals, which corresponds to (Y), is set to one out of (Kr), (Kg), or (Kb), then there is also one point along (U) and (V), at which (cRed), (cGreen), or (cBlue) is pure, and at its maximum amplitude. This is assuming, that (Umax) and (Vmax) have not been set to exceed the actual range of (U,V) that can be encoded, which is actually what the Android color-profile did – slightly – which I have cited.

Three distinct points in a 3D space, can define a plane. In this case, that plane would slice through the ‘YUV’ representation at some odd angle, which I never sought to compute. Within this slice, a triangle should form, for which ‘RGB’ are within their range, and at each vertex of which, (cRed), (cGreen), or (cBlue) are at their maximum. But then one problem with such an inclined slice would be, that the point within it, where (cRed == cGreen == cBlue), i.e., the point at which a certain shade of gray is obtained, will no longer be centered with respect to the derived (U’) and (V’). This is just logical, as all the three additive primary colors would remain linear functions of (U’) and (V’), and their maxima not equidistant from the center. And so what I did was just to ignore the question of ‘How bright that shade of gray should be,’ as well as to reduce the maximum amplitude of (cGreen), in the second visual of this posting, without changing its position with respect to (U’,V’), because I felt that the need for the ‘gray’ zone to be centered with respect to (U’) and (V’), was actually greater than the need for (cGreen) to reach its maximum amplitude.

Every time I multiplied or divided a linear function of (U’,V’) by something, it was by a constant-expression, itself independent of (U’) or (V’).


 

I suppose that there’s one more observation about ‘YUV’ color representation for me to add. It’s true that a maximum amplitude of (cBlue) gets encoded, to a maximum positive amplitude of (U). But then, this point does not coincide with a value of zero for (V). The reason for this is the same, as the logic by which I derived the values for (U) and (V) above, that corresponded to (cGreen == 1). Only in this case it would follow:

V = Vmax (-Kb) / (1 – Kr)

In other words, the corresponding value for (V) would be slightly negative, and just enough so, to shut off the amount of (cRed) output, when ‘YUV’ is decoded. And the converse is true, when (cRed == 1), but describing the effect of non-zero Luminance on (U).

For that reason, the arbitrary slice through ‘YUV’ which my triangles describe above, would also need to be transformed in 2D, so that in their arbitrary, 2D coordinate system, (+1, 0) and (0, +1) did represent the maximum values for (cBlue) and (cRed), respectively. So those (U’,V’) coordinates would truly become arbitrary in every sense, and the values for (U’,V’) that describe (cGreen == 1) could only follow in a way that I can’t program.


 

(Updated 08/05/2018, 14h10 : )

If the reader paid close attention, he or she might have noticed, that in my third visual above, a single pixel is missing from the ‘lit rectangle’, of in-gamut RGB-values. This pixel is actually missing, because its (cGreen) primary color component was too high.

AFAICT, This happened because the ‘YUV’ values generally start out as integers, such that the error of (U) and (V) was already (1/127,1/127). Yet, I had multiplied the fraction (Kg) by (256), to arrive at the floating-point value for (Y), masquerading as an integer. This would not have happened if the input ‘YUV’ values had all been floating-point.

Yet, I next divided each floating-point value by (256), before feeding it to the Euler-Math-Toolbox function ‘rgb()’. In spite of this, at some point, the condition ‘(cGreen > 255)’ tested as True.

In reality, there was a problem with my semantics. I should have set (Y) to (255*Kg), simply because the maximum value of (cGreen) would now have been the integer (255).

The easiest way to fix that would be, to make the decoding equation of (cGreen) specifically, slightly less-dependent on (-U) and on (-V). In other words, the multipliers that give rise to (cGreen) as a function of (U) and (V), can themselves be multiplied by (254.0/255.0) , or by (126.0/127.0) , whichever is appropriate, and that artefact will no longer occur in the decoding of this ‘YUV’ color profile.

Doing so would also make the decoding-equation differ, from the exact solution suggested by Linear Algebra, i.e., suggested by the Matrix computation I provided.

Simply changing (Y) will also solve this problem, which seems to be, that the row of pixels to the right is lit, the column of pixels above is lit, but the one pixel is not lit… However, the reader should already have noticed, that if I do set (Y) to (255*Kg), and then test for the constraint three times, (cGreen > 255) … , I may simply be asking for a repeat of the problem, for different values of (U,V).

My assumption is, that when an integer from [0..255] is being used to encode a real number from [0..+1.0) , the real number is first multiplied by (256), and the fractional part then ignored. At the same time, there was no risk in this exercise, that (Kg) would exactly equal (1.0) . Similarly, in reverse, the integer is just divided by (256.0) . In my world, integers usually work the same way in which binary fractions work, except that one is the other, multiplied by a power of two.

However, systems of representation exist, by which the real numbers in the range [0..+1.0] inclusively are represented by integers [0..255] , which means that when encoding, they are multiplied by (255), (+0.5) is added, and the integer-part taken / truncated. This system had remained quite foreign to me, until today. If we were to assume that such multiples of (1/255) are to be used for ‘YUV’ encoding, then we can obtain near-perfect results, while adhering to the principles of Linear Algebra exactly:

yuv_plot_7-001

http://dirkmittler.homeip.net/YUV_Plot_7.html

 

Now, when decoding from the integer range [-127..+127] to a real number, I did pay attention in some of my loops, to the question of whether the resulting real numbers should be in the range (-1.0..+1.0) , or in the range [-1.0..+1.0] . Because I intended for the latter case, I did divide those integers by (127.0) . Hence, there was a finite percentage of cases, where the real number was exactly equal to (±1.0) . If I had intended for the other case, I would have divided my integers by (128.0) .

Actually-recorded video-footage, can lead to wrong values just as easily. It’s just that actually-recorded video will also encode the (Y) value as an unsigned 8-bit integer. And then, one adaptation of the decoding Math in the player-application, will introduce fault-tolerance.


 

The matching question could be asked, whether a smart-phone would record video, with Luminance-values that actually reach (1.0) . The correct answer to that question would be, that if the smart-phone still had a channel-depth of 8 bits, each of its primary component-colors, (cRed), (cGreen) and (cBlue), would start out as integers in the range [0..255] , and therefore, the Luminance value (Y) should also be encoded as an integer, in the range [0..255] . If the primary colors’ maximum value was interpreted as (255/256), then this should also be the maximum value of (Y). The Math, in the recording device, which preserves this, would effectively be to compute (Y), as (255/256)*256 == (255). But, if the primary colors’ maximum value was interpreted internally as (255/255), the recording Math which preserves this would be to compute (Y) as ( int(255+0.5) / 255.0 )*255 == (255).

Dirk

 

Print Friendly, PDF & Email

One thought on “A way of visualizing YUV color-representation.”

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

This site uses Akismet to reduce spam. Learn how your comment data is processed.