[MPlayer-dev-eng] rgb<->yuv convertion question

[armin.gerritsen at philips.com]

Just some things I quickly saw. Don't shoot me if I miss something. :-)

>>>> Y = (0.257 * R) + (0.504 * G) + (0.098 * B) + 16
>>>> Cr = V = (0.439 * R) - (0.368 * G) - (0.071 * B) + 128
>>>> Cb = U = -(0.148 * R) - (0.291 * G) + (0.439 * B) + 128

I tried this quickly in one of my reference programs which converts
still-images and I see no red dots.
So I guess it goes indeed wrong when rounding.

> i put my Clamp function:
> unsigned char Clamp(float i) {
> if (i>256.0f)
> return 256;

if ( i>255.0f)
 return 255;

When you use 256 and 'round' it to a unsigned char, you get 0.
Codes 0 and 255 are, however, reserved for synchronisation purposes, and are
prohibited from appearing in video data in the CIR601 (most common)
YUV-standard.

> if (i<0.0f)
> return 0;
> return (unsigned char)i;
> }

> like : pYUV->Y = Clamp(0.257f * (float)pRGB[RINDEX] + 0.504f *
> (float)pRGB[GINDEX] + 0.098f * (float)pRGB[BINDEX] + 16.0f);

If the above doesn't fix it, it can still go wrong at two places

1) the values are too large (bigger than 8 bits). You handle that now.
2) in the end, you somehow end up with incorrect YUV-values (>235 of <16)

RGB = 255:0:0, will give for instance V = Clamp( 239.945 ) = 240

Which often doesn't wrong BTW, since the CIR601 standard knows headroom, but
I mention it anyway.
So, you could consider a sanity check at the end, but the damage of course
is done earlier.

If I see your used formula's, I see that they are pretty rounded already.
That may also be the cause of your problem. (I don't think MPlayer uses the
formula's like that anyway. Probably uses higher precision pre-calculated
lookup tables.)

Regards,

Armin