Let's talk about digital video sampling...come on in, I need help!

[Hulk]

I am trying to get a better understanding of video sampling and would appreciate a little help.

First of all is the 4:4:4 or 4:2:2 notation referred to as color space?

Okay, I’m going to go on a little rant regarding what I think I know (which is very little). Please feel free to help me out here.

Three discrete values are sampled in digitizing video, luminance, and two color difference channels. Unlike digital audio, digital video requires a lot more bandwidth for two reasons. One, a video screen is two dimensional whereas audio is a one dimensional stream of vales. Two, video data requires three values for each “sample” or pixel, each audio sample only requires one amplitude value.

If the full horizontal resolution is 720 (I believe that is called D1) and the sampling is 4:4:4 then the horizontal resolution of all three channels is 720. For DV video the sampling is 4:1:1 which means the color difference signals really only have an effective resolution of 720/4 or 180 pixels. Now, I have read that since the human eye is more sensitive to the luminance the grouping of color for these four pixels isn't that big a deal. Except for color keying, of course since it NEEDS the color resolution to discern edges/boundaries.

My first question is what exactly are the two color difference channels and how is the third color determined? If you have two equations and three unknowns, you have to know something else to get enough info to find the value of the three unknowns. Do the three colors equal the value of the luminance channel?

My next question is about the actual sampling rates. Somewhere on the web I read that video is sampled at 13.5 MHz. Or, 13.5 million pixels are sampled, with three corresponding values, per second.

13.5 million samples per second divided by 59.94 fields per second equals 2.25 x 10^5 pixels that are sampled per second for NTSC video.

If there are 720 pixels across the screen, then 2.25 x 10^5 divided by 720 equals 312.81 pixels of resolution per field vertically. It seems in the video world this is called the number of scan lines.

Now, I thought that NTSC video has 525 scan lines or 262.5 scan lines per field?

Also, why is this number not whole? Do I have the sampling frequency wrong?

Any thought/comments/help would be greatly appreciated. I’m a high school science teacher (physics/chemistry) with a background in mechanical engineering so we didn’t get into this in college. I have been getting prodded to teach a digital video course (develop one, actually) and if I do it, I want a decent background in the technical side of things. I can’t stand when teachers just teach the software without going into any of the science. I think spending some time on the basics will allows the kids to really understand what the software is doing, and how to use various video software without having to read every manual.

[Dr Mordrid]

Very basic, because there are variants:

Coefficient table:

Standard___Coef. Red____Coef. Green___Coef. Blue
Rec 601-1.....0.299...............0.587...............0.11 4
Rec 709........0.2125.............0.7154.............0 .0721
ITU..............0.2125.............0.7154........ .....0.0721

NTSC: RGB -> YCrCb:

Y = Coef. for red*Red + Coef. for green*Green + Coef. for blue*Blue
Cr = (Red-Y)/(2-2*Coef. for red)
Cb = (Blue-Y)/(2-2*Coef. for blue)

YCrCb -> RGB:

Red = Cr*(2-2*Coef. for red)+Y
Blue = Cb*(2-2*Coef. for blue)+Y
Green = (Y-Coef. for blue*Blue-Coef. for red*Red)/Coef. for green

The Green component must be computed after the Red and Blue components because its calculation uses their values.

PAL: RGB -> YUV

Y = 0.299*Red+0.587*Green+0.114*Blue
U = -0.147*Red-0.289*Green+0.436*Blue
V = 0.615*Red-0.515*Green-0.100*Blue

PAL: YUV -> RGB

Red = Y+0.000*U+1.140*V
Green = Y-0.396*U-0.581*V
Blue = Y+2.029*U+0.000*V

More info on TV systems worldwide...probably more than you care to know....with comparisons;

http://www.ee.surrey.ac.uk/Contrib/WorldTV/compare.html#pal

http://www.calfilm.com/list.htm

http://www.videouniversity.com (esp. the Engineering primer)

Dr. Mordrid

[Hulk]

Ahhh. I see, the value of the color channels add to 1, or the maximum value of the luminance. By grouping four samples worth of color data using 4:1:1 the data is reduced since normally those for samples would require 32 bits x 4 (including alpha channel for 4:4:4) whereas 4:1:1 only needs 8x4 (luminance) + 8x4 (alpha for each sample?) and 8x2x1 for both color channels for the group of four samples.

So, in my bad example,
4:1:1 requires 80 bits, assuming an alpha sample for each sample

4:4:4 requires 128 bits

A huge bandwidth saving right off the bat.

It's ashame there is no switch on our camcorders to record 4:4:4 for less time on the tape. The CCD is capturing the data but it is later being "thrown away" by the software.

Thanks for the info.

Mark

[wkulecz]

4:4:4 wouldn't be all that useful IMHO as your visual acuity for color is not as good as it it for intensity. The original color TV standards take advantage of this to reduce analog transmission bandwidth.

If you make black and white line chart you might resolve x line-pairs/cm at Z viewing distance (20-20 vision is generally taken to be 1 minute of arc resolving power, which is very close to 1mm at 5 ft, the mixed units come out about right :-)

Make the chart be alternating red and green lines and I doubt you'd be able to resolve x/2 line-pairs/cm at the same viewing distance Z.

Where the down sampling messes things up is "color fidelity" and when actual chroma values are important like with color key effects. When you use standard NTSC 4:2:2 you've pretty much thrown color fidelity out the window from the getgo because of the way NTSC is encoded/decoded.

I'll admit 4:4:4 has potential benefits for use in computer manipulations but I've never seen 4:4:4 camera, although it seems logical that a 3-CCD unit with seperate RGB outputs should be available, but if you have raw RGB you'd never mess around with YUV until you get ready for final output and then you're stuck with needing to use the TV standards.

720x480 at 29.97 frames/sec raw RGB at 8-bits per color would be about 30 MB/s compared to DV at ~3.6MB/s

--wally.

PS your single CCD camcorder has the 4:2:2 or 4:1:1 color space built into it by the color mosaic filter that is overlaid to turn a single channel sensor into a color camera so a "switch" would only work for three CCD units. I think your 13.5MHz sample clock is based on VGA 640x480 and may include some "invisible" pixels for the retrace clocks, so I think your assumption that DV uses a differnet sampling rate is probably correct.

[Hulk]

Wally,

You're right 4:4:4 might not be that useful for everyday video. All I'm saying it that it would be nice to have the option since the data is getting thrown away anyway.

Yes, uncompressed, that 4:4:4 stream would be about 30MB/sec. But, it could be compressed down to about 5 MB/sec, assuming the same compression as today's DV format.

I can't help wondering if our present DV format is a little short sighted. As technology increases at a faster and faster rate, two things will be working toward providing better video;

1. As memory gets smaller, faster, and cheaper, it will less costly to store more information.

2. As CPU power increases, the compression technology can also become more aggressive, providing smaller files with better quality.

Wouldn't it be nice to have a huffyuv-like option in your dv cam? Or a super high 1024progressive mode? Or a 300fps progressive mode? I think in the not to distant future we'll be seeing those features. How long will the manufacturers try to sell consumers on useless features like 400x digital zoom and in camera titling?

Mark

[wkulecz]

Hulk,

Your are preaching to the choir my friend.

I get so frustrated by the shortsighted lack of real features in order to save a buck or two in production on a several hundred dollar item when they spend multi-millions on marketing/advertizing of dubious "features" like digital zoom.

I doubt we "prosumers" will ever see better than things like the VX2000 -- since these are being used my news crews in Afganistan now -- anthing "better" will start eating into the cash cow of Betacam.

I'm afraid if we enthusists aren't careful when advising the more casual we'll end up with mpeg recording camcorders that are as uneditable as is VHS tapes, but it'll be on disks, BFD.

:-(

--wally.