[whatwg] Video source selection based on quality (was: <video> feedback)

[Hugh Guiney]

On Mon, Feb 15, 2010 at 8:09 PM, David Singer <singer at apple.com> wrote:
> I think I agree with Tim here.  When you ask to watch "360p" content, you are asking for content that has 360 lines of pixels to be displayed to you.

Right.

> You're not asking for whatever is displayed to you to occupy only 360 lines of pixels on your display.

That's what YouTube's doing, though. In compacted mode, YouTube
displays *everything* at 640x360. In expanded mode, everything is
displayed at 854x480. That's regardless of whether you select "360p",
"480p", "720p", or "1080p"—which are all *supposed* to be
abbreviations for different frame sizes.

What you are *actually* selecting is which of the multiple encodes
YouTube automatically creates of your video to act as the source
video, which is then scaled to 640x360 or 854x480 depending on the
selected player size.

Using the [pixel height][scan type] notation in this way is
misleading, since they're no longer semantic (UNLESS you know what's
going on behind-the-scenes, which most people don't). Video is already
very confusing and labels like that, with no explanation, don't help
matters.

> Yes, when it is shown larger, then filtering etc. is done to avoid pixel-doubling blocky artifacts;  this does not increase (and, we hope, not decrease) the amount of information in the scene.

Interpolation is not "pixel-doubling"—it can be done in that way
(i.e., where every source pixel is mapped to a multiple of itself),
but there are many different algorithms that determine how an image is
to be scaled and they all work differently with varying levels of
visual degradation and processor intensity.

While it is true that the amount of information in the SOURCE image
does not change, the amount of information in the RESULT image *does*,
simply by nature of the fact that it is no longer the same image.

This is far and away from what I was originally discussing though. If
you want to learn more about how image interpolation works this is a
good primer: <http://www.cambridgeincolour.com/tutorials/image-interpolation.htm>

> With the advent of higher-resolution displays, and the ability to use CSS with HTML to set 'sensible' sizes of video relative to other page content, the assumption that video will always be displayed in a 1:1 ratio of source (coded, transmitted) pixel to display pixel is increasingly untenable.

I never said that video will always be displayed 1:1; that's
completely unrealistic. Every TV currently in production scales the
incoming image in some way with default settings. This is because most
consumers can not cope with letterboxing and manufacturers need a way
to sell bigger and bigger screen sizes even even when the content
doesn't fit snugly between each corner. What I was saying was that we
shouldn't refer to something by a name that means something else.

Yes, if you select 1080p on an HD YT clip, it'll use the 1080p
file—but it WON'T fill up Joe's brand new 1080p computer monitor until
he goes into fullscreen mode. But how is he supposed to know that he
has to do that? It SAYS 1080p down at the bottom, after all. For all
he knows that's the size of the video. Similarly, Jane may be thinking
about buying a new monitor herself, but after seeing how "small" 1080p
is on YT, decides it's not worth it.

And when other established terms are used, like "480p"—which, in
virtually every other context, refers to 720x480, the most common of
the acceptable resolution for DVDs—yet the video is *854*x480, that's
also confusing. Ted could download such a "480p" clip and attempt to
burn it, only to discover that his authoring program won't accept that
format.

What is boils down to is that video formats are *already* very
confusing, and that fact does not need to be compounded by careless
labeling.

-Hugh