[whatwg] Quality Values for Media Source Elements

[Hugh Guiney]

On Sun, Dec 13, 2009 at 7:26 AM, Aryeh Gregor <Simetrical+w3c at gmail.com> wrote:
> JavaScript is an integral part of HTML to all intents and purposes.
> HTML itself does not and should not try to cover use-cases that are
> already adequately covered by HTML+JavaScript -- there will always be
> things that are better handled by a general-purpose scripting
> language.  Of course, moving something into HTML might be valuable
> because it makes the feature easier for authors to use, but that needs
> to be weighed against the cost of browsers having to implement it
> rather than some other feature.

JavaScript is a crutch that far too many applications are relying on
for major functionality lately. JavaScript should enhance a Web
experience, not supplant it.

> Well, yes.  On the other hand, almost nobody actually uses content
> negotiation, so I don't think that supports your case.

If no one uses content negotiation then there is no need to have the
<source> element at all.

> Well, no, because there's almost no functional difference between
> XHTML and HTML except that the former is more likely to break due to
> typos or minor bugs.  Plus, virtually no site actually provides both
> XHTML and HTML.  Actually, virtually no site provides real XHTML at
> all.  So I don't bother specifying a preference for either.  If you
> do, I rather suspect that makes you one of a few hundred people at
> most, out of billions of web users.  So maybe you could pick an
> analogy that's more realistic?

XHTML and HTML are interchangeable with any other two technologies in
that example. PDF and Word, HTML and RSS, RSS and XHTML... the point
isn't whether most site authors are offering those two in particular;
the point is that on a platform that supports content negotiation, it
makes no sense to outsource it to another technology, making authors
reinvent the wheel simply because not enough people are using wheels.

> On the other hand, every single video site already does allow you to
> specify quality, and I've never had a problem with this.  It's a
> simple control that's only there when you want it, and you can easily
> figure out if you actually want higher or lower quality in any given
> case.

It's simple for an end-user; not necessarily so simple for authors to implement.

On Sun, Dec 13, 2009 at 5:35 PM, Silvia Pfeiffer
<silviapfeiffer1 at gmail.com> wrote:
> The analogy with the source selection algorithm for mime types on a
> server doesn't work well, because there is only one dimension upon
> which to choose a source file: mime type. Here, we have several
> dimensions, making any automated choice a challenge.

I do agree with this.

On Mon, Dec 14, 2009 at 12:12 AM, Eric Carlson <eric.carlson at apple.com> wrote:
>  Certainly! WebKit evaluates the query in the 'media' attribute if it believes it can handle the MIME type. If the query evaluates to true, it uses that <source> element. If it evaluates to false it skips it, even though it could (in theory) open the movie. For example, one of our layout tests [1] has the following :
>
> <video controls>
>    <source src=content/error.mpeg media="print">
>    <source src=content/error2.mpeg media="screen and (min-device-width: 80000px)">
>    <source src=content/test.mp4 media="screen and (min-device-width: 100px)">
> </video>
>
>  The test fails if the video element is instantiated with anything but "test.mp4".

This seems extremely useful. How many media features are implemented?

Currently, though, the CSS3 Media Query spec doesn't cover enough
metadata to make this as useful as it could be.

On Mon, Dec 14, 2009 at 1:54 AM, Silvia Pfeiffer
<silviapfeiffer1 at gmail.com> wrote:
> Indeed it seems to me the solution to the "quality" problem should
> then be done through the media attribute. I am not sure yet how to,
> because we have no definition for what a "low quality" or "high
> quality" video is other than some form or SD vs HD and lower
> resolution vs higher resolution and lower bandwidth vs higher
> bandwidth.

Well, we could certainly define them as they'd be defined *today*, but
as HD becomes more and more commonplace it will effectively stop being
"high definition", even if the name sticks. And, what one person
considers "low quality", another person may consider "high quality",
and vice-versa, depending on the capabilities of their machine and the
type of content they're used to seeing. Which is why it doesn't make
sense to specify absolutes, and why I proposed using relative values.

If we're to be more granular though, the biggest barrier to
implementation is the fact that, as you said, video is
multi-dimensional: there are MANY different factors that can affect
quality, viewing preference, and/or playback compatibility. Here is a
non-comprehensive list off the top of my head:

* Aspect Ratio (or Width and Height)
* Pixel Aspect Ratio (or Relative Pixel Width and Relative Pixel Height)
* Display Aspect Ratio (or AR / W & H and PAR / PW & PH)
* Content Aspect Ratio (or Content Width and Content Height)
* Sample Rate (or Rate and Scale)
* Bit Rate
* Frame Type or Picture Type, i.e. spatial/temporal compression technology
* Frame Scan Type (Progressive / Interlaced)
* Field Order (None / Top Field First / Bottom Field First)
* Colorimetry
* Codec
* Container Format
* Telecine Pattern
* Dimensionality (2D, 3D, Panoramic, etc.)
* Audio Channels
* Audio Sample Rate
* Audio Bit Rate
* Audio Codec

Even if all of this metadata were converted to media features, most
people wouldn't even know what most of it means. There is a tremendous
amount of misinformation out there about video technology; when it's
accurate it is almost always confusing. Even as a digital video
professional, I often have to explain a lot of these concepts to my
peers. So unless HTML authors are also fairly technical people with
video backgrounds, chances are most of it wouldn't end up being used.

Still though, implementing even just a subset of these would improve
media selection by a lot.