[whatwg] video tag: pixel aspect ratio

[Ian Hickson]

On Mon, 17 Nov 2008, Pierre-Olivier Latour wrote:
> > >
> > > And the suggested "hack" is not even really usable: if you have a 
> > > video coming from a NTSC DV source as 720x480 improperly transcoded 
> > > to say MP4 720x480 square pixels, using the theoretical 10:11 pixel 
> > > aspect ratio will _not_ make it look right: it needs to be clipped 
> > > to 704x480 first.
> > 
> > Are you sure? If you don't clip it, you still get the right shape 
> > pixels, don't you? You don't get the right final video size, sure, 
> > because you didn't crop, but so what? We're just trying to do a 
> > last-ditch aspect ratio fix here, not get perfect video.
> 
> Well, the pixels will look right if you pass 10:11, but not the overall 
> video, or the video will look right but not the pixels if you pass an 
> aspect ratio to end up with 640x480 (the very nice 0.888888888888889)...

Anyone who's using this attribute has long given up on getting perfect 
output; they just want to have circles look like circles.


> > > Pixel aspect ratio has a precise meaning in the video world, and 
> > > using it outside of clean aperture does not make a lot of sense...
> > 
> > As far as I can tell, using it outside clean aperture works fine so 
> > long as you don't also expect the final output to be the "right" video 
> > size.
> 
> You're effectively saying that it works *fine* as long as you we don't 
> expect to work *right*. I have to admit, this is a concept that escapes 
> me ;)

We're dealing with videos that are mis-encoded here, by definition (that's 
what the attribute is for). The biggest problem with misencoded videos is 
the ratio. The goal here is just to provide a way to hack the ratio to be 
right so that at least the biggest mistake is fixed.


> > > If we start going in this direction, then <img> should have a "dpi" 
> > > attribute so you can "hack" around images uploaded at dpi > 72 ;)
> > 
> > We effectively do, it's the "height" (or "width") attribute.
> 
> Exactly my point: now replace, <img> by <video>, "dpi" by "aspectRatio" 
> and add a new boolean attribute to the video tag, so you can do 
> "fillToFit" instead of "scaleToFit" and you have a real solution that 
> allows you to resize the video the way you want and avoids half-baked 
> concepts like "it's pixel aspect ratio, but actually not really, and you 
> shouldn't be using it anyway".

I don't understand why that would be better. If we did this, then there'd 
be no way to have a consistent playback area across multiple pages and fix 
the ratio of busted clips. You'd be forced to change the layout, which is 
hardly a "quick fix".


> I might be missing something here, but:
> 1) I don't remember any major media system I've dealt with so far having 
> an explicit pixel aspect ratio override API,
> 2) on the web, neither QT plug-in nor Flash have it,

That might explain the large number of videos on the Web that are rendered 
at the wrong ratio without anyone doing anything about it. :-)


> 3) in the case of this spec, the way it's defined makes it behave 
> incorrectly

It doesn't support the clean aperture concept, and doesn't crop anything, 
certainly, but does it lead to the wrong ratio?


> 4) it's not straightforward to use (see very first reply above)

I'm open to better ideas.


> 5) there's no _actual_ data that proves it's necessary (shouldn't the 
> software or video web site fix the videos upfront?)

Anecdotally, I see this quite a lot (several times a week).


> Based on this, it seems to me this attribute should not be in the spec 
> by default, and we should switch the burden of the proof to people who 
> want it (rather than it being on people who don't want it as it seems to 
> be the case today), and finally wait to see 1) if there's a real need 
> for a solution here and 2) if the best solution is indeed a pixel aspect 
> ratio override.

I'm certainly open to other solutions. What do you suggest?


On Mon, 17 Nov 2008, Peter Kasting wrote:
> 
> I agree.  The more this attribute is discussed, the more "this is a hack 
> that no one should actually use" starts to sound like "we shouldn't put 
> this in the spec to begin with".  The potential for problems seems 
> greater than the upside from authors correctly using this to do 
> emergency-overrides of particular videos whose sources they don't 
> control.

I don't understand why this attribute would cause problems. Can you 
elaborate?


On Mon, 17 Nov 2008, Philip Jägenstedt wrote:
>
> I should point out that the pixelratio attribute isn't only for authors, 
> it's also useful when the media framework used doesn't recognize the 
> (pixel) aspect ratio even when it's correctly set. From reading the 
> mplayer man page I see that AVI files can have aspect ratio set in the 
> OpenDML vprp header, but I doubt e.g. DirectShow supports this (I could 
> be wrong though).
> 
> I don't feel very strongly about the attribute either way, but given 
> that video is scaled to fit inside its element box with aspect preserved 
> and not simply stretched then an incorrectly parsed/guessed aspect ratio 
> would make a big difference. This seems very similar to the width/height 
> attributes of an image and that usually isn't considered an ugly hack. 
> If the pixelratio attribute is removed then I would suggest also 
> removing the involvement of aspect ratio over the size of the element.
> 
> By the way, the "pixel-aspect-ratio" on video caps in the GStreamer 
> framework has precisely the same meaning as this attribute, overriding 
> it on a video sink also has an effect similar to what is suggested in 
> the HTML5 spec. In other words, it's not so outlanding from a media 
> framework's point of view.

Indeed.


On Mon, 17 Nov 2008, Dave Singer wrote:
> 
> I have a feeling that this is but one of a class of statements "that the 
> media file could have, and maybe should have, made" but that it did not. 
> I wonder if picking them off piece-meal, one by one is right.
> 
> Particularly, obviously, "missing" annotations come to mind (and this is 
> much more likely than missing video attributes), where you might indeed 
> want to author one file in say XML that contains a whole load of 
> metadata and associate it with several <source> elements, all of which 
> are versions (different compressions, bitrates etc.) of the same 
> conceptual content.
> 
> If we're not doing it for this rather more obvious use, and we're not 
> doing it generally, why are we doing it for pixelratio?

Because a video file having the wrong ratio is far more annoying than a 
video file having the wrong Genre.


> Another obvious over-ride, which I think also comes up more often, is 
> when the source doesn't tag its color space correctly ("It was designed 
> for traditional Mac gamma and it's now put on the web and viewed on 
> Windows systems").

This is far less of a problem than the wrong ratio, IMHO.

(Also, people's screens are likely to have gamma, brightness, and contrast 
settings far more out of wack than the difference that a gamma override 
would make.)


On Tue, 18 Nov 2008, Silvia Pfeiffer wrote:
> 
> I was under the impression that the attribute was requested by YouTube. 
> Does YouTube itself provide such an attribute? If now, why not? If so, 
> how often is it being used?

I have received feedback from YouTube developers who have commented 
positively on this feature, but I believe the feature was there before 
they commented on it. I don't believe YouTube has anything like this yet. 
(Can you stretch video in Flash?)

-- 
Ian Hickson               U+1047E                )\._.,--....,'``.    fL
http://ln.hixie.ch/       U+263A                /,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'