[whatwg] <video> element feedback

[Ian Hickson]

Thanks for all the feedback on <video>. There were several topics 
discussed. I'll cover the three most important ones first.

ON THE CODEC:

A number of people put forth many arguments for and against all kinds of 
codecs. However, very little of the feedback introduced any information 
that wasn't taken into account when I first drafted the proposal.

The spec can't require MPEG4 or other proprietary or patent-encumbered 
formats or suites of formats, because of the patent situation. Unless 
someone is willing to do the work and pay for licenses, or get MPEG-LA 
and/or similar groups to waive their rights for implementations of this 
spec, we can't require them, as it would require certain implementors to 
either expose themselves to known legal risks, or be non-conformant. This 
doesn't stop anyone from implementing such formats if they so desire, of 
course.

The spec doesn't require Dirac or any of the other freely implementable 
codecs and container formats other than Ogg, because requiring more than 
one would reduce the likelihood of any being implemented, and of those 
available, the Ogg family is more widely recognised and is arguably more 
mature.

The spec doesn't prohibit any codec from being implemented, because doing 
so would be an exercise in futility -- specifications are supposed to 
foster interoperability, not block innovation. After all, it is from 
extensions to specs that most of the HTML5 features were derived.



ON NATIVE UI:

A large portion of the feedback concerned the way that the current spec 
doesn't have any features for native browser-provided UI.

I completely agree that on the long term this is something we need to 
offer. However, we musn't bite off more than we can chew. There are 
several sets of use cases, some of which require browser-provided UI, and 
some of which need just video playback under the control of the author. If 
we want to have this implemented well, we need to focus the browser 
vendors' concentration on specific features. Since it is easier to 
implement first an API then a native UI, rather than first a native UI and 
then an API, the API has to come first. Similarly, because authors can 
make a UI with an API but can't sanely make an API with a native UI, 
providing an API provides authors with more options.

If LiveJournal on YouTube or some other site wanted to allow authors to 
embed video without requiring the authors to have to deal with JavaScript, 
they could create a page that contained just the video and video playback 
UI, with the video URI being taken out of the query parameters of the 
page, and then could give people markup that embedded the video using an 
<iframe>, as in:

   <iframe src="http://video.example.com/video.html?src=http://example.net/v193"></iframe>

This would be a safe mechanism, and would be relatively easy to set up. 
(It doesn't even require server-side support.)



ON HANDLING OTHER MEDIA FORMATS:

A lot of feedback concerned the necessity of introducing an element 
specifically for <video> in the first place.

We could use <object> for this, adding multiple APIs to <object> for each 
kind of media file, defining the semantics for changing from one to the 
other, for content-negotiation, for disambiguating similar media types 
that have overlapping but not identical APIs, and so forth.

However, the browser vendors would hate us. Browser vendors have 
repeatedly and loudly stated that overloading elements leads to 
implementation difficulties, resulting in poor interoperability, edge 
cases with strange behaviour, security bugs, and the works. Good examples 
of this in existing HTML browsers are <object> and <input>, both of which 
have had huge interoperability problems over the years, and both of which 
still have big issues. When it takes more than 10 years to get an element 
implemented well in every single browser that has tried to implement it, 
you have to look at why that is, and you have to learn from the mistake. 
In this case, the mistake is adding too much functionality to one element.

Similarly, for backwards-compatibility reasons, adding anything to 
<object> is a nightmare. We'd have to carefully examine every addition to 
make sure it didn't clash with existing content, for instance.

Furthermore, overloading an element with various APIs results in 
difficulties for authors. An author dealing with audio doesn't want to 
think about aspect ratios, and an author dealing with video doesn't want 
to think about plugin parameters.

This doesn't mean we have to specify everything as its own element. There 
are media types that it doesn't make sense to support with a specific 
element (at least not yet); we don't want to have six dozen elements with 
each type having its own set of features (and bugs). We _do_ have a 
generic element, <object>, which does work for generic inclusion. It 
doesn't support media-specific features (like the Video API) but it works 
as a stop-gap until the media in question is important enough to deserve 
special treatment, if that happens.



On Thu, 15 Mar 2007, Bjoern Hoehrmann wrote:
> 
> In case of video, there is no need to implement anything using style 
> sheets, behaviors, or scripting, you can use it directly, right now, 
> it's easy as pie,
> 
>   <html xmlns:t="urn:schemas-microsoft-com:time">
>   <?import namespace="t" implementation="#default#time2">
>   <body>
>     <t:video src='example.video'></t:video>
>   </body>

There are, unfortunately, a number of problems with doing this. The API is 
horrendously complicated. I have spoken to authors, as well as people who 
used to work on IE, who express nothing but muted horror when I ask about 
the HTML+TIME stuff (or SMIL). The HTML+TIME features have seen virtually 
no uptake with authors, despite having nearly the same market penetration 
as Flash for several years; even now, after more than 7 years of this 
feature being available, there are virtually no sites using <t:video> 
outside of tutorials or copies of MSDN.


> and based on an open W3C standard. No need for separate languages at 
> all. It's not perfect, but the terrible design of XMLHttpRequest, 
> <canvas>, and other features also did not prevent their inclusion in 
> "Web Applications 1.0".

(Could you elaborate on the problems you see in the <canvas> API? Also, 
note that XMLHttpRequest is now a W3C spec, not a WHATWG feature.)


> Don't you think the differences between the <video> features in IE5+, 
> SMIL, SVG, and HTML should be minimized, and using them in IE be made as 
> easy as technically feasible?

I don't see that attempting to get compatibility with SMIL is a good 
thing. Indeed, I think evidence suggests that we should avoid making the 
mistakes SMIL made, if we want to address author needs successfully.



On Sat, 17 Mar 2007, Håkon Wium Lie wrote:
>
> Also sprach Bjoern Hoehrmann [corrected by Ian Hickson]:
> 
>  >   +----------------+-----------------+-----------------+---------------+
>  >   | SMIL           | SVG             | IE              | "WHATWG"      |
>  >   +----------------+-----------------+-----------------+---------------+
>  >     beginElement() | beginElement()  | beginElement()  | play()
>  >     endElement()   | endElement()    | endElement()    | stop()
>  >     -              | pauseElement()  | pauseElement()  | pause()
>  >     -              | resumeElement() | resumeElement() | play()
>  >     -              | isPaused        | isPaused        | state == PAUSED
>  >    ...
> 
> I personallay think play, stop and pause are better names.

Agreed.



On Sat, 17 Mar 2007, Bjoern Hoehrmann wrote:
> 
> Do you really think using two different methods to trigger playback of 
> svg:video and xhtml:video elements is better than using a single method?

Given authors' opinions of SMIL features, yes, avoiding using the SMIL 
APIs would be a good thing. It is not yet clear that SVG video playback is 
an especially important feature on the Web; it is possible that HTML video 
playback will be more important and SVG video playback may simply want to 
be abandoned or restricted to certain specialty markets.


>   <t:video id='video'
>            begin='play.click'
>            end='stop.click'
>            src='example.video'>
> 
>     <t:transitionFilter begin="video.begin"
>                         type='barnDoorWipe'
>                         dur="5" />
> 
>   </t:video>
>   <p><input type='button' value='Play!' id='play' />
>      <input type='button' value='Stop!' id='stop' />

This is a highly contrived use case -- most uses of video on the Web today 
do not come close to this level of complexity, and optimising for this 
kind of complexity is likely to result in the feature lacking adoption.


> Here the playback of the video begins when the play control is clicked, 
> and the barnDoorWipe transition effect on the video will begin in turn 
> when playback of the video begins. The begin attribute is quite flex- 
> ible, I might change the example so playback of the video begins auto- 
> matically 2 seconds after the document began:
> 
>   <t:video id='video'
>            begin='2s; play.click'
>            end='stop.click'
>            src='example.video'>
> 
> or I might just drop the controls and just let it begin at 2s:
> 
>   <t:video begin='2s' src='example.video' />
> 
> You said 'play' might be a better name, so let's just use that for a 
> moment:
> 
>   <t:video play='2s' src='example.video' />
> 
> That does not look so much better to me, I would think this plays for
> two seconds, not to start playing after two seconds have elapsed.

I believe the argument was that play() is a better method name than 
beginElement(), not that the attribute for timeline alignment should be 
called play. Indeed, I would argue that we shouldn't have any features for 
timeline alignment. We already have technologies for that kind of thing -- 
such as SVG (using SMIL) -- and this is not what <video> is trying to 
address. Compatibility for compatibility's sake is pointless; features 
should be designed for the use cases they are trying to address and not to 
fit some theoretical uniform model.

With <video> we are trying to reduce the need for proprietary technologies 
(typically Flash) to be used for playback of video content, e.g. so that 
sites like YouTube, Vimeo, or Metacafe can transition from a Flash-based 
closed system to a more open system not dependent on a single vendor.


> I also would not consider a transition effect as I've used it above to 
> "play", and animation effects also don't really "play" for me.

I agree, but I wouldn't really consider transition effects to be in scope 
here.


> I do think that common timing control attributes and APIs are a good 
> thing,

They're a good thing when you're designing an animation format like SVG 
arguably is, but that's not in scope here.


> It just plays the video back, where the video is positioned and scaled 
> as the typical media player would do (it's scaled to fit the browser 
> window while preserving the aspect ratio, and centered in the space left 
> to fill). I would have given the "HTML5" equivalent but I could not 
> think of a simple solution for this. It would probably be some- thing 
> like [...]

Aspect ratio is automatically preserved. A page that displayed a video in 
the full frame (on visual UAs) with playback starting on load would look 
like:

   <!DOCTYPE HTML>
   <title>Example Video</title>
   <style>
     video { position: absolute; top: 0; right: 0; bottom: 0; left: 0; }
   </style>
   <p><video src="example.video">Your browser does not support video.</video>
   <script>
     document.getElementsByTagName('video')[0].play();
   </script>

...although of course this supports things that the WPF example you gave 
would not (like having a title for bookmarking, like supporting user 
overrides of styles, and so forth).



On Sat, 17 Mar 2007, Kornel Lesinski wrote:
> > 
> >   +----------------------------------+---------------------------------+
> >   | Flash/ActionScript               | "WHATWG"                        |
> >   +----------------------------------+---------------------------------+
> >     pause()                          | togglePause()
> >     pause(true)                      | pause()
> >     pause(false)                     | play()
> >     seek(s)                          | seek(1000 * s)
> >     time                             | position / 1000
> 
> This however is a good point - since Flash became de-facto standard for 
> publishing video on the web, authors are likely to know Flash's API 
> already. Having similar, but not exactly the same API may be source of 
> mistakes.

I agree, in principle. All the differences are intentional, though; I 
studied the Flash API before writing this spec.

Specifically:

     pause()                          | togglePause()
     pause(true)                      | pause()
     pause(false)                     | play()

I try to avoid overloading methods, because when I overload them, I get 
told off by browser vendors and people trying to implement the APIs in 
languages that don't have overloading.

We just have pause(true) and pause(false), but then pause(false) is 
equivalent to play() anyway, so that pause(false) is redundant. So it 
makes more sense just to have pause(true) be written as pause(). But then 
we don't have pause() left for the toggling feature, which we presumably 
want to include since Flash found a need for it, so we need a new name for 
that. And that's how we end up with the three names in the spec.


     seek(s)                          | seek(1000 * s)
     time                             | position / 1000

I agree that it would be nice to do what Flash does, but the cost of using 
floats for this is high. Using integers doesn't reduce our range, and 
increases our precision within that range relative to floats. 
Unfortunately if we use integers we can't use seconds, hence the 
difference between Flash and this API.



On Sat, 17 Mar 2007, Keryx Web wrote:
> 
> 1. That videos should never start to play without my consent. No more 
> bgsound-hellish experiences. Advertisers will protest, but I'd say it is 
> up to them to make their commercials so appealing that I'd want to play 
> the video in question.

The spec already allows this.


> 1b. If not (1) That video does not play automatically in a window/tab 
> that does not sit in the foreground. I tend to scroll wheel click on 
> links a lot. if there is video content on the page that has just opened 
> behind the one I am currently in and I would like to watch it, I'd 
> definitely prefer to start it when I flip tabs.

The spec already allows for this too.


> 2. I would like greater control than just start, stop, pause, forward, back
> and perhaps moving a slider. Looking at this:
> http://video.yahoo.com/video/play?vid=cccd4aa02a3993ab06e56af731346f78.2006940
> 
> Would it not be great if I could *navigate* to different parts of the 
> video according to some headlines:
> 
> - Douglas C: opening
> - Chris Wilson on...
> - Chris Wilson on...
> ...
> - HÃ¥kon Wium Lie on ACID 2
> - HÃ¥kon Wium Lie on the proposed <video> element
> Etc

The current API would allow sites to implement this. (That's one reason to 
start with an API and not a browser-provided UI.)


> Perhaps some sort of "goto" function with associated labels?

Well, the API can't really implement this directly since it would require 
support from the video itself. But nothing stops a site from including 
out-of-band metadata for this kind of thing.



On Mon, 19 Mar 2007, Simon Pieters wrote:
> 
> Therefore (to ease authoring and thus help adoption) I would like to 
> propose a new boolean attribute for <video>: autoplay="autoplay". It's 
> presence would be equivalent to invocing play() on the video ASAP. 

I agree that this would be good, but I think we should wait for browsers 
to implement what we have so far before adding more features. :-)

I've added it to the v2 list.



On Mon, 19 Mar 2007, Maik Merten wrote:
>
> This also sorta tangents the question if UAs should provide a default UI 
> that does not depend on JavaScript being enabled to fire the control 
> events. If there's no autostart feature and the user has disabled 
> JavaScript then there's no way to access the video content if I 
> understand the current draft correctly (correct me if I am wrong).

Browsers are allowed to provide UI (e.g. in context menus), just not UIs 
that show up inline, according to the current spec.



On Fri, 16 Mar 2007, Gareth Hay wrote:
>
> This topic is worrying me slightly, as I can only see two possible outcomes :-
> 
> using <object> for everything,
> 	images  <object type="image/jpeg" data="some.jpg">
> 	video     <object type="application/ogg" data="video.ogg">
> 
> or defining separate tags for all possible content :-
> 
> <image>
> <video>
> <sound>
> <etc...>
>
> As I can't see how it can be a mix and match of the two approaches.

Why not? A mix and match is exactly what we have today, and it works 
pretty well, considering.



On Fri, 16 Mar 2007, Gareth Hay wrote:
>
> Ok, I could understand that approach, with things like <img><video> 
> handled internally. Is there then a case for doing <object> properly by 
> specifying a replacement, something like <plugin> / <extern>?

<embed> is in HTML5 specifically for plugins.

However, for <embed>, <object>, <iframe>, and <video>, the spec doesn't 
require that UAs implement the features using plugins or using native 
code. For example, Mozilla natively supports SVG in <embed> (it doesn't 
require a plugin). Similarly, you could see <video> be implemented as a 
special-case plugin. That's an implementation detail and doesn't really 
affect the spec.



On Sat, 17 Mar 2007, Benjamin Hawkes-Lewis wrote:
> 
> Should we have attributes to indicate whether <video> content has 
> subtitles, captions, audio descriptions, and transcriptions embedded, so 
> that videos are only downloaded and played if they are in fact going to 
> be accessible to the user? Or to flag content that is NSFW, so UAs can 
> be configured to not play it? Or to point to higher quality but higher 
> bandwidth alternatives? Or maybe to indicate licencing, so that 
> authoring tools could throw up a warning if one tried to mix up or 
> deeplink to content with a restrictive licence. (I don't actually think 
> authoring tools should prevent one doing so, because authoring tools 
> aren't lawyers.) All of this is stuff one might very much wish to know 
> /before/ downloading the video. Particularly on the accessibility front, 
> when users with disabilities go through the palaver of downloading and 
> opening content which could be made accessible, but hasn't been, it 
> damages their faith in the relevant technology.

I don't think such metadata attributes would help. They would just be 
ignored by most authors and incorrectly set by many others.



On Sat, 17 Mar 2007, Marcos Caceres wrote:
> 
> I also think <video> is a great idea, however I would like to see the 
> ability to change the playback speed of the movie through the API (eg. 
> x0.5, x1.5, x2, etc). I think there is a strong use case for usability 
> in being able to change the playback rate.

I agree. However, in the interests of not overloading browser vendors on 
the first try, I'd rather keep this for a future version. I have it in the 
list of "v2" features in the spec as a comment.



On Sat, 17 Mar 2007, Matthew Ratzloff wrote:
> 
> I thought the comments raised some valid points.  Do Flash videos use the
> <video> element or the <object> element?

<object> (or <embed>), because they are really applications, not videos.


> Is there going to be an <audio> element as well?

This is something we could consider, yes, if a good set of use cases was 
provided for it, and if there was clear widespread use of audio on the 
Web the same way there is for video.


> Why not <media>, since the principles of video and audio are basically 
> the same?

They're not the same. For example, you wouldn't really show visualisation 
for a video the way you would for audio. You wouldn't preserve aspect 
ratios for audio they way you do for video. They have similarities, for 
sure, but they are different nonetheless.


On Mon, 19 Mar 2007, Asbjørn Ulsberg wrote:
> > 
> > But the mechanism should always be extensible, to allow plugin vendors 
> > to add functionality of their own where it is not specified.
> 
> For <object>, sure. But with <media>, <video>, <audio> or whatever the 
> element is going to be called, I think we should have a standard API 
> with a standard way to extend, e.g. setParameter('name', 'value'). That 
> way, the browser vendors will implement the required mechanisms to 
> support the playback, and if plugin authors wants to extend it, they can 
> do it with a standard mechanism that won't break in browser X because 
> the plugin vendor didn't implement support for it there.

I don't really follow what you mean here. Could you give a concrete 
example?



On Thu, 15 Mar 2007, Benjamin West wrote:
> > 
> > I have added such an element and its corresponding API (influenced by 
> > the other feedback received) to the specification. Thank you for the 
> > proposal and implementation experience!
> 
> What are the events?

The events for the load status are:
   begin, progress, stalled, load, abort, error

The playback state events are:
   playing, paused, autopaused, stopped

They're all available by either onfoo="" or addEventListener().


> I noticed the stop() method is used both to stop playback and to abort 
> any pending download.  Is this a good idea?  Wouldn't it be simpler to 
> add abort() to explicitly stop the download?

Well, to stop playback we have pause(). stop() just kills everything and 
resets it to the start.


> Are timeout's guaranteed to be in sync with the video?  For example:
> // assume myvid.position = 0;
> myvid.play();
> myvid.setTimeout('alert(myvid.position);', 10);

No, this is not guarenteed to be in sync.


> I have a hunch most authors will care more about time till the end than 
> time since the 0.  I don't have any evidence for this, and don't know 
> why I think it.  Except for...

That's a fair point. I've added the idea of an "ended" event to the list 
of things to consider in v2 -- maybe this would be important enough to 
have in v1, though. What do people think?

You can currently catch the end like this:

   video.onautopaused = function () { if (video.ended) {
     // it's the end
     // ...
   } };


> (BTW, one thing that is really annoying is not knowing when resources 
> [like images or scripts] failed to load.)

Doesn't onerror do this?


> What happens to the played range after we use seek()?

The video.played object returns a list of ranges.


> > On Mon, 30 Oct 2006, Charles Iliya Krempeaux wrote:
> > >
> > > #5: When to pre-fetch and when NOT to pre-fetch videos (and 
> > > "download" it at the last possible minute).
> > 
> > Could you elaborate on this?
> 
> My previous example makes a lame attempt at pre-fetching.  If you change 
> the src element to a previously dereferenced resource, how does the 
> cache behave?  Will we need to clone and delete nodes instead?

Certainly cloning and deleting (or, better, just having several <video> 
elements and switching between them) will be better, because you can track 
the downloads separately. But caching should work, sure. It's an HTTP 
thing, though, out of scope for our spec.



> > > The frame capturing would be cool (and useful).
> > 
> > Could you elaborate on the use case for this? Since the author will 
> > have the complete data on his end, there doesn't seem much use for 
> > actual frame capture on the client.
> 
> Only thing I can think of is the initial screen cap shown (or is this
> different?)  Perhaps for the image to show, you can just use an <img>
> in the content of the <video>.

I think that's different.


> Do videos encode what size they are best displayed in?  I hate entering 
> height and width for images.

Yes.



On Fri, 16 Mar 2007, Matthew Raymond wrote:
> >>> .loop, .startpos
> >>> loop = false | true
> >>> autostart = true | false
> >>> startpos = 0 | specified pos
> >>
> >> Could you elaborate on the use cases for these?
> >
> > Can't these be done in script?
> 
> Those attributes are basic values you'd want to use to initialize the 
> video. For those that don't know Javascript very well and just want a 
> video to play a certain way right away, it would be a real pain to only 
> have access to these properties via scripting. Besides, I can't see the 
> author wanting to set |autostart| and |startpos| dynamically, as they 
> are clearly intended to be used when the page is loading. (The |loop| 
> attribute might be set dynamically, but I can see situations where one 
> would want the video to play in the background in a continuous loop from 
> the moment the page is loaded. Granted, most of those uses are 
> presentational.)

That doesn't answer the question, though, which is: what are the use 
cases? Especially for loop.



On Fri, 16 Mar 2007, Nicholas Shanks wrote:
> 
> 1) If you specify both a width and a height, video content that doesn't 
> match that gets distorted.

With the current spec, the content is never stretched, it'll get 
letterboxed instead.


> 2) If you want an element to have a fixed width, but variable height 
> dependant on the aspect of the video, or fixed height and variable 
> width, yet still have a non-zero initial value for the variable 
> parameter (so that the <video> element occupies some screen area), you 
> cannot achieve this without distorting the final image by setting both 
> width and height.

You can achieve this with CSS, because the <video> element never distorts, 
but has intrinsic dimensions that CSS can hook into. (The rendering 
section, still to be written, will presumably talk about this in more 
detail.)



On Sat, 17 Mar 2007, Shadow2531 wrote:
> 
> For example, every video page on a site might have a 400 x 400 video 
> element (to fit with the layout for example), but the video that plays 
> in it will range in size and aspect ratio. [...]

Yup, the spec supports that today. I don't think we need control over 
this, certainly not in v1 and probably not even in v2, though.



On Sat, 17 Mar 2007, Shadow2531 wrote:
> 
> "Video content must be rendered inside the element's playback area such 
> that the video content is shown centered in the playback area at the 
> largest possible size that fits completely within it, with the video 
> content's aspect ratio being preserved. Thus, if the aspect ratio of the 
> playback area does not match the aspect ratio of the video, the video 
> will be shown letterboxed."
> 
> So, that also means that scaletofit in my examples is implied by that?

Yes.



On Fri, 16 Mar 2007, Stuart Langridge wrote:
> On 3/16/07, Ian Hickson <ian at hixie.ch> wrote:
> > 
> > Supporting only one encoding is not going to fly: you can't stop 
> > browser vendors from adding features; and you want to allow the 
> > standard to evolve over time.
> 
> I understand what you're saying here, but I think in practice that SVG's 
> decision to mandate support for PNG and JPEG makes the raster format 
> inclusions there useful.

But they don't prohibit other formats. The WHATWG spec requires Theora, 
but doesn't prohibit other formats. Which I think is what you're 
proposing...



On Fri, 16 Mar 2007, liorean wrote:
> > On Mon, 30 Oct 2006, Charles Iliya Krempeaux wrote:
> > > #3: Playlists.  (A single video file just won't cut it.)
> > 
> > These were the only requests for playlists. Could you elaborate on the 
> > use cases for playlists? What are the needs for playlists?
> 
> I immediately though of uses like this: 
> <http://www.scribemedia.org/2006/12/04/javascript-redux-brendan-eich/> 
> where the actual talk is split into 5 pieces for whatever reason, or 
> Douglas Crockford's JavaScript talks videos, among many others.

Interesting. It's something to consider for v2, then.



> How about the case where you want to provide a low and a high bandwidth 
> version, or a widescreen/normal screen aspect ratio version, or 
> something similar.

Wouldn't a bunch of links be good enough? Anything more than that and we 
have to start defining negotiation protocols, which is a huge rathole.


> How about if a user wants to view the video in a separate window, 
> separate tab or maybe fullscreen? Those are options where the size is 
> not necessarily corresponding to the video size. Another aspect here is 
> of course zooming/resizing/stretching and how to handle that.

The spec already handles these cases.


On Fri, 16 Mar 2007, Shadow2531 wrote:
> 
> A video might be split into separate files (chapters). A playlist is 
> needed to provide consecutive playback without user interaction and to 
> provide view of your choices. If you've ever watched episodes in parts 
> on youtube, you might see why this is important.

Does YouTube support this? I thought YouTube didn't support this, which 
would suggest it's not a high priority...


> One could use the events and states to determine when a video is done 
> playing. Then, the src attribute could be changed to the next video. The 
> list of videos could be gotten from a JS array or by parsing an xml 
> playlist via self-written code or some playlist parsing JS library.

Yep.


> However, what if JS is turned off and you want to do playlists?

If JS is turned off, applications won't work. :-) Just like when you turn 
JS off and try to use Google Calendar, or turn off Flash and try to use 
YouTube. In v2 we can add UI features to handle this, though, such as:

> <video src="playlist.xspf"></video>

...native XSPF support.


> > > .loop, .startpos
> > > loop = false | true
> > > autostart = true | false
> > > startpos = 0 | specified pos
> > 
> > Could you elaborate on the use cases for these?
> 
> <video src="VideoIWasWatching.ogg">
>    <param name="startpos" value="value gotten from cookie where I left off
> at">
> </video>

If the data has to be gotten from a cookie, wouldn't a JS seek() not be 
better? You'll need JS to save the position anyway.


> <video src="AwesomeMusicVideo.ogg">
>    <param name="loop" value="true">
> </video>

Wouldn't the decision of whether to loop or not be a user decision? UAs 
can provide a loop option as a checkable item on a context menu. I'd hate 
to go to a site and have a video keep looping.


> Page where a user is expecting a video to play.
> <video src="file.ogg">
>    <param name="autostart" value="true">
> </video>

We got a bunch of feedback from people saying "never autoplay!". :-) 
Again, the spec allows users to control this now.


> However, if JS is needed for the video element to function at all, then 
> the video element needs to fall back if JS is turned off.

Interesting point. You can do this with JS, of course (and that's the 
preferred way; hide the fallback when you have JS).


On Fri, 16 Mar 2007, Nicholas Shanks wrote:
>
> For the above usage case, I don't see why "remembering where you left 
> off" can't be a user-agent feature rather than part of the 
> specification.

That's certainly an option too. The spec allows this (arguably).


> All said, it would be very useful if the startpos param could be 
> returned to the server when starting the download, or when scrubbing 
> forward into parts of the video that haven't yet downloaded. as that 
> would help avoid using bandwidth unnecessarily.

HTTP requests with Range headers is the way the spec currently handles 
this. This may not be perfect, though. There are a number of other formats 
(as you point out). It's not clear to me what the best option is.



On Sat, 17 Mar 2007, Håkon Wium Lie wrote:
> > 
> > For now, the spec says that UAs SHOULD support Theora for video and 
> > Vorbis for audio, and SHOULD support the Ogg container format (it's 
> > not a MUST because some vendors may have legal reasons why they can't 
> > or won't support it, and there's no point making them non-conforming 
> > when they have no choice in the matter).
> 
> I'd rather make <video> and <audio> optional so that those who cannot 
> support these Ogg on these elements (for whatever reason) can still 
> comply with the spec. They can also support proprietary codecs through 
> <object>.

Why would we want to prevent the <video> API from being available to 
authors in walled gardens? Even if we make it legal to omit support for 
<video>, but require <video> to support Theora, UAs that don't want to 
implement Theora will still support <video>, they'll just do it without 
Theora, and be non-compliant. I don't see who wins here.



On Sun, 18 Mar 2007, Benjamin Hawkes-Lewis wrote:
> 
> Well user-agent developers should follow UAAG, W3C's User Agent 
> Accessibility Guidelines:
> 
> http://www.w3.org/TR/WAI-USERAGENT/
> 
> Guidelines 3 and 4 seem directly relevant. Really, UAAG compliance needs 
> to become the basis of public sector and corporate procurement decisions 
> to have much teeth, but any encouragement from WHATWG specs would help.

See the spec; it already does encourage that.



On Sun, 18 Mar 2007, Sander Tekelenburg wrote:
> > 
> > * If using Audio(), who's responsibility is it to provide that 
> > control/user interface?  UA? Authors?
> 
> The UA. Letting each and every content provider decide on the UI for their
> content just creates usablity hell for end users.

Well for Audio() the whole point is that the UI is provided by the author 
(e.g. because the audio clips are being used as sound effects in a game).

However if we introduce <audio>, I agree; and I agree that <video> needs a 
standard UI (in v2, at least).



On Sun, 18 Mar 2007, Benjamin Hawkes-Lewis wrote:
> 
> I suspect the fact that JS can't add to the context menu in browser 
> environments more generally is a shame.

Look up <menu> in the spec.



On Fri, 16 Mar 2007, Anne van Kesteren wrote:
>
> * INVALID_STATE_ERR needs <code> around it, not <span>. (This occurs 
> more than once.)

Fixed. Thanks.


> * It's unclear if the user agent can start buffering before play() is 
> invoked. Currently we do that although it doesn't delay the <body> load 
> event in any way.

I've made the spec allow this.


> * The stop() method is not in the IDL.

Fixed.


> * If the video is shown "letterboxes" what does the surrounding space 
> look like? Would that be the 'background' of the <video> element?

Black. Fixed.


> * I like how videoHeight and videoWidth work but are they not too 
> different from how similar constructs work in HTML (such as <img>, 
> <object>, <embed>, <iframe>)?

I don't think so; why would they be? They're different because we're 
introducing a new element and we don't have to make the same mistakes.



On Fri, 16 Mar 2007, Nicholas Shanks wrote:
> > 
> > * I like how videoHeight and videoWidth work but are they not too 
> > different from how similar constructs work in HTML (such as <img>, 
> > <object>, <embed>, <iframe>)?
> 
> how about inherentW/H or nativeW/H — then the same DOM attributes can be 
> reused on <img> too.

I think videoWidth/videoHeight is clearer than inherentWidth or 
nativeWidth. Is there really a use case for the equivalent on <img>?



On Sun, 18 Mar 2007, Alexey Feldgendler wrote:
> 
> A compromise solution that I propose is to include a standard set of 
> browser-provided controls, displayed by default and styleable through 
> CSS pseudo-elements and pseudo-classes:

That's not a bad idea necessarily, but it would be something to suggest to 
the CSSWG.


On Sun, 18 Mar 2007, Kornel Lesinski wrote:
> 
> How about controls overlaid on top of video, which are visible only when 
> mouse hovers it? Something like fullscreen controls in QuickTime Pro or 
> iTunes 7.

That's already allowed by the spec.



On Sun, 18 Mar 2007, Anne van Kesteren wrote:
> 
> I just played some more with our internal implementation (Opera's) and 
> noticed that our pause() really is like togglePause() in the HTML5 
> proposal. Looking at the specification I don't see much need for pause() 
> there. Perhaps togglePause() should just become pause() and pause() be 
> removed?

Having pause() always pause is better because it means that you're not 
likely to end up in state where the UI says one thing but the UA does 
another (especially when you combine this with UA-provided UI). I only 
included togglePause() because Flash supports it and some people asked for 
it; I'm not convinced we should keep it.



On Sun, 18 Mar 2007, Mihai Sucan wrote:
> 
> 1. I would suggest renaming "length" to "duration", because "length" is 
> a rather general term. "length" is also used for arrays in JavaScript, 
> and other languages. "duration" (or simply "dur") is more specific to 
> videos.

Changed.


> 2. Should the type attribute be added? It would be a good hint, anchors 
> have it, links have it, scripts have it, etc. Type would be useful for 
> UAs running on systems with limited bandwidth and/or limited resources, 
> they could choose not to download the video if the type is unrecognized.

Video types are extremely inadequate. I don't really see how it would 
work. Unless someone can suggest a precise processing model for it, I'd 
rather not include it.



If I missed any points people made, please let me know. I didn't 
explicitly quote and reply to every e-mail because a lot of them simply 
repeated the same arguments.

Cheers,
-- 
Ian Hickson               U+1047E                )\._.,--....,'``.    fL
http://ln.hixie.ch/       U+263A                /,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'