[whatwg] Video : Slow motion, fast forward effects

[Ian Hickson]

On Wed, 15 Oct 2008, Charles Iliya Krempeaux wrote:
> On Wed, Oct 15, 2008 at 2:08 AM, Ian Hickson <ian at hixie.ch> wrote:
> >
> > Regarding whether to play audio while rewinding:
> > 
> > Leaving this be something that is optional doesn't make sense to me. 
> > If we want interoperability, we need to either have all the browsers 
> > play sound while moving backwards through a sound or video file, or 
> > not play sound. Leaving things optional means that one author will 
> > test with one browser and expect one behavior, and then when the user 
> > uses that page on another browser, he will get unexpected behavior.
> > 
> > For example, if Alice uses Browser A, and finds that when rewinding 
> > the sound isn't played, and thus does a hack that fakes the sound 
> > playback by having a separate hidden <video> > element while 
> > rewinding, in which she seeks, plays forward for a bit, seeks, plays, 
> > etc, then when Bob loads that page in Browser B, which does play audio 
> > when rewinding, he'll get a horrible double audio effect.
> > 
> > Or similarly, if Alice, in an attempt to get a cool effect, plays a 
> > bunch of videos backwards, and when the user selects one, plays it 
> > forward, then Bob will find that all the audio from all the videos 
> > plays simultaneously instead of all being muted except the video that 
> > Alice is expecting to play.
> > 
> > So. We have to either require that backwards-playing audio be muted, 
> > or require that it play. I don't mind which. I've required that it be 
> > muted because implementors said playing would be hard.
> 
> Could we make it configurable though?...
> 
> So... the web developer could check if playing sound for this is available.
> 
> And if it is available, they can turn it on (if they wanted to).  And if 
> it is not available, they can "handle" it in some way.

We could, but this is something I'd leave for a future version, since 
browser vendors have already said that they don't want to support it 
today.


On Wed, 15 Oct 2008, Peter Kasting wrote:
>
> * These use cases are really a stretch, compared to the much more 
> plausible use case of simply allowing a user to scrub a video forwards 
> or backwards.

These aren't use cases, they're just examples of why we have to define 
this one way or the other.


> * All this still seems well-enough-solved if audio can be 
> programmatically muted.  The worst possible case is that the author, as 
> you say, does not test in one browser, and causes another browser to 
> have overlapped audio. Then users complain and the author is responsible 
> and fixes their JS to mute the audio (hopefully simple), or they are 
> lazy and don't.  Neither one seems like catastrophe.  As you say in your 
> mail, authors act irrationally, and not all will fix their pages.  But 
> so what?

If we don't define it, and some browsers do it and some don't, then 
eventually all browsers will be forced to do the same thing (probably mute 
when rewinding). Not defining things is an unstable equilibrium. In the 
meantime, we'll have wasted a bunch of engineering resources.


> By comparison, mandating that browsers play audio puts a great burden on 
> browsers and codec authors (since the feasibility of reversed audio 
> depends on the format, the OS, etc.), and mandating that they don't 
> completely rules out whole classes of use cases.  This cure is much 
> worse than the disease.

We can always add a new feature later that enables scrubbing audio when 
rewinding.

If we don't define it, we're forcing it just as much, it's just that the 
forcing will happen mostly out of our control, and might end up being much 
more of a mess than if we pick one first. That's how a lot of HTML's most 
crazy quirks came about.


> > We can get this by requiring that the user agent play _some_ audio, 
> > even if severely degraded, at all speeds including negative speeds. 
> > I'm happy to require that if people are ok with it.
> 
> I don't see how this is any better than the "no spec" case you wanted to 
> avoid.  Use your previous argument again: an author tests in one 
> browser, which plays reversed audio in a fashion he deems acceptable.  
> The released app, played in another browser, sounds hideous.  Users 
> complain.

But here, it's the less good browser that gets the bad effect, so the 
browser vendors end up forced towards the better behavior, not the worse 
behavior, as would happen otherwise.


> I don't see how an explicit attribute buys you anything more or less 
> than explicit muting until the day when every browser respects that 
> attribute in all cases -- and if our reason for not mandating reversed 
> playback is that it's technically infeasible in some circumstances 
> (which I think has merit), that's unlikely to ever happen.

We can design the feature to indicate whether or not rewinding is going to 
give audio or not.


On Wed, 15 Oct 2008, Kristof Zelechovski wrote:
>
> Requiring some audio to play is a good idea but it should be more 
> precise. If the implementors are required to play some audio, they can 
> do away by playing silence, or play white noise at level 0.

The spec defines that it has to be the media's audio.


> Authors should mute the audio if they don't actually want it.  Setting 
> "muted" to false is hardly the right way to do it.

This is somewhat orthogonal to the issue.


On Wed, 15 Oct 2008, Jeremy Doig wrote:
>
> personally, i think this edge case is complete nonsense. I totally 
> support making it optional. I can't think of a single site today that 
> has anything like this - or wants it. web video is about linear 
> consumption, not edit-type scenarios with frame level control.  We 
> absolutely need the ability to scrub/seek - but that's already in the 
> spec.it's easy for quicktime to talk about supporting it - because they 
> already have it implemented. but that doesn't mean anyone on the web 
> actually needs it.

The questions is not whether we should support it or not. It's an issue of 
making sure we have interoperability.


On Thu, 16 Oct 2008, WeBMartians wrote:
>
> Yeah, it's sort of like the date time situation: either you have to 
> support everything (or, well, at least, be consistent) or have some kind 
> of API to report what you do/don't support.
> 
> I hate big APIs and therefore vote for consistency (if consistency is 
> "the hobgoblin of small minds," I'm a flatworm).

I don't really follow what you mean.

-- 
Ian Hickson               U+1047E                )\._.,--....,'``.    fL
http://ln.hixie.ch/       U+263A                /,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'