[whatwg] Volume and Mute feedback on <video>

[Roger Hågensen]

On 2010-08-20 21:57, Ian Hickson wrote:
> On Mon, 31 May 2010, Silvia Pfeiffer wrote:
>    
>> I just came across a curious situation in the spec: IIUC, it seems the
>> @volume and @muted attributes are only IDL attributes and not content
>> attributes. This means that an author who is creating an audio-visual
>> Webpage has to use JavaScript to turn down (or up) the loudness of their
>> media elements or mute them rather than just being able to specify this
>> through content attributes.
>>      
> What is the use case for overriding the user's defaults in this way?
>
> I guess I could see a use case for muting (e.g. video ads often start
> off muted), but declaring the default volume seems very strange.
>    

I must say that content has no business messing wit the mute.
So whatever the user has chosen as default audio or video behavior 
should be respected at all times, no exceptions.
I.e. If the browser is "muted" then playback should be muted as well.

>> However, if I have multiple videos on a page, all on autoplay, it would
>> be nice to turn off the sound of all of them without JavaScript. With
>> all the new CSS3 functionality, I can, for example, build a spinning
>> cube of video elements that are on autoplay or a marquee of videos on
>> autoplay - all of which would require muting the videos to be bearable.
>> If we added @muted to the content attributes, it would be easy to set
>> the muted state without having to write any JavaScript.
>>      
> I guess that could make sense.
>
> Would we want to make .muted simply reflect the content attribute, so
> that the user enabling/disabling muting changes how the DOM is serialised?
> Or would we go for another attribute, say mute="", as the default, and
> have the IDL attribute be set by that attribute when loading, and then be
> independent of it? The latter seems better I guess.
>    

Multiple on autoplay? Outch, that's just bad design no matter how you 
think about it.
If anything only the first encountered media should autoplay, the other 
though should be autopause but possibly start pre-buffering in he 
background though.
Maybe multiple autoplay behavior could be user configurable. (defaulting 
to the behavior I just mentioned)

>> As for the @volume attribute, I think it would be similarly useful if an
>> author could control the loudness at which a video or audio file starts
>> playing back, in particular if he/she knows it is actually a fairly
>> loud/quiet file.
>>      
> Given that the default is 1.0 (the loudest setting), it would only really
> let authors make things quieter. It's not clear that there are that many
> use cases for that.
>
>
> On Wed, 9 Jun 2010, Kevin Marks wrote:
>    
>> Setting volume above 1.0 can be very useful if the original is too
>> quiet. For example, Quicktime allows a volume of 300% to amplify quiet
>> tracks
>>      
>
> On Thu, 10 Jun 2010, Ashley Sheridan wrote:
>    
>> Or you could just raise the volume of the audio track itself. I think
>> being able to raise the volume like this (beyond 100% of what it is)
>> with script just makes it something more likely to be abused (think how
>> the TV adverts always seem twice as loud as the programs they surround)
>> and so will end up getting blocked more often.
>>      
> Yeah.
>
>
> On Thu, 10 Jun 2010, Silvia Pfeiffer wrote:
>    
>> That requires editing the resource. Think about it from a process
>> point-of-view: you're a Web developer and have been given a set of media
>> resources to put on a Website. As you put it all together, you notice that
>> the volume of the different files is different and thus playing them back
>> next to each other will create a very confusing user experience. Do you
>> really want to shoot the files back to the production to adjust the volume
>> settings so they are all similar? If you're under time pressure, you'd
>> probably much prefer just setting a volume attribute on each so they all
>> play back with the same level.
>>      

There is no way to go beyond 0dBFS (digital 100%) anyway.
It is much better to force the loudness to the production using some 
standard. K-System K20 is my favorite which also matches the SMPTE movie 
standard for audio.

Also, doesn't the stream format itself allow loudness info? So there is 
no need to mess with the browser playback volume if the stream can hold 
info saying that it should be played back at 50% for example.
(the decoder would ensure the volume matches that, the browser volume is 
outside of that, and system volume outside of that again and possibly 
speakers system volume outside of that again).
And if WebM does not support volume info then WebM needs to have that 
added so streams can be matched in volume.

This would allow a the those delivering the stream to easily edit the 
volume without having the decode/re-code the stream, just editing extra 
stream info.
Or it could even be automated server side by doing a single decode 
originally and doing a simple RMS of the audio using 50ms window (and 
where 0dB = Full Signal Sine Wave),
this can even be easily checked in software like Adobe Audition or 
similar, and then just edit the stream info of the WebM being served.

One thing is for sure...do not mess with the user's browser defaults. If 
I where to set the playback volume to say 25% (be it globally, for that 
site or page only or the current stream)
then I do not want something to suddenly crank that up to 100% nor down 
to 0% without my explicit permission.
I recently mastered and released three albums, and I made sure that they 
sound consistently pleasant in volume, using RMS with a 50ms window (and 
0dB = Full Signal Sine Wave),
same as the K-20 standard using a flat (i.e. RMS aka no weighting) 
filter instead of A-weighted. Thus their loudness are all around -20dBFS 
(+/- 0.5 dB) RMS.

PS! Reason for plain RMS rather than weighted loudness curves is due to 
speed, simplicity and that it's flat aka neutral, it's also cheap to 
implement both in hardware and software.
Maybe WebM could take it one step further and actually pass along the 
RMS of the recording, thus allowing the user to tell a browser that 
their preferred level is say -20dBFS or -14dBFS etc.
Incidentally -20dBFS RMS is exactly 0.1 or -0.1 in floating point (where 
1.0 or -1.0 is 0dBFS) so the math is super simple here.

Note! I know many preffer loudness curves like ReplayGain that 
emphasizes common human hearing range, but it is actually biased due to 
that. I've yet to hear any music or video/movie sounds where unwanted 
audio was acually present in a dominant way.
Plain RMS is also less likely to cause clipping unlike loudness curves 
that ignore certain frequencies that could be very high indeed.)

So if the RMS of the audio where stated in the stream as being 0.5 
(floating point absolute value), and the user prefers a RMS of 0.1 
(-20dBFS) the modifier would be 0.2
The math for the volume change the browser would do for the stream 
playback could thus be: (0.1/RMS)=volume modifier for the stream, and 
applied as: (sample float value * modifier) = new sample value to send 
to the OS sound API.
And if for some reason the RMS info was not that correct or the user 
wants it louder or softer they could simply drag the volume slider up or 
down for that clip when listing to it. (as it'll act as a default user 
override).


Roger.

-- 
Roger "Rescator" Hågensen.
Freelancer - http://EmSai.net/