[whatwg] several messages regarding Ogg in HTML5

[Ian Hickson]

First, let me welcome all our new members. For those of you not familiar 
with how things work in the WHATWG, a quick recap: basically, I promise to 
(eventually) reply to all substantive e-mails sent to this list, and 
update the spec apropriately. Your part in this is to make sure you read 
the replies I send before responding, to make sure you are not recovering 
old ground -- we don't benefit from going around in circles.

If your e-mail hasn't yet received a reply, you can look at the list of 
outstanding e-mails (updated daily) on this page:

   http://www.whatwg.org/issues/

With that said:

On Tue, 11 Dec 2007, Maik Merten wrote:
> >
> > One would imagine that they would happily take new risks if the 
> > rewards were great (e.g. a better codec). Sadly the rewards in the 
> > case of Ogg Theora are low -- there isn't much content using Theora, 
> > and Theora isn't technically an especially compelling codec compared 
> > to other contemporary codecs like, say, H.264.
> 
> If keeping the web free of IP licensing horrors and being interoperable 
> with as many players as possible (commercial and non-commercial 
> entities, open source or not, free software or not) isn't much of a 
> reason things are looking cheerless for the web indeed.

Actually those are pretty much the only reasons being taken into account 
here. Sadly, Ogg doesn't keep the Web free of IP licensing horrors, due to 
the submarine patent issue -- as Microsoft experienced with MP3 and 
with the Eolas patent over the past few years, for instance, even things 
that seem to have well-understood patent landscapes can be unexpectedly 
attacked by patent trolls.

This does suggest we need patent reform, but in practice this is out of 
scope for HTML5's development. We can't design our spec on the assumption 
that the patent system will be reformed.


> I don't exactly see why the web should embrace non-free standards just 
> because the big players made the "mistake" of licensing 
> definitely-encumbered formats and are unwilling to "take further risks". 
> (I am aware this is a pretty hard wording and that things aren't quite 
> that easy.)

In the absence of IP constraints, there are strong technical reasons to 
prefer H.264 over Ogg. For a company like Apple, where the MPEG-LA 
licensing fee cap for H.264 is easily reached, the technical reasons are 
very compelling.


> The old wording was a SHOULD requirement. No MUST. If the big players 
> don't want to take the perceived risk (their decision) they'd still be 
> 100% within the spec. Thus I fail to see why there was need for action.

The problem is that if the big players don't follow the spec, even the 
SHOULD requirements, then the spec is basically pointless. What we want 
isn't that some people support Ogg, what we fundamentally want is that 
_everyone_ support the same codec, whatever that may be.


> > One way to get a company like Apple to want to take the risk of 
> > implementing Theora would be to cause there to be a large pool of 
> > existing Theora content out there. Obviously, this presents a 
> > bootstrapping problem (aka a "chicken and egg" problem).
> 
> In a world where content is served on a per-user basis (streaming, DRM 
> encrypted media files) I don't think this is much of an argument. HTML5 
> is a future standard which will serve future content.

I don't see how this affects Apple's stance here. Today they can get 
significant traction with just H.264 (for example, Google is also moving 
to H.264 and Apple can therefore implement YouTube applications on iPhone 
without using anything but H.264). With Ogg, they get very little 
traction, yet significant financial risk.


> > I think the current wording in the spec is actually biased towards the 
> > small players more than the big ones, but if you think it's the other 
> > way around then I probably have struck the right balance.
> 
> I was specifically thinking of the "additional submarine patent risk for 
> large companies" part. Nobody wants to get struck by a submarine, so 
> either this requirement should be extended to all sort of entities or 
> dropped completely (as its hard by definition to make an informed 
> statement about submarine patents).

Small companies aren't targetted by patent trolls. Only big (really big) 
companies are. It's a big-company concern, just like "no per-user 
licensing" is a small-company concern. That's just the reality of the 
situation, it's not intended to be a bias.


On Tue, 11 Dec 2007, Manuel Amador (Rudd-O) wrote:
> >
> > It is intended to be exactly truthful, actually. I apologise if you 
> > believe this to be fear mongering.
> 
> Well, the intentions certainly didn't match the actions.

I am sorry you perceive them this way.


> > > - The Xiph developers were extremely zealous and almost fiduciarily 
> > > diligent in researching all possible patent threats to Vorbis 
> > > technology, and for more than a year they found none -- they even 
> > > did the research *before* beginning to code, explicitly to avoid 
> > > submarine patents.
> >
> > While this is very true, and admirable, and impressive, it is sadly 
> > not a guarantee. Certain companies (Nokia and Apple among them) have 
> > reported that they still fear that undisclosed patents may exist that 
> > cover the relevant codecs,
> 
> I fear that an earthquake will make my building tumble and fall over.  
> That doesn't mean I'm gonna move to the open street.

No, but you presumably would avoid living in a house that had matchstick 
walls and a large concrete ceiling. It's a matter of risk management.


> In other words, here we go.  FEAR.

But it is fear felt by the companies themselves, not fear that they are 
attempting to pass onto the userbase.


> > as they might exist for other formats like MPEG4/H.264.
> 
> UNCERTAINTY.

Indeed, the entire issue here is one of uncertainty.


> > The difference is that while Apple (for example) have already assumed 
> > the risk of submarine patents with H.264, they currently have taken no 
> > risks with respect to the aforementioned codecs, and they do not wish 
> > to take on that risk. Given the extremely large sums of money that are 
> > awarded for patent violations (cf. Microsoft's recent settlements), it 
> > is understandable that companies with the high profile of Apple and 
> > Nokia would not wish to take on such risks.
> 
> DOUBT.

Yes, the big players here are fearful and doubtful of the uncertain patent 
situation surrounding Ogg. That is in fact the entire problem.


> It's all unsubstantiated terror-mongering. How curious that the very 
> same companies who are rallying against free multimedia and throwing 
> hissy fits because of the "patent monster" are the ones most deeply 
> supportive of software patents.  They have huge patent portfolios to 
> wage a war against any patent troll, so why are they trolling now?

Unfortunately patent portfolios don't help against trolling patent holding 
companies. It's also not clear to me that the companies to which you refer 
are in fact the most supportive of software patents; indeed, as far as I 
can tell most companies involved here have extensive lobbying efforts 
ongoing to encourage patent reform. However, that is quite out of scope 
for the matter at hand, namely the HTML5 specification. As noted above, we 
can't base what we do in the spec on ongoing lobbying efforts whose 
success is as yet highly unpredictable.


> > MP3 is and old codec as well, yet the threat of submarine patents 
> > covering MP3 surfaced recently, much to Microsoft's chagrin. Unless 
> > the codecs are older than the patent lifetime, there is unfortunately 
> > no guarentee.
> 
> Of course there is no guarantee.  There is *never* any guarantee.

There can be a guarentee, e.g. with technologies that have outlasted the 
patent lifetime (Motion JPEG, e.g., or, as I understand it, H.261). There 
is also a matter of risk vs reward; with 


> > Patent trolling companies are patient and will wait for bigger 
> > targets, as has been seen time and time again. (As an example of this, 
> > the Eolas patent case is still fresh on everyone's minds, I'm sure.)
> 
> Wow, sounds like something's wrong with software patents around the 
> world.

Hear hear.


> Watch out, someone may be holding a submarine patent on encoding 
> hypertext.

Luckily, patents on hypertext have long expired.


> Fact: Vorbis is the *only* codec whose patent status has been widely 
> researched, nearly to exhaustion.

Sadly there's really no such thing as an exhaustive patent search.


> Repeating the same FUD over and over again (which you just did) may lead 
> the world to believe this to be false, but it's TRUE.  You should at 
> least have talked to Monty @ Xiph before jumping to rash conclusions.

The first person I spoke to upon making the change to the spec was in fact 
from Xiph. They understand the concern, and, as I understand it, agree 
that the current text in the spec is an accurate summary of the issue.


> > Ogg is not necessarily the only solution that avoids patent 
> > encumbrence. There are codecs that have been in existence for longer 
> > than the patent lifetime, for instance. Dave Singer posted a quite 
> > thorough analysis of this issue recently.
> 
> Yes, but Vorbis is the only one offering competitive performance, which 
> you explicitly laid out as a requirement (and, frankly, I coulnd't agree 
> more with you).

Vorbis isn't really the problem. The real issue is video codecs (Theora).


> > Apple, Nokia, Microsoft and other large companies have stated that 
> > they will not support Theora purely based on the requirement in the 
> > spec. Having or not having this requirement in the spec thus makes no 
> > difference to independent authors.
> 
> Yes it does.

How?


> Let me rephrase your statement to be worded in a more *honest* way.  
> Vorbis provides the perfect escape for proprietary audio prisons.  
> Apple and Nokia are having problems with consumers and authors actually 
> waking up and using free, non-patent-encumbered, widely available, 
> unrestricted, non-proprietary technology.  Since Vorbis directly 
> threatens their ability to sell traps, they are extorting your 
> compliance with threats of not supporting the HTML5 spec.

I don't know what you base your conclusions on, but I assure you that to 
the best of my knowledge, that's not the current situation. I have been in 
this business a long time, and I've been played for a fool many times 
before. This particular issue does not have the tell-tale signs of players 
acting in bad faith. Indeed, Apple employees have probably done more to 
resolve this issue than anyone else so far.


> It doesn't.  The effort that would most effectively address these issues 
> would be a "patch -r" on the related spec revision, combined with a 
> polite mail: "no, Nokia, no, Apple, we're very sorry but you are wrong 
> and we're not going to let you break the spec's legs before it's even 
> born".

The spec is pointless if we don't have interoperability. The goal here 
isn't to support Ogg, or H.264, or anything else. The goal is to get a 
baseline video codec that everyone can use and implement, and that will 
work in all browsers. For that, we need a codec that is known to not 
require per-unit or per-distributor licensing, that is compatible with the 
open source development model, that is of sufficient quality as to be 
usable, and that is not an additional submarine patent risk for large 
companies.



On Tue, 11 Dec 2007, Manuel Amador (Rudd-O) wrote:
>
> I actually think this Slashdot comment summarizes the sentiment 
> perfectly:
> 
> "Methinks you are being a bit myopic here. Where would we be today if 
> the HTML spec didn't specify jpg, gif, and png as baseline standards for 
> the image tag? [...]"

Actually we'd be right where we are now, since the HTML specs never 
required any particular formats for <img>. However, it does seem like we 
need more guidance for video codecs.


> Exactly.  For audio, Ogg Vorbis should be the baseline standard.  
> Companies are free to implement their own technology and installable 
> kits or redistribution agreements that allow people to use their tech on 
> their computers.  And for video, likewise but replace Vorbis for Theora.
> 
> Otherwise, let's start preparing for 1995 and "To view this page, you 
> need to install this piece of crapware" all over again.  I lived that 
> (together with Windows 95, which in all fairness was rather good 
> compared to the alternatives -- thank god for Linux).  I don't want to 
> experience it all over again, especially since I know that even today, 
> that crapware isn't even gonna be made for Linux, and I'm going to be 
> screwed again.

The whole point of the change was to make the point that we need something 
that will not screw you. Ogg isn't a solution, as it won't be implemented 
by Apple and Microsoft. If we require Ogg, then what will happen is the 
big players will support something else, then that will become the 
de-facto standard, and you will get screwed. What we _want_ is for 
everyone to support the same codec. We don't get that by having a 
SHOULD-level requirement for Ogg.



On Tue, 11 Dec 2007, Jérôme Marchand wrote:
> 
> Please, anybody, tell me it's not true.Ogg Vorbis/Theora is perfect for 
> web applications. We need to suport those. Is there anybody else than me 
> that realise it cost 0.75$ US to of patents licensing LEGALLY have an 
> MP3 decoder? I devellop for embedded applications, and it cost 15000$ 
> Just to ship around 100 Units with a MP3 software Encoder. Ogg and 
> Theora does a comparable cost but are free. I still can't imagine why 
> anybody would retract those two great standarts from HTML5, unless they 
> have codecs they what royalties from!)

As should be clear from the text in the spec, I entirely agree with your 
concerns, and whatever codec we eventually settle on will have to be a 
royalty-free codec.



On Tue, 11 Dec 2007, alex wrote:
>
> I am a webdeveloper and a fierce supporter of opensource. I was under 
> the impression the standards were being designed in the same opensource 
> spirit, but I may have been wrong. Setting OGG as the de facto standard 
> is the best idea i've heard in a long time, and now it's all coming down 
> because a few companies (some of which are known for their vendor 
> lock-in tactics) want to keep their empire.

I don't think the reason you cite is the actual reason for the deadlock, 
but you are right in saying that we are failing to get a de facto standard 
because not everyone is agreeing on what to implement.

We can't, by definition, have a de facto standard unless everyone agrees. 
That's what "de facto" means.


> I am not saying that ogg should be enforced onto anyone, if nokia wishes 
> to keep using a different format, no problem, but by making it a 
> standard, we at least know that ogg will be supported by all 
> (standards-compatible) browsers, and as such it can be deployed by those 
> who are opposed to vendor lock-in or monopoly positions.

We know that all standards-compatible browsers will support the standards, 
but what about all the other browsers? Surely what we want isn't just for 
a small set of browsers to support a codec but for _all_ the browsers to 
suppor the same codec.


> OGG is the choice of freedom, enabling that freedom for all 
> webdevelopers is a must in my opinion, although in the same spirit, it 
> can not be enforced upon anyone, therefor the original text stating it 
> "should" instead of it "must" is probably the best way to go.
> 
> Freedom for those who choose, the alternative for the rest.

Ogg is _a_ choice, which provides freedom for some but not everyone. We 
need a codec that works for everyone.



On Tue, 11 Dec 2007, Ryan McLean wrote:
> 
> I must express my disappointment that w3c is caving to pressure to 
> remove an essentially "free" codec from the specification that was at 
> best a recommendation, after all it said *should* not *must*, as to the 
> preferred codec.

This isn't the W3C, this is the WHATWG. :-) I don't think the W3C has made 
any decisions either way on this, though they are hosting a conference 
tomorrow regarding the subject.


> Maybe Nokia would be as good as to point out which codec is better? wmv? 
> divx? mov?

Sadly today there are no codecs that address all the needs of everyone 
involved; if there was, we wouldn't be having this discussion.


> My argument in favour of a "free" codec is that all browsers could ship 
> with it, without fear of being sued, this would allow users to 
> watch/listen to clips/movies/music out of the box without scouring the 
> Internet for codec XYZ for a once off use.

I think that's what everyone wants. The problem is that Ogg is not such a 
codec -- Apple, for instance, can't implement Ogg without fear of being 
sued.


> A good question that jumps to mind though is who are Apple / Nokia to 
> tell anyone what should be a web standard [...]

Well, they're browser vendors. At the end of the day, the browser vendors 
have a very effective absolute veto on anything in the browser specs, 
since they get to decide what they implement and what they don't. We can 
only specify things that they're willing to implement. This goes for all 
browser vendors, Mozilla, Opera, Apple, etc.


> Really if *anyone* should have any sway here (and I personally think 
> that no 1 or 2 companies should) it should be Google lets face it they 
> are the largest power on the Internet whether you love em/hate em/dont 
> know who they are..

I work for Google. :-)



On Tue, 11 Dec 2007, alex wrote:
>
> The difference with the "should" is that the browsers who support 
> standards will support ogg natively. The fact that big companies like 
> nokia etc don't actually use OGG is less my concern, it's more about the 
> free developers knowing that ogg will be supported at the users' end.

But they won't know that, if Apple, for instance, refuse to implement it.


> Patents is less my area of expertise as I am (luckily) a resident of the 
> EU, but this whole submarine patent bussiness has got me thinking that 
> America better clean up ship quickly.

Sadly submarine patents are a problem in the EU as well, though luckily 
software patents in general are not as serious a problem.



On Tue, 11 Dec 2007, David wrote:
>
> Sorry for the spam and all, but I'd just like to weigh in with my 
> support for ogg, and utter dismay at how it seem the process has been 
> brought out by big company's.
>
> You have a chance to make a big impact on the web.  Do you really want 
> to go down in history as the people who sold out the web for a quick 
> buck?

I assure you that the change was made in good faith; I (sadly) received no 
money for the change. I really wish I had.

The idea here is to have a big impact on the Web. Ogg, sadly, can't have a 
big impact in the current climate, because some of the biggest players 
refuse to implement it. We must find a solution that really can have a big 
impact.


I hope this addresses most of the concerns that are raised. I understand 
that some people will see a conspiracy here; naturally I can't really 
reassure you that there isn't one, that's the nature of conspiracies.

-- 
Ian Hickson               U+1047E                )\._.,--....,'``.    fL
http://ln.hixie.ch/       U+263A                /,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'