[whatwg] metadata attribute for media

[Ian Hickson]

On Thu, 20 Sep 2012, Ralph Giles wrote:
>
> Back in June, I proposed[1] a new attribute to get metadata "tag" data 
> out of media resources.
> 
> I've done an experimental implementation of this which is now in the 
> Firefox Aurora (alpha) channel[2] and Nightly development builds.
> 
> The method is media.mozGetMetadata() and it returns a new object each 
> time, whose properties are key:value pairs representing the metadata 
> tags from the file. Aurora supports this for Ogg Vorbis (.ogg) streams. 
> Nightly supports Opus (.opus) files as well.
> 
> Right now the method returns whatever set of tags happen to be in the 
> media stream, at least if they can be interpreted as unicode text. There 
> are several things I'd like to fix about this. Media formats generally 
> have rules about how to map various tags to a standard vocabulary. Right 
> now, web content has to implement those rules, but the user-agent is in 
> a better position to do so.
> 
> Also, a number of Mozilla (C++) developers I've talked to prefer a fixed 
> schema for metadata, rather than the more dynamic approach.
> 
> So, I'd like to do a new method which returns an object with a fixed 
> schema we can describe in idl, representing standard fields derived from 
> the dublic core tag set. The raw metadata would be available under the 
> old method for advanced applications.
> 
> Some metadata is also per-track, rather than per-stream, so it makes 
> sense to have these calls available on track objects as well as media 
> objects.
> 
> The most essential tags are dc:creator and dc:title. That covers almost 
> all player use cases. For per-track metadata, the language is perhaps 
> also useful for selection purposes. Are there any other tags folks would 
> like to see in the initial proposal?
>
> [1] http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2012-June/036352.html
> [2] available from aurora.mozilla.org

This seems reasonable. If there's implementor interest from other vendors, 
and if there's a spec that I can point to for how to get the metadata out 
of the various formats, then I'd be happy to add it to the spec.

I don't want to be the one to maintain the mapping from media formats to 
metadata schema, because this isn't my area of expertise, and it isn't 
trivial work.

I don't think we should have an open-ended API without fixed names, 
because that is a recipe for an interoperability disaster. We'd just end 
up having to define the mapping later when major yet poorly-tested sites 
started depending on particular mappings of popular UAs.


On Mon, 26 Nov 2012, Ralph Giles wrote:
> On 12-09-27 1:44 AM, Philip Jägenstedt wrote:
> >
> > I'm skeptical that all that we want from ID3v2 or common VorbisComment 
> > tags can be mapped to Dublin Core, it seems better to define mappings 
> > directly from the underlying format to the WebIDL interface.
> 
> You're right.

Indeed.


> > Given the open-endedness of metadata contained in actual media 
> > resources, I'm personally a bit skeptical that there's something we 
> > could add to the Web platform that would be better than just letting 
> > authors pass that metadata out-of-band using any representation they 
> > like, but what use cases are you trying to cover here?
> 
> Two use cases I'm trying to address:
> 
> - A web application presents some view of a media library. If the libray 
> resides on a server, then yes, the server-side component of the app can 
> parse, cache, and deliver the metadata out-of-band. But the library 
> could also be local, in which case the webapp must do its own parsing, 
> e.g. from a list of blob urls returned by the file api.
> 
> - An author wants to display the embedded track title and artist name 
> with simple scripting on a webpage.

For music in particular this seems reasonable.

-- 
Ian Hickson               U+1047E                )\._.,--....,'``.    fL
http://ln.hixie.ch/       U+263A                /,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'