[whatwg] metadata attribute for media

[Ralph Giles]

On 12-09-27 1:44 AM, Philip Jägenstedt wrote:

> I'm skeptical that all that we want from ID3v2 or common VorbisComment
> tags can be mapped to Dublin Core, it seems better to define mappings
> directly from the underlying format to the WebIDL interface.

You're right.

> Given the open-endedness of metadata contained in actual media
> resources, I'm personally a bit skeptical that there's something we
> could add to the Web platform that would be better than just letting
> authors pass that metadata out-of-band using any representation they
> like, but what use cases are you trying to cover here?

Two use cases I'm trying to address:

- A web application presents some view of a media library. If the libray
resides on a server, then yes, the server-side component of the app can
parse, cache, and deliver the metadata out-of-band. But the library
could also be local, in which case the webapp must do its own parsing,
e.g. from a list of blob urls returned by the file api.

- An author wants to display the embedded track title and artist name
with simple scripting on a webpage.

One of the goals of html media was to make video and audio as simple to
include as images. User agents are generally parsing this metadata
anyway; exposing it is straightforward, and greatly simplifies the two
tasks above.

In any case, the media.mozGetMetadata() method I described earlier is
available in Firefox 17, released last week, with support for Vorbis
tags in .ogg files. Firefox 18, now in beta, adds support for .opus
files as well. Here's an example:

  https://people.xiph.org/~giles/2012/metadata.html

You should see 'Title', 'Album', etc. below the controls if your browser
supports mozGetMetadata().

We're continuing to implement this interface for other formats we support.

I still think it's useful to define both this 'raw' metadata interface,
returning just the data out of the file, and a more structured metadata
object with standardized tag names. I've left off implementing the
second one for lack of feedback on what the basic tags should be, and
how useful they are. There certainly wasn't consensus here.

So, what do you think of these two proposals:

1. Define the 'raw' getMetadata method in an unprefixed form:

interface HTMLMediaElement {
  ...
  object getMetadata();
};

After the metadataloaded event fires, this method would return a new
object containing a copy of the metadata read from the resource, in
whatever format the decoder implementation supplies. It would be up to
the caller to do any semantic interpretation. The same method should be
present on AudioTrack, VideoTrack, (TextTrack?) and Image elements.

2. Define a parsed metadata attribute with some standard tags:

interface Metadata {
  readonly attribute DOMString title;
  readonly attribute DOMString artist;
  readonly attribute DOMString album;
  readonly attribute Blob? artwork;
};

interface MediaElement {
  ...
  Metadata metadata;
};

So you could say something as simple as

  img.src = audio.metadata.artwork;

to display the cover art embedded in a download single. DOMString
attributes would have the value of an empty string if the underlying
data isn't available. These four attributes are the absolute minimum, I
think, for the use cases above. More could be added later as usage of
the api evolves. For example: date, publisher, tracknumber, tracktotal,
description, genre, sort-artist, sort-title, copyright, license, url.

If having both media.getMetadata() (raw) and media.metadata is
confusing, the first proposal could be named media.getRawMetadata() as well.

Does it make sense to include more technical metadata here? For example:
samplerate, channels, duration, width, height, framerate, aspect-ratio.
Firefox currently has prefixed properties for the number of channels and
the audio sample rate, and including these in the metadata interface
would let us deprecate the prefixed versions. On the other hand,
properties like duration, width, and height are available directly on
media elements today, so maybe it makes more sense to do the same for
the others.

In that case, do we want the indirection of the Metadata inferface?
Saying 'video.title' or 'img.src = audio.artwork' instead is shorter.

Feedback welcome,
 -r