[whatwg] Proposal for <canvas src> to allow images with structured fallback

[Tab Atkins Jr.]

Allow me to quickly summarize the state of affairs and the problem I
wish to solve:

1. <img> was designed in a broken manner, as it is a purely visual
element with no way for non-sighted users, such as blind people or
search engines, to view its contents.  To fix this issue, @alt was
added to <img>, to provide a textual alternative to the image.

2. @alt is fine for images with simple textual equivalents, but some
images are complex and can't be easily described in a single
unstructured paragraph.  For example, a complex graph may be best
represented in textual form by presenting the data used to generate it
in the form of a <table>.

3. To fix this, @longdesc was added, which is a link to another page
with a longer, and possibly structured, alternative representation of
the image.  This has several problems, however.  First, data shows
that many users of @longdesc don't realize that the value of the
attribute should be a link to a page representing the longer
description, and instead either point it to the embedding page or the
image itself, or just fill it with nonsense.  Second, the fact that
the long description is a separate page means that it's possible that
the linkage can be broken if the server is reorganized or the url
structure of the site is altered, or if the code containing the <img>
is copypasted between pages.  Since the long description is not
presented at all in all major visual user-agents, a breakage can go a
long time before being detected.

4. Some other elements, such as <object> and <video>, have
conceptually similar issues, where they want to present alternative
content in case their main content is unusable or unrecognized.  These
elements encode the fallback content as direct descendants, hiding
them when they're not necessary.

5. The <canvas> element's situation is *directly* analogous, as
<canvas> represents an image, and contains textual/structured
alternative content as descendants.  The problems with @longdesc
described in #3 are not present in <canvas> - the descendants are
clearly alternative content, and travel inline with the element,
making them immune to linkrot.

It thus seems that <canvas> represents a strictly better alternative
to <img> when structured accessibility fallback content is required,
except for one problem - <canvas> can only be scripted.  To use it as
an image, one must also run some javascript with creates an
out-of-document <img> element, points it at the appropriate image, and
then draws it into the canvas.  This is inaccessible for visual users
without javascript, or for user-agents like feed readers that don't
run script.

I suggest that we can retain all the benefits of <canvas> over <img
longdesc> while avoiding the above problem by adding an optional @src
attribute to <canvas>.  If present, the image linked by the attribute
is loaded and drawn into the <canvas> automatically, without any
script intervention required.  <canvas> would then fire the same
events that <img> currently does and generally expose the same API, in
addition to the current canvas API.  It would be used like:

<canvas src="complex-chart.png">
  <table>
    -data that the chart represents-
  </table>
</canvas>

I've discussed this privately already, and one of the objections to
this proposal was that @longdesc allows the user-agent to delay
loading the alternative until the user actively requests it, as it's
not always ideal to drop a large structured alternative into the
middle of content (similar to how sighted users can just skip over a
complex chart, returning to it later when they have more time or
attention).  I don't believe this is a valid objection - there is
nothing requiring a screen-reader to automatically present the
alternative content to the user.  UAs must already have ways to delay
showing in-band information until requested, so as to correctly
represent the <details> element, for example.  The same mechanism can
be used for <canvas>; while I'm not in any place to recommend UI for
screen-readers, it may even make sense to present <canvas> and
<details> in a unified fashion.

Thoughts on the problem or the proposed solution presented here?

~TJ