[whatwg] Proposal for <canvas src> to allow images with structured fallback

[Ian Hickson]

On Wed, 2 Mar 2011, Tab Atkins Jr. wrote:
> 
> <img> was designed in a broken manner, as it is a purely visual element 
> with no way for non-sighted users, such as blind people or search 
> engines, to view its contents.  To fix this issue, @alt was added to 
> <img>, to provide a textual alternative to the image.
> 
> @alt is fine for images with simple textual equivalents, but some images 
> are complex and can't be easily described in a single unstructured 
> paragraph.  For example, a complex graph may be best represented in 
> textual form by presenting the data used to generate it in the form of a 
> <table>.
> 
> To fix this, @longdesc was added, which is a link to another page with a 
> longer, and possibly structured, alternative representation of the 
> image. This has several problems, however.  First, data shows that many 
> users of @longdesc don't realize that the value of the attribute should 
> be a link to a page representing the longer description, and instead 
> either point it to the embedding page or the image itself, or just fill 
> it with nonsense.  Second, the fact that the long description is a 
> separate page means that it's possible that the linkage can be broken if 
> the server is reorganized or the url structure of the site is altered, 
> or if the code containing the <img> is copypasted between pages.  Since 
> the long description is not presented at all in all major visual 
> user-agents, a breakage can go a long time before being detected.

There's also a third problem, which is that this information, if useful to 
people who can't see the image, is usually _also_ useful to people who 
_can_ see the image. As such, the simplest solution is just to provide the 
information right there next to the image.


> 4. Some other elements, such as <object> and <video>, have conceptually 
> similar issues, where they want to present alternative content in case 
> their main content is unusable or unrecognized.  These elements encode 
> the fallback content as direct descendants, hiding them when they're not 
> necessary.
>
> 5. The <canvas> element's situation is *directly* analogous, as <canvas> 
> represents an image, and contains textual/structured alternative content 
> as descendants.  The problems with @longdesc described in #3 are not 
> present in <canvas> - the descendants are clearly alternative content, 
> and travel inline with the element, making them immune to linkrot.

Actually <canvas>' fallback isn't quite the same as <object>'s: it's still 
active even when the <canvas> is shown. You can tab into the <canvas> 
element's fallback content, screen readers render the fallback while 
letting the user interact with the image, etc. It's not fallback, like 
with <object>, or a description, like with longdesc=""; it's a layer on 
top of the canvas, that just augments it.


> It thus seems that <canvas> represents a strictly better alternative
> to <img> when structured accessibility fallback content is required,
> except for one problem - <canvas> can only be scripted.

It's not really fallback at all.


> I suggest that we can retain all the benefits of <canvas> over <img 
> longdesc> while avoiding the above problem by adding an optional @src 
> attribute to <canvas>.  If present, the image linked by the attribute is 
> loaded and drawn into the <canvas> automatically, without any script 
> intervention required.  <canvas> would then fire the same events that 
> <img> currently does and generally expose the same API, in addition to 
> the current canvas API.  It would be used like:
> 
> <canvas src="complex-chart.png">
>   <table>
>     -data that the chart represents-
>   </table>
> </canvas>

This seems exactly equivalent to:

   <object data=complex-chart.png">
     <table>
       -data that the chart represents-
     </table>
   </object>

...except that in the former case, the accessibility APIs would be quite 
confused since they would let visually impaired (but not blind) users 
navigate a table that wasn't displayed.


On Fri, 4 Mar 2011, Tab Atkins Jr. wrote:
> On Thu, Mar 3, 2011 at 6:29 AM, Anne van Kesteren <annevk at opera.com> wrote:
> >
> > Why not use <object>? It already works and avoids overloading <canvas> 
> > for something it was not designed for.
> 
> <object> doesn't expose image APIs like the 'complete' event.

Sure it does; it fires a 'load' event once the resource is loaded.

It doesn't have the .complete IDL attribute that HTMLImageElement does, 
but then nor does <canvas>. We could add it to <object> if it's really 
that useful.


> Also, it takes arbitrary content, not just images, so you can't use it 
> to contain user-supplied urls.

<canvas> doesn't currently take any URLs.

What user-supplied URLs are you expecting to display that have structured 
fallback content? Surely if you can sanitise the fallback you can sanitise 
the URL as well.


> Finally, <object> has bad interactivity behavior - if you drop SVG into 
> <object>, the SVG will swallow any clicks, while SVG-in-<img> doesn't.

If you drop SVG into <canvas>, it'll presumably lose any interactivity 
whatsoever, which seems even worse, especially from an accessibility 
point of view. Having said that, surely when using SVG rather than a 
bitmap you'd be better off using SVG's own built-in accessibility features 
rather than having a separate structured fallback solution.


> <canvas> doesn't *yet* expose the image APIs, but it wouldn't cause any 
> conflicts to make it have them.  A canvas without a @src would act just 
> like an <img> without a @src, which I believe is already consistent with 
> how <canvas> works.

If this problem truly exists, which I'm not convinced it does since I feel 
the far better solution is just to show everyone the structured fallback 
and not call it fallback, then it seems that it would be better to use 
<object> rather than try to overload <canvas> to do double-duty as both a 
direct-mode graphics API and an image embedding feature. If changes are 
needed for <object>, we can make those. I'm not sure any are, though.

-- 
Ian Hickson               U+1047E                )\._.,--....,'``.    fL
http://ln.hixie.ch/       U+263A                /,   _.. \   _\  ;`._ ,.
Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'