[whatwg] hit regions: limited set of elements for fallback content

[Dominic Mazzoni]

First a high-level thought.

I'm happy to keep chasing after "legitimate" use-cases instead of contrived
ones, but just because we can't think of one, doesn't mean it doesn't
exist. As Alan Perlis said, "Every program has (at least) two purposes: the
one for which it was written and another for which it wasn't. Maybe the
vast majority of web apps that use canvas for a grid, or a slider, or a
list box would be better off using standard html5 objects. But what if
there's one app that can't, for some reason we haven't anticipated? If we
wait until that app appears to allow that control to have a hit region,
then it will be months or years before that app can be accessible.

More below:

On Tue, Feb 18, 2014 at 1:16 PM, Ian Hickson <ian at hixie.ch> wrote:

> On Tue, 18 Feb 2014, Dominic Mazzoni wrote:
> > On Tue, Feb 18, 2014 at 10:51 AM, Ian Hickson <ian at hixie.ch> wrote:
> > > >
> > > > I'm curious if it's possible to implement an accessible list box or
> > > > other select control in a canvas. Wouldn't it be possible to make it
> > > > accessible if the canvas lets you focus the list box by clicking on
> > > > its hit region, and then change the selection using the arrow keys?
> > >
> > > What's the concrete use case?
> >
> > How can I get more concrete than there's a list box inside a canvas?
>
> Well for example, is the use case one of the controls on Bugzilla's
> advanced search page?:
>
>    https://www.w3.org/Bugs/Public/query.cgi?format=advanced
>
> If so, I feel comfortable saying that we don't need to make <canvas>
> support that, since that use case is already handled very well by the
> <select> element.
>

As I argued above, maybe we can't come up with a really good use-case, but
that doesn't mean one doesn't exist.

I'll try, though: what if I had a list of choices displayed as a pie chart?
Each slice of the pie is a focusable object that, when you click on it,
allows you to take an action on that pie slice.

Surely you'd agree that rendering a pie chart is a natural use-case for the
canvas element. I know it's technically possible in css, but it's quite
tricky - whereas it's simple and natural in canvas. And there are plenty of
shapes that are basically impossible in pure CSS.

But how should the list of pie slices behave if you were interacting with
the pie chart using the keyboard? Why not a list box? Conceptually you just
have a list of elements, one of which can be selected at a time.

> Whether or not we think it's a good idea, people are building entire UIs
> > using canvas. Sometimes it's in order to achieve pixel-perfect rendering
> > accuracy across browsers and platforms, sometimes it's for speed,
> > sometimes it's for visual effects that aren't possible in HTML,
> > sometimes it's because they're building a cross-platform UI.
>
> Just because people do it doesn't mean we need to design our APIs around
> it, or otherwise endorse it. For example, people use <table> elements for
> layout purposes, but this is expressively against the <table> conformance
> requirements. People use <blockquote> for indenting, but instead of
> providing a <blockquote indent-depth> attribute, we improve CSS to support
> indenting. And so on.
>

Even though you're not supposed to use <table> for layout, is there any
part of the HTML spec that forbids placing a certain element inside a table
cell because the only legitimate reason to do so would be if it was a
layout table? I'm not aware of one. Similarly, I think it'd be simplest to
just allow any element to be the target of a hit region. We can still
discourage it, but fail gracefully if authors decide to abuse it.

> Asking web developers not to build UIs in canvas is like asking Windows,
> > Mac, iOS, or Android developers to not build custom controls. It's not
> > going to happen.
>
> Sure, but that misses the point a little. We are developing technologies
> for custom controls -- Web components. To support a <select> drop-down,
> you need much more than just a canvas: you need a way to show a popup
> window, you need a way to draw shadows over underlying content, etc.
>

Yes, and people are building replacements for <select> using web components
now, like polymer-ui-menu:
http://www.polymer-project.org/components/polymer-ui-menu/smoke.html

It should be easy to make something like polymer-ui-menu accessible using
role=listbox and role=option under the hood.

The question is, why shouldn't it be possible to implement a
polymer-ui-canvas-menu that works the same way but uses a canvas to render?

So this is why I ask for the precise use cases. We can only evaluate if
> the technologies are actually addressing needs by going through and seeing
> how they do with precise use cases. If the use cases are very vague, we
> can at best guess, and we might guess poorly.
>
>
> > What if I do want a <select>, but I just want a canvas to render it
> > visually?
>
> Are Web Components and CSS unable to get the effects you need? Maybe we
> should be improving those rather than <canvas>. It's hard to tell without
> knowing precisely what you want to do.
>

Is my pie chart example good? We can keep brainstorming, but just because
we can't think of something really compelling doesn't mean it's not out
there.

> Yes, but people are implementing text controls in canvas too. We can't
> > ignore that.
>
> We shouldn't ignore it; we should provide better solutions that make it so
> they don't feel the need to do it.
>
> If a region has a car theft problem, you don't solve it by giving all the
> thieves the car keys. You solve it by improving the economy so that
> thieves have better things to do (like get an interesting job), and you
> solve the remainder by improving law enforcement. The same applies here.
> We solve it by providing better tools for custom text editing controls
> (e.g. better contenteditable APIs), and by making it non-conforming to
> abuse <canvas> for this purpose.
>

I'd suggest a different analogy: suppose your company makes foam pipe
insulation and you discover people are buying your product and using it as
a swimming pool flotation device. Do you try to stop them from using your
product and try to get them to purchase other pool toys, or do you start
selling your pipe insulation directly to the sporting goods stores?

Of course we should improve contenteditable APIs so that people who just
want basic text editing don't write their own poor replacement. But I don't
see why we shouldn't also allow people to build feature-complete
alternative text editing controls with features that might never make it
into contenteditable - like discontiguous selections, or complex wrapping
around objects.

> That suggests we also need a lower-level API that associates an element
> > in fallback content with a region in canvas, even without making it a
> > "hit" region. I don't see how that's an argument against hit regions,
> > though.
>
> Not sure what you mean. What's the difference between a region and a hit
> region, in this context?


A hit region is one that passes mouse events through to the element in
fallback content.

Instead, this would be a region that simply associates a region in the
canvas with an element in fallback content to indicate its accessible
bounds, not to propagate events.

Suppose you have some static text in your canvas. Associating the bounding
box of that text it with a <p> element containing an accessible version of
the same text could be very useful to screen magnifiers - ZoomText, for
example, lets you navigate to content that isn't focusable, and it will
optionally speak the text, even though it's not intended for totally blind
users.

> Browsers don't currently know what element has AT focus, but the browser
> > tells the AT if the coordinates of an element make it offscreen, and AT
> > can send a command telling the browser to scroll an element on-screen.
> > Currently browsers implement this automatically, without involving the
> > web author - but for completeness we may want canvas authors to have the
> > option to respond to a request to scroll a fallback content element into
> > view.
>
> What would such an API need? Do we need an event that fires on the
> <canvas> indicating that the AT focus has moved to a specific region, the
> same way we handle mouse events? (Should we just support this for all
> elements, not just canvas, while we're at it?)
>

Currently AT doesn't even tell the browser what has focus. This is partly
done on purpose - poorly-designed apps (not just web apps) will do things
like "trap" focus in a form control until you supply a valid response, and
AT needs to allow a user to explore the page without notifying the
underlying application where you are.

I think what's needed is a new event - "scrollToMakeVisible". Perhaps
whenever an element gets focus, this event could fire on that element, and
the default implementation would do what's done now - but an author could
choose to preventDefault and implement their own scrolling logic. Then what
would happen is that AT would sometimes fire a scrollToMakeVisible on an
element that isn't necessarily focused.

Are there other questions we should be asking to support this grid use
> case?
>
> So far it seems like to support grids in canvas, we need:
>
>  - enable focusable <td> elements to be hit region controls
>  - fire an event on an element when the AT focus moves to that element
>  - when this event is fired on a canvas, include the region information
>

I think a scrollToMakeVisible event would be better than a receivedATFocus
event.


> Anything else?
>

I think AT would like to know the region corresponding to the <table>
element too, not just the focusable <td>.

So, some API that just associates a canvas region with a fallback element -
purely for AT - would be very useful.

Now I wonder if it wouldn't be simplest to just allow *any* fallback
content element to be the target of a hit region . It would be simpler to
spec.

Here's another argument in favor of this: it's not uncommon on the web for
objects to do something on hover, even if they're not focusable or
clickable. For example, a static image might have a caption that appears on
hover, or a static paragraph that's partially truncated might grow to show
its full content on hover.

Allowing hit regions to map to any element would allow for the same type of
thing to happen within a canvas. An author might simply want to track what
region the mouse cursor is over, *even when that region isn't focusable or
clickable*.

This solves the accessibility use case too - if you want AT to know the
bounding box of an element in fallback content, just give it a hit region.
That allows AT to know where it is on the screen, and also allows you to
track when the mouse hovers on it.