[whatwg] <picture> redux

[Tab Atkins Jr.]

Simon Pieters wrote up Kornel's earlier approach to a saner, more
palatable source selection algorithm for <picture> (rather than
copying <video>/<audio>).  This approach also has a new wrinkle:
<picture> *requires* an <img> child, and it's the <img> that still
actually displays the image.  The <picture> element is just a wrapper
for the <img>+<source> elements, and provides a context for the source
selection algorithm.  This makes testing substantially easier, as we
can limit ourselves to testing the source selection algorithm, and
probably makes implementation easier as well.

This new approach seems to reasonably popular.  The major people
supporting src-N are for it, plus several new people; in particular,
Timothy from Apple has voiced support for it, as have several people
such as James Graham who disliked src-N for its use of multiple
attributes.

On Tue, Nov 19, 2013 at 2:07 PM, Simon Pieters <simonp at opera.com> wrote:
> On Tue, 19 Nov 2013 12:40:17 +0100, James Graham <james at hoppipolla.co.uk>
> wrote:
>> On 19/11/13 01:55, Kornel Lesiński wrote:
>>> On Tue, 19 Nov 2013 01:12:12 -0000, Tab Atkins Jr. <jackalmage at gmail.com>
>>> wrote:
>>>>> AFAIK it makes it as easy to implement and as safe to use as src-N.
>>>>>
>>>>> Simon, who initially raised concerns about use of <source> in <picture>
>>>>> found that solution acceptable[2].
>>>>>
>>>>> I'd love to hear feedback about simplified, atomic <source> from other
>>>>> vendors.
>>>>
>>>>
>>>> The cost there is that <picture><source> is now treated substantially
>>>> differently than <video><source>, despite sharing a name.
>>>
>>>
>>> The substantial difference is that it lacks JS API exposing
>>> network/buffering state, but IHMO that's not a big loss, as those
>>> concepts are not as needed for pictures.
>>>
>>> IMHO the important thing is that on the surface (syntactical level)
>>> they're the same - multiple <source> elements where the first one
>>> matches.
>>
>>
>> So the remaining objections I am aware of to atomic-source are:
>>
>> * Something related to animations. I don't actually understand this, so it
>> would be nice if someone who does would explain. Alternatively this might
>> not actually be an issue.
>>
>> * Verbosity. This proposal is clearly verbose, but it is also the one that
>> authors seem to prefer, largely because it uses the underlying markup syntax
>> in a natural way. It seems that people will likely deal with the verbosity
>> by copy and paste, templates or libraries to provide a convenient shorthand.
>> If the latter occurs we can look at standardising it later.
>>
>> * More testing is needed. Specifically it seems that tests will be needed
>> to use <source> elements (or <picture> elements?) where you can currently
>> use <img> elements. This is a real concern of course, but seems lower on the
>> priority of constituencies than authoring concerns, unless we think that
>> poor interop will poison the feature. With an atomic proposal this seems
>> much less likely, Hopefully implementations will be able to reuse the
>> existing <img> code so that the actual amount of new *code* to test is less
>> than you might think by looking at the extra API surface.
>
> Also see discussion in http://krijnhoetmer.nl/irc-logs/whatwg/20131119#l-537
>
> In http://lists.w3.org/Archives/Public/public-respimg/2013Oct/0045.html I
> discuss a problem that a new element would have, namely that it would
> require a new fallback mechanism and a lot of stuff would need to be
> duplicated from img.
>
> If we want to avoid that problem but still use <source> elements, that is
> possible by using <img> for rendering the image and <source> for providing
> the sources. There is precedent for this sort of thing in HTML, namely
> <input list=x> <datalist id=x><option>...</datalist>.
>
> My first idea along these lines was to use <img list=""> <sourcelist><source
> ...>...</sourcelist> where, if the img element has a list attribute, the
> next element sibling is used for the sources. However, Anne pointed out a
> problem that it wouldn't work so well if you want to create the image in
> script and have it load without appending it to the document, since they
> wouldn't be siblings. There are ways to make it work but it would be
> non-obvious and it's better to come up with something that just works.
>
> So my next idea is to put the img element inside the sourcelist element,
> like so:
>
>    <sourcelist> <img list=""> <source>... </sourcelist>
>
> The sourcelist element would be a normal inline element and only serve to
> bind together the img with the source elements. I've left the list attribute
> on img because currently the <img src> would start loading as soon as the
> element is created by the HTML parser, before the element is inserted to the
> document so there is no parent. If we want to get rid of the attribute, we
> can make the HTML parser set a "parser-created" flag on the img element and
> upon element creation, if the flag is set, do nothing, and when it is
> inserted to the document, if the flag is set, unset it and run 'update the
> image data' steps. (The spec says to await a stable state before downloading
> anything, I don't know if browsers want img to wait with downloading until
> the parser yields or would rather handle a flag and not await a stable state
> for parser-created img.)
>
> At this point we could change the name of the wrapping element to <picture>
> and basically have the same syntax as current <picture> except there would
> be a required <img> child element.
>
> If we want similar syntax to <video>, the <source>s should go first and the
> <img> last.
>
> So:
>
> <picture>
>  <source ...>
>  <source ...>
>  <img src="fallback" alt="...">
> </picture>
>
> The selection algorithm would only consider <source> elements that are
> previous siblings of the <img> if the parent is a <picture> element, and
> would be called in place of the current 'process the image candidates' in
> the spec (called from 'update the image data'). 'Update the image data' gets
> run when an img element is created, has its src or crossorigin (or srcset if
> we still want that on img) attributes changed/set/removed, is inserted or
> removed from its parent, when <source> is inserted to a <picture> as a
> previous sibling, or a <source> that is a previous sibling is removed from
> <picture>, or when a <source> that is a previous sibling and is in <picture>
> has its src or srcset (or whatever attributes we want to use on <source>)
> attributes changed/set/removed. 'Update the image data' aborts if the
> parser-created flag is set. When img is inserted to the document, if the
> parser-created flag is set, the flag is first unset and then 'update the
> image data' is run but without the await a stable state step.

In addition, we can integrate the additional benefits of src-N in a
less compact, more readable way now:

1. Add a <picture sizes> attribute which takes a list of MQ/width
pairs, plus optionally a final fallback width by itself.  This gives
the <img> a specified width, if it doesn't specify <img width> itself.

2. Allow <source src> to take, in addition to url/density pairs,
url/width pairs.  The widths are resolved into "effective densities",
as described in the src-N proposal, and used in the source selection
algorithm as normal.

The <picture sizes> attribute also naturally interoperates with
Client-Hints, if you want that.

~TJ