[whatwg] responsive images srcalt proposal

[pghj]

I'm resending this (slightly updated) message because the first didn't
appear to get delivered. Concerning responsive images, I will make the case
that:

* Art-direction & matching media features/types should not be part of a
responsive image solution.
* The benefits of a preload-scanner are overrated when it comes to images
from <img> elements.
* There is a more elegant solution than srcset, src-N and picture

ART-DIRECTION & MATCHING MEDIA FEATURES/TYPES

I don't think the Art-direction issue should be solved at the <img> level,
because it's just a special case of another problem and replicates a
solution already provided by CSS. For example, providing a less detailed
image might very well require a change in accompanying text as well:

<div class='desktop'>
<img src='moredetail.jpg'> the red arrows indicate your troop movements
</div>
<div class='mobile'>
<img src='lessdetail.jpg'> click on the image to view details
</div>

...where a stylesheet hides one of either representations.

I'd rather leave Art-direction in the CSS domain as a one-size-fits-all
solution, even if it concerns just images.

The same argument goes for "matching media features and media types". The
images in the pie-chart example given at
http://usecases.responsiveimages.org happen to contain the text about the
colour keys. Were this not the case, a solution like the one I gave above
would still be needed.



RESOLUTION-SIZE-DISCRIMINATION

Moving on to resolution/size discrimination: I don't like any of the
proposed solutions so far, because of their complexity, and because they
replicate functionality in CSS, requiring the web-designer to mark-up style
in two places and keep them in sync.

As I see it, the complexity of these solutions derives from the attempt to
feed preload-scanners with partial style information that would otherwise
only become available at a later stage.

As Maciej Stachowiak put it, "the fundamental thing that's missing is
ability to have one of several images correctly selected by the browser at
preload time"

But how important is this, really? Measurements show a ~20% speed increase
when a preload-scanner is enabled, but how much of that can be attributed
to preloading images from <img> elements? More importantly, does preloading
these images help with perceived speed? I'd say not much: often most of
them are outside the visible area, make up only a small surface relative to
the whole page, and the page renders fine before they are loaded,
especially when the dimensions are known through attributes or CSS.
Preloading them can actually delay the loading of more important images
from CSS, which does degrade the perceived speed very much.

In fact, many efforts are made to bypass the preload scanner to avoid
loading images before they are scrolled into view, to increase speed,
conserve bandwidth, or both.

For those convinced of the necessity of a preload-scanner for <img>
elements, I am willing to provide a detailed overview of why the available
test-results are misleading, how image preloading harms browser
performance, and what would be a better way to preload images.

Also, as a matter of principle, I don't think it is desirable that
optimizations in today's browsers justify extending the html specification
with constructs as complicated as the solutions proposed so far.

Therefore, under the assumption that the importance of a preload scanner is
overrated with respect to <img> images, I would like to propose the
following alternative approach:



PROPOSAL

For responsive images, we forsake any (real or imagined) benefits from the
preload-scanner, and just give the browser all the information we have on
image candidates to let it figure things out for itself. Example:

<img src="default.jpg" srcalt="default.jpg 200x150 10kB, small.jpg 100x75
4kB, large.jpg 400x300 16kB"/>

With this information, the browser can decide (after initial layout), based
on display properties and user preferences about quality, loading-time, and
bandwidth conservation, which image best suits its needs.

The exact syntax is of course open to discussion: I just wanted to put
forward the general idea.

Pros:
* very easy to write and understand
* very compact
* avoids complexity
* very good opportunity for quality/speed/bandwidth trade-offs
* doesn't turn HTML ugly (at least far less than other proposals)
* no preload scanner

Cons (maybe):
* no preload scanner

There are some interesting variations possible on this approach. In the
following example, only the alternative sources are enumerated, but not
their properties:

<img src="default.jpg" srcalt="small.jpg large.jpg"/>

In this case, the browser could make a series of pipelined HEAD requests on
these sources, and the server would report the size (through
Content-Length) and image dimensions (through a specialised response
header) for each image. This would eliminate the dependency of HTML-source
on properties of these images, or knowledge of them during page generation,
which some might find preferable.



Thank you for reading.

Josh