[whatwg] Implementation complexity with elements vs an attribute (responsive images)

[David Goss]

On 13 May 2012 10:14, James Graham <jgraham at opera.com> wrote:
>
>
> On Sun, 13 May 2012, David Goss wrote:
>
>> A common sentiment here seems to be that the two proposed responsive
>> image solutions solve two different use cases:
>>
>> - <img srcset> for serving different resolutions of a content image
>> (for bandwidth and dpi)
>> - <picture> for serving different versions of a content image (for art
>> direction)
>>
>> ...and that neither solution can deal with both issues. I disagree. I
>> would describe it as a single, broad use case:
>> Serving different sources of an image based on properties of the
>> client. These properties could include:
>> - Viewport width/height
>> - Containing element width/height
>> - Device orientation
>> - Colour capability
>> - Old-fashioned media type (screen/print)
>> - Connection speed
>> - Pixel density
>> - Things we haven't thought about/aren't an issue yet
>
>
> Which of hese things are actual requirements that people need to meet and
> which are hypothetical? For example I think it is uncontroversial that
> viewport width/height is a real requirement. On the other hand, I have never
> heard of a site that switches assets based on display colour capability. Can
> you point to sites actually switching assets based on each property you
> listed?

Before going on, I'll note that it's difficult to find many sites
actually doing these things right now, the main reason being that the
costs (wasted downloads, very hacky methods) outweigh the benefits
much of the time and stop it being worthwhile to do.

Viewport width/height
As you said, viewport width/height is the obvious one. The Boston
Globe (http://bostonglobe.com/) is an example of a site doing this now
(and using a necessarily hacky method), as described in Mat's article
(http://www.alistapart.com/articles/responsive-images-how-they-almost-worked-and-what-we-need/).

Containing element width/height:
This is kind of hypothetical, at least right now. But authors would
like to be able to apply different CSS (and different image sources)
based on the current context of the target (e.g. a module could be
used multiple times on a site in different places, with various widths
depending on where they are in the layout) - see
http://blog.andyhume.net/responsive-containers/. I appreciate it would
be difficult in terms of syntax and browser implementation. I'm sure
it's no coincidence that no such media query exists yet. But this is
the point: the <picture> syntax allows us to use future media queries
if/when they arrive. As I understand it, the <img srcset> syntax would
have to keep getting extended every time we wanted to test a different
property.

Device orientation
This is essentially a part of viewport width/height, as it effectively
denotes "is the viewport wider than it is tall, or vice versa".
Perhaps I shouldn't have listed it separately.

Pixel density
Higher pixel density displays are getting more common. The new iPad is
a good example. If you have a large, prominent image (especially a
photo) on a web page that takes up the majority of the screen at once,
you want to take advantage of the extra pixel density by serving a
high resolution image, but it'd be irresponsible to just serve an
<img> with the high res source to all users, making them wait longer
for the download even though they can't see the extra quality on their
screen. So, you serve the image at an ordinary resolution by default,
or serve the high res version if the pixel density is over a certain
number. Apple are doing this with a JS method, as documented by Jason
Grigsby (http://blog.cloudfour.com/how-apple-com-will-serve-retina-images-to-new-ipads/),
and in cases where they do serve the high res image, the low res one
is also downloaded. This is one of the problems <picture> solves:
<picture alt="The new iPad">
 <source src="lores.jpg">
 <source src="hires.jpg" media="min-resolution: 300dpi">
 <img src="lores.jpg" alt="The new iPad">
</picture>

Connection speed
As an extension of the iPad example above, it would also be
irresponsible to serve the high res image to users that do have a high
pixel density display but are not on a fast internet connection for
whatever reason. So you might write:
<picture alt="The new iPad">
 <source src="lores.jpg">
 <source src="hires.jpg" media="(min-resolution: 300dpi) and
(min-connection-speed: 1mbps)">
 <img src="lores.jpg" alt="The new iPad">
</picture>

>
> Also note that there is a great difference in implementation complexity
> between various properties above. For example, viewport width/height is
> rather easy to work with because one can assume it won't change between
> prefetching and layout, so one can prefetch the right asset. On the other
> hand switching based on containing element width/height requires layout to
> happen before the right asset can be selected, so it has to be loaded late.
> This will significantly decrease the perceived responsiveness of the site.
>
> Other properties like connection speed are very difficult to work with
> because they can have high temporal variability e.g. due to sharing of one
> connection by many consumers, due to temporary environmental conditions
> (train goes into a tunnel) or due to switching transports (wifi to 3G, for
> example). My suspicion is that trying to write a solution for switching
> based on connection speed would lead to people getting the "wrong" assets
> much of the time.

So, if someone's connection speed changes dramatically when the image
request has already started, they'll either get:
1 - a high res image, but slowly
2 - a low res image (relative to what their device can display)

I'm not an expert on internet connections. I'd like to know,
statistically, how likely it is that a user's connection speed will
stay consistent (say +/- 20%) for a ten second period. Does anyone
know of any research like this? If wireless internet connections were
constantly all over the place it would be a concern, but as someone
that uses them all the time it doesn't feel that way to me.

I do appreciate that standardised connection speed testing is going to
be hard. But if and when it comes, it should surely come in the form
of a media query (as well as a JS API) in which case <picture> would
support it automatically.

> Note that these concerns argue, to a certian extent, *against* reusing a
> very general syntax that can express constraints that aren't relevant to the
> actual use cases, or that provide an attractive nuisance that encourages
> developers to do things that can't be implemented in a performant way.

Dumping <picture> as it is won't prevent this, as it's already in the
door in the form of <video>.