[whatwg] Media queries, viewport dimensions, srcset and picture

[Mathew Marquis]

WHATWG,

The Responsive Images Community Group was recently asked to furnish a formal draft proposal for consideration by the HTML WG. I thought it best to post it here along with some details, where Ian Hickson has mentioned that he’ll be considering this issue again within a few days.

More and more it seems that it’s a waste of effort trying to retrofit the original srcset proposal to cover all the use cases of the original `picture` proposal. As we attempt to do so, the `srcset` msyntax grows more confusing, and shares an increasing amount of overlap with media queries — though with some obvious holes, for example: units other than `px`. To those ends, the Responsive Images Community Group has officially published a draft proposal based on Florian Rivoal’s proposed compromise ( http://lists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2012-May/036160.html ) between the two markup patterns. In this way, `srcset` retains the purpose for which it was originally proposed: a terse, easily-implemented syntax for switching image sources based on the client resolution.

The draft proposal can be found here:
http://www.w3.org/community/respimg/wiki/Picture_Element_Proposal

Discussion with the HTML WG can be found here:
http://lists.w3.org/Archives/Public/public-html/2012Jun/0113.html

## Proposed Markup

<picture alt="Description of image subject.">
<source srcset="small.jpg 1x, small-highres.jpg 2x">
<source media="(min-width: 18em)" srcset="med.jpg 1x, med-highres.jpg 2x">
<source media="(min-width: 45em)" srcset="large.jpg 1x, large-highres.jpg 2x"> 
<img src="small.jpg" alt="Description of image subject.">
</picture>  

The chain of events followed by the above markup pattern are:

1. If the `picture` element is unsupported, the `img` contained therein is shown as fallback markup.
2. If picture is supported, use `media` attributes to determine which source element best suits the user’s viewport, following the same logic as `video`’s specced use of `media` attributes.
3. Once an appropriate source element has been selected, the `srcset` attribute determines which image source is best suited to the user’s screen resolution. If only a single resolution is necessary, the `src` attribute will function as expected, instead.

In terms of selecting a source element, this markup leverages all the strengths of media queries — the syntax created for this very purpose — to handle the “art direction” use case.

However, as has been detailed at length here and elsewhere, `device-pixel-ratio` media queries are poorly suited towards these decisions. As an author, using vendor-prefixed `min-device-pixel-ratio` media queries in the example above would involve a massive amount of text and twice as many source elements. This could get unwieldy for authors very quickly, a concern voiced numerous times in these ongoing discussions. Further, implementation of MQ-based resolution switching is far more difficult on the UA side: a very real concern.

Once we’ve used media queries to determine the most appropriate source element, srcset’s originally intended usage becomes absolutely ideal for our purposes: simply determining the appropriate image source for a user’s resolution.

It’s worth noting that this example is, in fact, the most convoluted this element can ever be. This pattern in no way precludes the use of srcset on an `img` tag for simply preforming resolution switching, nor does it preclude the use of `picture` as originally proposed for the “art direction”/screen size use cases, with `src` in source elements rather than `srcset`.

## Bandwidth

We cannot reliably make assumptions about bandwidth based on client capabilities — a MacBook Pro with a Retina display may be tethered to a 3G phone; a high-resolution mobile device is as likely to be connected to WiFi as it is an EDGE connection.

Based on previous discussion on the list, I think we’re largely in agreement that bandwidth decisions are best left to the browser. It would assume a great deal if authors were to make this decision for the users. It would add a point of failure: we would be taking the bandwidth information afforded us by the browser, and selectively applying that information. Some of us may do it wrong; some of us may find ourselves forced to make a decision as to whether we account for users with limited bandwidth or not. To not account for it would be, in my opinion, untenable — I’ve expressed that elsewhere, in no uncertain terms. The decision to download high vs. standard resolution images should be made by user agents, depending on the bandwidth available — and further, I believe there should be a user settable preference for “always use standard resolution images,” “always use high resolution images,” ”download high resolution as bandwidth permits,” and so on.

In discussing the final markup pattern, we have to consider the above. Somewhere, that markup is going to contain a suggestion, rather than an imperative. I think `srcset` affords us that opportunity: a new syntax _designed_ to be treated as such. I wouldn’t want to introduce that sort of variance to the media query spec — a syntax long established as a set of absolutes.

It seems `srcset` won’t be going anywhere, and that’s not an indictment. There is a time and a place for `srcset`, and I feel that place is resolution switching — as it was originally intended. Our best efforts to bring srcset closer in-line with the originally proposed picture element only stand to leave us with a siloed microsyntax that inconsistently serves the purpose of media queries. With that comes further opportunity for errors by implementors and authors alike — countless new potential points of failure.

## Updated Polyfill

In order to better wrap my head around this pattern, I’ve updated Scott Jehl’s Picturefill to make use of the proposed syntax. The source code is available on GitHub ( https://github.com/Wilto/picturefill-proposal/ ), and I’ve posted a demo ( http://wil.to/picturefill/ ) as well.

Thank you for your ongoing consideration, sincerely. I look forward to finally putting this issue to rest.

Mat Marquis