[whatwg] Media queries, viewport dimensions, srcset and picture

[Scott Jehl]

I agree with Mat. 

This idea appears to elegantly satisfy the goals we originally set out to address in the responsive images group (and ended up subsequently coming to like the picture/source syntax), while also bringing the advantages of the pixel density notation.



> 
>> 
> From: "Florian Rivoal" <florianr at opera.com>
> Subject: [whatwg] Media queries, viewport dimensions, srcset and picture
> Date: May 23, 2012 11:21:44 AM EDT
> To: whatwg at lists.whatwg.org
> 
> Sorry for not replying to the right message in the thread,
> I was previously not subscribed to this list, so I can't.
> 
> As the editor of the CSS Media Queries spec, I've been asked
> to share my opinion about this debate on responsive images, srcset,
> media queries, etc.
> 
> Disclamer: I haven't followed the discussion too closely, and I haven't
> really done my homework of reading everything that's been said so far,
> so I'm sure I'll miss the point in a bunch of places, but here's my
> brain dump, for what it's worth.
> 
> 
> 
> The responsive image problem is made significantly harder by the fact that
> in most cases, the decision of whether you want to use a high res or low
> res image depends both on the resolution of the device and on the network
> bandwidth.
> 
> For a low res device, no matter what the bandwidth is, you're going to
> want a low res image (at least as long as you don't take zooming into
> account). For a high res device, you want a high res image, unless the
> bandwidth would make it to slow to load, in which case you might prefer a
> low res image. If you have a variable bandwidth, the last thing you want
> to do is to change your mind about which resolution you wanted half way
> through due to a change of bandwidth, and discard already downloaded data
> because it's the wrong one after all. This situation gets worse as with
> high latency.
> 
> Leaving syntax aside, I think media queries in relation to images can be
> useful. Providing a different image based on aspect ratio, color depth,
> viewport size, etc can certainly make sense.
> 
> But I am skeptical that media queries can solve the responsive image
> problem well, because I don't see how one could make a bandwidth or
> latency media query that doesn't cause already downloaded content to be
> discarded when the conditions change, other than by making it not update
> to reflect the current conditions, which would make it fairly useless.
> 
> Given a set of images of different qualities, browsers can have fairly
> advanced heuristics to pick the right one. For example switching from low
> res to high res halfway through the rendering of the page if the device is
> high resolution and the bandwith just went from bad to good and the
> latency is low enough. Most authors are not going to bother writing media
> queries of that complexity, and as media queries are stateless, even if
> they wanted they couldn't be as sophisticated as browsers. This gets even
> more true if you consider zooming.
> 
> Having said all that, I think srcset="foo.jpg 1x, foo2.jpg 2x" is quite
> good, because it does indeed provide the browser with a set of images with
> different quality, leaving it free to pick the appropriate one.
> 
> On the other hand, I think that including 600w 400h in there is misguided.
> It doesn't help for the problem of picking the right image for the right
> resolution/bandwidth combination, but is too crippled to be useful as
> media queries meant to serve different images to in different scenarios.
> <picture> serves these use cases much better.
> 
> Here's what I think we should do:
> 
> 1) simplyfy srcset to only accept the *x qualifier
> 
> 2) add support for srcset as an attribute of the <source> sub-element of
> the <picture> element (in addition to src, or instead of it? I am not
> sure).
> 
> Then you could do stuff like this:
> <picture>
> <source media="(orientation:landscape)" srcset="long.jpg 1x, long2.jpg 2x">
> <source media="(orientation:portrait)" srcset="tall.jpg 1x, tall2.jpg 2x">
> <img src="fallback.jpg" />
> </picture>
> 
> Note that it is different from:
> 
> <picture>
> <source media="(orientation:landscape) and (resolution:1dppx)"
> src="long.jpg">
> <source media="(orientation:landscape) and (resolution:2dppx)"
> src="long2.jpg">
> <source media="(orientation:portrait) and (resolution:1dppx)"
> src="tall.jpg">
> <source media="(orientation:portrait) and (resolution:2dppx)"
> src="tall2.jpg">
> <img src="fallback.jpg" />
> </picture>
> 
> because it allows the browser to be smart about loading the high or low
> res image depending on the network conditions. The solution purely based
> on media queries doesn't let you do that.
> 
> One final note: the "1x, 2x" ... solution still has one problem that I
> think cannot be properly solved purely in html/css. Even though the
> browser can be smart about which image to used based on network
> conditions, it still cannot escape the fact that to change its mind half
> way through, it will have wasted time downloading the wrong image. It
> may be worth it in some cases, but it is still wasteful.
> 
> I believe the only way out is through an image format that:
> - stores multiple resolutions in one file
> - stores the higher resolution as incremental information on top of the
> low res, so that downloading low res images isn't a waste of time even if
> you want the high res.
> - is designed so that the browser can stop downloading half way through
> the file, if it determines it got sufficiently high resolution given the
> environment.
> 
> This would allow browsers to switch from wanting a high res image to
> wanting a low res image without having to restart the download from
> scratch, which is really bad, as the main reason for switching from high
> to low is a bad network. When the browser is aiming for the high res
> image, it still gets some lower quality image to display temporarily while
> the higher quality image is being downloaded.
> 
> I am not enough of an image guy to know if progressive jpeg or webp or
> some other existing format has these characteristics.
> 
> The "1x, 2x..." syntax probably needs to be tweaked to accommodate such
> images:
> srcset="standard.jpg 1x, progressive.prog 0.1-4x"
> 
> Even if we don't have an existing format for that, the html syntax should
> probably anticipate it, so that soon-to-be implemented UAs don't get
> confused when they get served content from the longer term future that
> uses this syntax.
> 
> As an aside, This syntax might work to mix raster images and vector
> images: srcset="foo.svg 0.1-0.5x, foo.jpg 1x"
>