[whatwg] Media queries, viewport dimensions, srcset and picture

[Scott Jehl]

With this proposal, could "src" be used on a source element if you don't need the features srcset provides?

Or maybe, would that just be equivalent to srcset with a single source listed?

On May 23, 2012, at 2:56 PM, Matthew Wilcox wrote:

> I think this is a good step forward, however nless I am
> mis-understanding something (entirely possible given how much has been
> going on over this recently) there are problems still...
> 
> Resolution of an image and a device is not a guarantee of suitability
> of an image at a given physical size. This solution seems to take the
> art-directed aspect out of the equation. Just because there's enough
> resolution on the device does not mean that the image itself is
> suitable at the size the device is outputting the image. Without some
> form of other qualifier you end up in a situation where an iPhone 4
> with it's retina display will load an image intended for a device
> twice as big. Now, unless you've got perfect eyesight that image will
> be displayed at the correct resolution, but *half the size* on an
> iPhone 4. That's going to be a problem for some users, especially
> older users.
> 
> There needs to maintain an art-directed aspect, and it doesn't seem
> possible for a device to have the required intelligence to know which
> image is appropriate based solely on the device's pixel density and a
> collection of images at given dimensions.
> 
> -Matt
> 
> On 23 May 2012 18:27, Scott Jehl <scott at scottjehl.com> wrote:
>> I agree with Mat.
>> 
>> This idea appears to elegantly satisfy the goals we originally set out to address in the responsive images group (and ended up subsequently coming to like the picture/source syntax), while also bringing the advantages of the pixel density notation.
>> 
>> 
>> 
>>> 
>>>> 
>>> From: "Florian Rivoal" <florianr at opera.com>
>>> Subject: [whatwg] Media queries, viewport dimensions, srcset and picture
>>> Date: May 23, 2012 11:21:44 AM EDT
>>> To: whatwg at lists.whatwg.org
>>> 
>>> Sorry for not replying to the right message in the thread,
>>> I was previously not subscribed to this list, so I can't.
>>> 
>>> As the editor of the CSS Media Queries spec, I've been asked
>>> to share my opinion about this debate on responsive images, srcset,
>>> media queries, etc.
>>> 
>>> Disclamer: I haven't followed the discussion too closely, and I haven't
>>> really done my homework of reading everything that's been said so far,
>>> so I'm sure I'll miss the point in a bunch of places, but here's my
>>> brain dump, for what it's worth.
>>> 
>>> 
>>> 
>>> The responsive image problem is made significantly harder by the fact that
>>> in most cases, the decision of whether you want to use a high res or low
>>> res image depends both on the resolution of the device and on the network
>>> bandwidth.
>>> 
>>> For a low res device, no matter what the bandwidth is, you're going to
>>> want a low res image (at least as long as you don't take zooming into
>>> account). For a high res device, you want a high res image, unless the
>>> bandwidth would make it to slow to load, in which case you might prefer a
>>> low res image. If you have a variable bandwidth, the last thing you want
>>> to do is to change your mind about which resolution you wanted half way
>>> through due to a change of bandwidth, and discard already downloaded data
>>> because it's the wrong one after all. This situation gets worse as with
>>> high latency.
>>> 
>>> Leaving syntax aside, I think media queries in relation to images can be
>>> useful. Providing a different image based on aspect ratio, color depth,
>>> viewport size, etc can certainly make sense.
>>> 
>>> But I am skeptical that media queries can solve the responsive image
>>> problem well, because I don't see how one could make a bandwidth or
>>> latency media query that doesn't cause already downloaded content to be
>>> discarded when the conditions change, other than by making it not update
>>> to reflect the current conditions, which would make it fairly useless.
>>> 
>>> Given a set of images of different qualities, browsers can have fairly
>>> advanced heuristics to pick the right one. For example switching from low
>>> res to high res halfway through the rendering of the page if the device is
>>> high resolution and the bandwith just went from bad to good and the
>>> latency is low enough. Most authors are not going to bother writing media
>>> queries of that complexity, and as media queries are stateless, even if
>>> they wanted they couldn't be as sophisticated as browsers. This gets even
>>> more true if you consider zooming.
>>> 
>>> Having said all that, I think srcset="foo.jpg 1x, foo2.jpg 2x" is quite
>>> good, because it does indeed provide the browser with a set of images with
>>> different quality, leaving it free to pick the appropriate one.
>>> 
>>> On the other hand, I think that including 600w 400h in there is misguided.
>>> It doesn't help for the problem of picking the right image for the right
>>> resolution/bandwidth combination, but is too crippled to be useful as
>>> media queries meant to serve different images to in different scenarios.
>>> <picture> serves these use cases much better.
>>> 
>>> Here's what I think we should do:
>>> 
>>> 1) simplyfy srcset to only accept the *x qualifier
>>> 
>>> 2) add support for srcset as an attribute of the <source> sub-element of
>>> the <picture> element (in addition to src, or instead of it? I am not
>>> sure).
>>> 
>>> Then you could do stuff like this:
>>> <picture>
>>> <source media="(orientation:landscape)" srcset="long.jpg 1x, long2.jpg 2x">
>>> <source media="(orientation:portrait)" srcset="tall.jpg 1x, tall2.jpg 2x">
>>> <img src="fallback.jpg" />
>>> </picture>
>>> 
>>> Note that it is different from:
>>> 
>>> <picture>
>>> <source media="(orientation:landscape) and (resolution:1dppx)"
>>> src="long.jpg">
>>> <source media="(orientation:landscape) and (resolution:2dppx)"
>>> src="long2.jpg">
>>> <source media="(orientation:portrait) and (resolution:1dppx)"
>>> src="tall.jpg">
>>> <source media="(orientation:portrait) and (resolution:2dppx)"
>>> src="tall2.jpg">
>>> <img src="fallback.jpg" />
>>> </picture>
>>> 
>>> because it allows the browser to be smart about loading the high or low
>>> res image depending on the network conditions. The solution purely based
>>> on media queries doesn't let you do that.
>>> 
>>> One final note: the "1x, 2x" ... solution still has one problem that I
>>> think cannot be properly solved purely in html/css. Even though the
>>> browser can be smart about which image to used based on network
>>> conditions, it still cannot escape the fact that to change its mind half
>>> way through, it will have wasted time downloading the wrong image. It
>>> may be worth it in some cases, but it is still wasteful.
>>> 
>>> I believe the only way out is through an image format that:
>>> - stores multiple resolutions in one file
>>> - stores the higher resolution as incremental information on top of the
>>> low res, so that downloading low res images isn't a waste of time even if
>>> you want the high res.
>>> - is designed so that the browser can stop downloading half way through
>>> the file, if it determines it got sufficiently high resolution given the
>>> environment.
>>> 
>>> This would allow browsers to switch from wanting a high res image to
>>> wanting a low res image without having to restart the download from
>>> scratch, which is really bad, as the main reason for switching from high
>>> to low is a bad network. When the browser is aiming for the high res
>>> image, it still gets some lower quality image to display temporarily while
>>> the higher quality image is being downloaded.
>>> 
>>> I am not enough of an image guy to know if progressive jpeg or webp or
>>> some other existing format has these characteristics.
>>> 
>>> The "1x, 2x..." syntax probably needs to be tweaked to accommodate such
>>> images:
>>> srcset="standard.jpg 1x, progressive.prog 0.1-4x"
>>> 
>>> Even if we don't have an existing format for that, the html syntax should
>>> probably anticipate it, so that soon-to-be implemented UAs don't get
>>> confused when they get served content from the longer term future that
>>> uses this syntax.
>>> 
>>> As an aside, This syntax might work to mix raster images and vector
>>> images: srcset="foo.svg 0.1-0.5x, foo.jpg 1x"
>>> 
>>