[whatwg] img srcset / css image-set issues

[Fred Andrews]

...

> But that's exactly my point! The w/h descriptors are sufficient *only* for
> art direction, where you want to swap between completely different images
> (or at least different crops of the same image), based on viewport size, to
> match other changes occurring in your responsive design as the viewport
> size changes (and they do a fine job at art direction).
> 
> They are *not* suitable (even combined with x descriptors) as a general
> purpose mechanism for dealing with flexibly-sized images, where you need to
> match the number of device pixels taken up by the image to the number of
> image pixels in the image you load (while allowing the browser to adjust
> this based on bandwidth constraints etc), since doing that properly
> *requires* additional information, i.e. knowing the intrinsic size (or at
> least intrinsic width or height) of the image before choosing which to
> download.

Agreed.  The width and height of the images is what is needed.  The browser needs to fill pixels so needs to know the sizes of the available images in pixels.

Forget the x scale descriptions and all the other proposed extensions and just include a set of images with their pixel height and width.  Forget dpi etc, its just not needed for this.

Consider adding a 'selected' flag to give JS control of the selection.   If a particular image is 'selected' then it is the default choice. If no image is 'selected' then the browser can use a default algorithm to make a choice based on the target image box size and user choices relating to image quality versus bandwidth tradeoffs.  This allows JS to override the browser default algorithm.  For example a web gallery can offer visitors a 'quality' choice and use this to flag an appropriate selected image.

Image cropping and centering is something best left to JS algorithms.  Fluid cropping and centering may need an algorithm that takes into account particular areas of interest and focus within an image, and user choices which would be better handled within the webpage and not at the browser level.  A web gallery may allow visitors to choose to scale versus crop images and to pan cropped image.

The fact that many people disable JS because they feel it is abused is a fault in the design and should be corrected elsewhere - simple data-flow calculations could handle many of these algorithms safely and still give people confidence to enable them.

Regarding bandwidth choices, is there are proposal that the image sizes in bytes could be declared in the HTML, or is there an expectation that the browser will query the server for the size of each image, or would web browser just use a heuristic to estimate the image byte size scaling based on the image dimensions?

...
> No, it's not fine to just toss in an additional breakpoint. If you do this
> for all important device categories you go back to the monstrous 16-entry
> srcset I showed earlier, which is clearly unsustainable.

Agreed.  MQs do not map well to the task.  The image box size is not just a matter of the media, but depends on the scaled image box size on the webpage.

> The art direction use case can be entirely orthogonal. It should be handled
> with the w/h descriptors as currently specified. What I'm proposing would
> operate after any w/h descriptors have narrowed down the set of allowable
> images, and let the browser choose between the remaining images more
> intelligently in the case of flexible-size images, where currently the
> browser has no idea which to use.

Yes, this sounds like a better approach.  It there are proposal taking this view into account?

cheers
Fred