[whatwg] Adding features needed for WebGL to ImageBitmap

[Gregg Tavares]

On Wed, Jun 19, 2013 at 3:13 PM, Rik Cabanier <cabanier at gmail.com> wrote:

>
> On Wed, Jun 19, 2013 at 2:47 PM, Gregg Tavares <gman at google.com> wrote:
>
>> In order for ImageBitmap to be useful for WebGL we need more options
>>
>> Specifically
>>
>> premultipliedAlpha: true/false (default true)
>> Nearly all GL games use non-premultipiled alpha textures. So all those
>> games people want to port to WebGL will require non-premultipied textures.
>> Often in games the alpha might not even be used for alpha but rather for
>> glow maps or specular maps or the other kinds of data.
>>
>
> When would premultipliedAlpha ever be true?
> 2D Canvas always works with non-premultiplied alpha in its APIs.
>

AFAIK the canvas API expects all images to be premultiplied. Certainly in
WebKit and Blink images used in the canvas and displayed in the img tag are
loaded premultiplied which is why we had to add the option on WebGL since
we needed those images before they had lost data.


>
>
>>
>> flipY: true/false (default false)
>> Nearly all 3D modeling apps expect the bottom left pixel to be the first
>> pixel in a texture so many 3D engines flip the textures on load. WebGL
>> provides this option but it takes time and memory to flip a large image
>> therefore it would be nice if that flip happened before the callback from
>> ImageBitmap
>>
>
> Couldn't you just draw upside down?
>

No, games often animate texture coordinates and other things making it far
more complicated. There are ways to work around this issue in code yes,
they often require a ton of work.

Most professional game engines pre-process the textures and flip them
offline but that doesn't help when you're downloading models off say
http://sketchup.google.com/3dwarehouse/


>
>
>>
>> colorspaceConversion: true/false (default true)
>> Some browsers apply color space conversion to match monitor settings.
>> That's fine for images with color but WebGL apps often load heightmaps,
>> normalmaps, lightmaps, global illumination maps and many other kinds of
>> data through images. If the browser applies a colorspace conversion the
>> data is not longer suitable for it's intended purpose therefore many WebGL
>> apps turn off color conversions. As it is now, when an image is uploaded
>> to
>> WebGL, if colorspace conversion is
>> off<
>> http://www.khronos.org/registry/webgl/specs/latest/#PIXEL_STORAGE_PARAMETERS
>> >,
>>
>> WebGL has to synchronously re-decode the image. It would be nice if
>> ImageBitmap could handle this case so it can decode the image without
>> applying any colorspace manipulations.
>>
>
> Shouldn't the color space conversion happen when the final canvas bit are
> blitted to the screen?
> It seems like you should never do it during compositing since you could
> get double conversions.
>

Maybe but that's not relevant to ImageBitmap is it? The point here is we
want the ImageBitmap to give us the data in the format we need. It's
designed to be async so it can do this but it we can't prevent it from
applying colorspace conversions.
Some browsers did that for regular img tags which pointed out the original
problem. The browser can't guess how the image is going to be used and
since it's a lot of work to decode an image you'd like to be able to tell
it what you really need before it guesses wrong.


>
>
>>
>> If it was up to me I'd make createImageBitmap take on object with
>> properties so that new options can be added later as in
>>
>>     createImageBitmap(src, callback, {
>>        premultipliedAlpha: false,
>>        colorspaceConversion: false,
>>        x: 123,
>>     });
>>
>> But I'm not familiar if there is a common way to make APIs take a options
>> like this except for the XHR way which is to create a request, set
>> properties on the request, and finally execute the request.
>
>
>  Like Tab said, it's fine to implement it that way.
> Be aware that you might have to do some work in your idl compiler since I
> *think* there are no other APIs (in Blink) that take a dictionary.
>
>