[whatwg] Adding features needed for WebGL to ImageBitmap

[Rik Cabanier]

On Wed, Jun 19, 2013 at 3:24 PM, Gregg Tavares <gman at google.com> wrote:

>
>
>
> On Wed, Jun 19, 2013 at 3:13 PM, Rik Cabanier <cabanier at gmail.com> wrote:
>
>>
>> On Wed, Jun 19, 2013 at 2:47 PM, Gregg Tavares <gman at google.com> wrote:
>>
>>> In order for ImageBitmap to be useful for WebGL we need more options
>>>
>>> Specifically
>>>
>>> premultipliedAlpha: true/false (default true)
>>> Nearly all GL games use non-premultipiled alpha textures. So all those
>>> games people want to port to WebGL will require non-premultipied
>>> textures.
>>> Often in games the alpha might not even be used for alpha but rather for
>>> glow maps or specular maps or the other kinds of data.
>>>
>>
>> When would premultipliedAlpha ever be true?
>> 2D Canvas always works with non-premultiplied alpha in its APIs.
>>
>
> AFAIK the canvas API expects all images to be premultiplied. Certainly in
> WebKit and Blink images used in the canvas and displayed in the img tag are
> loaded premultiplied which is why we had to add the option on WebGL since
> we needed those images before they had lost data.
>
>
>>
>>
>>>
>>> flipY: true/false (default false)
>>> Nearly all 3D modeling apps expect the bottom left pixel to be the first
>>> pixel in a texture so many 3D engines flip the textures on load. WebGL
>>> provides this option but it takes time and memory to flip a large image
>>> therefore it would be nice if that flip happened before the callback from
>>> ImageBitmap
>>>
>>
>> Couldn't you just draw upside down?
>>
>
> No, games often animate texture coordinates and other things making it far
> more complicated. There are ways to work around this issue in code yes,
> they often require a ton of work.
>
> Most professional game engines pre-process the textures and flip them
> offline but that doesn't help when you're downloading models off say
> http://sketchup.google.com/3dwarehouse/
>
>
>>
>>
>>>
>>> colorspaceConversion: true/false (default true)
>>> Some browsers apply color space conversion to match monitor settings.
>>> That's fine for images with color but WebGL apps often load heightmaps,
>>> normalmaps, lightmaps, global illumination maps and many other kinds of
>>> data through images. If the browser applies a colorspace conversion the
>>> data is not longer suitable for it's intended purpose therefore many
>>> WebGL
>>> apps turn off color conversions. As it is now, when an image is uploaded
>>> to
>>> WebGL, if colorspace conversion is
>>> off<
>>> http://www.khronos.org/registry/webgl/specs/latest/#PIXEL_STORAGE_PARAMETERS
>>> >,
>>
>>
OK, I see what you're trying to accomplish. You want to pass
non-premultiplied data and color converted (from sRGB to monitor) pixels to
WebGL
I think your API looks fine, except that the defaults should all be false...


>
>>> WebGL has to synchronously re-decode the image. It would be nice if
>>> ImageBitmap could handle this case so it can decode the image without
>>> applying any colorspace manipulations.
>>>
>>
>> Shouldn't the color space conversion happen when the final canvas bit are
>> blitted to the screen?
>> It seems like you should never do it during compositing since you could
>> get double conversions.
>>
>
> Maybe but that's not relevant to ImageBitmap is it? The point here is we
> want the ImageBitmap to give us the data in the format we need. It's
> designed to be async so it can do this but it we can't prevent it from
> applying colorspace conversions.
> Some browsers did that for regular img tags which pointed out the original
> problem. The browser can't guess how the image is going to be used and
> since it's a lot of work to decode an image you'd like to be able to tell
> it what you really need before it guesses wrong.
>
>
>>
>>
>>>
>>> If it was up to me I'd make createImageBitmap take on object with
>>> properties so that new options can be added later as in
>>>
>>>     createImageBitmap(src, callback, {
>>>        premultipliedAlpha: false,
>>>        colorspaceConversion: false,
>>>        x: 123,
>>>     });
>>>
>>> But I'm not familiar if there is a common way to make APIs take a options
>>> like this except for the XHR way which is to create a request, set
>>> properties on the request, and finally execute the request.
>>
>>
>>  Like Tab said, it's fine to implement it that way.
>> Be aware that you might have to do some work in your idl compiler since I
>> *think* there are no other APIs (in Blink) that take a dictionary.
>>
>>
>