[whatwg] Drag-and-drop feedback

[Charles Pritchard]

On 11/1/2010 6:03 PM, Ian Hickson wrote:
> On Mon, 22 Feb 2010, Ian Hickson wrote:
>> On Thu, 4 Feb 2010, Ian Hickson wrote:
>>> On Sat, 23 Jan 2010, Eduard Pascual wrote:
>>>> Would it be possible to provide a list of "drag items" (to call them
>>>> somehow) instead of, or in addition to, the current info provided by
>>>> the DataTransfer object?
>>> That's a pretty good idea. I think we should probably do this when we
>>> add more types to the DataTransfer object.
>> Some engineers at Google discussed this a bit and came up with the
>> following proposal:
>>
>>     dataTransfer.items = DataTransferItems
>>
>>      DataTransferItems.length
>>                       .getItem(n) = DataTransferItem
>>                       .add(stringData, type)
>>                       .add(blobData)
>>                       .add(fileData)
>>                       .add(dataTransferItem)
>>                       .clear()
>>
>>      DataTransferItem.kind = 'string', 'file', 'blob', ...
>>                      .type = MIME type
>>                      .binary = boolean
>>                      .getTextData(function callback (data)) - throws if binary is true
>>                      .getBlob() - returns File or Blob
>>
>> When we add promises later, this can easily be extended to support that
>> as well (basically, just by adding a new add() method for the promise
>> case).
> I've added a simple version of this (no Blob support, and no way to add a
> dataTransferItem, but otherwise more or less the same).
>

How close are we to adding promises?

.add(fileEntry,callOnTransfer);

This would allow the script to write to the contents of a file entry
upon request, instead of doing it ahead of time.

callOnTransfer = function( dataTransferPromiseEvent ) {
     var dest = dataTransferPromiseEvent.fileEntry;
     /// file writer API....
     dataTransferPromiseEvent.flush();
     )
}

The fileEntry would be written ahead of time, either as a blank file,
or it'd be an already existing file.

>> DataTransfer.addFile(fileData);
> It's now DataTransfer.items.add(fileData);
>
>
>> When interacting with non-DOM apps or pages, some platforms can't easily
>> convert arbitrary MIME types to native data transfer types for
>> copy/paste or DnD. For this reason, I think the spec should explicitly
>> list MIME types for which UAs should handle the conversion to native
>> data transfer types. A couple that come to mind: text/plain,
>> text/uri-list, text/rtf, application/rtf, text/html, text/xml,
>> image/png, and image/svg+xml. UAs can make a best-effort attempt to
>> convert the other types, but it won't be guaranteed that they will be
>> there for interaction with non-DOM applications.
> I'm not sure what this means exactly. Could you elaborate?

I don't think these need to be "converted" by a UA -- the application which
receives the data does that conversion on its own.

This list of transfer types reminds me of all the redundancy that can 
take place in a data transfer.

A sufficiently large XML content file may be transferred in ~4 different 
file formats
for compatibility with the destination.

This is a good use case for "promise"-based data callbacks.


>
> On Mon, 28 Jun 2010, Daniel Cheng wrote:
>> It's pretty common for there to be non-text data in a drag-and-drop
>> operation or copy-and-paste operation. DataTransfer doesn't allow for that
>> currently, since it only sets and returns DOMStrings.
>>
>> It'd be nice if we could extend setData/getData to allow for Blobs. Some
>> random thoughts:
>> 1. Add a bool parameter to setData/getData. If false, treat the data as a
>> DOMString; if true, treat the data as a Blob.
>> 2. Add an encoding parameter to setData/getData. Encoding can be a string
>> value naming a text encoding like UTF-8 or ISO-8859-1, or it can be the
>> string value "binary". If encoding names a text encoding, the UA will
>> transcode the requested data into/from a DOMString. Otherwise, if the
>> encoding value is "binary", the UA will treat data as a Blob.
>> 3. Create new DataTransfer functions instead of overloading them, e.g.
>> setDataBlob, getDataBlob.
>>
>> I'm not sure which one is the preferred approach. It seems like it'd be nice
>> to have native support for whatever text encodings the browser understands,
>> but it seems complicated and I'm not sure it's necessary. Thoughts?
> The new DataTransferItems feature could let you do this using
> dataTransfer.items.add(blob), if we added that. Right now you can add just
> text strings and File objects. We could also one day support arbitrary JS
> objects (with the structured clone stuff) the same way, too.
>
>
>> Also, if I wanted to go ahead and implement a prototype in WebKit, should I
>> prefix it with a UA-specific string, e.g. webkitSetDataBlob?
> Yes.
This relates to my comments later in this e-mail -- About allowing 
.add(fileEntry,callback)
as a means for a "promise-based" download.

>
>> * I am worried about the effectAllowed attribute. Needless to say that
>> the number of value is exponencially proportional to the number of
>> possible values for the dropEffect attribute. Wouldn't it be better to
>> have a linear number of booleans ?
> Yes, but it's about 10 years too late to change that.

This section should (and is?) be kept up with WAI-ARIA.

The concept here, of effectAllowed, is semantic, much like HTML.
If your use-case doesn't fit the minimal vocabulary, you can use your 
own cursor and other visualizations.
Even in that case, you should still include some amount of semantic data 
about the possible drop event.


> On Thu, 26 Aug 2010, Charles Pritchard wrote:
>> On 8/25/2010 2:02 PM, Ian Hickson wrote:
>>> On Mon, 2 Aug 2010, Charles Pritchard wrote:
>>>>> [ UAs can use<input type=file>   to let the user enter remote URLs ]
>>>> When a user through selection, click+drag or manual entry of a URL
>>>> should the browser still submit an Origin request header? It seems
>>>> that CORS doesn't come into effect here -- but at the same time,
>>>> it'd be handy for logging purposes and added security.
>>> I don't think there'd be an origin, but that's rather up to the user
>>> agent. (In this case it's acting on behalf of the user, not the page,
>>> so I don't think it makes sense to give the page's origin.)
>> Sounds like an implementer would not include a Referer header, either.
> Possibly.
At this point, <input type="file"> with a remote URL is something left 
up to the operating system.

I personally like how it works, but I don't like that it's undefined in 
the standards, and left up to the OS --
it takes more steps on OS X than on Windows: in OSX you must first save 
the target to the desktop.
On windows you may enter a URL, which it then fetches into a temporary 
file directory.

Is this an area you'd like to flesh out, or leave undefined? 
<intentionally left blank>

>> Continuing on with tweaking URLs to work with with the File API:
>>
>> Chrome has gone ahead with their setData proposal, enhancing the
>> event.dataTransfer object so that users may drag a file from within the
>> browser onto their desktop.
>>
>> The extension uses setData with a key of DownloadURL and a value
>> including a mime type, file descriptor and URI.
>>
>> I'd like this interface to work within ondrop; if getData(DownloadURL)
>> is set, then a FileList would be returned in event.dataTransfer.files,
>> much like it is when users drag files from their desktop into the
>> browser.
>>
>> This would of course require Origin checks; whereas dragging onto the
>> desktop does not require an Origin check.
> I'm not quite sure I follow what you are proposing. However, in a future
> version of this API we should definitely add a "promise"-like feature that
> lets you specify drag data without having it already downloaded, so that
> when the user drops the data somewhere, the browser can then ask the JS
> for the data. I'm not sure using setData('DownloadURL') is a good way to
> do it; that just seems to add more hacks to an already pretty hacky API.
Perhaps passing a FileEntry and/or DirectoryEntry, with function 
pointers, would work?

The file would be completely written onto the user's temporary / 
permanent filesystem when the callback is used
to signal to the browser that the file is ready to be transferred to the 
drop destination.

Constraints are already be defined by the FileSystem API, and the 
process would work with Web Workers.

> On Mon, 18 Oct 2010, Daniel Cheng wrote:
>> I've been working on better support of arbitrary MIME types in WebKit
>> for some time, and I had some implementation questions. In the past, UAs
>> seem to have gone out of their way to make sure full filesystem paths
>> aren't exposed to the Javascript (e.g. in the file input control). When
>> I did the work for WebKit, I implemented the web dragging clipboard as a
>> simple reflection of the native dragging clipboard.
>>
>> However, this leads to issues like file system paths being exposed
>> through properties like "x-special/gnome-icon-list" or even
>> "text/plain". What is the expected behavior here? Mirroring the native
>> dragging clipboard allows for a much richer interaction with the system,
>> but I'm not sure if we need to go out of our way to try to scrub all
>> paths from the drag. After all, if you're dropping the file on the page,
>> you're already exposing the contents of the file, which are probably
>> much more interesting than just the path. Thoughts?
Set the behavior as a second "File", which the drop object receives.
Existing sites should already be looping through file mime types on drop,
so this shouldn't break existing standards.

For instance, when I loop through ondrop files, I check for content type.

This is not going to help a use-case for "exposing" the clipboard 
contents before a drop,
as that should be handled through other routes (such as browser extensions).
It would help transfer meta-data to the scripting environment, and/or host.

Use case: An OS may store additional meta-data in file properties, such 
as a title,
or comments. When a user drags that file into a web app, they'd like 
those properties exposed.

This is something up to the individual implementer, and the environments 
they target.
Security is out of scope here, but implementers should be reminded that 
sensitive data should be exposed with great care.