[whatwg] Referer header sent with <a ping>?

[Julian Reschke]

Ian Hickson wrote:
> 
> This e-mail is a response to all the recent ping="" feedback.
> 
> I carefully took into account all the feedback mentioned below, as well as 
> past feedback and comments outside these mailing lists. In response to 
> these comments, I've made Referer: be #PING for all pings, and added 

A value that MUST NOT be used, according to RFC2616:

"The URI MUST NOT include a fragment." -- 
<http://greenbytes.de/tech/webdav/rfc2616.html#rfc.section.14.36.p.5>

> Ping-From and Ping-To to all pings that use the same origin or that aren't 
> encrypted. I know this doesn't fulfill everyone's desires, but 
> unfortunately the requests from people here were in some cases mutually 
> exclusive and so it was not possible to please everyone. If I disagreed 
> with something you said, either explicitly below or implicitly in the way 
> the specification was changed, please do not take that as meaning your 
> feedback was not welcome or not considered.

That makes it sound a bit like a working group decision was made, which 
I think is not the case.

> On Fri, 1 Feb 2008, Kornel Lesinski wrote:
>> Theoretically it does, but I haven't seen UA nor application that 
>> supports it. Anyway, it could be made an URI with useless scheme, like 
>> about:ping.
> 
> That could work, though really here we're trying to do something that's 
> not a URI at all.

...in which case you shouldn't use the Referer header:

"The Referer[sic] request-header field allows the client to specify, for 
the server's benefit, the address (URI) of the resource from which the 
Request-URI was obtained..." -- 
<http://greenbytes.de/tech/webdav/rfc2616.html#rfc.section.14.36.p.1>

> On Sat, 2 Feb 2008, Julian Reschke wrote:
>> How is that better compared not to send the Referer header at all?
> 
> As noted by others, no Referer header is treated as a local Referer 
> header, which makes it susceptible to XSRF.

Not sure what a "local" referer header would be.

I'm not sure which mail you are referring to (pointer, please), is it 
<http://lists.w3.org/Archives/Public/public-html/2008Feb/0014.html>?

That states that a bogus (sic) Referer header could be used to filter 
ping requests. There are many other ways to do that, so I don't accept 
it as a valid argument.

>>> The point of it all is to make abuse of ping for CSRF harder, so 
>>> standard body formats like www-form-urlencoded or XML are undesirable, 
>>> but non-standard formats will require acceess to raw post data and 
>>> custom parsers, which isn't as easy as reading headers.
>> So define a custom format.
> 
> As noted by Kornel, this is a significantly higher barrier to entry than 
> just using headers.

Yes, sometimes doing something properly is more work than just hacking 
around.

>>> Another advantage of headers is that Apache could log pings without 
>>> help of any scripts or non-standard modules - LogFormat directive 
>>> allows logging of arbitrary headers.
>> I'm not sure how this is relevant...
> 
> It seems extremely relevant, as it enables cheap server-side use instead 
> of requiring heavy lifting for the author.

For the author?

It seems the additional work would be for the implementor of the web 
server, implementing a module for ping tracking. I'm not sure why that's 
considered a major issue.

> On Sat, 2 Feb 2008, Kornel Lesinski wrote:
>> Special Content-Type might work equally well -- it can be detected by 
>> tools scanning headers only, and should prevent applications from 
>> accepting unexpected POST.
> 
> Sadly I fear most sites don't check the Content-Type headers.

If you fear vulnerabilities because of servers misinterpreting POST, 
than this would be a good argument for not using POST, it seems.

> It's not a matter of abuse. We need to do a POST request, since that is 
> the most appropriate method for this case (we don't want to add a PING 

There was no consensus for that.

> method, just like we wouldn't want to add a LOGIN method, a SEARCH method, 
> a POSTEMAIL method, a SAVEPREFERENCES method, etc). However, there is a 

Ironically, there is a SEARCH method, although only used in WebDAV land.

As a matter of fact, PING *has* been suggested.

> minor risk with doing a POST request in a new way, which is that 
> XSS blacklisting code wouldn't stop the new feature. This is a problem 
> whether the Referer header is absent, or whether it contains a valid 
> value. Hence, we have an invalid value. This seems emminently pragmatic, 
> and not at all something I would classify as abuse.

Ok, we'll continue to disagree on that. But please do not claim that 
there was some kind of consensus for it.

> On Sat, 2 Feb 2008, Julian Reschke wrote:
>>> I see two ways forward here. One would be to redefine Referer to 
>>> remove the relative URI thing, since, to my knowledge at least, nobody 
>>> uses it.
>> That's IMHO not sufficient reason to remove it. It's not broken.
> 
> Lack of use is a reason to remove a feature according to IETF process, 
> actually. It's even been used as a reason for HTTP -- Link: headers were 
> removed from HTTP a while back for this very reason. (I've since worked 
> hard to get browsers to implement it, and we can probably get it back in 
> at this point. But that's another story.)
> 
> (RFC 2026, section 4.1.2.)

The reason it was removed was because HTTP/1.1 was advancing on the 
standards track, which requires evidence of things being in use. Next 
time this happens with HTTP/1.1 (draft -> full), questions like these 
will be asked again.

And yes, I agree that Link is useful and needs to get revived in some 
way. People do not seem to understand that it's still part of HTTP.

>>> The other is that we can define the magic value to be "#PING" instead, 
>>> since that's a non-conforming Referer value right now.
>> It's not conforming, so are you suggesting to use a non-conforming 
>> value?
> 
> Right; by using a previously illegal value, we can ensure that nobody is 
> relying on it already. (I presume your objection to using a relative value 
> "ping" is that it could be misinterpreted by existing implementations due 
> to its current definition.)

The value is illegal until you make it legal. To make that, you'll have 
to change the definition in HTTP/1.1.

The risk is that recipients that expect a compliant Referer header will 
break in some way.

>> Could you please state what problem you are trying to solve, and why it 
>> needs to involve the Referer header?
> 
> I believe the problem has been stated a number of times in this thread 
> already. I refer you to dolphinling's original e-mail.

Pointer, please.

> Not at all; the risks are easily mitigated.

Apparently we disagree on this.

> On Sat, 2 Feb 2008, Adam Barth wrote:
>> Perhaps this has been suggested before, but another option is to use a 
>> new verb, such as PING, instead of GET when making the request. Servers 
>> unaware of the ping attribute will likely ignore this verb, mitigating 
>> the request-forgery attack vector.
> 
> Sadly this is more than likely to cause problems with transparent proxies. 

Evidence, please.

> It also seems silly to invent a new method, as noted earlier in this 
> message. POST is the appropriate method here.

Disagreed.

I do not believe a new method is needed (as I still think that GET/HEAD 
would be fine); but it certainly would be better than POST because of 
all the issues we're discussing here.

> On Sat, 2 Feb 2008, dolphinling wrote:
>> If (X-)Ping-From/Ping-To are present, why is a referer needed at all? 
>> I'd say just leave it out. If not, #PING works for me.
> 
> We can't remove it since then it would be treated as a local request.

Please explain "local request".

The absence of a Referer header means that no Referer information is 
available, not that it's a local request. So please clarify.

> On Sun, 3 Feb 2008, Julian Reschke wrote:
>> I think it's a strange way to design a network protocol. When ping 
>> requests in access logs are a problem, there are many ways not to have 
>> it in the first place (not using HTTP, for instance) or by using a 
>> different HTTP verb. Forcing something into a header it's not designed 
>> for is even worse, in particular if the only reason given is to make it 
>> easier to parse logs.
> 
> It's not clear exactly what your concern is here.

My concern is that you're raking ease of implementation higher than 
consistency of specifications.

BR, Julian