[whatwg] Referer header sent with <a ping>?

Julian Reschke julian.reschke at gmx.de
Sat Feb 9 02:44:48 PST 2008


Ian Hickson wrote:
> 
> This e-mail is a response to all the recent ping="" feedback.
> 
> I carefully took into account all the feedback mentioned below, as well as 
> past feedback and comments outside these mailing lists. In response to 
> these comments, I've made Referer: be #PING for all pings, and added 

A value that MUST NOT be used, according to RFC2616:

"The URI MUST NOT include a fragment." -- 
<http://greenbytes.de/tech/webdav/rfc2616.html#rfc.section.14.36.p.5>

> Ping-From and Ping-To to all pings that use the same origin or that aren't 
> encrypted. I know this doesn't fulfill everyone's desires, but 
> unfortunately the requests from people here were in some cases mutually 
> exclusive and so it was not possible to please everyone. If I disagreed 
> with something you said, either explicitly below or implicitly in the way 
> the specification was changed, please do not take that as meaning your 
> feedback was not welcome or not considered.

That makes it sound a bit like a working group decision was made, which 
I think is not the case.

> On Fri, 1 Feb 2008, Kornel Lesinski wrote:
>> Theoretically it does, but I haven't seen UA nor application that 
>> supports it. Anyway, it could be made an URI with useless scheme, like 
>> about:ping.
> 
> That could work, though really here we're trying to do something that's 
> not a URI at all.

...in which case you shouldn't use the Referer header:

"The Referer[sic] request-header field allows the client to specify, for 
the server's benefit, the address (URI) of the resource from which the 
Request-URI was obtained..." -- 
<http://greenbytes.de/tech/webdav/rfc2616.html#rfc.section.14.36.p.1>

> On Sat, 2 Feb 2008, Julian Reschke wrote:
>> How is that better compared not to send the Referer header at all?
> 
> As noted by others, no Referer header is treated as a local Referer 
> header, which makes it susceptible to XSRF.

Not sure what a "local" referer header would be.

I'm not sure which mail you are referring to (pointer, please), is it 
<http://lists.w3.org/Archives/Public/public-html/2008Feb/0014.html>?

That states that a bogus (sic) Referer header could be used to filter 
ping requests. There are many other ways to do that, so I don't accept 
it as a valid argument.

>>> The point of it all is to make abuse of ping for CSRF harder, so 
>>> standard body formats like www-form-urlencoded or XML are undesirable, 
>>> but non-standard formats will require acceess to raw post data and 
>>> custom parsers, which isn't as easy as reading headers.
>> So define a custom format.
> 
> As noted by Kornel, this is a significantly higher barrier to entry than 
> just using headers.

Yes, sometimes doing something properly is more work than just hacking 
around.

>>> Another advantage of headers is that Apache could log pings without 
>>> help of any scripts or non-standard modules - LogFormat directive 
>>> allows logging of arbitrary headers.
>> I'm not sure how this is relevant...
> 
> It seems extremely relevant, as it enables cheap server-side use instead 
> of requiring heavy lifting for the author.

For the author?

It seems the additional work would be for the implementor of the web 
server, implementing a module for ping tracking. I'm not sure why that's 
considered a major issue.

> On Sat, 2 Feb 2008, Kornel Lesinski wrote:
>> Special Content-Type might work equally well -- it can be detected by 
>> tools scanning headers only, and should prevent applications from 
>> accepting unexpected POST.
> 
> Sadly I fear most sites don't check the Content-Type headers.

If you fear vulnerabilities because of servers misinterpreting POST, 
than this would be a good argument for not using POST, it seems.

> It's not a matter of abuse. We need to do a POST request, since that is 
> the most appropriate method for this case (we don't want to add a PING 

There was no consensus for that.

> method, just like we wouldn't want to add a LOGIN method, a SEARCH method, 
> a POSTEMAIL method, a SAVEPREFERENCES method, etc). However, there is a 

Ironically, there is a SEARCH method, although only used in WebDAV land.

As a matter of fact, PING *has* been suggested.

> minor risk with doing a POST request in a new way, which is that 
> XSS blacklisting code wouldn't stop the new feature. This is a problem 
> whether the Referer header is absent, or whether it contains a valid 
> value. Hence, we have an invalid value. This seems emminently pragmatic, 
> and not at all something I would classify as abuse.

Ok, we'll continue to disagree on that. But please do not claim that 
there was some kind of consensus for it.

> On Sat, 2 Feb 2008, Julian Reschke wrote:
>>> I see two ways forward here. One would be to redefine Referer to 
>>> remove the relative URI thing, since, to my knowledge at least, nobody 
>>> uses it.
>> That's IMHO not sufficient reason to remove it. It's not broken.
> 
> Lack of use is a reason to remove a feature according to IETF process, 
> actually. It's even been used as a reason for HTTP -- Link: headers were 
> removed from HTTP a while back for this very reason. (I've since worked 
> hard to get browsers to implement it, and we can probably get it back in 
> at this point. But that's another story.)
> 
> (RFC 2026, section 4.1.2.)

The reason it was removed was because HTTP/1.1 was advancing on the 
standards track, which requires evidence of things being in use. Next 
time this happens with HTTP/1.1 (draft -> full), questions like these 
will be asked again.

And yes, I agree that Link is useful and needs to get revived in some 
way. People do not seem to understand that it's still part of HTTP.

>>> The other is that we can define the magic value to be "#PING" instead, 
>>> since that's a non-conforming Referer value right now.
>> It's not conforming, so are you suggesting to use a non-conforming 
>> value?
> 
> Right; by using a previously illegal value, we can ensure that nobody is 
> relying on it already. (I presume your objection to using a relative value 
> "ping" is that it could be misinterpreted by existing implementations due 
> to its current definition.)

The value is illegal until you make it legal. To make that, you'll have 
to change the definition in HTTP/1.1.

The risk is that recipients that expect a compliant Referer header will 
break in some way.

>> Could you please state what problem you are trying to solve, and why it 
>> needs to involve the Referer header?
> 
> I believe the problem has been stated a number of times in this thread 
> already. I refer you to dolphinling's original e-mail.

Pointer, please.

> Not at all; the risks are easily mitigated.

Apparently we disagree on this.

> On Sat, 2 Feb 2008, Adam Barth wrote:
>> Perhaps this has been suggested before, but another option is to use a 
>> new verb, such as PING, instead of GET when making the request. Servers 
>> unaware of the ping attribute will likely ignore this verb, mitigating 
>> the request-forgery attack vector.
> 
> Sadly this is more than likely to cause problems with transparent proxies. 

Evidence, please.

> It also seems silly to invent a new method, as noted earlier in this 
> message. POST is the appropriate method here.

Disagreed.

I do not believe a new method is needed (as I still think that GET/HEAD 
would be fine); but it certainly would be better than POST because of 
all the issues we're discussing here.

> On Sat, 2 Feb 2008, dolphinling wrote:
>> If (X-)Ping-From/Ping-To are present, why is a referer needed at all? 
>> I'd say just leave it out. If not, #PING works for me.
> 
> We can't remove it since then it would be treated as a local request.

Please explain "local request".

The absence of a Referer header means that no Referer information is 
available, not that it's a local request. So please clarify.

> On Sun, 3 Feb 2008, Julian Reschke wrote:
>> I think it's a strange way to design a network protocol. When ping 
>> requests in access logs are a problem, there are many ways not to have 
>> it in the first place (not using HTTP, for instance) or by using a 
>> different HTTP verb. Forcing something into a header it's not designed 
>> for is even worse, in particular if the only reason given is to make it 
>> easier to parse logs.
> 
> It's not clear exactly what your concern is here.

My concern is that you're raking ease of implementation higher than 
consistency of specifications.

BR, Julian



More information about the whatwg mailing list