[whatwg] Referer header sent with <a ping>?
julian.reschke at gmx.de
Sat Feb 9 02:44:48 PST 2008
Ian Hickson wrote:
> This e-mail is a response to all the recent ping="" feedback.
> I carefully took into account all the feedback mentioned below, as well as
> past feedback and comments outside these mailing lists. In response to
> these comments, I've made Referer: be #PING for all pings, and added
A value that MUST NOT be used, according to RFC2616:
"The URI MUST NOT include a fragment." --
> Ping-From and Ping-To to all pings that use the same origin or that aren't
> encrypted. I know this doesn't fulfill everyone's desires, but
> unfortunately the requests from people here were in some cases mutually
> exclusive and so it was not possible to please everyone. If I disagreed
> with something you said, either explicitly below or implicitly in the way
> the specification was changed, please do not take that as meaning your
> feedback was not welcome or not considered.
That makes it sound a bit like a working group decision was made, which
I think is not the case.
> On Fri, 1 Feb 2008, Kornel Lesinski wrote:
>> Theoretically it does, but I haven't seen UA nor application that
>> supports it. Anyway, it could be made an URI with useless scheme, like
> That could work, though really here we're trying to do something that's
> not a URI at all.
...in which case you shouldn't use the Referer header:
"The Referer[sic] request-header field allows the client to specify, for
the server's benefit, the address (URI) of the resource from which the
Request-URI was obtained..." --
> On Sat, 2 Feb 2008, Julian Reschke wrote:
>> How is that better compared not to send the Referer header at all?
> As noted by others, no Referer header is treated as a local Referer
> header, which makes it susceptible to XSRF.
Not sure what a "local" referer header would be.
I'm not sure which mail you are referring to (pointer, please), is it
That states that a bogus (sic) Referer header could be used to filter
ping requests. There are many other ways to do that, so I don't accept
it as a valid argument.
>>> The point of it all is to make abuse of ping for CSRF harder, so
>>> standard body formats like www-form-urlencoded or XML are undesirable,
>>> but non-standard formats will require acceess to raw post data and
>>> custom parsers, which isn't as easy as reading headers.
>> So define a custom format.
> As noted by Kornel, this is a significantly higher barrier to entry than
> just using headers.
Yes, sometimes doing something properly is more work than just hacking
>>> Another advantage of headers is that Apache could log pings without
>>> help of any scripts or non-standard modules - LogFormat directive
>>> allows logging of arbitrary headers.
>> I'm not sure how this is relevant...
> It seems extremely relevant, as it enables cheap server-side use instead
> of requiring heavy lifting for the author.
For the author?
It seems the additional work would be for the implementor of the web
server, implementing a module for ping tracking. I'm not sure why that's
considered a major issue.
> On Sat, 2 Feb 2008, Kornel Lesinski wrote:
>> Special Content-Type might work equally well -- it can be detected by
>> tools scanning headers only, and should prevent applications from
>> accepting unexpected POST.
> Sadly I fear most sites don't check the Content-Type headers.
If you fear vulnerabilities because of servers misinterpreting POST,
than this would be a good argument for not using POST, it seems.
> It's not a matter of abuse. We need to do a POST request, since that is
> the most appropriate method for this case (we don't want to add a PING
There was no consensus for that.
> method, just like we wouldn't want to add a LOGIN method, a SEARCH method,
> a POSTEMAIL method, a SAVEPREFERENCES method, etc). However, there is a
Ironically, there is a SEARCH method, although only used in WebDAV land.
As a matter of fact, PING *has* been suggested.
> minor risk with doing a POST request in a new way, which is that
> XSS blacklisting code wouldn't stop the new feature. This is a problem
> whether the Referer header is absent, or whether it contains a valid
> value. Hence, we have an invalid value. This seems emminently pragmatic,
> and not at all something I would classify as abuse.
Ok, we'll continue to disagree on that. But please do not claim that
there was some kind of consensus for it.
> On Sat, 2 Feb 2008, Julian Reschke wrote:
>>> I see two ways forward here. One would be to redefine Referer to
>>> remove the relative URI thing, since, to my knowledge at least, nobody
>>> uses it.
>> That's IMHO not sufficient reason to remove it. It's not broken.
> Lack of use is a reason to remove a feature according to IETF process,
> actually. It's even been used as a reason for HTTP -- Link: headers were
> removed from HTTP a while back for this very reason. (I've since worked
> hard to get browsers to implement it, and we can probably get it back in
> at this point. But that's another story.)
> (RFC 2026, section 4.1.2.)
The reason it was removed was because HTTP/1.1 was advancing on the
standards track, which requires evidence of things being in use. Next
time this happens with HTTP/1.1 (draft -> full), questions like these
will be asked again.
And yes, I agree that Link is useful and needs to get revived in some
way. People do not seem to understand that it's still part of HTTP.
>>> The other is that we can define the magic value to be "#PING" instead,
>>> since that's a non-conforming Referer value right now.
>> It's not conforming, so are you suggesting to use a non-conforming
> Right; by using a previously illegal value, we can ensure that nobody is
> relying on it already. (I presume your objection to using a relative value
> "ping" is that it could be misinterpreted by existing implementations due
> to its current definition.)
The value is illegal until you make it legal. To make that, you'll have
to change the definition in HTTP/1.1.
The risk is that recipients that expect a compliant Referer header will
break in some way.
>> Could you please state what problem you are trying to solve, and why it
>> needs to involve the Referer header?
> I believe the problem has been stated a number of times in this thread
> already. I refer you to dolphinling's original e-mail.
> Not at all; the risks are easily mitigated.
Apparently we disagree on this.
> On Sat, 2 Feb 2008, Adam Barth wrote:
>> Perhaps this has been suggested before, but another option is to use a
>> new verb, such as PING, instead of GET when making the request. Servers
>> unaware of the ping attribute will likely ignore this verb, mitigating
>> the request-forgery attack vector.
> Sadly this is more than likely to cause problems with transparent proxies.
> It also seems silly to invent a new method, as noted earlier in this
> message. POST is the appropriate method here.
I do not believe a new method is needed (as I still think that GET/HEAD
would be fine); but it certainly would be better than POST because of
all the issues we're discussing here.
> On Sat, 2 Feb 2008, dolphinling wrote:
>> If (X-)Ping-From/Ping-To are present, why is a referer needed at all?
>> I'd say just leave it out. If not, #PING works for me.
> We can't remove it since then it would be treated as a local request.
Please explain "local request".
The absence of a Referer header means that no Referer information is
available, not that it's a local request. So please clarify.
> On Sun, 3 Feb 2008, Julian Reschke wrote:
>> I think it's a strange way to design a network protocol. When ping
>> requests in access logs are a problem, there are many ways not to have
>> it in the first place (not using HTTP, for instance) or by using a
>> different HTTP verb. Forcing something into a header it's not designed
>> for is even worse, in particular if the only reason given is to make it
>> easier to parse logs.
> It's not clear exactly what your concern is here.
My concern is that you're raking ease of implementation higher than
consistency of specifications.
More information about the whatwg