Should I use Request.UrlReferrer when determining referrals

I came upon an interesting discussion with my team around the use of HttpRequest.UrlReferrer and wanted to solicit feedback from the community. According to the W3C spec:

The Referer[sic] request-header field
allows the client to specify, for the
server’s benefit, the address (URI) of
the resource from which the
Request-URI was obtained (the
“referrer”, although the header field
is misspelled.) The Referer
request-header allows a server to
generate lists of back-links to
resources for interest, logging,
optimized caching, etc. It also allows
obsolete or mistyped links to be
traced for maintenance. The Referer
field MUST NOT be sent if the
Request-URI was obtained from a source
that does not have its own URI, such
as input from the user keyboard.as input from the user keyboard.

The Request.UrlReferrer object does the work of converting referral strings that contain well formed URIs to an object with properties on every request.

According to our logs there are requests that come in that contain invalid data in the referral such as:

localhost
app:/BeamBackTest.swf
app:/multtiple.swf
app:/AFriendFeed.swf
ALToolBar
app:/index.html
mhtml:file://C:\Documents+and+Settings\User\Desktop\oracle\What+is+a+View+in+Oracle+-+Stack+Overflow.mht

Using Request.UrlReferrer would mean the above cases would be NULL. Is it better to discard the invalid data based on the W3C spec by using Request.UrlReferrer or preserve it by using Request.ServerVariables[“HTTP_REFERER”] even though the data may be interesting, but potentially useless.

Add Comment
2 Answer(s)

Its subjective based on your needs. We store everything even if its junk. One day we might know how to process it better and it won’t be junk anymore. If we didn’t store it because we didn’t understand it at the time we’d never be able to get it back.

Answered on July 11, 2016.
Add Comment

Well, it depends on what you’re doing with the data. If you’re doing something that requires a valid URI, then you might as well use Request.UrlReferrer since you have to strip it down anyway. But if all you’re doing is logging it, and your logging tool can handle weird things, I’d recommend the second method- especially for the app:/ data, which is likely to yield useful patterns.

Answered on July 11, 2016.
Add Comment

Your Answer

By posting your answer, you agree to the privacy policy and terms of service.