Search code examples
asp.netvb.netcachinguser-agenthttp-caching

User Agent is being cached on the server, why and how to stop it?

I have some code on my website that is supposed to only run when the user agent isn't a bot. For the most part it works, at least in VS debug mode:

ASP.Net:

Response.Write("<span style=""display:none;"" id=""test-UA"">" & ClassData.UserAgent & "</span>")
Response.Write("<span style=""display:none;"" id=""test-IP"">" & ClassData.IPAddress & "</span>")
If Not ClassData.BotUser Then
    Dim PageHistory As String() = WebSite.GetDBField.LastVisits(CInt(Session("VisitorID")), 5)
    If PageHistory IsNot Nothing Then
       Response.Write("<ul id=""PageHistory"" class=""noicon"">")
       For intPage As Integer = 0 To PageHistory.Length - 1
          Response.Write("<li><a href=""" & PageHistory(intPage) & """>" & WebSite.GetDBField.PageNameByURL(PageHistory(intPage)) & "</a></li>")
       Next
       Response.Write("<li onclick=""HistoryHider()"" class=""mover"">Page History ▲</li>")
       Response.Write("<li onclick=""HistoryHider()"" class=""mover hide"">Page History ▼</li>")
       Response.Write("</ul>")
       End If
End If

My code in particlar is failing on the "Classdata.BotUser" which is in a VB file in the App_Code folder:

Private Shared ReadOnly Requester As System.Web.HttpRequest = System.Web.HttpContext.Current.Request
Public Shared ReadOnly IPAddress As String = Requester.ServerVariables("REMOTE_ADDR")
Public Shared ReadOnly UserAgent As String = Requester.UserAgent 'ServerVariables("HTTP_USER_AGENT")
Public Shared ReadOnly BotUser As Boolean = (UserAgent.ToLower.Contains("bot") OrElse UserAgent.ToLower.Contains("spider") OrElse UserAgent.ToLower.Contains("crawl") OrElse UserAgent.ToLower.Contains("seo"))

It appears that the reason why it's failing is because the server appears to be caching the User Agent. I change my user agent within Firefox and it doesn't change to what I've set it to, but then I notice that the User Agent is actually Chrome (colleague next to me was using Chrome), eventually I managed to get it use the correct User Agent (somehow writing the user agent to the page did it) and at first thought it might be down to the fact we're both on the same connection. So I then connected to the website from my Phone, seemed to work correctly. I then re-tried it on Firefox on my computer and Firefox now thought that it was Google Chrome on Android!?

Because my pages are dynamic, I have added the below lines of code for caching purposes:

Response.Cache.SetLastModified(CDate(GetDBField.Pages(ID, "Last Modified")))
Response.Cache.SetETagFromFileDependencies()
Response.Cache.SetCacheability(HttpCacheability.[Public])
Response.Cache.SetMaxAge(New TimeSpan(356, 0, 0, 0))

Could it be that one of these is causing the strange caching or is it something else? Could someone please enlighten me with what on earth is going on because it's been driving me round the bend for the past few hours...

Solution

  • Your problem is using static (Shared) variables in a web application.

    The static variables exist once, so they are shared between all visitors to the web site. They get initialised the first time that someone visits the site, then they remain the same until the site shuts down.

    You need to get these values for each request, and not store them in static variables.

    Static variables should generally not be used in web applications, unless you want to store some data that actually should be shared by all requests. In that case you also need to synchronise the access to the data if it ever changes.