Search code examples
excelvbainternet-explorerselenium-chromedriver

Can MSXML2.XMLHTTP be used with Chrome


I have been using the following Excel VBA macro to bring back data from a website. It worked fine until a few days ago when the website stopped supporting IE. Of course the macro just fails now as there is no data on the webpage to bring back to Excel, just a message saying, "Your browser, Internet Explorer, is no longer supported." Is there a way to have the "Get method" (MSXML2.XMLHTTP) use Chrome instead of IE to interact with the website? BTW, my default browser is already set to "Chrome".

Dim html_doc As HTMLDocument        ' note: reference to Microsoft HTML Object Library must be set

Sub KS()    
' Define product url
    KS_url = "https://www.kingsoopers.com/p/r-w-knudsen-just-blueberry-juice/0007468210784"
    
' Collect data
        Set html_doc = New HTMLDocument
        Set xml_obj = CreateObject("MSXML2.XMLHTTP")
        
        xml_obj.Open "GET", KS_url, False
        xml_obj.send
        html_doc.body.innerHTML = xml_obj.responseText
        Set xml_obj = Nothing
  
        KS_product = html_doc.getElementsByClassName("ProductDetails-header")(0).innerText
        KS_price = "$" & html_doc.getElementsByClassName("kds-Price kds-Price--alternate mb-8")(1).Value

      do Stuff

End Sub

Solution

  • The check for this is a basic server check on user agent. Tell it what it wants to "hear" by passing a supported browser in the UA header...(or technically, in this case, just saying the equivalent of: "Hi, I am not Internet Explorer".)

    It can be as simple as xml.setRequestHeader "User-Agent", "Chrome". I said basic because you could even pass xml.setRequestHeader "User-Agent", "I am a unicorn", so it is likely an exclusion based list on the server for Internet Explorer.

    Option Explicit
    
    Public Sub KS()
        Dim url As String
    
        url = "https://www.kingsoopers.com/p/r-w-knudsen-just-blueberry-juice/0007468210784"
        Dim html As MSHTML.HTMLDocument, xml As Object
        
        Set html = New MSHTML.HTMLDocument
        Set xml = CreateObject("MSXML2.XMLHTTP")
           
        xml.Open "GET", url, False
        xml.setRequestHeader "User-Agent", "Mozilla/5.0"
        xml.send
        html.body.innerHTML = xml.responseText
    
        Debug.Print html.getElementsByClassName("ProductDetails-header")(0).innerText
        Debug.Print "$" & html.getElementsByClassName("kds-Price kds-Price--alternate mb-8")(1).Value
    
        Stop
    
    End Sub
    

    Compare that with adding no UA or adding xml.setRequestHeader "User-Agent", "MSIE".