Search code examples
vbaweb-scrapinggetelementsbyclassname

getElementsByClassName fails

I have below code but it fails and I am not sure why? The error is "Object variable or With block variable not set". Used to work previously not sure what changed. I have checked the class name but it has not changed on the website.

Public sub tester()

Dim oHtml As HTMLDocument
Dim oElement As Object

Set oHtml = New HTMLDocument
With CreateObject("MSXML2.ServerXMLHTTP.6.0")
    .Open "GET",”https://www.cigarsofcuba.co.uk/acatalog/”, False
    .send
    oHtml.body.innerHTML = .responseText
End With
v = 0
log = "section-list"

Set dados = oHtml.getElementsByClassName(log)(v).getElementsByTagName("a")
For Each oElement In dados
    Debug.Print dados(i).href
Next oElement
End sub
Solution

  • The following should fetch you the product links from that webpage:

    Public Sub FetchLinks()
    Const base = "https://www.cigarsofcuba.co.uk/acatalog/"
    Dim oHtml As HTMLDocument, targetLink$, I&
    
    Set oHtml = New HTMLDocument
    
    With CreateObject("MSXML2.XMLHTTP")
        .Open "GET", "https://www.cigarsofcuba.co.uk/acatalog/", False
        .setRequestHeader "User-Agent", "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.135 Safari/537.36"
        .send
        oHtml.body.innerHTML = .responseText
    End With
    
    With oHtml.querySelectorAll(".section-list > li > a")
        For I = 0 To .Length - 1
            targetLink = Replace(.item(I).href, "about:", base)
            Debug.Print targetLink
        Next I
    End With
End Sub

    First few results are like:

    https://www.cigarsofcuba.co.uk/acatalog/Buy-Cuban-Cigars.html
https://www.cigarsofcuba.co.uk/acatalog/Christmas-Santa-s-Grotto-p1.html
https://www.cigarsofcuba.co.uk/acatalog/New-World-Sweet---Aromatic-Cigars-p1.html
https://www.cigarsofcuba.co.uk/acatalog/Cigar-Samplers-p1.html
https://www.cigarsofcuba.co.uk/acatalog/Cigar-Connoisseurs-Collection-p1.html
https://www.cigarsofcuba.co.uk/acatalog/Cohiba-Cigars-Gift-boxes.html
https://www.cigarsofcuba.co.uk/acatalog/Romeo-Y-Julieta-Cigar-Gift-Boxes.html
https://www.cigarsofcuba.co.uk/acatalog/Montecristo-Cigar-Gift-Boxes.html
https://www.cigarsofcuba.co.uk/acatalog/Machine-Made-Packet-Cigars-p1.html