Search code examples
excelvbaweb-scrapingscreen-scraping

VBA Scraping div elements


So, I've trying to scrape data from a website but I simply can't reach my goal...
I'm new with VBA and i've tried to search the basics of vba in order to understand some code.
So far I got this code but it's only scraping the data from the 1st div and it scrap all the data to one cell, and I need the macro to run trought all the page and scrap all the data that has the className I input on the code on diferent cells (eg: 1st div to cell A:1, 2nd div to cell A2... and so on)
Could you help me or give me some "lights" of what I'm doing wrong pls? Thank you!
Code:

Sub BoschRoupa()

    Dim ieObj As InternetExplorer
    Dim htmlEle As IHTMLElement
    Dim i As Integer
   

    i = 1
   

    Set ieObj = New InternetExplorer
    ieObj.Visible = False
    ieObj.navigate "https://www.worten.pt/grandes-eletrodomesticos/maquinas-de-roupa/maquinas-de-roupa-ver-todos-marca-BALAY-e-BOSCH-e-SIEMENS?per_page=100"
   

    Application.Wait Now + TimeValue("00:00:05")
   

    For Each htmlEle In ieObj.document.getElementsByClassName("w-product__content")(0).getElementsByTagName("div")
        With ActiveSheet
            .Range("A" & i).Value = htmlEle.Children(0).textContent
        End With
       
        i = i + 1
    Next htmlEle
   
End Sub

Solution

  • You can use xmlhttp, rather than a browser, then the following loop to write out all the div info. I would probably be more selective in how I grab only data of interest but the following, I hope, is in the spirit of what you have asked for.

    Option Explicit
    Public Sub GetInfo()
        Dim data As Object, i As Long, html As HTMLDocument, r As Long, c As Long, item As Object, div As Object
        Set html = New HTMLDocument                  '<== VBE > Tools > References > Microsoft HTML Object Library
        With CreateObject("MSXML2.XMLHTTP")
            .Open "GET", "https://www.worten.pt/grandes-eletrodomesticos/maquinas-de-roupa/maquinas-de-roupa-ver-todos-marca-BALAY-e-BOSCH-e-SIEMENS?per_page=100", False
            .send
            html.body.innerHTML = .responseText
        End With
        Set data = html.getElementsByClassName("w-product__content")
        For Each item In data
            r = r + 1: c = 1
            For Each div In item.getElementsByTagName("div")
                With ThisWorkbook.Worksheets("Sheet1")
                    .Cells(r, c) = div.innerText
                End With
                c = c + 1
            Next
        Next
    End Sub