Search code examples
excelvbaweb-scrapinggetxmlhttprequest

No response form website using MSXML2.XMLHTTP with POST and GET


I am trying to transfer data with MSXML2.XMLHTTP from the second of multiple websites to Excel without opening the browser. It is meant that the main homepage creates a query page (or query-content) after data input.

My code:

Sub GetData()

Dim HTML As New HTMLDocument, HTMLDoc As New HTMLDocument
    Dim elmt As Object
    Dim L As Long
    Const URL = "http://www.zvg-portal.de/index.php?button=Termine%20suchen"
    Const URL1 = "http://www.zvg-portal.de/index.php?button=Suchen"
         
    With CreateObject("MSXML2.XMLHTTP")
    .Open "POST", URL, False
    .send "land_abk=be&button=suchen"
    .Open "GET", URL1, False
    .send
    HTML.body.innerHTML = .responseText
    End With
               
    Set elmt = HTML.querySelectorAll("TR")
    For L = 0 To elmt.Length - 1
    ActiveSheet.Cells(L + 2, 2) = elmt.Item(L).innerText
    Next

End Sub

Nothing happens. I think it is because there is no access to the query-content or the query-content is not built. Maybe becuase of the wrong BUTTON code? I also tried button=submit and submit=true and suchen=true, without success. How can this problem be solved? THX!

HTML of the main-site for data-input:

<button type=submit>Suchen</button><button type=reset onClick="reset_form()">Zurücksetzen</button><br></h3></nobr><p>

<input type=hidden name=ger_name id=name=ger_name>

<!-- -->
<table border=0><tr><td ><b>Sortiert nach:&nbsp;</></td><td>
                <SELECT size=1 name=order_by><option value=2>Termin</option>
<option value=1>Aktualisierung</option>
<option value=3>Aktenzeichen</option>

</td> </tr>
</table>

<!--Land-->
<br>
<table border=0>
<tr><td><font color=red>*</font>&nbsp;<b>Land:</b></td></tr>
<tr><td>
<select size=1 name=land_abk onChange="updateAmtsgericht(this.value);" style="width:643px">
<option value=0>-- Bitte Bundesland ausw&auml;hlen --</option><option value='bw' >Baden-Wuerttemberg</option>
<option value='by' >Bayern</option>
<option value='be' >Berlin</option>


Solution

  • You missed the header which plays an important role here to fetch the content. Try this way instead:

    Sub GetData()
        Const URL = "http://www.zvg-portal.de/index.php?button=Suchen"
        Dim HTML As New HTMLDocument
        Dim elmt As Object, L As Long
    
        With CreateObject("MSXML2.XMLHTTP")
            .Open "POST", URL, False
            .setRequestHeader "Content-Type", "application/x-www-form-urlencoded"
            .send ("land_abk=be")
            HTML.body.innerHTML = .responseText
        End With
                   
        Set elmt = HTML.querySelectorAll("tr > td")
        For L = 0 To elmt.Length - 1
            Debug.Print elmt.Item(L).innerText
        Next L
    End Sub