Search code examples
phpweb-scrapingget

web scraping a website in php is not working


I want to get the Currencies Table​​​ data from this link but I can't get the data in the table ... I had tried different ways but all in vain

the library that I'm using: https://sourceforge.net/projects/simplehtmldom/

here is the php code:

require "simple_html_dom.php";
$html = file_get_html("http://www.cibeg.com/English/Pages/CIBCurrencies.aspx");
$c = $html->find("#divCurrTableContainer",0)->innertext; 
echo $c;

Solution

  • Here is the code that their site uses to get the information in that table:

      $.ajax({
            type: "POST",
            url: '/_layouts/15/LINKDev.CIB.CurrenciesFunds/FundsCurrencies.aspx/GetCurrencies',
            async: true,
            data: "{'lang':'" + document.getElementById("ctl00_ctl48_g_5d7fc52f_a66d_4aa2_8d6c_c01fb4b38cb2_hdnLang").value + "'}",
            contentType: "application/json; charset=utf-8",
            dataType: "json",
            success: function (msg) {
                if (msg.d != null && msg.d.length > 0) {
                    var contentHTML = "<table class='currTable' cellspacing='0' rules='all' style='border-width:0px;border-collapse:collapse;'>"
                            + "<tbody><tr class='currHeaderRow'>"
                                + "<th scope='col'>" + document.getElementById("ctl00_ctl48_g_5d7fc52f_a66d_4aa2_8d6c_c01fb4b38cb2_hdnCurrency").value + "</th><th scope='col'>" + document.getElementById("ctl00_ctl48_g_5d7fc52f_a66d_4aa2_8d6c_c01fb4b38cb2_hdnBuy").value + "</th><th scope='col'>" + document.getElementById("ctl00_ctl48_g_5d7fc52f_a66d_4aa2_8d6c_c01fb4b38cb2_hdnSell").value + "</th>"
                            + "</tr>";
    
                    for (var i = 0; i < msg.d.length; i++) {
                        if (msg.d[i].CurrencyID.length > 0) {
                            contentHTML += "<tr class='currRow'>"
                                    + "<td>" + msg.d[i].CurrencyID + "</td><td>" + msg.d[i].BuyRate + "</td><td class='lastCell'>" + msg.d[i].SellRate + "</td>"
                                + "</tr>";
                        }
                    }
                    contentHTML += "</tbody></table>";
                    $("#divCurrTableContainer").html(contentHTML);
                    if ($(".bannerElements").length > 0)
                        FixCurrenciesRatesScroll();
                }
            },
            error: function (msg) {
            }
        });
    

    In php, we can replicate this request with curl:

    <?php
    $ch = curl_init('http://www.cibeg.com/_layouts/15/LINKDev.CIB.CurrenciesFunds/FundsCurrencies.aspx/GetCurrencies');
    
    $data = array("lang" => "en");                                                                   
    
    $data_string = json_encode($data);
    
    curl_setopt($ch, CURLOPT_CUSTOMREQUEST, "POST");    
    curl_setopt($ch, CURLOPT_POSTFIELDS, $data_string);
    curl_setopt($ch, CURLOPT_HTTPHEADER, array(                                                                          
      'Content-Type: application/json',                                                                                
      'Content-Length: ' . strlen($data_string))                                                                       
    
    );
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, true); 
    $server_output = curl_exec ($ch);
    
    curl_close ($ch);
    var_dump($server_output);
    
    ?>
    

    paste this into http://phpfiddle.org/ and see the result you get back, which you'll have to parse with php