how to parse a endless pagination webpage using HTML DOM parser?

Question

Hi i am trying to parse a webpage which has endless pagination (scrolling gives more items) with simple html dom parser. But i am able to get data only for first page only. How to get data of another webpages.

<?php

require  'simple_html_dom.php';

$html = file_get_html('http://www.flipkart.com/mobiles/pr?sid=tyy,4io&otracker=ch_vn_mobile_filter_Mobile%20Brands');

foreach ($html->find("div.pu-visual-section a") as $el) {

        $product_url = "http://flipkart.com".$el->href;

        echo $product_url;echo "<br>";

    }


 ?>

Answer 1

Disable javascript, go to the site and check if there is a "more"-button. Use the link from it to fetch new results.

EDIT: I disabled javascript and checked your url.

There is a "next" link at the bottom of the page:

http://www.flipkart.com/mobiles/~new-releases/pr?sid=tyy%2C4io&start=21&ref=436ee817-3fca-44b8-9b53-777f12126701

The &start=21 will be the necessary part for you to fetch new items. &start=41 etc

EDIT 2: So you didn't want to parse all existing items but rather get the item count.

preg_match('/class=\"items\">(.*?)</', $result, $match);

With this you should get the desired result. I did not tested it with the pagecontent itself. Let me know if it helped!