I have this script of scraping from website through PHP. All I want is to show result with comma separator and there are so many pages with paging. Just to show them with.
my code is
$ch = curl_init('http://www.qatarliving.com/v3/classifieds/search/category/mobile-devices');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
/*
* XXX: This is not a "fix" for your problem, this is a work-around. You
* should fix your local CAs
*/
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
/* Set a browser UA so that we aren't told to update */
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/34.0.1847.116 Safari/537.36');
$res = curl_exec($ch);
if ($res === false) {
die('error: ' . curl_error($ch));
}
curl_close($ch);
$d = new DOMDocument();
@$d->loadHTML($res);
$output = array(
'class' => '',
);
$x = new DOMXPath($d);
$myspan = $x->query('//span[@class="b-card b-card-mod-h item "]');
if($myspan->length > 0){
foreach($myspan as $row){
echo $row->nodeValue . "<br/>";
}
}
and result is
2,000 QAR Mobile phones, Al Gharrafa iPhone 6 128 like new By professional76
75 QAR Mobile phones, Other Virtual Reality Cardboard [NEW] By 1StopGulf
...
try below solution $data_array
will contains desired output array:
<?php
$ch = curl_init('http://www.qatarliving.com/v3/classifieds/search/category/mobile-devices');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
/*
* XXX: This is not a "fix" for your problem, this is a work-around. You
* should fix your local CAs
*/
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
/* Set a browser UA so that we aren't told to update */
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/34.0.1847.116 Safari/537.36');
$res = curl_exec($ch);
if ($res === false) {
die('error: ' . curl_error($ch));
}
curl_close($ch);
$d = new DOMDocument();
@$d->loadHTML($res);
$output = array(
'class' => '',
);
$x = new DOMXPath($d);
$myspan = $x->query('//span[@class="b-card b-card-mod-h item "]');
$data_array = array();
if ($myspan->length > 0) {
foreach ($myspan as $row) {
$data = $row->getElementsByTagName('p');
$array = array();
foreach ($data as $dt) {
$tag = '';
$class = $dt->getAttribute('class');
$value = $dt->nodeValue;
if ($class == 'b-card--el-deposit-val') {
$tag = 'price';
} else if ($class == 'b-card--el-deposit-time') {
$tag = 'deposittime';
} else if ($class == 'b-ad-excerpt b-par-mod-clear b-line-mod-thin--mix-item') {
$tag = 'category';
} else if ($class == 'b-card--el-description') {
$tag = 'name';
}
if ($tag) {
$array[$tag] = $value;
}
}
$data = $row->getElementsByTagName('a');
foreach ($data as $dt) {
$tag = '';
$class = $dt->getAttribute('class');
$value = $dt->nodeValue;
if (trim($class) == 'b-card--el-agency-title') {
$tag = 'addedby';
}
if ($tag) {
$array[$tag] = $value;
}
}
$data_array[] = $array;
}
echo '<pre>';
print_r($data_array);
}
Output:
Array
(
[0] => Array
(
[price] => 1,200
[deposittime] => QAR
[category] => Tablets, West Bay
[name] => iPad Air 64gb Silver with Leather cover
[addedby] => rocknrolla
)
[1] => Array
(
[price] => 2,500
[deposittime] => QAR
[category] => Mobile phones, Fereej Al Ameer / Muraikh
[name] => iPhone 6 Plus 64gb
[addedby] => nabbool
)
[2] => Array
(
[price] => 2,300
[deposittime] => QAR
[category] => Mobile phones, Al Sadd
[name] => lady use 6 plus gold 64 gb
[addedby] => nijumok
)
[3] => Array
(
[price] => 2,050
[deposittime] => QAR
[category] => Mobile phones, Old Airport
[name] => LG v10 blue for sale
[addedby] => ramsah92
)
[4] => Array
(
[price] => 1,750
[deposittime] => QAR
[category] => Mobile phones, Industrial Area
[name] => Neat and cleaned iPhone 6 16gb
[addedby] => ali murtza
)
[5] => Array
(
[price] => 1,350
[deposittime] => QAR
[category] => Mobile phones, Ain Khaled
[name] => Brand new honour 7 sell or sawp...4g with 1 year warenty
[addedby] => makbool_khan
)
[6] => Array
(
[price] => 250
[deposittime] => QAR
[category] => Mobile phones, Al Sadd
[name] => NOTE 3 ACCESSORIES
[addedby] => MRS70
)
[7] => Array
(
[price] => 0
[deposittime] => QAR
[category] => Tablets, West Bay
[name] => Hi, I'm looking for a Sony Xperia Z4 Tablet
[addedby] => carl_albrecht
)
[8] => Array
(
[price] => 50
[deposittime] => QAR
[category] => Mobile phones, Doha
[name] => SN0009 -Luxury Ultra-thin Shockproof Armor Back Case Cover for Apple iPhone 6S
[addedby] => Qesale
)
[9] => Array
(
[price] => 75
[deposittime] => QAR
[category] => Mobile phones, Doha
[name] => SN0003 - Dual Fast Adaptive USB Car Charger Adapter + Lightning Cable for iPhone Samsung
[addedby] => Qesale
)
[10] => Array
(
[price] => 2,000
[deposittime] => QAR
[category] => Mobile phones, Al Gharrafa
[name] => iPhone 6 128 like new
[addedby] => professional76
)
[11] => Array
(
[price] => 75
[deposittime] => QAR
[category] => Mobile phones, Other
[name] => Virtual Reality Cardboard [NEW]
[addedby] => 1StopGulf
)
)