Search code examples
phparray-unique

Remove duplicates from PHP array (array_unique)


I am getting a specific field from 4 different tables.

<?php 

//I connect to the database and the 4 tables

// Location of CSV  
$location = 'path.csv';

// List creation that will be updated with the fields and be put into my CSV file
$list = array();

// Read csv file to avoid adding duplicates
$file = fopen($location, 'r');
$data = array();

while($row = fgetcsv($file)) 
{
   $data[] = $row;
}

// Query 1
$sql = ('select distinct(field) as field from '.$table1.'');

// Run the query
$query = $Db->query($sql);

// Check for SQL errors
if ($Db->error) 
{
    return ($Db->error);
}

// Put data in the list
while ($row = $query->fetch_assoc()) 
{
    array_push($list,array($row['field'], ''));
}

// Query 2
$sql = ('select distinct(field) as field from '.$table2.'');

// Run the query
$query = $Db->query($sql);

// Check for SQL errors
if ($Db->error) 
{
    return ($Db->error);
}

// Put data in the list
while ($row = $query->fetch_assoc()) 
{
    array_push($list,array($row['field'], ''));
}

// Query 3
$sql = ('select distinct(field) as field from '.$table3.'');

// Run the query
$query = $Db->query($sql);

// Check for SQL errors
if ($Db->error) 
{
    return ($Db->error);
}

// Put data in the list
while ($row = $query->fetch_assoc()) 
{
    array_push($list,array($row['field'], ''));
}

// Query 4
$sql = ('select distinct(field) as field from '.$table4.'');

// Run the query
$query = $Db->query($sql);

// Check for SQL errors
if ($Db->error) 
{
    return ($Db->error);
}

// Put data in the list
while ($row = $query->fetch_assoc()) 
{
    array_push($list,array($row['field'], ''));
}


// Save list in the csv file without overwriting
$fp = fopen($location, 'a');

foreach (array_unique($list) as $fields) 
{
    if (in_array($fields, $data)) 
    {
        echo "Duplicate found";
    }
    else
    {
        echo "Save to file";
        fputcsv($fp, $fields);
    }           
}

fclose($fp);    

?>

In the end I check if the fields are already in the file. The only problem is that I still have duplicates because some tables might have exactly the same field. So, I want to remove the duplicates from the PHP array "list".

I am using :

$cleanlist = array_unique($list);

but I am getting an error:

PHP Notice: Array to string conversion

More specifically the change in my code is :

    $cleanlist = array_unique($list);

// Save list in the csv file without overwriting
$fp = fopen($location, 'a');

foreach ($cleanlist as $fields) 
{
    if (in_array($fields, $data)) 
    {
        echo "Duplicate found";
    }
    else
    {
        echo "Save to file";
        fputcsv($fp, $fields);
    }           
}

Solution

  • You could reduce amount of code a lot by using UNION in SELECT statement.

    SELECT field FROM table1
    UNION
    SELECT field FROM table2
    UNION
    SELECT field FROM table3
    UNION
    SELECT field FROM table4
    

    UNION returns distinct results by default.