Search code examples
phpmysqlcsvload-data-infile

how to increment variable value by 1 for each record with php mysql LOAD DATA INFILE


I am trying to save an incrementing value for one of the columns using LOAD DATA INFILE method (csv data to database). Nothing I am trying is working. without the extra serial_no field everything imports fine, and if i set a hard coded value to serial_no, that works too. Just can't get it to increase the numerical value by 1 for each record.

--

$loadsqlfiletodb = "SET @a:=".$serial_no_max_new.";
        LOAD DATA LOCAL INFILE '".$target_file."'
        INTO TABLE ".DB_NAME2.".list_data
        FIELDS TERMINATED BY ','
        OPTIONALLY ENCLOSED BY '\"' 
        LINES TERMINATED BY '\n'
        IGNORE 1 LINES 
        (first,last,address,city,state,zip,company,tracking_number,vin,year,make,model,rough,average,clean,misc1,misc2,misc3,misc4,misc5,misc6,misc7,misc8,misc9,misc10,list,email,phone,image_name,image_path)
        SET cid = ".$campaignid.",
            jid = ".$jobid.",
            serial_no = @a:=@a+1 ";

        mysql_query($loadsqlfiletodb) or die(mysql_error());

--

$loadsqlfiletodb = "
        SET @a:='".$serial_no_max_new."'
        LOAD DATA LOCAL INFILE '".$target_file."'
        INTO TABLE ".DB_NAME2.".list_data
        FIELDS TERMINATED BY ','
        OPTIONALLY ENCLOSED BY '\"' 
        LINES TERMINATED BY '\n'
        IGNORE 1 LINES 
        (first,last,address,city,state,zip,company,tracking_number,vin,year,make,model,rough,average,clean,misc1,misc2,misc3,misc4,misc5,misc6,misc7,misc8,misc9,misc10,list,email,phone,image_name,image_path)
        SET cid = ".$campaignid.",
            jid = ".$jobid.",
            serial_no = @a:=@a+1 ";

        mysql_query($loadsqlfiletodb) or die(mysql_error());

--

$loadsqlfiletodb = "LOAD DATA LOCAL INFILE '".$target_file."'
        INTO TABLE ".DB_NAME2.".list_data
        FIELDS TERMINATED BY ','
        OPTIONALLY ENCLOSED BY '\"' 
        LINES TERMINATED BY '\n'
        IGNORE 1 LINES 
        (first,last,address,city,state,zip,company,tracking_number,vin,year,make,model,rough,average,clean,misc1,misc2,misc3,misc4,misc5,misc6,misc7,misc8,misc9,misc10,list,email,phone,image_name,image_path)
        SET cid = ".$campaignid.",
            jid = ".$jobid.",
            serial_no = ".$serial_no_max_new."+1 ";

        mysql_query($loadsqlfiletodb) or die(mysql_error());

--

$loadsqlfiletodb = "LOAD DATA LOCAL INFILE '".$target_file."'
        INTO TABLE ".DB_NAME2.".list_data
        FIELDS TERMINATED BY ','
        OPTIONALLY ENCLOSED BY '\"' 
        LINES TERMINATED BY '\n'
        IGNORE 1 LINES 
        (first,last,address,city,state,zip,company,tracking_number,vin,year,make,model,rough,average,clean,misc1,misc2,misc3,misc4,misc5,misc6,misc7,misc8,misc9,misc10,list,email,phone,image_name,image_path)
        SET cid = ".$campaignid.",
            jid = ".$jobid.",
            serial_no = ".$serial_no_max_new." = ".$serial_no_max_new."+1 ";

        mysql_query($loadsqlfiletodb) or die(mysql_error());

update:

here is the full thing. serial number is always going to be 9 digits. no more, no less. when serial reaches 999999999, the next recorded serial will be 000000000. however, don't worry about the nines to zeros part. at this time i am trying to figure out how to increment value by 1 for that column using the LOAD DATA INFILE method of importing file to database.

$query_max = mysql_fetch_array(mysql_query("SELECT id, latest_serial_no FROM ".DB_NAME2.".latest_serial_number WHERE id=1"), MYSQL_ASSOC);
        $serial_no_max = $query_max["latest_serial_no"];
        $serial_no_max_new = $serial_no_max;
        if ($serial_no_max > 999999998) {$serial_no_max_new = 000000000;}

        $padd = "";
        $serial_len = strlen($serial_no_max_new);
        if($serial_len < 9) {
            $serial_len = (9 - $serial_len);
            for($is=0;$is<$serial_len;$is++) {
                $padd .= "0";
            }
            $serial_no_max_new = $padd . $serial_no_max_new;
        }
        $query_serial = trim($serial_no_max_new);
        //echo "q_s: ".$query_serial;

        //Import uploaded file to Database
        //$target_file is where the uploaded file is located

        $loadsqlfiletodb = "SET @a:=".$query_serial.";
        LOAD DATA LOCAL INFILE '".$target_file."'
        INTO TABLE ".DB_NAME2.".list_data
        FIELDS TERMINATED BY ','
        OPTIONALLY ENCLOSED BY '\"' 
        LINES TERMINATED BY '\n'
        IGNORE 1 LINES 
        (first,last,address,city,state,zip,company,tracking_number,vin,year,make,model,rough,average,clean,misc1,misc2,misc3,misc4,misc5,misc6,misc7,misc8,misc9,misc10,list,email,phone,image_name,image_path)
        SET cid = ".$campaignid.",
            jid = ".$jobid.",
            serial_no = @a:=@a+1 ";

        mysql_query($loadsqlfiletodb) or die(mysql_error());

Solution

  • Maybe like this:

     /*1 get last serial number.
    
     open csv file.
     foreach row in csv file:
        increment serial number.
        if max number
           set serial number to 0.
        add new field to csv row
    
     close csv file
    
     insert csv to database*/
    
    
        $query_max = mysql_fetch_array(mysql_query("SELECT id, latest_serial_no FROM ".DB_NAME2.".latest_serial_number WHERE id=1"), MYSQL_ASSOC);
        $serial_no_max = $query_max["latest_serial_no"];
        $serial_no_max_new = $serial_no_max;
        if ($serial_no_max > 999999998) {$serial_no_max_new = 000000000;}
    
        $padd = "";
        $serial_len = strlen($serial_no_max_new);
        if($serial_len < 9) {
            $serial_len = (9 - $serial_len);
            for($is=0;$is<$serial_len;$is++) {
                $padd .= "0";
            }
            $serial_no_max_new = $padd . $serial_no_max_new;
        }
        $query_serial = trim($serial_no_max_new);
        //echo "q_s: ".$query_serial;
    
        // Add serail number to csv code
        $csv = fopen($target_file, "r");
        $tmp_csv_addr = $target_file . "2";
        $tmp_csv = fopen($tmp_csv_addr, "a");
        while (($data = fgetcsv($csv, 0, ",")) !== FALSE) {
            $data[] = $query_serial;
            fputcsv($tmp_csv, $data, ",");
        }
        fclose($csv);
        fclose($tmp_csv);
    
        // Insert into database
        $loadsqlfiletodb = "LOAD DATA LOCAL INFILE '".$tmp_csv_addr."'
        INTO TABLE ".DB_NAME2.".list_data
        FIELDS TERMINATED BY ','
        OPTIONALLY ENCLOSED BY '\"' 
        LINES TERMINATED BY '\n'
        IGNORE 1 LINES 
        (first,last,address,city,state,zip,company,tracking_number,vin,year,make,model,rough,average,clean,misc1,misc2,misc3,misc4,misc5,misc6,misc7,misc8,misc9,misc10,list,email,phone,image_name,image_path,serial_no)
        SET cid = ".$campaignid.",
            jid = ".$jobid;
    
        mysql_query($loadsqlfiletodb) or die(mysql_error());
    
        unlink($tmp_csv);