Search code examples
powershell

Loop through ConvertFrom-Csv returned object


I have below code which splits files based on certain column that we pass to it. I am facing issues while outputting results to csv file. As of now I am getting this output in csv

Company : ABC

Desc : DEF

Region : GHI

I need it in row column format

Company,Desc,Region

ABC,DEF,GHI

Code:-

function fBreakFiles() {
    param(
        [String]$pDirectory,
        [String]$pFileNameString,
        [String]$pDateColumnName,
        [String]$pNewFileString,
        [String]$pLogFile,
        [String]$pDateFormat
    )


    Get-ChildItem $pDirectory -Force | Where-Object { $_.Name -ilike "$pFileNameString" } | ForEach-Object {
        $FilePath = $_.FullName
        Write-Output "################ $(Get-Date -Format 'yyyy/MM/dd hh:mm:ss:fff') :: Starting to extract source file at location $FilePath  `r`n" | Out-File $pLogFile -Append

        # Open the CSV file for reading
        $streamReader = [System.IO.StreamReader]::new($FilePath)

        # Get header line
        $header = $streamReader.ReadLine()

        # Set the buffer size based on your requirements
        $bufferSize = 2000
        $buffer = @()

        while ($streamReader.Peek() -ge 0) {
        # Read a block of lines into the buffer
        $buffer = 1..$bufferSize | ForEach-Object { $streamReader.ReadLine() }

        # Group the buffer data by required column
        $csvString = $buffer -join "`n"
        $csvObject = $csvString | ConvertFrom-Csv -Header ($header -split ',')
        $groupedData = $csvObject | Group-Object { $_.$pDateColumnName.Trim() }

        foreach ($group in $groupedData) {
            $DateString = ([datetime]::parseexact(($group.name), $pDateFormat, $null)).ToString("yyyy-MM-dd")
            $outputPath = Join-Path $pSourceDirectory "$pNewFileString$DateString.csv"
            if ($Date -ne $DateString)
            {
            $header | Out-File -Append -FilePath $outputPath
            }
            $Date = $DateString
            $group.Group | Out-File -Append -FilePath $outputPath
        }
    }
}

Solution

  • As @Mathias mentioned you need convert the objects back to CSV format before writing them to the file. $group.Group | ConvertTo-Csv -NoTypeInformation | Select-Object -Skip 1 | Out-File -Append -FilePath $outputPath

    foreach ($group in $groupedData) {
        $DateString = ([datetime]::parseexact(($group.name), $pDateFormat, $null)).ToString("yyyy-MM-dd")
        $outputPath = Join-Path $pSourceDirectory "$pNewFileString$DateString.csv"
        if ($Date -ne $DateString)
        {
            $header | Out-File -Append -FilePath $outputPath
        }
        $Date = $DateString
        $group.Group | ConvertTo-Csv -NoTypeInformation | Select-Object -Skip 1 | Out-File -Append -FilePath $outputPath
    }