Search code examples
powershellcsvparsingpowershell-4.0text-parsing

Issue : Output the same line over and over


Below is the code, the idea is to grab a raw output of a differents storage arrays, and parse them in to a csv, the last part of the function is a contribution from this link:

check csv for blank fields and write output if exist blank

But when i output the information, the result is the same hostname writen over and over again, with the same results, using the same lines of the raw file but with only one host info. The idea would be to have the output with the last check for each different host.

Something like this:

2018-11-13,host1,7.1.80-3,IP_INFO, 1023.6445.64.2513, 255.255.255.0, 1340.6565.6124.1, 1340.6565.6124.1,
2018-11-13,host2,7.1.80-3,IP_INFO, 1023.6445.64.2513, 255.255.255.0, 1340.6565.6124.1, 1340.6565.6124.1,No-Compliance
2018-11-13,host3,7.1.80-3,IP_INFO, 1023.6445.64.2513, 255.255.255.0, 1340.6565.6124.1, 101340.6565.6124.1,Compliance
2018-11-13,host4,7.1.80-3,IP_INFO, 1023.6445.64.2513, 255.255.255.0, 1340.6565.6124.1, 1340.6565.6124.1,Compliance

$OK = "Compliance"
$FAIL = "No-Compliance"
 function Parse_IP_NAS {

    #Write-Output "Date, Hostname,version, Config, Value, value 2, value 3, value 4 " | Out-File $finalpatch\outputip.txt -Encoding ASCII

        # "Network Information VNX"
    $contentnasip = ($ParseFileIP -split "`n") 

    For($i=0;$i -lt $contentnasip.count;$i++){

        if($contentnasip[$i] -match "Hostname:")
        {
            Write-Host "$infohostname,$infoipv4,$infonetmask,$gatewayinfo,$dnsinfo,$date"
            Write-Output "$date,$infohostname,IP_INFO,$infoipv4,$infonetmask,$gatewayinfo,$dnsinfo" | Out-File $destination_PARSE_NAS\IP_NAS_PARSED.txt -Append  -Encoding ASCII
            Write-Output "$date,$infohostname,IP_INFO,$infoipv4,$infonetmask,$gatewayinfo,$dnsinfo,$MatchDNS" | Out-File $destination_PARSE_NAS\IP_NAS_PARSED_2.txt -Append  -Encoding ASCII

            #"Hostname Information"
            $global:infohostname = $contentnasip[$i+1]
        }

        elseif($contentnasip[$i] -match "IPv4 Address")
        {
            #"Ipv4 Address"
            $global:infoipv4 = ($contentnasip[$i]  -split "=")[1]
        }
        elseif($contentnasip[$i] -match "IPv4 Netmask")
        {
            #"Netmask Information"
            $global:infonetmask = ($contentnasip[$i] -split "=")[1]
        }
        elseif($contentnasip[$i] -match "IPv4 Gateway")
        {
            #"Gateway Information"
            $global:gatewayinfo = ($contentnasip[$i] -split "=")[1]
        }
        elseif($contentnasip[$i] -match "DNS Servers")
        {
           # "DNS Servers Information"
            $global:dnsinfo = ($contentnasip[$i] -split "=")[1]
        }   
    }

    $csvContent = Import-Csv -Path "$destination_PARSE_NAS\IP_NAS_PARSED.txt" -Header @("Date","Hostname","version","Config","Value","value 2","value 3","value 4")
    $csvContent | ForEach-Object {
        Write-Output "$date,$infohostname,IP_INFO,$infoipv4,$infonetmask,$gatewayinfo,$dnsinfo,$MatchDNS" | Out-File $destination_PARSE_NAS\IP_NAS_PARSED_3.txt -Append  -Encoding ASCII
        if ([string]::IsNullOrEmpty($_.Value)) {
            Write-Host "$($_.HostName) = FAIL" -ForegroundColor Red

            $Global:MatchDNS = $FAIL
        }
        else {
            Write-Host "$($_.HostName) = OK" -ForegroundColor Green
            $Global:MatchDNS = $OK

        }
    }

Output ( $destination_PARSE_NAS\IP_NAS_PARSED.txt ) :

2018-11-07,host1-CS0,8.1.9-184,IP_INFO, 10.143.17.205, 255.255.254.0, 10.13443.16.15, 1299.493.100.1342,199.52342.1724.12341
2018-11-07,host2,8.1.9-184,IP_INFO, 101.20432.40.6512, 255.255.255.0, 130.2104.4034.1, 10.146.11.58,10.146.11.74,202343.2234.12340.37
2018-11-07,host3,8.1.9-184,IP_INFO, 120.204.430.658, 255.255.255.0, 10.20344.40.1, 10.146.11.58,10.14436.11.7234,203.2342.12340.37
2018-11-07,host4,7.1.79-8,IP_INFO, 1412.2221.89.1432, 255.255.255.0, 141232.212321.81239.254, 142.221.921.223,13499.2349.100.12
2018-11-07,host5,7.1.80-7,IP_INFO, 1452.2421.922.817, 255.255.255.0, 141232.2421.9122.2514, 142.221.119.92,142.221.119.91
2018-11-07,host6,8.1.9-155,IP_INFO, 1242.22211.92.21232, 255.255.255.0, 14232.2231.942.2524, 142.221.119.92,142.2321.1149.91
2018-11-07,host7,8.1.9-184,IP_INFO, 130.1392.423.2144, 255.255.255.0, 4210.19412.324.1, 101.14246.11.58,10.1414 62.131.744,2033.22.104.37
2018-11-07,host8,8.1.9-184,IP_INFO, 12310.12395.560.4213, 255.255.255.0, 110.191245.61240.241, 11240.12446.111.4258,1240.12446.2411.2474,20243.12.110.3427
2018-11-07,,,IP_INFO, 10.123.444.1, 25515.255.255.0, 10.195.60.1, 11230.14216.111.51248,10124.142416.14211.7424,212403.21.1240.3247
2018-11-07,host9,8.1.9-184,IP_INFO, 110.123.443.51, 255.255.255.0, 10124.11250.042.124, 11230.22312.2320.542,1042.21212.142129.2414

The idea:

2018-11-07,host1-CS0,8.1.9-184,IP_INFO, 10.143.17.205, 255.255.254.0, 10.13443.16.15, 1299.493.100.1342,199.52342.1724.12341, compliance
2018-11-07,host2,8.1.9-184,IP_INFO, 101.20432.40.6512, 255.255.255.0, 130.2104.4034.1, 10.146.11.58,10.146.11.74,202343.2234.12340.37, compliance
2018-11-07,host3,8.1.9-184,IP_INFO, 120.204.430.658, 255.255.255.0, 10.20344.40.1, 10.146.11.58,10.14436.11.7234,203.2342.12340.37, compliance
2018-11-07,host4,7.1.79-8,IP_INFO, 1412.2221.89.1432, 255.255.255.0, 141232.212321.81239.254, 142.221.921.223,13499.2349.100.12, compliance
2018-11-07,host5,7.1.80-7,IP_INFO, 1452.2421.922.817, 255.255.255.0, 141232.2421.9122.2514, 142.221.119.92,142.221.119.91 , compliance
2018-11-07,host6,8.1.9-155,IP_INFO, 1242.22211.92.21232, 255.255.255.0, 14232.2231.942.2524, 142.221.119.92,142.2321.1149.91 , compliance
2018-11-07,host7,8.1.9-184,IP_INFO, 130.1392.423.2144, 255.255.255.0, 4210.19412.324.1, 101.14246.11.58,10.1414 62.131.744,2033.22.104.37 , compliance
2018-11-07,host8,8.1.9-184,IP_INFO, 12310.12395.560.4213, 255.255.255.0, 110.191245.61240.241, 11240.12446.111.4258,1240.12446.2411.2474,20243.12.110.3427 , compliance
2018-11-07,,,IP_INFO, 10.123.444.1, 25515.255.255.0, 10.195.60.1, 11230.14216.111.51248,10124.142416.14211.7424,212403.21.1240.3247 , compliance
2018-11-07,host9,8.1.9-184,IP_INFO, 110.123.443.51, 255.255.255.0, 10124.11250.042.124, 11230.22312.2320.542,1042.21212.142129.2414 , compliance










 This is the example of $ParseFileIP





Hostname:
    host1-CS0,8.1.9-184
    Network:
    IPv4 Address            = 1.1.1.1.1
    IPv4 Netmask            = 255.255.254.0
    IPv4 Gateway            = 1.1.1.1.1
    DNS Servers             = 1.1.1.1.1,1.1.1.1.1
    Hostname:
    Host2,8.1.9-184
    Network:
    IPv4 Address            = 1.1.1.1.1
    IPv4 Netmask            = 255.255.255.0
    IPv4 Gateway            = 1.1.1.1.1
    DNS Servers             = 1.1.1.1.1,1.1.1.1.1
    Hostname:
    Host3,8.1.9-184
    Network:
    IPv4 Address            = 1.1.1.1.1
    IPv4 Netmask            = 255.255.255.0
    IPv4 Gateway            = 1.1.1.1.1
    DNS Servers             = 1.1.1.1.1,1.1.1.1.1

Solution

  • Ok, so if your function Parse_IP_NAS creates a file called IP_NAS_PARSED.txt that looks like this:

    2018-11-07,host1-CS0,8.1.9-184,IP_INFO, 10.143.17.205, 255.255.254.0, 10.13443.16.15, 1299.493.100.1342,199.52342.1724.12341
    2018-11-07,host2,8.1.9-184,IP_INFO, 101.20432.40.6512, 255.255.255.0, 130.2104.4034.1, 10.146.11.58,10.146.11.74,202343.2234.12340.37
    2018-11-07,host3,8.1.9-184,IP_INFO, 120.204.430.658, 255.255.255.0, 10.20344.40.1, 10.146.11.58,10.14436.11.7234,203.2342.12340.37
    2018-11-07,host4,7.1.79-8,IP_INFO, 1412.2221.89.1432, 255.255.255.0, 141232.212321.81239.254, 142.221.921.223,13499.2349.100.12
    2018-11-07,host5,7.1.80-7,IP_INFO, 1452.2421.922.817, 255.255.255.0, 141232.2421.9122.2514, 142.221.119.92,142.221.119.91
    2018-11-07,host6,8.1.9-155,IP_INFO, 1242.22211.92.21232, 255.255.255.0, 14232.2231.942.2524, 142.221.119.92,142.2321.1149.91
    2018-11-07,host7,8.1.9-184,IP_INFO, 130.1392.423.2144, 255.255.255.0, 4210.19412.324.1, 101.14246.11.58,10.1414 62.131.744,2033.22.104.37
    2018-11-07,host8,8.1.9-184,IP_INFO, 12310.12395.560.4213, 255.255.255.0, 110.191245.61240.241, 11240.12446.111.4258,1240.12446.2411.2474,20243.12.110.3427
    2018-11-07,,,IP_INFO, 10.123.444.1, 25515.255.255.0, 10.195.60.1, 11230.14216.111.51248,10124.142416.14211.7424,212403.21.1240.3247
    2018-11-07,host9,8.1.9-184,IP_INFO, 110.123.443.51, 255.255.255.0, 10124.11250.042.124, 11230.22312.2320.542,1042.21212.142129.2414
    

    and all you want is code that reads that IP_NAS_PARSED.txt to do a check on the existance if the IPv4 value and append a field after each line, I think all you have to do is this AFTER the IP_NAS_PARSED.txt is created:

    # read the IP_NAS_PARSED.txt created by the 'Parse_IP_NAS' function and create a comma delimited .TXT file from it with the value '$MatchDNS' appended
    
    # check if the file "$destination_PARSE_NAS\IP_NAS_PARSED_3.txt" already exists. delete it if it does.
    if (Test-Path -Path "$destination_PARSE_NAS\IP_NAS_PARSED_3.txt") {
        Remove-Item -Path "$destination_PARSE_NAS\IP_NAS_PARSED_3.txt" -Force -ErrorAction SilentlyContinue
    }
    
    $OK = "Compliance"
    $FAIL = "No-Compliance"
    $csvContent = Import-Csv -Path "$destination_PARSE_NAS\IP_NAS_PARSED.txt" -Header @("Date","Hostname","version","Config","IPv4","SubnetMask","Gateway","DNS1","DNS2")
    $csvContent | ForEach-Object {
        # you can use IsNullOrEmpty() too in your case, but use 'IsNullOrWhiteSpace()' for safety if ever you decide to quote the values.
        if ([string]::IsNullOrWhiteSpace($_.IPv4)) {
            $color = 'Red'
            $MatchDNS = $FAIL
        }
        else {
            $color = 'Green'
            $MatchDNS = $OK
        }
    
        # create a comma delimited line and show it in the console
        $newline = '{0},{1},{2},{3},{4},{5},{6},{7},{8},{9}' -f $_.Date, $_.Hostname, $_.version, $_.Config, $_.IPv4, $_.SubnetMask, $_.Gateway, $_.DNS1, $_.DNS2, $MatchDNS
        Write-Host $newline -ForegroundColor $color
    
        # and add it to file "IP_NAS_PARSED_3.txt" manually (Add-Content wil create the file if it does not exist)
        $newline | Add-Content -Path "$destination_PARSE_NAS\IP_NAS_PARSED_3.txt" -Force
    }
    

    Note: this code follows your function Parse_IP_NAS and is not meant to be inside that function.


    EDIT

    Maybe it is even a better idea to redo the Parse_IP_NAS function a bit, so we can streamline the parsing more efficiently and we don't need any 'in-between' files.

    Just a thought, but how about this:

    function Parse_IP_NAS {
        [CmdletBinding(DefaultParameterSetName = 'ByContent')]
        Param(
            [Parameter(Position = 0, Mandatory = $true, ParameterSetName = 'ByContent')]
            [string]$Content,
    
            [Parameter(Position = 0, Mandatory = $true, ParameterSetName = 'ByFileName')]
            [ValidateScript({Test-Path -Path $_ -PathType Leaf})]
            [string]$Path
        )
        if ($PSCmdlet.ParameterSetName -eq 'ByFileName') {
            $Content = Get-Content -Path $Path -Raw
        }
    
        # create an array to store the resulting objects
        $result = @()
        # split the content of the "Network Information VNX" in $Content into blocks 
        # Each block starts with "Hostname:"
        foreach ($hostBlock in ($Content -split "Hostname:")) {
            # skip if we have nothing in the block
            if ([string]::IsNullOrWhiteSpace($hostBlock)) { continue }
    
            # create a PSCustomObject with these properties:
            $parsedHost = '' | Select-Object 'Date','Hostname','Version','Config','IPv4','SubnetMask','Gateway','DNS1','DNS2','MatchDNS'
    
            ## I'm not sure where the Date came from.. Is that today's date?
            $parsedHost.Date   = (Get-Date).ToString('yyyy\-MM\-dd')
            $parsedHost.Config = 'IP_INFO'
    
            # split the $hostBlock textblock on newlines (\r = CR, \n = LF) to get an array
            # loop through the lines, but skip the line 'Network:' and any possible empty or whitespace-only strings
            ($hostBlock -split '\r?\n') | Where-Object { $_ -match '\S' -and $_ -notmatch 'Network:' } | ForEach-Object {
                if ($_ -match '=') {
                    $value = ($_ -split '=')
                    switch -regex ($value[0]) {
                        'IPv4 Address' { $parsedHost.IPv4 = $value[1].Trim() }
                        'IPv4 Netmask' { $parsedHost.SubnetMask = $value[1].Trim() }
                        'IPv4 Gateway' { $parsedHost.Gateway = $value[1].Trim() }
                        'DNS Servers'  { $parsedHost.DNS1, $parsedHost.DNS2 = ($value[1] -split ',') | ForEach-Object { $_.Trim()}}
                    }
                }
                else {
                    $parsedHost.Hostname, $parsedHost.Version = ($_ -split ',') | ForEach-Object { $_.Trim() }
                }
            }
    
            # now check if the IPv4 value is set for this host
            if ([string]::IsNullOrWhiteSpace($parsedHost.IPv4)) {
                $color = 'Red'
                $parsedHost.MatchDNS = 'No-Compliance'  # $FAIL
            }
            else {
                $color = 'Green'
                $parsedHost.MatchDNS = 'Compliance'     # $OK
            }
    
            # add the object to the $result array
            $result += $parsedHost
    
            # add some visual output to the console
            $message = '{0},{1},{2},{3},{4},{5},{6},{7},{8},{9}' -f $parsedHost.Date, $parsedHost.Hostname, $parsedHost.Version, 
                                                                    $parsedHost.Config, $parsedHost.IPv4, $parsedHost.SubnetMask, 
                                                                    $parsedHost.Gateway, $parsedHost.DNS1, $parsedHost.DNS2, $parsedHost.MatchDNS
            Write-Host $message -ForegroundColor $color
        }
        return $result
    }
    

    Now you can execute the Parse_IP_NAS function to use either parameter 'Content' if you already have the content in a string variable like:

    $parsedResult = Parse_IP_NAS -Content $ParseFileIP
    

    Or feed it the full path and name of the file so it will read the content for you:

    $parsedResult = Parse_IP_NAS -Path '<path to the Network Information VNX file>'
    

    After this call to the function, we have an array of objects with all values for each host found in $parsedResult as objects, so this is where you choose what to do with this info:

    Scenario 1
    Create the output manually so it will become a file of comma delimited values without quotes around them:

    $outputFile = "$destination_PARSE_NAS\IP_NAS_PARSED_COMPLETE.txt"
    
    # check if the output file already exists. delete it if it does.
    if (Test-Path -Path $outputFile -PathType Leaf) {
        Remove-Item -Path $outputFile -Force -ErrorAction SilentlyContinue
    }
    
    # if you DO want headers in this file, uncomment the next line:
    # Add-Content -Path $outputFile -Value 'Date,Hostname,Version,Config,IPv4,SubnetMask,Gateway,DNS1,DNS2,MatchDNS'
    
    # loop through the $parsedResult array and build a line for each object in there
    $parsedResult | ForEach-Object {
        # create a comma delimited line and add it to the outputfile manually
        # (Add-Content wil create the file if it does not exist and default)
        $newline = '{0},{1},{2},{3},{4},{5},{6},{7},{8},{9}' -f $_.Date, $_.Hostname, $_.Version, 
                                                                $_.Config, $_.IPv4, $_.SubnetMask, 
                                                                $_.Gateway, $_.DNS1, $_.DNS2, $_.MatchDNS
        $newline | Add-Content -Path $outputFile -Force
    }
    

    Scenario 2
    Use PowerShells Export-Csv cmdlet and output the $parsedResult array as proper CSV file. The field values will have quotes around them.

    $outputFile = "$destination_PARSE_NAS\IP_NAS_PARSED_COMPLETE.csv"
    $parsedResult | Export-Csv -Path $outputFile -NoTypeInformation -Encoding ASCII -Delimiter ',' -Force