I have been struggling with this for several hours now and after reading many threads about ScriptBlocks, Closures, scopes etc I still don't see what's wrong in my code.
Let me explain: I have a main script that dynamically generates an HTML page using PSWriteHTML module and ScriptBlocks.
As I have a lot of PSWriteHTML pages to write, I use an arrayList of ScriptBlocks to generate the code with different set of values each time (corresponding to different servers), these ScriptBlocks being executed into a foreach loop.
This is done using the Save-utilizationReport function (I have only kept the relevant code):
function Save-utilizationReport ($currentDate, $navLinksScriptBlock, $htmlScriptBlockArray, $emeaTotalNumberOfCalls, $namTotalNumberOfCalls, $apacTotalNumberOfCalls, $path, $logFilePath) {
[...]
# Using Script Blocks, add the pages generated during the analysis to the HTML Report
foreach($htmlScriptBlock in $htmlScriptBlockArray){
Invoke-Command -ScriptBlock ($htmlScriptBlock)
}
[...]
}
The ScriptBlock are created using set of values gathered from a list of servers' logs and added to the arrayList of ScriptBlocks in the Create-utilizationReportPage function (again, I've only kept the relevant code):
function Create-utilizationReportPage ($matchedLines, $ipAddress, $hostname, $pageId, $utilisationReportTemplate, $htmlScriptBlockArray) {
# Retrieve the content of the Utilisation Report Template as a RAW string (Here-String)
$htmlPageCodeBlock = Get-Content $utilisationReportTemplate -Raw
# Create the Script Block that contains the HTML page
$htmlPageScriptBlock = {
# Get the "Total number of calls" information
$timeArray = $matchedLines.time
$participantNumberArray = $matchedLines.participantNumber
[...]
# Update the page ID in the template
$htmlPageCodeBlock = $htmlPageCodeBlock -replace '%PAGE_ID%', "$pageId"
# Update the page header information in the template
$htmlPageCodeBlock = $htmlPageCodeBlock -replace '%PAGE_HEADER%', "$ipAddress [$hostname]"
Invoke-Command -ScriptBlock ([scriptblock]::Create($htmlPageCodeBlock))
}.GetNewClosure()
# Add the Page's script block to the Script Blocks array
$htmlScriptBlockArray.Add($htmlPageScriptBlock)
}
These are called in the script below:
$currentDate = Get-CorrectDate $latestFolder
# The global Log file
$logFilePath = "$scriptPath/Logs/logs_$currentDate.txt"
$serversList = "$scriptPath/Config/$configFileName"
# If the Servers list exists, retrieve the Servers list
if (Test-Path $serversList) {
# Get the data from the file
[xml]$servers = Get-Content $serversList
# Select only the Servers information
$nodes = $servers.SelectNodes("//server")
try {
[...]
# Iterate through the Servers list
foreach ($node in $nodes) {
# Get the Server IP Address
$ipAddress = $node.ip
# Get the Server Hostname
$hostname = $node.hostname
# Get the "Debug Utilization" lines from the logbundle's syslog files
$matchedLines = [System.Collections.ArrayList]@(Find-UtilizationLinesInLogs $currentDate "$scriptPath\Data\$currentDate\logs_$ipAddress" "host:server: \[USAGE\] : \{`"1`" : ")[-1]
# Add the Server's participants throughout the day
switch ($hostname) {
{$_.Contains("emea")} {
# Update EMEA total number of participants
Update-ZoneTotalParticipants ([ref]$emeaServer) ([ref]$emeaTotalNumberOfCalls) $matchedLines
}
{$_.Contains("nam")} {
# Update NAM total number of participants
Update-ZoneTotalParticipants ([ref]$namServer) ([ref]$namTotalNumberOfCalls) $matchedLines
}
{$_.Contains("apac")} {
# Update APAC total number of participants
Update-ZoneTotalParticipants ([ref]$apacServer) ([ref]$apacTotalNumberOfCalls) $matchedLines
}
}
# Get the CPU Utilization values lines from the logbundle's sysdebug files
$cpu = Get-CpuUsage "$scriptPath\Data\$currentDate\logs_$ipAddress\sysdebug"
# Get the Memory Utilization values lines from the logs' sysdebug files
$memory = Get-MemoryUsage "$scriptPath\Data\$currentDate\logs_$ipAddress\sysdebug"
# Export the "Debug Utilization" to a CSV file
Save-csvUtilizationReport $matchedLines "$scriptPath\Output\$currentDate" "$ipAddress" "$hostname" $logFilePath
# Draw graphs from the "Debug Utilization" information and then export it to an HTML File
Create-utilizationReportPage $matchedLines "$ipAddress" "$hostname" $pageId $utilisationReportTemplate $htmlScriptBlockArray
# Add the new navigation link to the Navigation Links array
Add-htmlNavLink $navLinksArray "$ipAddress" "$hostname" $pageId $logFilePath
# Increment the Page ID counter
$pageId += 1
}
# Create an Here-String from the Navigation Links array
$OFS = ""
$navLinksCode =@"
$($navLinksArray)
"@
$OFS = " "
# Create a script block from the Navigation Links
$navLinksScriptBlock = [scriptblock]::Create($navLinksCode)
# Save the daily HTML utilization report
Save-utilizationReport $currentDate $navLinksScriptBlock $htmlScriptBlockArray $emeaTotalNumberOfCalls $namTotalNumberOfCalls $apacTotalNumberOfCalls "$scriptPath\Output\$currentDate" $logFilePath
}
catch
{
Write-Logs $logFilePath "Error: $($_.Exception.Message)"
exit 1
}
}
Everything is working fine and as expcted except for the first set of values in the first page which is somehow the sum of all the other set of values...
For example when I have 3 pages, I can see that the collected values are correct when the Find-UtilizationLinesInLogs function is executed:
matchedLines.participantNumber: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 5 5 5 5 4 4 4 4 4 12 14 14 15 16 16 16 7 7 7 7 8 14 17 18 19 18 19 19 20 16 16 16 15 7 7 7 7 5 4 4 4 4 4 4 4 4 1 1 0 0 0 3 6 5 5 5 5 9 14 16 18
matchedLines.participantNumber: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 8 9 10 10 10 10 9 9 9 9 9 9 8 8 8 8 8 8 8 8 8 8 9 4 12 15 11 14 14 13 12 12 12 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0
matchedLines.participantNumber: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 2 0 0 0 3 9 10 10 9 8 10 9 9 10 10 10 10 10 11 11 11 11 8 7 8 8 7 7 7 7 7 7 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 4 16 17 16 16
But when the ScriptBlocks are executed using the Invoke-Command in the foreach loop, the first batch of values is systematically the sum of the 3 sets of values while the following ones are correct:
matchedLines.participantNumber inside Create-utilizationReportPage: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 5 6 6 7 7 5 5 5 8 29 33 34 34 34 36 34 25 26 26 26 27 32 36 37 38 37 35 34 36 32 31 32 26 26 29 25 22 20 18 17 17 17 6 6 6 6 3 3 2 2 2 5 8 7 7 7 10 25 31 32 34
matchedLines.participantNumber inside Create-utilizationReportPage: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 1 1 1 8 9 10 10 10 10 9 9 9 9 9 9 8 8 8 8 8 8 8 8 8 8 9 4 12 15 11 14 14 13 12 12 12 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 0 0 0 0
matchedLines.participantNumber inside Create-utilizationReportPage: 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 1 2 0 0 0 3 9 10 10 9 8 10 9 9 10 10 10 10 10 11 11 11 11 8 7 8 8 7 7 7 7 7 7 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 4 16 17 16 16
I have tried many things without success so if someone has any hint of what can go wrong, it would be great!
Thanks for your help!
So! I finally found out what was wrong.
I suspected that my problem was related to arrayList copies or at least variable copies... so I tried to remove this part of the script where I extract the $macthedLines values and copy them into global arrays using references:
# Add the Server's participants throughout the day
switch ($hostname) {
{$_.Contains("emea")} {
# Update EMEA total number of participants
Update-ZoneTotalParticipants ([ref]$emeaServer) ([ref]$emeaTotalNumberOfCalls) $matchedLines
}
{$_.Contains("nam")} {
# Update NAM total number of participants
Update-ZoneTotalParticipants ([ref]$namServer) ([ref]$namTotalNumberOfCalls) $matchedLines
}
{$_.Contains("apac")} {
# Update APAC total number of participants
Update-ZoneTotalParticipants ([ref]$apacServer) ([ref]$apacTotalNumberOfCalls) $matchedLines
}
}
And bingo, this time the values written down in the first PSWriteHTML page are the correct one!
So I focused on the Update-ZoneTotalParticipants function which is doing the values copy:
function Update-ZoneTotalParticipants ([ref][int]$ServerNumber, [ref][System.Collections.ArrayList]$totalNumberOfCalls, $values) {
# If this is the first server to be analysed in the zone
if ($ServerNumber.value -eq 1) {
# Copy the Utilization lines of the server
$totalNumberOfCalls.value = $values
# Increment the zone's server counter
$ServerNumber.value += 1
}
# If this at least the 2nd server to be analysed in the zone
elseif ($ServerNumber.value -gt 1) {
# Parse the server matched lines, get the participantNumber value and add it to the total
0..($totalNumberOfCalls.value.Count - 1) | ForEach-Object {
if ($_ -le ($values.Count - 1)) {
$totalNumberOfCalls.value[$_].participantNumber = [int]($totalNumberOfCalls.value[$_].participantNumber) + [int]($values[$_].participantNumber)
}
# If there are less objects in the current server matched lines, add a 0 instead
else {
$totalNumberOfCalls.value[$_].participantNumber = [int]($totalNumberOfCalls.value[$_].participantNumber) + 0
}
}
}
}
The only part of the code where $matchedLines is involved is $totalNumberOfCalls.value = $values so it certainly is were the array manipulation goes wrong.
So I dug around ArrayList copies or clones and found out that I was not doing a deep copy of the object and that it could cause issues.
I used Petru Zaharia's solution in this thread to update the function:
# Copy the Utilization lines of the server
$totalNumberOfCalls.value = $values
# replaced with:
# Copy the Utilization lines of the server : Serialize and Deserialize data using PSSerializer:
$_TempCliXMLString = [System.Management.Automation.PSSerializer]::Serialize($matchedLines, [int32]::MaxValue)
$totalNumberOfCalls.value = [System.Management.Automation.PSSerializer]::Deserialize($_TempCliXMLString)
And now everything works as expected.
Thanks guys for your support!