The following Powershell command fails to copy the entire file; a few characters are always missing from the end.
[System.IO.StreamWriter]::new('C:\TEMP\b.csv', [System.Text.Encoding]::UTF8).Write([System.IO.StreamReader]::new('C:\Temp\a.csv', [System.Text.Encoding]::GetEncoding('iso-8859-1')).ReadToEnd())
I suspect it's because the writer doesn't flush the last bits because this does copy the entire file:
$X = [System.IO.StreamReader]::new('C:\Temp\a.csv', [System.Text.Encoding]::GetEncoding('iso-8859-1'))
$Y = [System.IO.StreamWriter]::new('C:\TEMP\b.csv', [System.Text.Encoding]::UTF8)
$Y.Write($X.ReadAll())
$X.Dispose()
$Y.Dispose()
Is it possible to dispose of (and flush) the reader & writer without having created variables to reference them?
EDIT: I tried this one-liner using streamreader/writer hoping the reader's read buffer would directly transfer to the writer's write buffer rather than waiting for the reader to read the entire file into memory and then write. What technique might achieve that?
I personally find code that does not declare a single-use object to often be cleaner / more succinct, but my focus is on understanding whether/how objects dispose of themselves, not the style.
There's no need to eschew variables or write on one line, but this behaviour isn't what I expected. In VBA one can copy a file like so and trust it will dispose of itself properly without having to declare a variable and explicitly flush (I think).
Sub Cpy()
With New Scripting.FileSystemObject
.CreateTextFile("c:\Temp\Out.txt").Write .OpenTextFile("C:\Temp\In.txt", ForReading).ReadAll
End With
End Sub
One can achieve similar behaviour in a custom VBA class by writing appropriate 'clean-up' code in a Class_Terminate() procedure. I assumed the Streamwriter would similarly flush data upon termination via garbage collection once the line executes and there's no longer a variable associated with it.
I also noticed that the file remains locked and I cannot delete it until I close the powershell session. Is there a way to flush contents and release the file without having declared a variable to work with?
For the specific use case given, Santiago Squarzon's helpful answer is indeed the best solution: using the static methods of the static System.IO.File class obviates the need for instances representing files that require calling a .Close() method or explicit disposing of.
To read lazily and therefore support overlapping reading and writing, line by line, you can use the static [System.IO.File]::ReadLines() and [System.IO.File]::WriteAllLines() methods, but note that this approach (a) invariably uses platform-native [Environment]::NewLine-format newlines in the output file, irrespective of what newline format the input file uses, and (b) invariably adds a trailing newline in this format, even if the input file had no trailing newline.
Overcoming these limitations would require use of a lower-level, raw-byte API, System.IO.FileStream - which again requires explicit disposal (see bottom section).
Given that your approach reads the entire input file into memory first and then writes, you could even make do with PowerShell cmdlets, assuming you're running PowerShell (Core) 7+, which writes BOM-less UTF-8 files by default, and whose -Encoding parameter accepts any supported encoding, such as ISO-8859-1 in your case:
# PowerShell (Core) 7+ only
Get-Content -Raw -Encoding iso-8859-1 C:\TEMP\a.csv |
Set-Content -NoNewLine C:\TEMP\b.csv
As for your general question:
As of PowerShell (Core) 7.2.1:
PowerShell has no construct equivalent to C#'s using statement that allows automatic disposing of objects whose type implements the System.IDisposable interface (which, in the case of file I/O APIs, implicitly closes the files).
GitHub issue #9886 discusses adding such a statement, but the discussion suggests that it likely won't be implemented.
Note: While PowerShell does have a family of statements starting with keyword using, they serve different purposes - see the conceptual about_Using help topic.
A future PowerShell version will support a clean { ... } (or cleanup { ... }) block that is automatically called when an advanced function or script terminates, which allows performing any necessary function-script-level cleanup (disposing of objects) - see RFC #294.
It is up to each type implementing the IDisposable interface whether it calls the .Dispose() methods from the finalizer. Only if so is an object automatically disposed of eventually, by the garbage collector.
For System.IO.StreamWriter and also the lower-level System.IO.FileStream class, this appears not to be the case, so in PowerShell you must call .Close() or .Dispose() explicitly, which is best done from the finally block of a try / catch / finally statement.
You can cut down on the ceremony somewhat by combining the aspects of object construction and variable assignment, but a robust idiom still requires a lot of ceremony:
$x = $y = $null
try {
($y = [System.IO.StreamWriter]::new('C:\TEMP\b.csv', [System.Text.Encoding]::UTF8)).
Write(
($x = [System.IO.StreamReader]::new('C:\Temp\a.csv', [System.Text.Encoding]::GetEncoding('iso-8859-1'))).
ReadToEnd()
)
} finally {
if ($x) { $x.Dispose() }
if ($y) { $y.Dispose() }
}
A helper function, Use-Object (source code below) can alleviate this a bit:
Use-Object
([System.IO.StreamReader]::new('C:\Temp\a.csv',[System.Text.Encoding]::GetEncoding('iso-8859-1'))),
([System.IO.StreamWriter]::new('C:\TEMP\b.csv', [System.Text.Encoding]::UTF8)) `
{ $_[1].Write($_[0].ReadToEnd()) }
Note how the disable objects passed as the first argument are referenced via $_ as an array in the script-block argument (as usual you may use $PSItem in lieu of $_).
A more readable alternative:
Use-Object
([System.IO.StreamReader]::new('C:\Temp\a.csv',[System.Text.Encoding]::GetEncoding('iso-8859-1'))),
([System.IO.StreamWriter]::new('C:\TEMP\b.csv', [System.Text.Encoding]::UTF8)) `
{
$reader, $writer = $_
$writer.Write($reader.ReadToEnd())
}
Or, perhaps even better, albeit with slightly different semantics (which will rarely matter),[1] as Darin suggests:
Use-Object
($reader = [System.IO.StreamReader]::new('C:\Temp\a.csv',[System.Text.Encoding]::GetEncoding('iso-8859-1'))),
($writer = [System.IO.StreamWriter]::new('C:\TEMP\b.csv', [System.Text.Encoding]::UTF8)) `
{
$writer.Write($reader.ReadToEnd())
}
Use-Object source code:
function Use-Object {
param(
[Parameter(Mandatory)] $ObjectsToDispose, # a single object or array
[Parameter(Mandatory)] [scriptblock] $ScriptBlock
)
try {
ForEach-Object $ScriptBlock -InputObject $ObjectsToDispose
}
finally {
foreach ($o in $ObjectsToDispose) {
if ($o -is [System.IDisposable]) {
$o.Dispose()
}
}
}
}
[1] With this syntax, you're creating the variables in the caller's scope, not in the function's, but this won't matter, as long as you don't try to assign different objects to these variables with the intent of also making the caller see such changes. (If you tried that, you would create a function-local copy of the variable that the caller won't see) - see this answer for details.