How to avoid the 1GB $env:TEMP) in Azure Automation Powershell

Question

I'm having a problem in a Powershell Runbook. We scan the O365 AuditLogs and write the results to disk, up to this moment we userd the $env:TEMP disk. After the scan we upload the csv files into a storage account. But not our csv are getting bigger than 1GB. THis s a part of the code ..

    $results = Search-UnifiedAuditLog -StartDate $currentStart -EndDate $currentEnd -SessionId $sessionID -SessionCommand ReturnLargeSet -ResultSize 5000
    if (($results | Measure-Object).Count -ne 0) {
        $outputFile = $outputPath + $recordType + "/Data_" + $currentStart.ToString("yyyyMMddHHmm") + "_" + $currentEnd.ToString("yyyyMMddHHmm") + "_" + $fileloopCount.ToString() + ".csv"
        Check-Dir -dirName (Split-Path -Path $outputFile)
        $results | export-csv -Path $outputFile -Append -NoTypeInformation

Is there another way of working, I could not append and create multiple small files, upload them after creation and delete them, but I'm wondering if there would be another way.

Use Export-Csv directly to a blob-storage ?

Kr, Harry

Answer 1

You can follow below approach to upload 1GB file to storage account using powershell.

You cannot access the file located in local path from Azure Runbook, follow the Stack Link for more details by @Pankaj Rawat

I also tried to upload 1GB file from local machine to Azure storage using Azure Runbook, getting below result.

Use Export-Csv directly to a blob-storage ?

Export-Csv command in PowerShell is used to export data to a CSV file, not directly to a blob storage in Azure.

You can use below powershell script to upload file from local to Azure Storage account.

#Authenticate to Azure
Connect-AzAccount -Identity

 Set the storage account name and container name
$storageAccountName = "emobro"
$containerName = "rithwik"
$storageAccountKey ="StoarageaccountKey"

#Get the storage account context

$storageContext = New-AzStorageContext -StorageAccountName $storageAccountName -StorageAccountKey $storageAccountKey


 Set the file path and name of the CSV file to upload
$localFilePath = "C:\Users\v-vallepuv\Downloads\samplefile.csv"
$blobName = "sample.csv"

 Upload the CSV file to Azure Blob Storage
Set-AzStorageBlobContent -Context $storageContext -Container $containerName -File $localFilePath -Blob $blobName

Once ran the above powershell code, file got uploaded to Azure storage account successfully as below.