How can I download a folder from a webserver with authentication?

Question

I need a script that downloads a certain folder and all its subfolders + files to my pc from a webserver. It needs to be in powershell. I searched a bit and found this:

Invoke-WebRequest http://www.example.com/package.zip -OutFile package.zip

I get this error when I try to run it. But I can't figure out how I can pass the username and password with it. If anyone can help me that would be greatly appreciated! Also how can I specify the folder it should be saved to? Thanks in advance

Answer 1

Following up regarding my comment.

I ask that question because some sites require you to be specific in the credential presentation vs just one blob of stuff. For example:

$credentials = Get-Credential

$webServerUrl               = 'http://SomeWebSite'
$r                          = Invoke-WebRequest $webServerUrl -SessionVariable my_session
$form                       = $r.Forms[0]
$form.fields['Username']    = $credentials.GetNetworkCredential().UserName
$form.fields['Password']    = $credentials.GetNetworkCredential().Password

$InvokeWebRequestSplat = @{
    Uri        = $($webServerUrl + $form.Action) 
    WebSession = $my_session 
    Method     = 'GET '
    Body       = $form.Fields
}
$r = Invoke-WebRequest @InvokeWebRequestSplat

Update

The follow-up to the comment. This is using IE with PowerShell for site automation.

# Scrape the site to find form data
$url = 'https://pwpush.com'
($FormElements = Invoke-WebRequest -Uri $url -SessionVariable fe)  
($Form = $FormElements.Forms[0]) | Format-List -Force
$Form | Get-Member
$Form.Fields

# Use the info on the site
$IE = New-Object -ComObject "InternetExplorer.Application"

$FormElementsequestURI = "https://pwpush.com"
$Password = "password_payload"
$SubmitButton = "submit"

$IE.Visible = $true
$IE.Silent = $true
$IE.Navigate($FormElementsequestURI)
While ($IE.Busy) {
    Start-Sleep -Milliseconds 100
}

$Doc = $IE.Document
$Doc.getElementsByTagName("input") | ForEach-Object {
    if ($_.id -ne $null){
        if ($_.id.contains($SubmitButton)) {$SubmitButton = $_}
        if ($_.id.contains($Password)) {$Password = $_}
    }
}

$Password.value = "1234"
$SubmitButton.click()

Invoke-WebRequest is Powershell's version of curl. Its alias is even named curl.

SO, in the IVR use case, all you really need to do something like the Facebook and Linkedin examples:

$cred  = Get-Credential
$login = Invoke-WebRequest 'facebook.com/login.php' -SessionVariable 'fb'
$login.Forms[0].Fields.email = $cred.UserName
$login.Forms[0].Fields.pass = $cred.GetNetworkCredential().Password
$mainPage = Invoke-WebRequest $login.Forms[0].Action -WebSession $fb -Body $login -Method Post 

$cred = Get-Credential
$login = Invoke-WebRequest 'https://www.linkedin.com/uas/login?goback=&trk=hb_signin' -SessionVariable 'li'
$login.Forms[0].Fields.email = $cred.UserName
$login.Forms[0].Fields.pass = $cred.GetNetworkCredential().Password
$mainPage = Invoke-WebRequest $login.Forms[0].Action -WebSession $LI -Body $login -Method Post 

Yet, notice I on the FB/LI login page, and I'd need to know that even existed before trying this. Note this is old code, That I've not used in a very long while and I don't have a FB account. I passed this on to someone who did.