I am new to PowerShell, and just for practice I got instructions to, "Read in fortune 100 URLs to an array...then using runspaces, connect to each one and write the page you get with Invoke-WebRequest to file." Then I was given this link: https://www.zyxware.com/articles/4344/list-of-fortune-500-companies-and-their-websites
I have been reading all about runspaces. Right now I'm working on getting my links into an array that I can then pull the first 100 URLs out of. Any help/advice would be appreciated.
The issue that I have now is that when I call the link variable, it won't give me the links. It seems like I should be able to just call the $link variable so that I can get all the links and put it into an array. The only way that I can get all the links to come up is by using "$page.Links.href". Can someone please explain to me why calling that variable won't work?
$page = Invoke-WebRequest https://www.zyxware.com/articles/4344/list-of-fortune-500-companies-and-their-websites
foreach($1 in $page.Links){
if($1.href -like '*www*com'){
Write-Host $1.href
$link = $1.Links.href
}
}
$RunspacePool = [RunspaceFactory]::CreateRunspacePool(1, $Throttle)
$RunspacePool.Open()
Go at this one step at a time.
Before ever doing something as advanced as runspaces, you should take a few quick PowerShell online courses and materials so you have a baseline. Especially since you say you've never really used PoSH.
https://mva.microsoft.com/en-us/training-courses/getting-started-with-microsoft-powershell-8276
https://youtube.com/results?search_query=beginner+powershell
There are several no-cost eBooks online ...
...as well as a ton of downloadable scripts for many things.
and long list's of resources...
Windows PowerShell Survival Guide https://social.technet.microsoft.com/wiki/contents/articles/183.windows-powershell-survival-guide.aspx
First understand how to use the PoSH WebCmdlets.
Get-Command -Name Invoke-WebRequest| Format-Table -AutoSize
CommandType Name Version Source
----------- ---- ------- ------
Cmdlet Invoke-WebRequest 3.1.0.0 Microsoft.PowerShell.Utility
# Get paramter, example, full and Online help for a cmdlet or function
(Get-Command -Name Invoke-WebRequest|).Parameters
Get-help -Name Invoke-WebRequest| -Examples
Get-help -Name Invoke-WebRequest| -Full
Get-help -Name Invoke-WebRequest| -Online
Get-Command -Name Invoke-RestMethod | Format-Table -AutoSize
CommandType Name Version Source
----------- ---- ------- ------
Cmdlet Invoke-RestMethod 3.1.0.0 Microsoft.PowerShell.Utility
# Get paramter, example, full and Online help for a cmdlet or function
(Get-Command -Name Invoke-RestMethod ).Parameters
Get-help -Name Invoke-RestMethod -Examples
Get-help -Name Invoke-RestMethod -Full
Get-help -Name Invoke-RestMethod -Online
Then search using the string 'Invoke-WebRequest to parse website for URLs'
Do that website URL parsing first, get that where you need it then move on to runspaces.
If you can't make the first part happen the runspaces is moot.
So, try to get the URL's with code you put together, then if you have issue with that, post what you've tried and the forum will try an pitch in. The same thing will apply to your second step of the runspace part.