Download Technical WhitePapers from VMware Site – PowerShell

Here is an automated script which will download all the technical white papers through vmware.com,

We are using a combination of both the Invoke-WebRequest cmdlet and the Start-BitsTransfer cmdlet.

I had to build a logic to download the files as we have multiple pages to download them.

The logic built below was based on the website to download the tech resource paper.

Every page contains links to download the file and it shows up in the format

“https://www.vmware.com/resources/techresources/cat/91?from=$i”, where $i increments in the count of 30, as we have 30 contents per page and the maximum count is 630.

So based on this we are using the Invoke-WebRequest cmdlet to get the links of the pdf files and once we get the links for the pdf files we download them to a destination location using Start-BitsTransfer.

We have enclosed the Start-Bitstransfer cmdlet inside a try catch statement so that i can catch and errors and those are redirected to a notepad file.

So lets take a look at the script.

function Download-VMWWhitePapers ($location) {


# Gathering the Script start time which would be used later to download 
# to calculate the time script ran.

$ScriptStart = (Get-Date)

# Credentials to connect to the site and start the bits transfer.
$credential = Get-Credential


<# 

The logic built below was based on the website to download the tech 
resource paper. Every page contains links to download the file and it 
shows up in the format

"https://www.vmware.com/resources/techresources/cat/91?from=$i", where $i 
 increments in the count of 30 as we have 30 content per
 page and the maximum count is 630

So based on this we are using the Invoke-WebRequest cmdlet to get the links 
of the pdf files and once we get the links for the pdf files
we download them to a destination location using Invoke-Webrequest

We have enclosed the Start-Bitstransfer cmdlet inside a try catch statement so
that i can catch and errors and those are redirected to a notepad file

#>

$i=0

$hrefs = @()

while ($i -le 600)

{

$counter = $i

$rxml=Invoke-WebRequest -Uri "https://www.vmware.com/resources/techresources/cat/91?from=$i" -Credential $credential

$hrefs += $rxml.Links | ?{$_.href -match "pdf"} | select href

$i = $i + 30

}


foreach ($href in $hrefs)

{

try {

Start-BitsTransfer -Source $href.href -Destination $location -Credential $credential -ErrorAction Stop

}

catch {

Write-Host "Error Downloading file $($href.href) " -ForegroundColor Red

$ErrorMessage = $_.Exception.Message

$ErrorMessage | Out-File -FilePath $location\error.txt -Append

}

}

$ScriptEnd = (Get-Date)

$RunTime = New-Timespan -Start $ScriptStart -End $ScriptEnd

“Elapsed Time: {0}:{1}:{2}” -f $RunTime.Hours,$Runtime.Minutes,$RunTime.Seconds


}

No manual download of the whitepapers, all downloaded and saved automatically at one shot to your local pc.

When i executed it in my Test Environment, it took about 13 minutes to complete!, isn’t that Neat!!.

Please disregard the colour coding for the errors i used, i used the “Dark Red”, and it was giving me colour blindness when i tried to read through it on my powershell console.

Here’s a view of the error notepad file created.