Auto-Scale vRA workloads with vROPS,vRO and NSX.

Auto-Scale vRA workloads with vROPS,vRO and NSX.

I am excited to put out a post on one of the greatest idea which had been in my mind for long 🙂 and its a solution to create a self healing data-center.

Before i start with my post i would like to mention a special thanks to my colleagues at VMware mentioned below who made this possible, Carlsen, Dean,  Sajal Debnath, Diwan Chandrabose and Ajay Kalla who assisted me with SNMP trap logic and the Auto Scale Workflow which made this post possible.

Ok so lets get started, so we had a customer who had a requirement to auto scale his virtual infrastructure based on the number of hits on his VMware NSX based load balancer hits.

Here’s a view on the blueprint itself, as you can see below, the maximum deployment limit of web-servers has been set to 10, which means it can scale out to maximum count of 10.

Well right now in vRA 7.2 we have an option to Scale-IN and Scale-Out , which is a manual option . the end user needs to manually click on Scale-IN or Scale-OUT to scale the virtual machines.

Now if we need to get this automated we need to basically using vROPS (vRealize Operations) with a NSX management pack installed combined with vRO (vRealize Orchestrator) to take a remediation action to auto scale out / in on the set of servers.

Ok so lets go take a step by step approach on how you can do this in your labs.

Setup an Alarm in vROPS with NSX management pack installed.

Lets create a symptom for the base object type NSX-vSphere Edge Load Balancer Service HTTP Request Rate.

Next create a alert definition with the below characteristics taking into account the symptom definition you created earlier.

Once you complete the above two steps, setup the notifications as below for this particular alert as below with SNMP trap plugin as the method of output.

Next lets set up the Outbound settings to send the SNMP traps for alerts generated via vROPS to be captured by vRO SNMP policy, also note vRA 7.2 comes with and internal vRO hence we would be redirecting the SNMP to this, this can be done as below

Ok so we are all good and done from the vROPS side, next lets move on to creating a SNMP policy in vROPS which would trap these alerts, decipher them and extract the Loadbalancer name and pass it onto a vRO workflow which would get the pool members associated with it , get the first server which is part of the pool and scale it out 🙂

Setting up vROPS SNMP policy and Workflow to perform the auto-scale.

First lets start by importing the vRO workflow package into your vRO setup, use the below link to download it.

Once you download the package import it into your vRO setup, there are numerous blogs out there which detail this so i wont be going into details on these.

Next lets setup the SNMP policy which would listen for the traps and take appropriate action to invoke the necessary workflow.

First lets add the vROPS server as an SNMP device, this is pretty straighforward, head over the SNMP folder and select the Register an snmp device workflow and feed in the below details.

Also make sure you select PORT number 4000, as that’s the port number we had configured in vROPS under the outbound settings tab.

Next lets get the Workflow ID for the ScaleDeploymentByLB workflow, once you import the vRO package from the link above the ID may differ, hence this is an important step.

Once you get the ID, head over to the SNMP policy under the administer tab, edit the SNMP policy Scripting and make sure you type in the correct workflow id, and save and close the SNMP policy.

Next apply this SNMP policy on the SNMP device which is our vROPS server.

Once we complete this step, next head over to the “Run” tab and start the policy created, you would immediately see that TRAPS are being captured and would get listed on the logs tab 🙂

Ok, next lets start to simulate some load on the web-server, there is a basic web page i created for this, go ahead and download this link for http request it would show in yellow and green for https calls 🙂

also the below link highlights on how to setup an ASP .NET site with IIS, ASP Application IIS

Basically, its a PowerShell script which used the Invoke-WebRequest cmdlet to generate the loads, also note just to go berserk! and ballistic i made it to 30000 hits 😀

If you need to invoke a HTTP request use the below.

1..30000 | % {Invoke-WebRequest -Uri http://172.16.10.12/lb/ -UseBasicParsing}

Similarly if we need to invoke a HTTPS request you need to use the below script.

### Ignore TLS/SSL errors
 
add-type @"
    using System.Net;
    using System.Security.Cryptography.X509Certificates;
    public class TrustAllCertsPolicy : ICertificatePolicy {
        public bool CheckValidationResult(
            ServicePoint srvPoint, X509Certificate certificate,
            WebRequest request, int certificateProblem) {
            return true;
        }
    }
"@
[System.Net.ServicePointManager]::CertificatePolicy = New-Object TrustAllCertsPolicy
 

1..30000 | % {Invoke-WebRequest -Uri https://172.16.10.12/lb/ -UseBasicParsing}

Once the load comes in, you would see a critical alert generated in vROPS , which would trigger the vRO workflow via SNMP trap

Also you would see the logs as below in vRO which states the alert was trapped and the Auto Scale workflow was successfully launched 🙂

Here’s the view from vRO where you can see the workflow successfully launches, also the view from vRA where the auto-scale action was triggered.

view from vRO Logs

And the final result you would see the number of web-servers in our environment have incremented.

view from deployment view.

I ran the script to generate load two times, hence you see two web-servers 🙂

Here’s the video which showcases the entire solution!

I hope you enjoyed this explosive blogpost and found the information useful, hope this gives you great ideas on how to perform similar automation to address various self healing scenarios raised in your environment 🙂

If you found this information useful, do check out how to use Web Hook shims to enhance this solution.

VMware WebHook Shim and AutoScale with vRO

Keep Rocking!