VMware Cloud Foundation with kubernetes: Consolidated build on VxRail

Example of VMware VCF 4 on VxRail 7.

Perform VxRail using internal VCenter server and prepare for CloudBuilder. Deploy Cloudbuilder.

Bring up validation. Note: In the Cloudbuilder spreadsheet, select No for AVN network. This means you will need to deploy the Edge Cluster manually after bringup.

complete brinup


Log into SDDC manager. Also log into vCenter and NSX-T to make sure all looks well. Enable CEIP in VCenter. In NSX-T there will not be a Teir-0 or Tier-1 and no Edge nodes or Edge Cluster yet. We will do that next, but first review SDDC manager and vCenter. Update Repository Settings with VMware credentials. Download and apply any updates through SDDC manager.

Deploy the Edge cluster in SDDC manager.

You will need to know 4 vlan’s / IP subnets to complete. 1. vCenter subnet, 2. Uplink 1 subnet/vlan. 3. Uplink 2 subnet/vlan. 4. Edge Node TEP subnet / vlan. You will also need to configure your BGP neighbor IP and password for the upstream BGP pair. Likewise, from upstream router, configure the downstream BGP ASN neighbor and password.

Validate and complete the Edge Cluster deployment, making sure your edge transport nodes have DNS entry for FQDN.

In NSX-T click on the T0 deployed and select the hyperlink for BGP Neighbors


Verify BGP is established upstream

Edge Node to look at the one of the BGP connections on 1 of 2 subnets

Click on the Edge Node dropdown menu to ensure BGP is Established state


Enable trust in NSX-T to the management vcenter.

Click EDIT and Select Enable Trust toggle to on – Green.

Ensure the Edge Cluster has WCPReady tag.

Click Manage select tag and save.

You’ll need to create a IP Prefix on the Tier-0.


Create Custom Route Map and add the previously created Prefix.


Verify Custom Route Map is associated to the Custom Route Map.

Add the customer route map to route re-distribution.

Deploy Kubernetes via SDDC manager Solutions link. It will verify things are in order and direct you to vCenter to complete the deployment. Ensure the Ingress and Egress CIDRs are routable in your network. In other words, these subnets will be used for pods to communicate with repos like github, so it needs to be a block of IP’s that are capable of being routed. You will not need to create anything physically, no VLAN’s, no gateway.. just two blocks of IP’s with nothing on them.


After 15 minutes / 20 minutes or so, you should see the config status as running. Click refresh.


If you’ve made it this far, you should be able to create a namespace for workload test.

Create a namespace in vCenter, assign a user with edit privilege and associate a vSAN storage policy.

VCenter hosts and clusters view shows new demo namespace.


Get logged into Kubernetes from a client system. I chose to stand up a linux vm for this. Once logged in, download the yelb file. This will deploy the test yaml application with a NSX-T load balanced frontend IP for service.


https://github.com/mreferre/yelb/blob/master/deployments/platformdeployment/Kubernetes/yaml/yelb-k8s-loadbalancer.yaml


Helping others modernize datacenter infrastructure.

Thanks,