Container Service Extension 2.5 Installation: Part 3

In Parts 1 and 2 of my series on installing and configuring the Container Service Extension for VMware Cloud Director, I focused on setting the CSE server up to support CSE Standard Kubernetes cluster creation.

CSE Standard clusters are comprised of deployed vApps that utilize NSX-V networking resources, utilizing Weave as the Container Network Interface for the Kubernetes clusters. In Part 3 of my series, I wanted to take some time to look at configuring the CSE Server to support the creation of CSE Enterprise Kubernetes clusters. CSE Enterprise clusters are comprised of VMware Enterprise PKS Kubernetes clusters deployed on top of NSX-T networking resources, utilizing the NSX Container Plugin as a CNI. CSE Enterprise brings enterprise grade features and functionality to CSE that include, but are not limited to:

  • HA, multi-master Kubernetes clusters
  • Dynamic persistent storage provisioning with the vSphere Cloud Provider integration
  • Automated Day 1 and Day 2 Kubernetes cluster management via Bosh Director
  • Microsegmentation capability for Kubernentes resources via integration with NSX-T
  • Automated creation of Kubernetes service type LoadBalancer and ingress resrouces via NSX-T L4/L7 load balancers
  • Support for Harbor, an open source cloud native registry

Continue reading “Container Service Extension 2.5 Installation: Part 3”

Exploring the Nirmata Kubernetes Extension for VMware Cloud Director

If you’ve been following my blog, you know that a lot of the content I publish focuses on VMware’s Container Service Extension and it’s integration with VMware Cloud Director, which allows service providers to create a Kubernetes-as-a-Service experience for their tenants utilizing their existing VCD-managed infrastructure.

Recently, myself and my colleague at VMware, Daniel Paluszek partnered with Nirmata to perform some testing on their new Kubernetes Extension for VMware Cloud Director. The Nirmata Kubernetes Extension for VCD builds on the rich UI experience already present in the VCD tenant portal by providing a workflow for provisioning Kubernetes clusters via CSE using the native UI.

The Native CSE Experience

As I’ve written about in my previous posts on CSE, once a service provider enables a tenant to provision Kubernetes clusters via CSE, tenants will use the vcd-cli with a CSE extension enabled to provision and manage Kubernetes clusters. For example, a tenant would log in to their VCD Org through the vcd-cli and issue the following command to create a Kubernetes cluster via CSE:

$ vcd cse cluster create k8-cluster-1 --network outside --nodes 1

where k8-cluster-1 is the name of the cluster, --network is the OvDC network the cluster will nodes will utilize, and --nodes 1 defines the number of worker nodes the cluster will contain.

While many users are familiar enough with a CLI to adapt to this method of resource provisioning, one piece of feedback we get from our partner community is that they’d like to offer a native UI experience in the tenant portal to allow their end customers to more intuitively provision Kubernetes clusters via VCD. That’s where the Nirmata Kubernetes Extension for VCD comes in…

Continue reading “Exploring the Nirmata Kubernetes Extension for VMware Cloud Director”

Container Service Extension 2.5 Installation: Part 2

Building on Part 1 of my series on installing VMware’s Container Service Extension 2.5.0, in this post, I’ll walk through the process of configuring a client server to interact with CSE via the vcd-cli tool. I’ll also walk through the process of onboarding a tenant as well as the workflow, from the tenant’s perspective, of provisioning and managing a Kubernetes cluster.

Configuring a CSE Client

Now that I’ve deployed my CSE server, I’ll need to utilize the the vcd-cli tool with the CSE client extension enabled in order to interact with the CSE service. For the client server, I am, again, utilizing a CentOS 7.6 server and a Python 3.7.3 virtual environment to install and utilize the vcd-cli tool in this walkthrough.

The first thing I’ll need to do is create and activate my virtual environment, which I will install in the ~/cse-client directory:

$ python3.7 -m virtualenv ~/cse-client
$ source ~/cse-client/bin/activate

Continue reading “Container Service Extension 2.5 Installation: Part 2”

Container Service Extension 2.5 Installation: Part 1

With the recent release of the Container Service Extension 2.5.0, I wanted to take some time to walk through the installation and configuration of the Container Service Extension (CSE) server in conjunction with VMware vCloud Director 10.

This will be a series of 3 blog posts that cover the following topics:

Container Service Extension Overview

Before we get started, I wanted to talk a bit about CSE and what purpose it serves in a Service Provider’s environment. The Container Service Extension is a VMware vCloud Director extension that helps tenants create, lifecycle manage, and interact with Kubernetes clusters in vCloud Director-managed environments.

There are currently two versions of CSE: Standard and Enterprise. CSE Standard brings Kubernetes-as-a-Service to vCD by creating customized vApp templates and enabling tenant/organization administrators to deploy fully functional Kubernetes clusters in self-contained vApps. CSE Standard cluster creation can be enabled on existing NSX-V backed OrgVDCs in a tenant’s environment. With the release of CSE Enterprise in the CSE 2.0 release, VMware has also added the ability for tenants to provision VMware Enterprise PKS Kubernetes clusters back by NSX-T resources in vCloud Director managed environments. In this blog post, I am going to focus on the enablement of CSE Standard Kubernetes cluster creation in an existing vCloud Director OvDC.

For more information on CSE, have a look at the Kubernetes-as-a-Service in vCloud Director reference architecture (authored by yours truly 😄) as well as the CSE Installation Documentation.

Continue reading “Container Service Extension 2.5 Installation: Part 1”

Creating a PvDC for Enterprise PKS in vCloud Director

If you read up on my recent blog post regarding RBAC in the new release of VMware’s Container Service Extension for vCloud Director, you may have noticed that I mentioned a follow-up post regarding the steps required to add an Enterprise PKS controlled vCenter Server to vCloud Director. I wanted to take a little bit of time to go through that process as it’s a relatively new workflow.

First of all, in our lab deployment, we are using an NSX-T backed vSphere environment to provide networking functionality to the Enterprise PKS deployment. As you may know, NSX-T integration is fairly new in the vCloud Director world (and growing every day!). With this in mind, the process of adding the vSphere/NSX-T components into vCD are a little bit different. Let’s have a look at the workflow for creating a Provider Virtual Datacenter (PvDC) that will support our tenant using CSE to provision Enterprise PKS kubernetes clusters.

Logging into the HTML5 vCloud Director Admin Portal

The first point to note is that we can only add a vSphere environment backed by NSX-T in the HTML5 admin portal in the current release of vCD (9.7 at the time of writing). Let’s navigate to https://vcd-director-url.com/provider and login:

Continue reading “Creating a PvDC for Enterprise PKS in vCloud Director”

Implementing RBAC with VMware’s Container Service Extension 2.0 for vCloud Director

In case you haven’t heard, VMware recently announced the general availability of the Container Service Extension 2.0 release for vCloud Director. The biggest addition of functionality in the 2.0 release is the ability to use CSE to deploy Enterprise PKS clusters via the vcd-cli tool in addition to native, upstream Kubernetes clusters. I’ll be adding a blog post shortly on the process required for enabling your vCD environment to support Enterprise PKS deployments via the Container Service Extension.

Today, we are going to talk about utilizing the RBAC functionality introduced in CSE 1.2.6 to assign different permissions to our tenants to allow them to deploy Enterprise PKS (CSE Enterprise) clusters and/or native Kubernetes clusters (CSE Native). The cloud admin will be responsible for enabling and configuring the CSE service and enabling tenant admin/users to deploy CSE Enterprise or CSE Native clusters in their virtual datacenter(s).

Prerequisites

  • The CSE 2.0 server is installed and configured to serve up native Kubernetes clusters AND Enterprise PKS clusters. Please refer to the CSE documentation for more information on this process.
  • Must have at least two organizations present and configured in vCD. In this example, I’ll be utilizing the following orgs:
    • cse-native-org (native k8 provider)
    • cse-ent-org (PKS Enterprise k8 provider)
  • This example also assumes none of the organizations have been enabled for k8 providers up to this point. We will be starting from scratch!

Continue reading “Implementing RBAC with VMware’s Container Service Extension 2.0 for vCloud Director”

Deploying VMware vCloud Director on a Single Virtual Machine with a Single Network Interface

Recently, while testing the new Container Service Extension 2.0 Beta release, I found myself needing a quick (and easily replicable) instantiation of vCloud Director in my lab environment. Being this needed to be deployed in my lab environment, I wanted to do this while using the least amount of resources and virtual machines possible to keep things simple. I decided to deploy a single CentOS virtual machine that housed the postgresdb, rabbitmq server (for my subsequent deployment of CSE), and the actual vCD server itself. I also decided to deploy using a single network interface to keep things simple.

Before we get started, I want to lay out some assumptions I’ve made in this environment that will need to be taken in consideration if you’d like to replicate this deployment as documented:

  • All of my servers hostnames are resolvable (I’m using dnsmasq to easily provide DNS/dhcp support in my lab)

  • I’ve disabled firewalld as well as this lab is completely isolated from outside traffic. This is NOT secure and NOT recommend for a production deployment. See the installation documentation for port requirements for vCD.

  • I’ve also persistently disabled SElinux. Again, this is NOT secure and NOT recommending for production but just wanted one less thing to troubleshoot barring config issues.

  • I’ve configured an NTP server in my lab that all the servers connect to. NTP is a requirement for vCD installation.

  • I am going to use the tooling provided by vCD to create self-signed SSL certs for use with vCD. Again, this is NOT secure and NOT recommending for production, but better suited for quick test deployments in a controlled lab environment.

I’ve configured a CentOS 7.6 server with 4 vCPU, 8GB of memory and a 20GB hard drive. After installation of my OS, I verify the configuration stated above and update my server to the latest and greatest:

yum update -y

Continue reading “Deploying VMware vCloud Director on a Single Virtual Machine with a Single Network Interface”