Setting up Multicluster Tanzu Application Platform 1.1

Tanzu Application Platform delivers a superior developer experience for enterprises building and deploying cloud-native applications on Kubernetes. It enables application teams to get to production faster by automating source-to-production pipelines. It clearly defines the roles of developers and operators so they can work collaboratively and integrate their efforts.

VMware released TAP 1.1 with several new features that includes setting up TAP on multicluster environment. In this blog post, I will be going through the step by step process to setup TAP v1.1 on 3 AKS clusters.

Below is the reference diagram of TAP running in a multicluster environment. In this post, I have used AKS cluster for all three profiles but you can use different one too.

What is covered in this post:

  • Deploy 3 Azure Kuberentes Clusters
  • Setup Cluster Essentials required before TAP package installation
  • Setting up View Profiled cluster
  • Setting up Build Profiled cluster
  • Setting up Run Profiled cluster
  • Integrate Build and Run clusters with View cluster
  • Build a sample app using Build Cluster, run it on the Run cluster and then view it using TAP running on View profiled cluster

Deploy 3 Azure Kubernetes Clusters

For simplicity purpose, I will use simple az command with limited options.

  • Create a resource group. You can use name as per your choice.
 az group create \                                                                                   
   --name=tkoworkshop-tsm \
   --location=westus2 \
   --output table
  • Create an AKS Cluster for TAP View Profile Installation
az aks create \
   --name tap11-aks-viewcluster \
   --resource-group tkoworkshop-tsm \
   --node-count 2 \
   --node-vm-size Standard_B4ms \
   --output table
  • Create an AKS Cluster for TAP Build Profile Installation
az aks create \
   --name tap11-aks-buildcluster \
   --resource-group tkoworkshop-tsm \
   --node-count 2 \
   --node-vm-size Standard_B4ms \
   --output table

Create an AKS Cluster for TAP Run Profile Installation

 az aks create \
   --name tap11-aks-runcluster \
   --resource-group tkoworkshop-tsm \
   --node-count 2 \
   --node-vm-size Standard_B4ms \
   --output table

Once clusters are created, list them.

az aks list --output table | grep -i tap11
tap11-aks-buildcluster  westus2     tkoworkshop-tsm  1.21.9               Succeeded            tap11-aks--tkoworkshop-tsm-deda07-87cb4b7e.hcp.westus2.azmk8s.io
tap11-aks-runcluster    westus2     tkoworkshop-tsm  1.21.9               Succeeded            tap11-aks--tkoworkshop-tsm-deda07-c0282b93.hcp.westus2.azmk8s.io
tap11-aks-viewcluster   westus2     tkoworkshop-tsm  1.21.9               Succeeded            tap11-aks--tkoworkshop-tsm-deda07-a873d2ec.hcp.westus2.azmk8s.io

You can view them on Azure portal too.

  • Update the kubeconfig file on your local system or bootstrap node to talk to clusters by running the following commands.
az aks get-credentials --resource-group tkoworkshop-tsm --name tap11-aks-viewclusteraz aks get-credentials --resource-group tkoworkshop-tsm --name tap11-aks-runcluster az aks get-credentials --resource-group tkoworkshop-tsm --name tap11-aks-buildcluster

TAP 1.1 Pre-requirements

  • Complete step 1,2,3 from here

Install Cluster Essential on all 3 clusters

  • Switch to the View Profiled AKS cluster
kubectl config use-context tap11-aks-viewcluster
Switched to context "tap11-aks-viewcluster".
  • Sign in to Tanzu Network.
  • Go to Cluster Essentials for VMware Tanzu on VMware Tanzu Network.
  • Accept or confirm that you have accepted the EULA for the product
  • Select a download according to your Kubernetes provider and operating system:
    • For macOS, download tanzu-cluster-essentials-darwin-amd64-1.1.0.tgz.
    • For Linux, download tanzu-cluster-essentials-linux-amd64-1.1.0.tgz.
  • Unpack the TAR file into the tanzu-cluster-essentials directory:mkdir $HOME/tanzu-cluster-essentials tar -xvf DOWNLOADED-CLUSTER-ESSENTIALS-BUNDLE -C $HOME/tanzu-cluster-essentials Where DOWNLOADED-CLUSTER-ESSENTIALS-BUNDLE is the name of the bundle you downloaded.
  • Export the required variables and update your Tanzu Network username and password.
export INSTALL_BUNDLE=registry.tanzu.vmware.com/tanzu-cluster-essentials/cluster-essentials-bundle@sha256:ab0a3539da241a6ea59c75c0743e9058511d7c56312ea3906178ec0f3491f51d
export INSTALL_REGISTRY_HOSTNAME=registry.tanzu.vmware.com
export INSTALL_REGISTRY_USERNAME=TANZU-NET-USER
export INSTALL_REGISTRY_PASSWORD=TANZU-NET-PASSWORD
  • Run install command
cd $HOME/tanzu-cluster-essentials 
./install.sh --yes
  • Notice the additional namespace those are created during cluster essential components deployment.
❯ kubectl get ns
NAME                        STATUS   AGE
default                     Active   21m
kapp-controller             Active   4m10s
kube-node-lease             Active   21m
kube-public                 Active   21m
kube-system                 Active   21m
secretgen-controller        Active   40s
tanzu-cluster-essentials    Active   4m24s
tanzu-package-repo-global   Active   4m11s
  • Do the same steps as above on remaining Build and Run clusters as well.

Additionally, You can look for a reference document here.

After cluster essentials are installed on all three clusters, we need to add tap packages repository on all three clusters.

Add TAP Packages Repository

  • Login to VMware Tanzu Network Registry and your image registry that you will be using to store TAP images. e.g. ACR, dockerhub, gcr etc. In my case. I have used ACR.
docker login registry.tanzu.vmware.com
docker login <acr-registry-name>
  • Once login to both registry is successful, export the below variables
export INSTALL_REGISTRY_USERNAME=<your-registry-username>
export INSTALL_REGISTRY_PASSWORD=<your-registry-password>
export INSTALL_REGISTRY_HOSTNAME=<your-registry-name>
export TAP_VERSION=1.1.0
  • Relocate the images with the Carvel tool imgpkg by running the following command.
imgpkg copy -b registry.tanzu.vmware.com/tanzu-application-platform/tap-packages:${TAP_VERSION} --to-repo ${INSTALL_REGISTRY_HOSTNAME}/tap11/tap-packages

- tap11 is image repository available on your registry. You can use based on your environment. Above steps will take sometime as there are many images.
  • Once image copy is successful, you will see the following message.
copy | exported 421 images
copy | importing 421 images...
 0 B / ? [----------------------------------------------------------------------------------------------------------------------------=]  0.00% 2562047h47m16s
copy | done uploading images
copy | Warning: Skipped layer due to it being non-distributable. If you would like to include non-distributable layers, use the --include-non-distributable-layers flag
Succeeded
  • Validate Images on your registry too. e.g. in my ACR I can see the following

Now, let’s add this TAP repository on all three clusters.

  • Switch to the View Profiled AKS cluster
kubectl config use-context tap11-aks-viewcluster
Switched to context "tap11-aks-viewcluster".
  • Create a namespace
kubectl create ns tap-install
namespace/tap-install created
  • Add tap-registry Secret. Here tap-registry is ACR.
tanzu secret registry add tap-registry \
  --username ${INSTALL_REGISTRY_USERNAME} --password ${INSTALL_REGISTRY_PASSWORD} \
  --server ${INSTALL_REGISTRY_HOSTNAME} \
  --export-to-all-namespaces --yes --namespace tap-install
Warning: By choosing --export-to-all-namespaces, given secret contents will be available to ALL users in ALL namespaces. Please ensure that included registry credentials allow only read-only access to the registry with minimal necessary scope.


\ Adding registry secret 'tap-registry'... 
 Added registry secret 'tap-registry' into namespace 'tap-install'
 Exported registry secret 'tap-registry' to all namespaces
  • Add TAP Repository
tanzu package repository add tanzu-tap-repository \
  --url ${INSTALL_REGISTRY_HOSTNAME}/tap11/tap-packages:$TAP_VERSION \
  --namespace tap-install
- Adding package repository 'tanzu-tap-repository' 
- Validating provided settings for the package repository 
\ Creating package repository resource 
/ Waiting for 'PackageRepository' reconciliation for 'tanzu-tap-repository' 
/ 'PackageRepository' resource install status: Reconciling 

| 'PackageRepository' resource successfully reconciled 
Added package repository 'tanzu-tap-repository' in namespace 'tap-install'
  • You can list the newly added repository by running the following command
tanzu package repository list -n tap-install
  • Do the same steps as above on remaining Build and Run clusters as well.

Install multicluster Tanzu Application Platform profiles

If you are new to TAP multicluster, have a quick view here to get a basic understanding.

Multicluster Installation Order of Operations

Due to dependency between TAP components, we need to follow a specific order for TAP installation on different clusters.

  1. Install View profile cluster
  2. Install Build profile cluster
  3. Install Run profile cluster

Install View profile on TAP View Cluster

  • Ensure that you are in view cluster context
 kubectl config get-contexts
  • Prepare View profile yaml and save it to view-tap11-values.yaml file
profile: view
ceip_policy_disclosed: true # Installation fails if this is set to 'false'

learningcenter:
  ingressDomain: "tapmcr.tanzupartnerdemo.com"

tap_gui:
  service_type: ClusterIP
  ingressEnabled: "true"
  ingressDomain: "tapmcr.tanzupartnerdemo.com"
  app_config:
    app:
      baseUrl: http://tap-gui.tapmcr.tanzupartnerdemo.com
    catalog:
      locations:
        - type: url
          target: https://github.com/dineshtripathi30/tanzuapp/blob/main/blank/catalog-info.yaml
    backend:
      baseUrl: http://tap-gui.tapmcr.tanzupartnerdemo.com
      cors:
        origin: http://tap-gui.tapmcr.tanzupartnerdemo.com
    kubernetes:
      serviceLocatorMethod:
        type: 'multiTenant'
      clusterLocatorMethods:
        - type: 'config'
          clusters:
           - url: CLUSTER_URL
             name: CLUSTER_NAME
             authProvider: serviceAccount
             serviceAccountToken: CLUSTER_TOKEN
             skipTLSVerify: true
              
metadata_store:
  app_service_type: LoadBalancer # (optional) Defaults to LoadBalancer. Change to NodePort for distributions that don't support LoadBalancer

contour:
  envoy:
    service:
        type: LoadBalancer

appliveview:
  ingressEnabled: true
  ingressDomain: tapmcr.tanzupartnerdemo.com

Note:
1. CLUSTER_URLCLUSTER_NAME and CLUSTER_TOKEN are described in the Viewing resources on multiple clusters in Tanzu Application Platform GUI. We will update values for all three clusters and update the package after setting up all profiles.
2. Update the correct value where text is marked in bold.

  • Run the TAP package install on a View cluster
❯ tanzu package install tap -p tap.tanzu.vmware.com -v 1.1.0 --values-file view-tap11-values.yaml -n tap-install
- Installing package 'tap.tanzu.vmware.com' 
/ Getting package metadata for 'tap.tanzu.vmware.com' 
- Creating service account 'tap-tap-install-sa' 
- Creating cluster admin role 'tap-tap-install-cluster-role' 
- Creating cluster role binding 'tap-tap-install-cluster-rolebinding' 
- Creating secret 'tap-tap-install-values' 
- Creating package resource 
\ Waiting for 'PackageInstall' reconciliation for 'tap' 
| 'PackageInstall' resource install status: Reconciling 


 Added installed package 'tap'
  • List the packages installed on view cluster
tanzu package installed list -n tap-install
- Retrieving installed packages... 
  NAME                      PACKAGE-NAME                               PACKAGE-VERSION  STATUS               
  accelerator               accelerator.apps.tanzu.vmware.com          1.1.2            Reconcile succeeded  
  api-portal                api-portal.tanzu.vmware.com                1.0.15           Reconcile succeeded  
  appliveview               backend.appliveview.tanzu.vmware.com       1.1.0            Reconcile succeeded  
  cert-manager              cert-manager.tanzu.vmware.com              1.5.3+tap.2      Reconcile succeeded  
  contour                   contour.tanzu.vmware.com                   1.18.2+tap.2     Reconcile succeeded  
  fluxcd-source-controller  fluxcd.source.controller.tanzu.vmware.com  0.16.4           Reconcile succeeded  
  learningcenter            learningcenter.tanzu.vmware.com            0.2.0            Reconcile succeeded  
  learningcenter-workshops  workshops.learningcenter.tanzu.vmware.com  0.2.0            Reconcile succeeded  
  metadata-store            metadata-store.apps.tanzu.vmware.com       1.1.2            Reconcile succeeded  
  source-controller         controller.source.apps.tanzu.vmware.com    0.3.3            Reconcile succeeded  
  tap                       tap.tanzu.vmware.com                       1.1.0            Reconcile succeeded  
  tap-auth                  tap-auth.tanzu.vmware.com                  1.0.1            Reconcile succeeded  
  tap-gui                   tap-gui.tanzu.vmware.com                   1.1.0            Reconcile succeeded  
  tap-telemetry             tap-telemetry.tanzu.vmware.com             0.1.4            Reconcile succeeded  

Install Build profile on TAP Build Cluster

  • Ensure that you are in View cluster context
 kubectl config get-contexts
  • Prepare view profile yaml and save it to build-tap11-values.yaml file. Ensure to fill the values
profile: build
ceip_policy_disclosed: true # Installation fails if this is set to 'false'
buildservice:
  kp_default_repository: ""
  kp_default_repository_username: ""
  kp_default_repository_password: ""
  tanzunet_username: ""
  tanzunet_password: ""
supply_chain: basic

ootb_supply_chain_basic:
  registry:
    server: ""
    repository: ""
  gitops:
    ssh_secret: ""

grype:
  namespace: "default" # (optional) Defaults to default namespace.
  targetImagePullSecret: "registry-credentials"

Note:
To know more about above parameters, refer the document here https://docs.vmware.com/en/Tanzu-Application-Platform/1.1/tap/GUID-multicluster-reference-tap-values-build-sample.html
– Fill the values in above config file. you can take a reference from link above to know more about those parameters.

  • Run the TAP package install on a Build cluster
tanzu package install tap -p tap.tanzu.vmware.com -v 1.1.0 --values-file build-tap11-values.yaml -n tap-install
| Installing package 'tap.tanzu.vmware.com' 
/ Getting package metadata for 'tap.tanzu.vmware.com' 
- Creating service account 'tap-tap-install-sa' 
- Creating cluster admin role 'tap-tap-install-cluster-role' 
- Creating cluster role binding 'tap-tap-install-cluster-rolebinding' 
- Creating secret 'tap-tap-install-values' 
- Creating package resource 
\ Waiting for 'PackageInstall' reconciliation for 'tap' 
- 'PackageInstall' resource install status: Reconciling 

| 'PackageInstall' resource successfully reconciled 

 Added installed package 'tap'
  • List the packages installed on Build cluster
tanzu package installed list -n tap-install
- Retrieving installed packages... 
   NAME                      PACKAGE-NAME                                  PACKAGE-VERSION  STATUS               
  appliveview-conventions   conventions.appliveview.tanzu.vmware.com      1.1.0            Reconcile succeeded  
  buildservice              buildservice.tanzu.vmware.com                 1.5.0            Reconcile succeeded  
  cartographer              cartographer.tanzu.vmware.com                 0.3.0            Reconcile succeeded  
  cert-manager              cert-manager.tanzu.vmware.com                 1.5.3+tap.2      Reconcile succeeded  
  contour                   contour.tanzu.vmware.com                      1.18.2+tap.2     Reconcile succeeded  
  conventions-controller    controller.conventions.apps.tanzu.vmware.com  0.6.3            Reconcile succeeded  
  fluxcd-source-controller  fluxcd.source.controller.tanzu.vmware.com     0.16.4           Reconcile succeeded  
  grype                     grype.scanning.apps.tanzu.vmware.com          1.1.0            Reconcile succeeded  
  ootb-supply-chain-basic   ootb-supply-chain-basic.tanzu.vmware.com      0.7.0            Reconcile succeeded  
  ootb-templates            ootb-templates.tanzu.vmware.com               0.7.0            Reconcile succeeded  
  scanning                  scanning.apps.tanzu.vmware.com                1.1.0            Reconcile succeeded  
  source-controller         controller.source.apps.tanzu.vmware.com       0.3.3            Reconcile succeeded  
  spring-boot-conventions   spring-boot-conventions.tanzu.vmware.com      0.4.0            Reconcile succeeded  
  tap                       tap.tanzu.vmware.com                          1.1.0            Reconcile succeeded  
  tap-auth                  tap-auth.tanzu.vmware.com                     1.0.1            Reconcile succeeded  
  tap-telemetry             tap-telemetry.tanzu.vmware.com                0.1.4            Reconcile succeeded  
  tekton-pipelines          tekton.tanzu.vmware.com                       0.33.2           Reconcile succeeded  

Install Run profile on TAP Run Cluster

  • Ensure that you are in Run cluster context
 kubectl config get-contexts
  • Prepare view profile yaml and save it to run-tap11-values.yaml file. Ensure to fill the values
profile: run
ceip_policy_disclosed: true # Installation fails if this is set to 'false'
supply_chain: basic  
cnrs:
  domain_name: "tapmcr.tanzupartnerdemo.com"
  
contour:
  envoy:
    service:
        type: LoadBalancer

appliveview_connector:
  backend:
    sslDisabled: true
    host: appliveview.tapmcr.tanzupartnerdemo.com

Note:
To know more about above parameters, refer the document here https://docs.vmware.com/en/Tanzu-Application-Platform/1.1/tap/GUID-multicluster-reference-tap-values-run-sample.html
– Replace the domain name in the above file as per your environment.

  • Run the TAP package install on a Run cluster
tanzu package install tap -p tap.tanzu.vmware.com -v 1.1.0 --values-file run-tap11-values.yaml -n tap-install
- Installing package 'tap.tanzu.vmware.com' 
/ Getting package metadata for 'tap.tanzu.vmware.com' 
- Creating service account 'tap-tap-install-sa' 
- Creating cluster admin role 'tap-tap-install-cluster-role' 
- Creating cluster role binding 'tap-tap-install-cluster-rolebinding' 
- Creating secret 'tap-tap-install-values' 
- Creating package resource 
\ Waiting for 'PackageInstall' reconciliation for 'tap' 
- 'PackageInstall' resource install status: Reconciling
  • List the packages installed on Run cluster
tanzu package installed list -n tap-install
- Retrieving installed packages... 
   NAME                      PACKAGE-NAME                                  PACKAGE-VERSION  STATUS               
 appliveview-connector     connector.appliveview.tanzu.vmware.com              1.1.0            Reconcile failed: Error (see .status.usefulErrorMessage for details)  
  cartographer              cartographer.tanzu.vmware.com                       0.3.0            Reconcile succeeded                                                   
  cert-manager              cert-manager.tanzu.vmware.com                       1.5.3+tap.2      Reconcile succeeded                                                   
  cnrs                      cnrs.tanzu.vmware.com                               1.2.0            Reconcile succeeded                                                   
  contour                   contour.tanzu.vmware.com                            1.18.2+tap.2     Reconcile succeeded                                                   
  fluxcd-source-controller  fluxcd.source.controller.tanzu.vmware.com           0.16.4           Reconcile succeeded                                                   
  image-policy-webhook      image-policy-webhook.signing.apps.tanzu.vmware.com  1.1.1            Reconcile succeeded                                                   
  ootb-delivery-basic       ootb-delivery-basic.tanzu.vmware.com                0.7.0            Reconcile succeeded                                                   
  ootb-templates            ootb-templates.tanzu.vmware.com                     0.7.0            Reconcile succeeded                                                   
  service-bindings          service-bindings.labs.vmware.com                    0.7.1            Reconcile succeeded                                                   
  services-toolkit          services-toolkit.tanzu.vmware.com                   0.6.0            Reconcile succeeded                                                   
  source-controller         controller.source.apps.tanzu.vmware.com             0.3.3            Reconcile succeeded                                                   
  tap                       tap.tanzu.vmware.com                                1.1.0            Reconcile failed: Error (see .status.usefulErrorMessage for details)  
  tap-auth                  tap-auth.tanzu.vmware.com                           1.0.1            Reconcile succeeded                                                   
  tap-telemetry             tap-telemetry.tanzu.vmware.com                      0.1.4            Reconcile succeeded   

Note: In my case, appliveview-connector is failing and i will keep you updated on the resolution steps.

Now, since all clusters are running with their respective profiles. Let’s do the post install part and integrate build and run cluster with the View cluster

Viewing resources from Build and Run clusters in Tanzu Application Platform GUI (on View Cluster)

  • Set up a Service Account to view resources on a cluster
  • Switch to the Build cluster context
kubectl config use-context tap11-aks-buildcluster
Switched to context "tap11-aks-buildcluster".
  • Create a file called tap-gui-viewer-service-account-rbac.yaml with the following content on your local system
apiVersion: v1
kind: Namespace
metadata:
  name: tap-gui
---
apiVersion: v1
kind: ServiceAccount
metadata:
  namespace: tap-gui
  name: tap-gui-viewer
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: tap-gui-read-k8s
subjects:
- kind: ServiceAccount
  namespace: tap-gui
  name: tap-gui-viewer
roleRef:
  kind: ClusterRole
  name: k8s-reader
  apiGroup: rbac.authorization.k8s.io
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: k8s-reader
rules:
- apiGroups: ['']
  resources: ['pods', 'services', 'configmaps']
  verbs: ['get', 'watch', 'list']
- apiGroups: ['apps']
  resources: ['deployments', 'replicasets']
  verbs: ['get', 'watch', 'list']
- apiGroups: ['autoscaling']
  resources: ['horizontalpodautoscalers']
  verbs: ['get', 'watch', 'list']
- apiGroups: ['networking.k8s.io']
  resources: ['ingresses']
  verbs: ['get', 'watch', 'list']
- apiGroups: ['networking.internal.knative.dev']
  resources: ['serverlessservices']
  verbs: ['get', 'watch', 'list']
- apiGroups: [ 'autoscaling.internal.knative.dev' ]
  resources: [ 'podautoscalers' ]
  verbs: [ 'get', 'watch', 'list' ]
- apiGroups: ['serving.knative.dev']
  resources:
  - configurations
  - revisions
  - routes
  - services
  verbs: ['get', 'watch', 'list']
- apiGroups: ['carto.run']
  resources:
  - clusterconfigtemplates
  - clusterdeliveries
  - clusterdeploymenttemplates
  - clusterimagetemplates
  - clusterruntemplates
  - clustersourcetemplates
  - clustersupplychains
  - clustertemplates
  - deliverables
  - runnables
  - workloads
  verbs: ['get', 'watch', 'list']
- apiGroups: ['source.toolkit.fluxcd.io']
  resources:
  - gitrepositories
  verbs: ['get', 'watch', 'list']
- apiGroups: ['source.apps.tanzu.vmware.com']
  resources:
  - imagerepositories
  verbs: ['get', 'watch', 'list']
- apiGroups: ['conventions.apps.tanzu.vmware.com']
  resources:
  - podintents
  verbs: ['get', 'watch', 'list']
- apiGroups: ['kpack.io']
  resources:
  - images
  - builds
  verbs: ['get', 'watch', 'list']
- apiGroups: ['scanning.apps.tanzu.vmware.com']
  resources:
  - sourcescans
  - imagescans
  verbs: ['get', 'watch', 'list']
- apiGroups: ['tekton.dev']
  resources:
  - taskruns
  - pipelineruns
  verbs: ['get', 'watch', 'list']
- apiGroups: ['kappctrl.k14s.io']
  resources:
  - apps
  verbs: ['get', 'watch', 'list']

This YAML content creates the NamespaceServiceAccountClusterRole, and ClusterRoleBinding.

  • Apply the yaml to Build cluster profile
kubectl create -f tap-gui-viewer-service-account-rbac.yaml

namespace/tap-gui created
serviceaccount/tap-gui-viewer created
clusterrolebinding.rbac.authorization.k8s.io/tap-gui-read-k8s created
clusterrole.rbac.authorization.k8s.io/k8s-reader created
  • Get the CLUSTER_URL and CLUSTER_TOKEN values
CLUSTER_URL=$(kubectl config view --minify -o jsonpath='{.clusters[0].cluster.server}')

CLUSTER_TOKEN=$(kubectl -n tap-gui get secret $(kubectl -n tap-gui get sa tap-gui-viewer -o=json \
| jq -r '.secrets[0].name') -o=json \
| jq -r '.data["token"]' \
| base64 --decode)

echo CLUSTER_URL: $CLUSTER_URL
echo CLUSTER_TOKEN: $CLUSTER_TOKEN
  • Do the same steps on a Run Cluster and note down the CLUSTER_URL and CLUSTER_TOKEN values.

Update Tanzu Application Platform GUI to view resources on Build and Run clusters

Update the view-tap11-values.yaml file to add Build and Run profiled cluster under clusters section.

app_config:
  kubernetes:
    serviceLocatorMethod:
      type: 'multiTenant'
    clusterLocatorMethods:
      - type: 'config'
        clusters:
          - url: CLUSTER-URL-from-build-cluster
            name: tap-aks-build-cluster
            authProvider: serviceAccount
            serviceAccountToken: "CLUSTER-TOKEN-from-build-cluster"
            skipTLSVerify: true
          - url: CLUSTER-URL-from-run-cluster
            name: tap-aks-run-cluster
            authProvider: serviceAccount
            serviceAccountToken: "CLUSTER-TOKEN-from-run-cluster"
            skipTLSVerify: true
  • Switch to View cluster context
kubectl config use-context tap11-aks-viewcluster
Switched to context "tap11-aks-viewcluster".
  • Update the tap-gui package by running the following command
tanzu package installed update tap -p tap.tanzu.vmware.com -v 1.1.0 --values-file view-tap11-values.yaml -n tap-install
| Updating installed package 'tap' 
/ Getting package install for 'tap' 
/ Getting package metadata for 'tap.tanzu.vmware.com' 
| Updating secret 'tap-tap-install-values' 
- Updating package install for 'tap' 
| Waiting for 'PackageInstall' reconciliation for 'tap' 

Updated installed package 'tap' in namespace 'tap-install'
  • Run the following command to get tap-gui url and access the TAP GUI. You need to ensure that your DNS is correctly updated for accessing TAP GUI.
kubectl get proxy -n tap-gui
NAME      FQDN                                  TLS SECRET   STATUS   STATUS DESCRIPTION
tap-gui   tap-gui.tapmcr.tanzupartnerdemo.com                valid    Valid HTTPProxy
  • Here is TAP 1.1 GUI running on View profiled cluster.

We are done with the installation and integrating Build and Run profiled cluster with TAP GUI.

Deploying Workload on a TAP Multicluster

Pre-requirement

  • Complete Installation of all three profiles
  • For simplicity, we will use application based on sample accelerator provided out of the box

We need to setup Developer Namespace on both the Build and Run profiled clusters.

Setup a Developer Namespace on a Build Profiled Cluster

Create a registry credentials where application image will be pushed to. You need to create this in a namespace where you will be deploying tanzu workload. in this example, i have used default namespace.

  • Add your registry secret to the default namespace
tanzu secret registry add registry-credentials --server REGISTRY-SERVER --username REGISTRY-USERNAME --password REGISTRY-PASSWORD --namespace default
  • You can validate it by running the following command:
tanzu secret registry list
  • Add secrets, a service account to execute the supply chain, and RBAC rules to authorise the service account to the developer namespace by running: In this case, default namespace is being used.
cat <<EOF | kubectl -n default apply -f -
apiVersion: v1
kind: Secret
metadata:
  name: tap-registry
  annotations:
    secretgen.carvel.dev/image-pull-secret: ""
type: kubernetes.io/dockerconfigjson
data:
  .dockerconfigjson: e30K
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: default
secrets:
  - name: registry-credentials
imagePullSecrets:
  - name: registry-credentials
  - name: tap-registry
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: default-permit-deliverable
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: deliverable
subjects:
  - kind: ServiceAccount
    name: default
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: default-permit-workload
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: workload
subjects:
  - kind: ServiceAccount
    name: default
EOF
secret/tap-registry created
serviceaccount/default configured
rolebinding.rbac.authorization.k8s.io/default-permit-deliverable created
rolebinding.rbac.authorization.k8s.io/default-permit-workload created
  • Give developers namespace-level access and view access to appropriate cluster-level resources by applying the following yaml in a cluster.
cat <<EOF | kubectl -n default apply -f -
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
 name: dev-permit-app-viewer
roleRef:
 apiGroup: rbac.authorization.k8s.io
 kind: ClusterRole
 name: app-viewer
subjects:
- kind: Group
  name: "namespace-developers"
  apiGroup: rbac.authorization.k8s.io
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
 name: namespace-dev-permit-app-viewer
roleRef:
 apiGroup: rbac.authorization.k8s.io
 kind: ClusterRole
 name: app-viewer-cluster-access
subjects:
- kind: Group
  name: "namespace-developers"
  apiGroup: rbac.authorization.k8s.io
EOF
rolebinding.rbac.authorization.k8s.io/dev-permit-app-viewer created
clusterrolebinding.rbac.authorization.k8s.io/namespace-dev-permit-app-viewer created
  • Do the same steps on your Run profiled cluster.
  • Switch back to the Build Profiled cluster
  • Export Developer Namespace by running the following command
export DEVELOPER_NAMESPACE=default

Start the workload on the Build profile cluster

The Build cluster starts by building the necessary bundle for the workload that is delivered to the Run cluster.

  1. Switch to Build profile cluster context
  2. Use Tanzu CLI to start creating first sample workload
tanzu apps workload create tanzu-java-web-app \
--git-repo https://github.com/sample-accelerators/tanzu-java-web-app \
--git-branch main \
--type web \
--label app.kubernetes.io/part-of=tanzu-java-web-app \
--yes \
--namespace ${DEVELOPER_NAMESPACE}
Create workload:
      1 + |---
      2 + |apiVersion: carto.run/v1alpha1
      3 + |kind: Workload
      4 + |metadata:
      5 + |  labels:
      6 + |    app.kubernetes.io/part-of: tanzu-java-web-app
      7 + |    apps.tanzu.vmware.com/workload-type: web
      8 + |  name: tanzu-java-web-app
      9 + |  namespace: default
     10 + |spec:
     11 + |  source:
     12 + |    git:
     13 + |      ref:
     14 + |        branch: main
     15 + |      url: https://github.com/sample-accelerators/tanzu-java-web-app

Created workload "tanzu-java-web-app"
  • Get the Status of workload
❯ tanzu apps workload get tanzu-java-web-app
# tanzu-java-web-app: Unknown
---
lastTransitionTime: "2022-04-16T08:30:20Z"
message: waiting to read value [.status.latestImage] from resource [image.kpack.io/tanzu-java-web-app]
  in namespace [default]
reason: MissingValueAtPath
status: Unknown
type: Ready

Pods
NAME                                   STATUS    RESTARTS   AGE
tanzu-java-web-app-build-1-build-pod   Pending   0          33s
❯ kubectl get po
NAME                                   READY   STATUS     RESTARTS   AGE
tanzu-java-web-app-build-1-build-pod   0/1     Init:4/6   0          42s

  • You can also view the log by running the following command
tanzu apps workload tail tanzu-java-web-app --since 10m --timestamp --namespace ${DEVELOPER_NAMESPACE}
  • Login to TAP GUI and view the supply chain status.
  • Click on it to see more detail

  • Verify that your supply chain has produced the necessary Deliverable for the Workload by running:
kubectl get deliverable --namespace ${DEVELOPER_NAMESPACE}
--- You can ignore the READY Status false and Reason DeliveryNotFound message for now.
  • Copy the content of Deliverable that you need to run on a Run profiled cluster
kubectl get deliverable tanzu-java-web-app --namespace ${DEVELOPER_NAMESPACE} -oyaml > deliverable.yaml
  • Delete the ownerReferences and status sections from the deliverable.yaml. Here is how the yaml looks like
apiVersion: carto.run/v1alpha1
kind: Deliverable
metadata:
  creationTimestamp: "2022-04-16T08:30:18Z"
  generation: 1
  labels:
    app.kubernetes.io/component: deliverable
    app.kubernetes.io/part-of: tanzu-java-web-app
    app.tanzu.vmware.com/deliverable-type: web
    apps.tanzu.vmware.com/workload-type: web
    carto.run/cluster-template-name: deliverable-template
    carto.run/resource-name: deliverable
    carto.run/supply-chain-name: source-to-url
    carto.run/template-kind: ClusterTemplate
    carto.run/workload-name: tanzu-java-web-app
    carto.run/workload-namespace: default
  name: tanzu-java-web-app
  namespace: default
  resourceVersion: "90727"
  uid: xxxxxxxxxxx # you will see the UID value here
spec:
  source:
    image: <your image name will be displayed here>
  • Switch to the Run profiled cluster context and then apply the yaml
kubectl apply -f deliverable.yaml --namespace ${DEVELOPER_NAMESPACE}
deliverable.carto.run/tanzu-java-web-app created
  • Verify the status
kubectl get deliverables --namespace default
NAME                 SOURCE                                                                                                               DELIVERY         READY   REASON   AGE
tanzu-java-web-app   your-source-image-here   delivery-basic   True    Ready    24s

  • To test the application, run the following command and get the url.
kubectl get httpproxy --namespace ${DEVELOPER_NAMESPACE}
  • Once you hit the url, you will see the output like below
  • Now, Let’s register a Catalog to the TAP UI and and validate the operations ran on Build and Run profiled clusters..
  • Grab the catalog-info.yaml url file from here
  • Go to the TAP UI, Click Home -> Register Entity
  • Click ANALYZE
  • Click IMPORT
  • Click on Home again and you will see two Catalog entries. Click on tanzu-java-web-app
  • Click on Runtime Resources and notice the Cluster column data. You will notice the cluster names.
  • If you notice under cluster column , you will find that there are few option that ran on Build profiled cluster and application is running on Run profiled clusters.

We are done here. Hope you have enjoyed reading the post.

Reference Docs and Videos

What’s new in TAP 1.1

TAP 1.1 Official Document

https://docs.vmware.com/en/Tanzu-Application-Platform/1.1/tap/GUID-overview.html

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s