Worker node in a Supervisor Cluster is showing Not Ready?

vSphere with Tanzu supervisor cluster does have master nodes as VM’s but the worker nodes are ESXi hosts. In order to bring ESXi hosts in a cluster, it uses CRX as container runtime (instead of Docker, CRI-O, containerd) and Spherelet in place of Kubelet. Similar to Kubelet, Spherelet post the node status to master.

One of the recent error i have seen is, where one of the ESXi node was showing not ready as below.

So, where to start troubleshooting ?

Here is how i did

  1. Click on the (1) after Not Ready and it will give you some clue.

It gives you an idea on what can be the potential problem. (btw, error message could have been show Spherelet instead of kubelet)

2. Login to ESXi node via putty

3. Check the status of spherelet using below command

[root@esxi02:~] /etc/init.d/spherelet status
2021-09-08 16:30:25,479 init.d/spherelet Log fetcher support: True
2021-09-08 16:30:25,506 init.d/spherelet spherelet is not running
2021-09-08 16:30:25,506 init.d/spherelet spherelet is not running

4. Start there spherelet process

[root@esxi02:~] /etc/init.d/spherelet start
2021-09-08 17:05:39,467 init.d/spherelet Log fetcher support: True
2021-09-08 17:05:39,687 init.d/spherelet Reserved 6 VMs for Spherelet
2021-09-08 17:05:39,687 init.d/spherelet Reserved 6 VMs for Spherelet
2021-09-08 17:05:39,731 init.d/spherelet spherelet started
2021-09-08 17:05:39,731 init.d/spherelet spherelet started

5. Check the status again

[root@esxi02:~] /etc/init.d/spherelet status
2021-09-08 17:06:16,408 init.d/spherelet Log fetcher support: True
2021-09-08 17:06:16,434 init.d/spherelet spherelet is running
2021-09-08 17:06:16,434 init.d/spherelet spherelet is running

6. Check the node status in vCenter after a min.

So, everything is good now.

This is just one of the issue, there can be other issues for node not ready and better place would be to look at spherelet log too.

$ tail -f /var/log/spherelet.log

I also found lot of good troubleshooting information in VMware documentation below.

https://docs.vmware.com/en/VMware-vSphere/7.0/vmware-vsphere-with-tanzu/GUID-75B06734-4D2C-4219-B220-413ADE8808C6.html

That’s all for this post.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s