Setting up Kubernetes multi node cluster on top of amazon EC2 instance

6 min readFeb 18, 2021

This guide will take you through the manual process for getting started with multi node cluster kubernetes on top of Amazon EC2 instance. We’ll run the nodes inside the Amazon linux 2 AMI, but you can also use a different amazon machine image depends in which OS you are familiar.

This application will have one master node and two worker node. On my up coming article we will walk through how to connect our local machine worker node to the master node.

If you want to learn more about what kubernetes actually is and what it can do for, go and check the official documentation

Kubernetes Documentation

© 2021 The Kubernetes Authors | Documentation Distributed under CC BY 4.0 Copyright © 2021 The Linux Foundation ®. All…

kubernetes.io

Pres-requisite

Follow the different step below to create and launch Amazon AMI machine. We have to login to our amazon AWS account before proceed , go to ec2 service and start process to launch amazon ami.

Choose an Amazon Machine Image(AMI)

Configure Instance Details

Number of instance equal 3, we need 3 instance (one master node and two worker node)

Just follow all step to launch all three instance

Topology

Notice: In case you are facing some difficulty to launch your amazon ami, you can go through this guide in which i explain how to launch ec2 instance using automation tools ansible.

Once all three nodes are already install, depend on your preference, use putty to connect with each of them.

Install Docker

Docker is an open platform for developing , shipping and running applications. Docker enables you to separate your applications form your infrastructure in the same ways you manage your applications.

Docker engine is available on a variety of distribution.

For more details you can go with:

Install Docker Engine

Docker Engine is available on a variety of Linux platforms, macOS and Windows 10 through Docker Desktop, and as a…

docs.docker.com

Using yum command to install docker on master node.

After installation make sure to enabled and start docker service

yum install docker -y
systemctl enable docker
systemctl start docker

Build Kubernetes

To quickly launch and deploy kubernetes cluster on master node, we are going to use kubeadm. Kubeadm is a tool use to create kubernetes cluster .

According to the official documentation, each node in the cluster should have at least two CPU and 2 Gb of Ram. But in case of our demonstration, we will use one CPU and 1Gb Ram. We will solve the issue regarding the limitation during implementation.

To install kubeadm follow :

Installing kubeadm

This page shows how to install the kubeadm toolbox. For information how to create a cluster with kubeadm once you have…

kubernetes.io

Add Kubernetes yum repo

To make kubernetes packages available to the yum package manager, you will add a kubernetes.repo file to the /etc/yum.repo.d/ directory as showing below.

You should be able to just copy and paste the following command with no change:

cat <<EOF | sudo tee /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-\$basearch
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
exclude=kubelet kubeadm kubectl
EOF

kubernetes.repo set successfully. now we can proceed with kubectl, kubeadm, kubelet installation

yum install -y kubelet kubeadm kubectl --disableexcludes=kubernetes 
systemctl enable --now kubelet

kubelet is the primary “node agent” that runs on each node. It can register the node with the api-server using one of: the hostname; a flag to override the hostname; or specific logic for a cloud provider.

Notice: kubelet service is not yet activate

Use docker images command to check if some images are already upload.

docker images

if not run :

kubeadm config images pull

This command pull all the images we need for master.

Some images are pull successfully on docker

Additional instruction for the Master Node

Initialize a kubernetes control panel

Initialize a kubernetes control panel in order to setup a cluster nodes. Kubeadm init bootstraps a kubernetes control panel node by executing different module.

In my next topic, i will explain in very detail how to troubleshooting kubeadm init command.

From now, just go and follow this step.

#Make sure daemon.json contain the following line
cat /etc/docker/daemon.json{"exec-opts": ["native.cgroupdriver=systemd"]}
#Use echo command to enable bridgeecho 1 > /proc/sys/net/bridge/bridge-nf-call-iptables#Run this two command to ignore cpu and ram warning
kubeadm init --pod-network-cidr=10.240.0.0/16 --ignore-preflight-errors=NumCPU --ignore-preflight-errors=Mem

output of kubeadm init command

On the same output notice this information at the end :

Kubernetes control node has initialized successfully!

we need to run the following command as a regular user (On Master node):

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

Master node is successful configure.

Let’s check with kubectl command how many node are connect

kubectl get nodes

Notice the status of master node or controller node is not ready.

Reason for this warning is now we have to setup and configure worker node. Use putty to connect to your workup node. Make sure you have repeater the process form installing docker, kubelet, kubeadm and kubectl.

Worker node

Run the following command on your worker node

#Install docker engine yum install docker -t
#----------------------------------------
#Setup kubernetes.repo filecat <<EOF | sudo tee /etc/yum.repos.d/kubernetes.repo[kubernetes]name=Kubernetesbaseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-\$basearchenabled=1gpgcheck=1repo_gpgcheck=1gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpgexclude=kubelet kubeadm kubectlEOF#Install kubelet, kubeadm and kubectlyum install -y kubelet kubeadm kubectl --disableexcludes=kubernetes#Enable kubelet service
systemctl enable --now kubelet#Make sure daemon.json file contain following line
cat /etc/docker/daemon.json
{
  "exec-opts": ["native.cgroupdriver=systemd"]
}
#Enable bridge
echo 1 > /proc/sys/net/bridge/bridge-nf-call-iptables

Join Worker node to Master node

To join worker node to master node, kubeadm provide some option call join.

Go to your master node and execute

kubeadm token list
kubeadm token create --print-join-command

Copy the output of kubeadm token create — print-join-command and paste it on worker node terminal

means:

kubeadm join 172.31.33.51:6443 --token lrgr97.mi6c2rhedjhi70tz     --discovery-token-ca-cert-hash sha256:843418995616480a1d06e76f5ba404aa22794d3c32a29a8c996a67d9b4365083

Before paste, used kubectl get nodes command on master node to check how many nodes is available and the status of each node

Run kubectl join command on worker node:

Output will be:

Validation with Kubectl get nodes

Congratulation, we have successfully setup our multi node cluster!!!!!!!!!!!!! 😃 😃 👍

In my up coming topic i will continuous with How to deploy, run and expose pods on multi node clusters.