Scaling out Jenkins is imperative, more so when it comes to improving the aspect of performance. Because bottlenecks obstruct productivity in a rapidly changing landscape of software development, this article discusses why multi-node agents are important in Jenkins and how you can go about implementing them most effectively.

Why we use Multi-Node Agents in Jenkins.

The potential of multi-node agents is to distribute workloads between several machines or nodes. This is a setup that comes with several benefits and particularly fits well into extensive CI/CD pipelines. Let's find out why multi-node agents are used below:

Single-Node Jenkins: Limitations:

Long Build Times: Everything happens on one server, causing delays.
Resource Contention: Multiple jobs compete for limited resources, leading to crashes.
Limited Scalability: As teams grow, a single node cannot handle increased workload.

Power of Multi-Node Agents:

Load Distribution Multi-node agents allow Jenkins to distribute builds across multiple machines, thereby reducing the load on a single system and ensuring better resource utilization.
Scalability Multi-node agents help Jenkins operations scale when the number of projects and builds grows since more nodes that can cope with the workload can be added in this way.
Parallel Builds It allows Jenkins to run several builds on various nodes at the same time, by implication, allowing CI/CD processes to happen more quickly.
Platform-specific Builds Different nodes may be configured with specific operating systems or environments. For example: A Linux node to test Linux-based applications. A Windows node for Windows-specific builds. A Mac node for iOS builds-all ensuring that builds are tested on the appropriate platforms.
Specialized Resources For certain types of builds, some nodes can be configured with certain tools and hardware or software dependencies. For example: Nodes utilizing a GPU for workloads relating to machine learning or the rendering of graphics. Nodes pre-installed with databases or application servers.
Improved Tolerance to Failure The potential of multi-node agents is extended to the evenness of tasks across these agents. If any one of the nodes dies, the build continues on other nodes.
Better Resource Utilization Multi-node agents open networks of machines to better use computing resources: the significant underutilization of top-bracket computing in most cases.
Isolation In this case, builds come in silos instead of being forced to converge globally or in a common env. (For example, library versions or environment contamination cannot match.)

Let’s Setup Multi-Nodes using Jenkins

Prerequisites:

1) EC2

Master Node :

256 MB of RAM
1 GB of drive space (although 10 GB is a recommended minimum if running Jenkins as a Docker container)

Install Java :

sudo apt update
sudo apt install fontconfig openjdk-17-jre
java -version

Install Jenkins in Host Machine:

sudo wget -O /usr/share/keyrings/jenkins-keyring.asc \
  https://pkg.jenkins.io/debian-stable/jenkins.io-2023.key
echo "deb [signed-by=/usr/share/keyrings/jenkins-keyring.asc]" \
  https://pkg.jenkins.io/debian-stable binary/ | sudo tee \
  /etc/apt/sources.list.d/jenkins.list > /dev/null
sudo apt-get update
sudo apt-get install jenkins
sudo usermod -aG docker jenkins

Verify Your Jenkins is Installed:

sudo systemctl start jenkins
sudo systemctl status jenkins

Now We need Docker and Docker Compose:

sudo apt-get update
sudo apt-get install docker.io -y 
sudo usermod -aG docker $USER
docker ps
sudo reboot

Now install Docker Compose

docker-compose-v2

Now in Worker Node:

sudo apt update
sudo apt install fontconfig openjdk-17-jre
java -version

sudo apt-get update
sudo apt-get install docker.io -y 
sudo usermod -aG docker $USER
docker ps
sudo reboot

Installation is Completed.

Copy your Master Node IP address 0.0.0.0:8080

 sudo cat /var/lib/jenkins/secrets/initialAdminPassword

After that Jenkins asks to create a first user admin.

Create a job

Paste your worker node IP address

Save it

Before using this Please setup Docker Credential


pipeline {
    agent { label "dev-server" }
    stages {
        stage("cloning") {
            steps {
                git url: "https://github.com/LondheShubham153/node-todo-cicd.git", branch: "master"
            }
        }
        stage("building") {
            steps {
                sh "docker build -t node-app ."
            }
        }
        stage("push to docker hub") {
            steps {
                withCredentials([usernamePassword(
                    credentialsId: "dockerHubcreds",
                    usernameVariable: "dockerHubUser",
                    passwordVariable: "dockerHubPass"
                )]) {
                    sh "docker login -u ${env.dockerHubUser} -p ${env.dockerHubPass}"
                    sh "docker image tag node-app:latest ${env.dockerHubUser}/node-app:latest"
                    sh "docker push ${env.dockerHubUser}/node-app:latest"
                }
            }
        }
        stage("deploy") {
            steps {
                sh "docker compose down && docker compose up --build"
            }
        }
    }
}