Ctrlk
Start your 30-day Pentaho Enterprise Evaluation ..
For the complete documentation index, see llms.txt. This page is also available as Markdown.

Jenkins

Used as an open-source scheduler ..

Jenkins is an open-source automation server that enables developers to build, test, and deploy software efficiently. It's mainly used for Continuous Integration (CI) and Continuous Delivery (CD), automating the parts of software development related to building, testing, and deploying, facilitating continuous improvement and collaboration.

CI/CD

The following section is for Reference only.

Jenkins + plugins have been installed and configured.

Docker is a platform designed to help developers build, share, and run container applications.

Docker Compose is a tool for defining and running multi-container applications.

  1. Run update & upgrade (optional).

sudo apt update -y && sudo apt upgrade -y

  1. Create a Jenkins folder.

cd
mkdir ~/Jenkins

  1. Create docker-compose.yml

cd
cd ~/Jenkins
nano docker-compose.yml
services:
  jenkins:
    image: jenkins/jenkins:lts
    restart: always
    privileged: true
    user: root
    ports:
      - 9080:8080
      - 50000:50000
    container_name: jenkins-1
    volumes:
      - /home/pentaho/Jenkins/jenkins_configuration:/var/jenkins_home
      - /var/run/docker.sock:/var/run/docker.sock

  1. Save.

  1. Create jenkins container.

  1. Verify the logs

  1. Make a note of the password: 3d56d7ddc87c459ebaafe1117f7f56e7

  2. You can also retrieve the password with:

Configure Jenkins

  1. Access Jenkins UI.

http://localhost:9080localhost
Link to Jenkins UI

  1. Copy and paste the password.

Setup Admin account

  1. Click 'Continue' & keep the default option 'Install suggested plugins'.

Install suggested plugins
Default plugins

  1. Create Admin User.

Username

admin

Password

Welcome123!

Full name

system admin

admin user

  1. Save & Finish. You are now able to access Jenkins with the default admin account.

Jenkins URL confirmtion

This section is for Reference only.

Best Practice is to configure a specific system account - Jenkins

The Jenkins architecture is designed for distributed build environments. It allows us to use different environments for each build project balancing the workload among multiple agents running jobs in parallel.

The Jenkins controller is the original node in the Jenkins installation. The Jenkins controller administers the Jenkins agents and orchestrates their work, including scheduling jobs on agents and monitoring agents. Agents may be connected to the Jenkins controller using either local or cloud computers.

The agents require a Java installation and a network connection to the Jenkins controller.

  1. Generate an SSH key. This key will allow the controller to access the agent via SSH.

This command creates two files:

jenkins_agent - private key.

jenkins-agent.pub - public key.

  1. Navigate to your Jenkins dashboard.

http://localhost:9080/localhost
Link to jenkins UI

  1. Log in as user: Jenkins

Jenkins UI

  1. Go to Manage Jenkins option in main menu and click on Credentials button.

Jenkins UI - Credentials

  1. From the drop down select: Add Credentials

Add credentials to

  1. Fill in the form:

Configure connection to Agent

Kind

SSH Username with private key

id

jenkins

Description

jenkins ssh key

Username

jenkins - jenkins' user already exists by default in the jenkins_agent-1 container.

Private Key

select Enter directly and press the Add button to insert the content of your private key file at ~/.ssh/jenkins_agent

Paraphrase

fill your passphrase used to generate the SSH key pair.

  1. Click on 'Create'.

Credentials to access agents - SSH Key

Jenkins Agent on Docker

  1. Run the command to start your first agent.

• Replace the tag [your-public-key] for your own SSH public key.

• Your public key value in this example could be found by issuing : cat ~/.ssh/jenkins_agent_key.pub on the machine your created it. Do not add the square brackets [] around the key value

• The value of [your-public-key] MUST include the full contents of your .pub file, including the ssh-XXXX prefix.

• Ex: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAAQQCo9+BpMRYQ/dL3DS2CyJxRF+j6ctbT3/Qp84+KeFhnii7NT7fELilKUSnxS30WAvQCCo2yU1orfgqr41mM70MB

• If your machine already has a ssh server running on the 22 port (if you logged onto this machine thanks to the ssh command, that’s the case), you should use another port for the docker command, such as -p 4444:22

or you can modify the docker-compose.yml to include the agent and public key.

  1. Bring the jenkins-1 container down.

  1. Edit the docker-compose.yml

Add the ~/.ssh/jenkins_agent.pub - the public key that will authentiacte the connection to the agent.

  1. Check that the Jenkins Docker Agent is up and running.

Portainer - jenkins containers

Configure the Agent

This is a very basic setup - everything is installed on a single server, i.e. from the master node we can deploy a new slave node & run our Jobs using the deployed Agent.

  1. Navigate to your Jenkins dashboard.

  2. Log in as user: Jenkins

Jenkins UI

  1. Go to Manage Jenkins option in main menu and click on Nodes button.

Or click on the Set Up agent message.

Configure Node

  1. Click on New Node button.

Create New Node

  1. Enter the following details:

New Node

x

Node1 configuration

Let's run through a couple of Jenkins scenarios:

Periodic Build - demonstrates the first stages of the CI/CD pipeline. Developers commit to a Github repository which, periodically, triggers a 'build'.

The 'build' executes a shell 'PUT' script which uploads the Jobs & Transformations into the Pentaho Repository.

Commit Build - a Github commit triggers a 'build'.

You could simply set a CRON schedule to trigger the 'build' process ..

  1. Log into Jenkins.

http://localhost:9080localhost
Link to Jenkins
Username
admin

Password

Welcome123!

  1. Click on 'Create a Job' or 'New Item'.

  2. Provide a name - Periodic Build - for the item and then choose: 'Freestyle project' & click: 'OK'

Freestyle project - Periodic Build

  1. Set the following project options - execute every 5 mins:

Section
Option
Value

Source Management

Git

Repository URL

https://github.com/jporeilly/Jenkins-Github

Branches to build

Branch Specifier

**

Build Triggers

Build periodically

Schedule

H/5 * * * *

Build Steps

Execute shell

Command

curl -H "Content-Type: application/xml" -d @Job1.kjb -X PUT -u admin:password -i http://pentaho.pentaho.lab:8080/pentaho/api/repo/files/:public:Jenkins:Job1.kjb

curl -H "Content-Type: application/xml" -d @Transformation1.ktr -X PUT -u admin:password -i http://pentaho.pentaho.lab:8080/pentaho/api/repo/files/:public:Jenkins:Transformation1.ktr

curl -H "Content-Type: application/xml" -d @Transformation2.ktr -X PUT -u admin:password -i http://pentaho.pentaho.lab:8080/pentaho/api/repo/files/:public:Jenkins:Transformation2.ktr

After specifying the Repository URL, and the error "Failed to connect to repository..." is displayed - indicates Git has not been installed.

If the Repository is private, Jenkins wil validate the Github credentials when pulling the source code.

Source Code Management

Configure Git Repository

Build Triggers

Branches to build
Build periodically - every 5 minutes

Build Steps

Build Steps

Before you click Save, just check:

• Pentaho Server is up and running.

• Create a Public/Jenkins folder & is empty.

  1. Check the 'Build Status' in the Dashboard.

  1. Click on a build

  1. View the 'Console Ouput'.

Console Output

  1. Finally .. check the Pentaho Repository.

Pentaho Repsoitory

In CI/CD we want to trigger the build once we commit our code to git.

There are at least two approaches to achieving this:

You could have Jenkins continuously polling the git repository for changes or more efficiently, have Github tell Jenkins about any commits that happen.

  1. Log into Jenkins.

http://localhost:9080localhost
Link to Jenkins
Username
admin

Password

Welcome123!

  1. Click on 'New Item'.

New Item

  1. Provide a name - Commit Build - for the item and then choose: 'Freestyle project' & click: 'OK'

Freestyle Project - Commit Build

  1. Set the following project options - Git commit:

Section
Option
Value

Source Management

Git

Repository URL

https://github.com/jporeilly/Jenkins-Github

Branches to build

Branch Specifier

**

Build Triggers

Build periodically

Schedule

H/5 * * * *

Build

Execute shell

Command

curl -H "Content-Type: application/xml" -d @Job1.kjb -X PUT -u admin:password -i http://pentaho.pentaho.lab:8080/pentaho/api/repo/files/:public:Jenkins:Job1.kjb

curl -H "Content-Type: application/xml" -d @Transformation1.ktr -X PUT -u admin:password -i http://pentaho.pentaho.lab:8080/pentaho/api/repo/files/:public:Jenkins:Transformation1.ktr

curl -H "Content-Type: application/xml" -d @Transformation2.ktr -X PUT -u admin:password -i http://pentaho.pentaho.lab:8080/pentaho/api/repo/files/:public:Jenkins:Transformation2.ktr

After specifying the Repository URL, and the error "Failed to connect to repository..." is displayed - indicates Git has not been installed.

If the Repository is private, Jenkins wil validate the Github credentials when pulling the source code.

x

x

PreviousUse CasesNextData Sources

Last updated

Was this helpful?