Storage

Setup Object Stores & SMB ..

Object Stores

Object storage systems like Amazon S3 and MinIO provide a way to store and retrieve large amounts of unstructured data such as files, images, videos, and backups through a simple web-based API. Unlike traditional file systems that organize data in hierarchical folders, object stores use a flat namespace where each piece of data (called an object) is stored in containers called buckets and accessed via unique keys or URLs.

Amazon S3 is AWS's flagship object storage service that offers virtually unlimited scalability, multiple storage classes for different use cases, and integration with other AWS services.

MinIO is an open-source alternative that provides S3-compatible APIs and can be deployed on-premises or in private clouds, making it popular for organizations that want object storage capabilities without vendor lock-in.

Both systems are designed for high durability, availability, and can handle massive scale while providing simple REST API access for applications to store and retrieve data programmatically.

MinIO

Follow the instructions below to setup a MinIO Docker Container.

Create a MinIO folder and copy the required files.

Create directory & copy - PowerShell

cd \
cd C:\Workshop--Data-Integration\Setup\MinIO
.\copy-minio.ps1

Check the Directory has been created and the files copied over.
Execute the docker-compose script to create the container.

MinIO Container - PowerShell

cd \
cd C:\MinIO
.\run-docker-minio.ps1

Check the container is up and running in Desktop Docker.

Access MinIO UI:

Username: minioadmin

Password: minioadmin

The MinIO port has been changed to prevent conflicts.

In this hands-on workshop, you'll learn to deploy and configure an SMB (Server Message Block) server using Docker Desktop and on Linux.

Follow the instructions below to deploy an SMB Docker container. We're going to deploy a very simple SMB server with 2 users:

Alice -

Bob -

As Windows already has a default SMB server running on port:445 so this is changed to 1445.

Pentaho Data Integration

(Optional) Download the latest jcifs driver.

(Optional) Copy the JCIFS JAR file into Pentaho Data Integration "lib" folder.

Download CIFS driver

Pentaho Data Integration ships with jcifs-1.3.3.jar

If you wish to replace the current driver, rename to: jcifs-1.3.3.jar -> jcifs-1.3.3.jar.bak

jcifs 2.1.40.jar driver has been downloaded to the Workshop--Data-Integration/Drivers

Create SMB Share Directories

Create a SMB folder and copy the required files. Will also add some sample data.

Create directory & copy - PowerShell

cd \
& "C:\Workshop--Data-Integration\Setup\SMB\copy-smb.ps1"

Check the Directory has been created and the files copied over.

Creating Local Windows Users

Let's add our SMB users to the system:

Bob -

Alice -

Right-click on the Start button.
Select Computer Management from the context menu.
Alternatively, press Win + X and select Computer Management.
In Computer Management, expand System Tools.
Click on: Local Users and Groups.
Select Users folder.
Right-click in the Users pane (right side).
Select New User...
Fill in the following details:

Username: bob

Full Name: Bob Smith

Password: password

Click Create
Click Close
Repeat the workflow to create: alice

SMB Share

Right-click on the C:\SMB\Bob folder.
Select Properties.
Click the Sharing tab.
Click Advanced Sharing...

☑ Check "Share this folder"
Share name: Bob (default is fine).

Click Permissions.
Remove "Everyone" if present (select and click Remove).
Click Add...

Add the following user:
- Type bob, click Check Names, click OK

Set permissions for Bob Smith:
- Select Bob Smith: ☑ Full Control

Click OK three times.

In the Sharing tab, note the Network Path: \\[Computer Name]\Bob

From another computer: \\[your-computer-ip]\Bob

Obviously if you want to play around with the SMB shares for Alice & 'Shared' then you will have to repeat the workflow.

Test

Time to test ..

Let's see if we can access C:\SMB\Bob from across the Network.

In the File Explorer or Run command , enter the following UNC path to access C:\SMB\Bob

\\[Computer Name]\Bob or \\localhost\Bob

You should see the following popup message displayed:

Follow the instructions outlined below to deploy an SMB server on Ubuntu 24.04.

Ensure all installed Packages are up-to-date.

sudo apt update && sudo apt upgrade -y

Install Samba server.

sudo apt install tasksel
sudo tasksel install samba-server

Make a copy of the existing configuration file and create a new /etc/samba/smb.conf configuration file

sudo cp /etc/samba/smb.conf /etc/samba/smb.conf_backup
sudo bash -c 'grep -v -E "^#|^;" /etc/samba/smb.conf_backup | grep . > /etc/samba/smb.conf'

Any user existing on the samba user list must also exist within the /etc/passwd file.

sudo smbpasswd -a pentaho

New SMB password: password
Retype new SMB password: password
Added user pentaho.

Add the home directory share.

sudo nano /etc/samba/smb.conf

Copy & paste the following to the bottom of the file - private home & public access.

[homes]
   comment = Home Directories
   browseable = yes
   read only = no
   create mask = 0700
   directory mask = 0700
   valid users = %S
[public]
  comment = public anonymous access
  path = /var/samba/
  browsable =yes
  create mask = 0660
  directory mask = 0771
  writable = yes
  guest ok = yes

Save.

CTRL + O
Enter
CTRL + X

Create a directory that mounts public share and change its access permission.

sudo mkdir /var/samba
sudo chmod 777 /var/samba/

Restart your samba server.

sudo systemctl restart smbd

SMB Server

We're going to setup the Samba server with access to shareable, public directory - /var/samba/ - that can be accessed anonymously.

Next .. access to the 'pentaho user' - /pentaho/home directory. Obviously you'll need to be a registered user with a password to access the directory.

Let’s create some test files.

touch /var/samba/public-share 
touch /home/pentaho/home-share

Public

A 'public' directory that be accessed from any machine ..

In File Explorer, select: + Other Locations.
Enter the following connection details:

smb://pentaho.local/public/

Connect as: Anonymous.

You should see the public-share file.

Registered

Only registered users can access the /pentaho/home directory ..

In File Explorer, select: + Other Locations.
Enter the following connection details:

smb://pentaho.local/homes/

Connect as: Registered User.

Username: pentaho

Domain: WORKGROUP

Password: password

You should see the public-share file somewhere in the /home directory.
Follow the instructions below to deploy:

PreviousData Sources NextBig Data

Last updated 1 month ago

Was this helpful?

Good evening