Workflows | Pentaho Academy

Clone Workshop--LLM git repository

The easiest installation method is to execute the setup.sh script which will automate the process.

Ensure the prerequisite steps have been completed: WSL + Docker

Clone the GitHub repository:

gh repo clone jporeilly/Workshop--LLM

Create a GenAI directory & copy over the GenAI files.

cd
mkdir ~/GenAI
cd ~/GenAI

Copy GenAI STACK files.

cd
cd ~/GenAI
cp -rvpi ~/Workshop--LLM/GenAI/* ~/Workshop--LLM/GenAI/.* .
ls -al

Deploy AI Stack

Check you have the required resources.

watch nvidia-smi

Deploy containers.

Deploy AI Stack

To make life easier, in the GenAI directory you will find 4 scripts:

Docker Compose configuration as docker-compose.yml
Environment template as .env.template
Setup script as setup.sh
GPU helper as configure-gpus.sh

The setup script will:

✅ Check for Docker and Docker Compose

✅ Install Ollama if needed

✅ Create all necessary directories

✅ Generate your .env file with secure random keys

✅ Create the external volume

✅ Pull all Docker images

✅ Start Ollama and pull AI models

✅ Start all services

✅ Test connections

Run the automated setup.

cd \
cd GenAI

# Make scripts executable
chmod +x setup.sh configure-gpus.sh

# Run the main setup (this does everything automatically)
./setup.sh

2. Configure Your GPUs (Optional)

If you have multiple GPUs:

# Run the GPU configuration helper
./configure-gpus.sh

This will show your GPUs and let you choose an allocation strategy.

Check Docker Desktop that the containers are up and running.

Flowise

Flowise is an open-source, low-code platform that enables users to build and deploy AI workflows and chatbots using a visual drag-and-drop interface. Built on top of LangChain, it allows developers and non-technical users to create custom AI applications by connecting various components like language models, vector databases, memory systems, and tools without extensive coding.

The platform supports integration with multiple AI providers including OpenAI, Anthropic, and local models, making it accessible for businesses and individuals looking to implement AI solutions quickly.

Log into Flowise:

The first time you log in you'll be asked to create an account.

You will need your deploy credentials - set in the .env file

The password policy requires 1 capital letter and number.

Setting

Value

Existing Username

admin

Existing Password

password

Once completed you will be able to login as Administrator with new account credentials:

Open WebUI

Open WebUI is a feature-rich, and user-friendly self-hosted AI platform designed to operate entirely offline. It supports various LLM runners like Ollama and OpenAI-compatible APIs, with built-in inference engine for RAG, making it a powerful AI deployment solution.

Open WebUI

x

Explore the Features ..

x

Tools extend the abilities of LLMs, allowing them to collect real-world, real-time data like weather, stock prices, etc.
Functions extend the capabilities of the Open WebUI itself, enabling you to add new AI model support (like Anthropic or Vertex AI) or improve usability (like creating custom buttons or filters).
Pipelines are more for advanced users who want to transform Open WebUI features into API-compatible workflows—mainly for offloading heavy processing.

Tools are like plugins that the LLM can use to gather real-world, real-time data. So, with a "weather tool" enabled, the model can go out on the internet, gather live weather data, and display it in your conversation.

Real-time weather predictions 🛰️.
Stock price retrievers 📈.
Flight tracking information ✈️.

Download and import manually

Navigate to the community site: https://openwebui.com/tools/

Click on the Tool you wish to import
Click the blue “Get” button in the top right-hand corner of the page
Click “Download as JSON export”
You can now upload the Tool into Open WebUI by navigating to Workspace => Tools and clicking “Import Tools”

Import via your Open WebUI URL

Navigate to the community site: https://openwebui.com/tools/

Click on the Tool you wish to import.

Click the “Get” button.

Enter the IP address of your Open WebUI instance - http://localhost:3000 - and click “Import to WebUI” which will automatically open your instance and allow you to import the Tool.

Click Save.

How can I use Tools?

Once installed, Tools can be used by assigning them to any LLM that supports function calling and then enabling that Tool.

Navigate to Workspace => Models.

Click the pencil icon to edit the model settings, scroll down to the Tools section and check any Tools you wish to enable.

Once done you must click save.

Now that Tools are enabled for the model, you can click the “+” icon when chatting with an LLM to use various Tools.

x

WebUI

Ollama Docker Connection

To manage your Ollama instance in Open WebUI, follow these steps.

Ensure your Ollama server is up and running - Powershell.

Get-NetTCPConnection -LocalPort 11435 -ErrorAction SilentlyContinue

Log into Open WebUI.

Navigate to: Settings > Connections.
Click on the + sign and configure a connection to Ollama.

Use the container name in the URL - http://ollama:11434

The API key can be anything as this is a local installation.

Pull Models

Follow the instructions below to check the ollama connection and pull models.

Go to Admin Settings in Open WebUI.

Navigate to Connections > Ollama > Manage (click the wrench icon).

You can also download models using the Model Selector.

Using the Ollama CLI to pull Models

You can directly access the Ollama container and pull models using the CLI:

# Connect to the running Ollama container
docker exec -it ollama /bin/sh

# Once inside the container, pull the model you want
ollama pull modelname

Replace modelname with the specific model you want to add (like llama3, mistral, gemma, etc.).

Ngrok

Ngrok is a powerful developer tool that creates secure tunnels between local development environments and the public internet. It essentially allows developers to expose local servers to the internet, even when they're behind NAT or firewalls, making it invaluable for development, testing, and demonstrations.

At its core, Ngrok works by creating a secure tunnel from your local machine to Ngrok's servers, which then provide a public URL that can route traffic back to your local application.

This is particularly useful when developing webhook integrations, testing mobile applications, sharing work-in-progress features with clients, or debugging applications in different environments.

For example, if you're running a local web server on port 3000, Ngrok can create a public URL like "https://92832de0.ngrok.io" that forwards all traffic to your local server.

ngrok will allow public access to your personal computer (via the port you specify), even if your computer is behind a firewall.

If you are uncomfortable with the security implications, you should skip the ngrok steps and only access your servers locally via localhost.

Sign up for a Ngrok account - its free.

Install Ngrok agent.

Linux

curl -s https://ngrok-agent.s3.amazonaws.com/ngrok.asc | \
  sudo gpg --dearmor -o /etc/apt/keyrings/ngrok.gpg && \
  echo "deb [signed-by=/etc/apt/keyrings/ngrok.gpg] https://ngrok-agent.s3.amazonaws.com buster main" | \
  sudo tee /etc/apt/sources.list.d/ngrok.list && \
  sudo apt update && sudo apt install ngrok

Windows

x

MacOS - use Homebrew

brew install ngrok

Test everything is working by running ngrok -h

Copy the value and run this command to add the authtoken in your terminal.

ngrok config add-authtoken TOKEN

Start ngrok tunnel by running the following command - Open Web UI listens on port 3000.

ngrok http 3000

Now open the Forwarding URL in your browser and you should see your local web service.

That URL is available to anyone in the world.
You are now using TLS (notice the 🔒 in your browser window) with a valid certificate without making any changes to your local service.

For free accounts, a static Ngrok domain can also be assigned to the application.

From the menu, select: Universal Gateway > Domains.

Click on the ... > Start Tunnel.

Copy and paste the command -adjust for app port: 3000

ngrok http --url=warm-anchovy-growing.ngrok-free.app 3000

The OpenWeb Chatbot will always be available on the static URL:

http://warm-anchovy-growing.ngrok-free.app

Traffic Dashboard

To inspect the traffic.

Qdrant

Log into Qdrant.

x

Open http://localhost:5678/ in your browser to set up n8n.

Enter your admin account

⚡️ Quick start and usage

The core of the Self-hosted AI Starter Kit is a Docker Compose file, pre-configured with network and storage settings, minimizing the need for additional installations. After completing the installation steps above, simply follow the steps below to get started.

The main component of the self-hosted AI starter kit is a docker compose file pre-configured with network and disk so there isn’t much else you need to install. After completing the installation steps above, follow the steps below to get started.

Open http://localhost:5678/ in your browser to set up n8n. You’ll only have to do this once. You are NOT creating an account with n8n in the setup here, it is only a local account for your instance!
Open the included workflow: http://localhost:5678/workflow/vTN9y2dLXqTiDfPT
Create credentials for every service:
Ollama URL: http://ollama:11434 - using container name
Postgres: use DB, username, and password from .env. Host is postgres
Qdrant URL: http://qdrant:6333 (API key can be whatever since this is running locally)
Google Drive: Follow this guide from n8n. Don't use localhost for the redirect URI, just use another domain you have, it will still work! Alternatively, you can set up local file triggers.
Select Test workflow to start running the workflow.
If this is the first time you’re running the workflow, you may need to wait until Ollama finishes downloading Llama3.1. You can inspect the docker console logs to check on the progress.
Make sure to toggle the workflow as active and copy the "Production" webhook URL!
Open http://localhost:3000/ in your browser to set up Open WebUI.
Go to Workspace -> Functions -> Add Function -> Give name + description then paste in the code from n8n_pipe.py
The function is also published here on Open WebUI's site.
Click on the gear icon and set the n8n_url to the production URL for the webhook you copied in a previous step.
Toggle the function on and now it will be available in your model dropdown in the top left!

To open n8n at any time, visit http://localhost:5678/ in your browser.

To open Open WebUI at any time, visit http://localhost:3000/.

With your n8n instance, you’ll have access to over 400 integrations and a suite of basic and advanced AI nodes such as AI Agent, Text classifier, and Information Extractor nodes. To keep everything local, just remember to use the Ollama node for your language model and Qdrant as your vector store.

x

Workflows Setup

Deploy AI Stack

Deploy AI Stack

Flowise

Open WebUI

Open WebUI

Download and import manually​

Import via your Open WebUI URL​

How can I use Tools?​

Ollama Docker Connection

Pull Models

Using the Ollama CLI to pull Models

Ngrok

Linux

Windows

MacOS - use Homebrew

Traffic Dashboard

Qdrant

⚡️ Quick start and usage

Download and import manually

Import via your Open WebUI URL

How can I use Tools?