More Resources
{% columns %} {% column %} #### Pentaho Documentation Site Documenetaion site provides comprehensive technical guides and resources for the Pentaho Platform, covering data integration, analytics, metadata management, data quality, and business intelligence capabilities to help users discover, manage, and derive insights from their data. {% endcolumn %} {% column %} #### Customer Support Portal Registered customers can: * download product releases and service packs * access technical documentation * submit and track support cases * view security updates {% endcolumn %} {% endcolumns %} {% columns %} {% column %} {% endcolumn %} {% column %} {% endcolumn %} {% endcolumns %} # Pentaho 11 Installation Pentaho Enterprise 11 Installation .. #### Introduction This page is intended for system administrators, DevOps engineers, and technical users who need to install or evaluate Pentaho Enterprise Edition 11 on Linux or Windows. {% hint style="info" %} Pentaho Enterprise Edition 11 is a data integration and analytics platform with tools for data ingestion, transformation, visualization, and reporting. Pentaho Enterprise runs on multiple operating systems, including Linux and Windows. This page is the starting point for installing Pentaho Enterprise on **Linux** and **Windows**. For production environments, you can install Pentaho Enterprise in one of the following ways: * [**Archive (Linux)**](/pentaho-11-installation-en/installation/archive-installation) – Run the Pentaho Server on the Tomcat instance supplied by Pentaho. * **Manual (advanced)** – Deploy the Pentaho Server on an existing Tomcat or JBoss web application server. This option is intended for experienced administrators and is not covered step-by-step in this workshop. * [**Client tools**](/pentaho-11-installation-en/installation/archive-installation/install-client-tools) – Install only the Business Analytics (BA) or Data Integration (DI) client components. For evaluation scenarios on Windows, you can use the guided installer: * [**Enterprise evaluation (Windows)**](/pentaho-11-installation-en/installation/evaluation-installation) – Install a preconfigured evaluation environment using the Windows wizard. {% endhint %} {% hint style="warning" %} The Manual installation option is intended for experienced administrators who manage their own application servers (Tomcat or JBoss). This documentation does not provide a full step-by-step guide for Manual installations. {% endhint %} *** # Whats New .. Pentaho 11 Pentaho Data Integration (PDI) 11.0 and Pentaho Business Analytics (PBA) 11.0 deliver major updates. This release focuses on a modern UI, stronger security, and simpler deployments. 11.0 is a long-term support (LTS) release. For service pack and update schedules, see the [Pentaho Support Lifecycle page](https://support.pentaho.com/hc/en-us/articles/205789159-Pentaho-Product-Lifecycle-Overview). For the full change list, see the [11.0 release notes](https://docs.pentaho.com/whats-new/release-notes-11.0). ### Highlights * Author ETL pipelines in a browser with Pipeline Designer. * Organize and deploy ETL assets with project-based lifecycle management. * Preview the redesigned Pentaho User Console (PUC). * Build Mondrian models in the browser with Semantic Model Editor (SME). * Enable SSO with built-in OpenID Connect (OIDC) and OAuth 2.0. * Simplify container deployments with standardized, prebuilt images. * Run on Java 21 and Tomcat 10. * Manage features as plugins with Plugin Manager. * Monitor ETL runs with OpenTelemetry-based observability. ### Pipeline Designer Pipeline Designer is a browser-based UI for authoring ETL pipelines. Build PDI transformations and jobs in a web browser. Use Spoon only for legacy tooling.  It is similar to Spoon. The UI uses a modern framework. It remains compatible with transformations and jobs created in Spoon. Learn more about [Pipeline Designer](https://docs.pentaho.com/pba/11.0-pba/pipeline-designer). ### Project-based lifecycle management Before 11.0, PDI had no defined structure for organizing transformations, jobs, and configuration. That made collaboration and environment promotion harder for ETL and DevOps teams. Configuration resolution could also feel inconsistent.  Project-based lifecycle management addresses these gaps. Learn more about [configuring ETL with Projects](https://docs.pentaho.com/pdia-data-integration/pdia-11.0-data-integration/organizing-etl-with-projects#manageability). ### Modern Pentaho User Console (preview) {% hint style="info" %} This feature is in preview. Behavior and UI might change in later updates. {% endhint %} 11.0 introduces a redesigned user experience (UX) for Pentaho User Console (PUC). It aligns with the broader Pentaho platform UX. It addresses pain points in PUC 10.2 and earlier.  The existing PUC stays available until feature parity. Learn more about [Modern PUC](https://docs.pentaho.com/pba/11.0-pba/pentaho-user-console/modern-design). ### Semantic Model Editor (SME) 11.0 introduces a new tool for building and managing Mondrian data models. Previously, customers used Schema Workbench or Data Source Wizard. Semantic Model Editor (SME) provides a modern, web-based workflow. SME works for both new and advanced users. It improves the modeling experience in PBA, especially Analyzer. It supports existing Mondrian models and adds new capabilities.  Learn more about [Semantic Model Editor](https://docs.pentaho.com/pba/11.0-pba/semantic-model-editor). ### Built-in OIDC and OAuth 2.0 11.0 supports OpenID Connect (OIDC) and OAuth 2.0 authentication for Pentaho Server. This enables single sign-on (SSO) with identity providers such as Google, Okta, and Azure. It supports any OIDC-compliant identity provider (IdP).  Learn more about [OIDC and OAuth 2.0](https://docs.pentaho.com/pdia-admin/pdia-11.0-admin/administer/secure-the-pentaho-system/user-security/advanced-security-providers/oidc-oauth-2.0). ### Granular permissions for Pentaho Server 11.0 adds more granular and flexible access control across the platform. This addresses long-standing challenges: * Permissions were not fine-grained enough. For example, **Read Content** lets users see content from any plugin. File and folder permissions can still block access. * Permissions could not cleanly allow or block individual plugins. * Permissions were too broad for data sources and similar assets. * Execute permissions were too broad.  11.0 addresses these issues in Pentaho Server. Combined with OIDC and OAuth 2.0, it provides a stronger authentication and authorization model. Learn more about [granular permissions](https://docs.pentaho.com/pba/11.0-pba/semantic-model-editor/sharing-a-semantic-model/permissions-for-semantic-models). ### Simplified container deployment 11.0 simplifies Docker-based deployments. It introduces optimized, prebuilt images for on-premises deployments and major Kubernetes platforms. This includes plain Docker, Kubernetes, EKS, AKS, and GKE. Images use standardized installation paths and variables. Entrypoint scripts support runtime overrides for configuration files and licenses. Learn more about [Docker deployment](https://docs.pentaho.com/install/pdia-11.0-installation/pentaho-installation-overview-cp/docker-container-deployment-of-pentaho-installation-cp). ### Java 21 and Tomcat 10 support Java 21 is supported in 11.0. You can use Oracle JDK, OpenJDK, or other supported JVMs. Pentaho Server 11.0 ships with Tomcat 10. This addresses vulnerabilities and defects associated with Tomcat 9. Learn more in the [Components reference](https://docs.pentaho.com/install/pdia-11.0-installation/components-reference). ### Plugin Manager 11.0 introduces a Plugin Manager for both PDI and PBA plugins. Pentaho will ship more functionality as plugins over time. This makes it easier to identify, deploy, and update plugins.  Learn more about [Plugin Manager](https://docs.pentaho.com/pba/11.0-pba/pentaho-user-console/modern-design/plugin-manager). ### Karaf and OSGi removed, big data plugins delivered separately In 11.0, Karaf and OSGi are removed from PDI. Big data components are now delivered as plugins. There is no separate PDI distribution for big data add-ons. Deploy big data components the same way as other PDI plugins. This reduces both the PDI client and Pentaho Server deployment size by more than 1 GB. Learn more about [Plugin Manager](https://docs.pentaho.com/pba/11.0-pba/pentaho-user-console/modern-design/plugin-manager). ### OpenTelemetry-based observability [OpenTelemetry](https://opentelemetry.io/docs/languages/java/) (OTel) is an open standard for sharing telemetry data such as traces, metrics, and logs. Many tools can consume OpenTelemetry data, including Datadog, Splunk, Elastic, Amazon CloudWatch, and Azure Monitor. With the OTel plugin, you can monitor Pentaho ETL processes with: * Logs that are consolidated in a single place and represented hierarchically * Traces to view task timing, execution hierarchy, and variables during execution * Metrics to track data flow trends at specified points of interest Learn more about [Plugin Manager](https://docs.pentaho.com/pba/11.0-pba/pentaho-user-console/modern-design/plugin-manager). ### Other enhancements 11.0 also includes smaller enhancements and defect fixes. See the [11.0 release notes](https://docs.pentaho.com/whats-new/release-notes-11.0) for details. # Getting Started Archive installation of Pentaho Enterprise on Linux - Ubuntu 24.04 LTS .. {% stepper %} {% step %} **Download packages (EE)** Select your download binaries option: {% tabs %} {% tab title="Enterprise Edition" %} {% hint style="info" %} **30-day Enterprise Edition** The 30-day Pentaho Enterprise download is for: {% endhint %} 1. Navigate to the Pentaho download page & fill in the form. {% embed url="Fill out form
Download - Pentaho EE onPrem
Pentaho Enterprise OS versions
Pentaho Support Portal
Pentaho Customer Support Portal - Log In
Packages
| Cover image | |
|---|---|
| Windows 11 Installation | /files/45hzmlZbZPOvPHCR7ZIV |
| Linux Archive Installation | /files/H9P2DRCqPfgYBRFbNkGY |
| Cover image | Cover image (dark) | |||
|---|---|---|---|---|
| :screwdriver-wrench: Installation | NEW | Install Pentaho Enterprise Edition 11.x on Linux & Windows. | /files/ieWw0fkisLJYDec1ZxlZ | |
| :arrow-progress: Pipeline Designer | NEW | Pipeline Designer is a web-based interface that manages data integration pipelines. | /files/Af2wkqOTpXxswbOfoMKv | |
| :diagram-sankey: Data Integration | Simplify hybrid data estates using advanced data pipeline techniques. | /files/YJ5e3vAD4UrGZsKVlBJm | ||
| :kaaba: Semantic Model Editor | NEW | Data Modeling is the first step in understanding the reporting requirements. | /files/CbN4wwnC8iVnlvouTC26 | |
| :cube: Schema Workbench | A visual design tool used to create and edit Mondrian 3 OLAP schemas. | /files/NCD5ZZrCWh8cCuYpzChC | ||
| :clipboard-list-check: Metadata Editor | A graphical modeling tool that creates and manages semantic business models. | /files/P2J4wHyzVwPYg5OZVM6S | ||
| :chart-pie-simple: Business Analytics | Turn your raw data into actionable insights. | /files/JAc3lYCBs884RQ4nypaQ | ||
| :chart-mixed: CTools | A suite of visualisation tools for creating highly interactive dashboards. | /files/N5l6lblCLxZ9cdH1x3NX | /files/JAc3lYCBs884RQ4nypaQ |
Pentaho Architecture
| Port Number | Description |
|---|---|
| 5432 | PostgreSQL Server |
| 8080 | Pentaho Server Tomcat Web Server Startup Port |
| 8012 | Pentaho Server Shutdown Port |
| 9001 | HSQL Server Port |
| 9092 | Embedded H2 Database |
Pentaho Client / Server Architecture
Metadata Editor
Schema Workbench
Aggregation Designer
Analyzer Report
Interactive Report
Dashboard
Pipeline Designer
Carte Cluster
Pentaho paths
What's the difference bewteen Oracle JDK & OpenJDK?
Oracle JDK and OpenJDK are both implementations of the Java Platform, but they have some important differences: **Licensing and Cost:** * **OpenJDK** is completely free and open source under the GPL license. You can use it for any purpose without restrictions. * **Oracle JDK** changed its licensing model in 2019. It's now free for development and personal use, but requires a paid subscription for commercial production use (Oracle Java SE Subscription). **Source and Development:** * **OpenJDK** is the reference implementation of Java and serves as the base for most JDK distributions. Oracle actually contributes significantly to OpenJDK development. * **Oracle JDK** is built from the OpenJDK source code but includes some additional proprietary components and commercial features. **Performance and Features:** * In modern versions (Java 11+), the performance differences are negligible. Oracle has contributed most of its performance improvements back to OpenJDK. * Oracle JDK historically included some additional tools and features (like Java Flight Recorder and Java Mission Control), but many of these have been open-sourced and are now available in OpenJDK. **Support and Updates:** * **OpenJDK** receives community support and updates for about 6 months per release (except for LTS versions maintained by various vendors). * **Oracle JDK** offers Long Term Support (LTS) with commercial subscriptions, providing updates and security patches for extended periods. **Other Distributions:** Many vendors offer their own builds of OpenJDK with long-term support, including Amazon Corretto, Azul Zulu, Eclipse Temurin (formerly AdoptOpenJDK), and Red Hat OpenJDK. For most developers and organizations, OpenJDK or vendor-supported OpenJDK distributions are the go-to choice unless you specifically need Oracle's commercial support.OpenJDK versions
Java 21
Set PENTAHO_JAVA_HOME
PostgreSQL 17.7
\q
Set listener
SED: pg_hba.conf
Create server group
Server group
Connection details
pgAdmin 4 UI
Validate PostgreSQL 17 service ..
Pentaho Pro Suite
Unzip Pentaho Server
Pentaho databases
Pentaho User Console
NEW - Pentaho User Console
Validate Repository after first run (click to expand)
```sql -- Quartz example \c quartz SELECT COUNT(*) FROM qrtz_scheduler_state; -- Hibernate example (expect many tables after first start) \c hibuser \dt ```Troubleshooting (click to expand)
* HTTP 404 on `/pentaho` after startup: confirm `"$TOMCAT_HOME/webapps/pentaho"` exists, check `catalina.*.log` for deployment errors, and verify file permissions under `$PENTAHO_SERVER`. * Port 8080 already in use: change Tomcat port in `server.xml` or stop the conflicting service. * JDBC driver not found: verify the driver JARs (e.g., `postgresql-*.jar`, `mysql-*.jar`) exist in `"$TOMCAT_HOME/lib"`. * Authentication to PostgreSQL fails: prefer `scram-sha-256`; review `pg_hba.conf`, restart PostgreSQL, and test `psql -h 127.0.0.1` with the target user. * Jackrabbit schema issues: re-check `repository.xml` sections are set to `postgresql`. * License activation errors: check Tomcat logs in `tomcat/logs/` for `license`/`elm` messages.Licenses
Add license
License Manager
Plugin Manager
Switch to Modern Design
NEW - Pentaho User Console
Analyzer
Analyzer Plugin
Interactive Reporting
Interactive Reporting
Interactive Reporting Plugin
Dashboard Designer
Dashboard Designer Plugin
Pipeline Designer
Pipeline Designer Plugin
Piopeline Designer
Transformation - Hello World
Semantic Model Editor
Semantic Model Editor
SteelWheels Model
Scheduler
Carte Status
Carte details
SampleData connection
SampleData connection details
Create new database connection
Add Connection
Configure connection
| Plugin | Description |
|---|---|
| Analyzer | |
| Interactive Reporting | |
| Dashboard | |
| Pipeline Designer | |
| Semantic Model Editor | |
| Pipeline Designer | |
| Pipeline Designer Carte | |
| Elastic MapReduce |
Pentaho Pro Suite
~/Pentaho/design-tools
├── design-tools
└── data-integration
└── metadata-editor
└── schema-workbench
Spoon UI
Pentaho Metadata Editor
Schema Workbench
Aggregation Designer
General Troubleshooting (click to expand)
* Spoon fails to start or shows GTK/WebKit errors: * Install GTK/WebKit libs (see the note below) and try again. * Launch with additional SWT/GTK flags if needed, or test on a different desktop flavor. * Fonts/UI rendering issues on HiDPI displays: * Try `GDK_SCALE=2 ./spoon.sh` or adjust your desktop scaling. * Missing JDBC drivers in clients: * Copy the required driver JARs into the tool‑specific folders (see driver locations in the Server page) and restart the tool. * License prompts or feature disabled: * Ensure the License Manager has activated client entitlements; verify `PENTAHO_LICENSE_INFORMATION_PATH` if required. * Slow startup or out‑of‑memory errors: * Increase `-Xms`/`-Xmx` in the tool’s `*.ini` or startup script.Missing GTK/WebKit modules
You will need to add a version from previous Jammy release: 1. Add package repository. ```bash sudo apt-get install -qq software-properties-common ``` 2. Add repository entry. ```bash sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 3B4FE6ACC0B21F32 sudo add-apt-repository 'deb [trusted=yes] http://cz.archive.ubuntu.com/ubuntu bionic main universe' ``` 3. Update repositories. ```bash sudo apt-get update ``` 5. Install package. ```bash sudo apt-get install -qq libwebkitgtk-1.0-0 ``` ```bash sudo apt-get install libcanberra-gtk-module ``` 6. Start PDI. ```bash cd cd ~/Pentaho/design-tools/data-integration ./spoon.sh ```PDI Plugin Manager
| Plugin | Description |
|---|---|
| Databricks | The Bulk load into Databricks entry loads large volumes of data from cloud storage files directly into Databricks tables. How it works: It accomplishes this by using Databricks' COPY INTO command behind the scenes. |
| Salesforce Bulk Operation | The Salesforce bulk operation step performs large-scale data operations (insert, update, upsert, and delete) on Salesforce objects using the Salesforce Bulk API 2.0. How it works: The step reads data from an input stream, creates a CSV file of the changes, and executes the bulk job against Salesforce. After the job completes, you can optionally route three types of results to separate output streams: successful records, unprocessed records, and failed records. Requirements: You must have a Salesforce Client ID and Client Secret to use this step. |
| Google Analytics v4 | The Google Analytics v4 step retrieves data from your Google Analytics account for reporting or data warehousing purposes. How it works: The step queries Google Analytics properties through the Google Analytics API v4 and sends the resulting dimension and metric values to the output stream. |
| Hierarchical Data Type | Pentaho supports a hierarchical data type (HDT) through the Pentaho EE Marketplace plugin. This plugin adds the HDT data type and includes five specialized steps for working with it. What it does: These steps simplify working with complex, nested data structures. They can convert between HDT fields and formatted strings, and let you directly access or modify nested array indices and keys. Performance benefits: The steps significantly improve performance compared to handling hierarchical data as plain strings. Data structure: HDT can store nested or complex data built from objects and arrays, as well as single elements. It's compatible with any PDI step that processes hierarchical data. |
| Kafka Job | |
Hide Tomcat Server header
By default, Tomcat sends a `Server` header exposing version information. You can override it to reduce information leakage. 1. Edit the Tomcat connector in `server.xml`. ```bash sudo nano /opt/pentaho/server/pentaho-server/tomcat/conf/server.xml ``` 2. Add or update the `server` attribute on the HTTP connector and (if used) AJP connector, then save. ```xmlJava Security Manager (deprecated/removed)
The legacy Java Security Manager is deprecated and not available on modern Java LTS versions (including Java 21). Do not use `-security` with Tomcat on Java 21. Prefer OS‑level hardening, least‑privilege users, network scoping, and container/AppArmor/SELinux policies as appropriate.Change the web application context path
Change the context path if you do not want the application accessible at `/pentaho`. 1. Stop the Pentaho Server. ```bash cd /opt/pentaho/server/pentaho-server sudo ./stop-pentaho.sh ``` 2. Edit `context.xml`. ```bash sudo nano /opt/pentaho/server/pentaho-server/tomcat/webapps/pentaho/META-INF/context.xml ``` 3. Update the context path. ```xmlChange to HTTPs
Default port is 8080. 1. Stop the Pentaho Server. ```bash cd /opt/pentaho/server/pentaho-server sudo ./stop-pentaho.sh ``` 2. Change the connector port. ```bash sudo nano /opt/pentaho/server/pentaho-server/tomcat/conf/server.xml ``` ```xmlChange default HTTP port
Default port is 8080. 1. Stop the Pentaho Server. ```bash cd /opt/pentaho/server/pentaho-server sudo ./stop-pentaho.sh ``` 2. Change the connector port. ```bash sudo nano /opt/pentaho/server/pentaho-server/tomcat/conf/server.xml ``` ```xmlHarden or disable the Tomcat shutdown port
By default Tomcat listens on a local shutdown port (8005) for the `SHUTDOWN` command. * Disable the port by setting `port="-1"`, or * Change both the port and the shutdown command to unpredictable values. 1. Edit the `Custom error pages (404, 403, 500)
Define application‑level error pages to avoid exposing defaults. 1. Create an error page in your webapp. ```bash sudo tee /opt/pentaho/server/pentaho-server/tomcat/webapps/pentaho/error.jsp >/dev/null <<'EOF'Something went wrong
Please contact your administrator.
EOF ``` 2. Add error mappings in the webapp `web.xml`. ```bash sudo nano /opt/pentaho/server/pentaho-server/tomcat/webapps/pentaho/WEB-INF/web.xml ``` ```xmlSession timeout
Set a global session timeout for the application. 1. Edit the webapp `web.xml`. ```bash sudo nano /opt/pentaho/server/pentaho-server/tomcat/webapps/pentaho/WEB-INF/web.xml ``` ```xmlIncrease Karaf startup wait time
If server startup times out while Karaf installs features, increase the wait time. 1. Stop the server. ```bash sudo systemctl stop pentaho-server ``` 2. Edit `server.properties`. ```bash sudo nano /opt/pentaho/server/pentaho-server/pentaho-solutions/system/server.properties ``` Uncomment or add: ``` # Time (ms) to wait for Karaf to install features before timing out karafWaitForBoot=180000 ``` 3. Start the server. ```bash sudo systemctl start pentaho-server ```Remove sample data from the server
Remove evaluation samples before moving to production. 1. Stop the server. ```bash sudo systemctl stop pentaho-server ``` 2. Delete the `samples.zip` from default content (path may vary by version). ```bash sudo rm -f /opt/pentaho/server/pentaho-server/pentaho-solutions/system/default-content/samples.zip || true ``` 3. Edit the webapp `web.xml` and remove the HSQLDB sample definitions and the SystemStatusFilter (dev‑only). ```bash sudo nano /opt/pentaho/server/pentaho-server/tomcat/webapps/pentaho/WEB-INF/web.xml ``` Remove blocks similar to: ```xmlHide Home perspective widgets
Hide Getting Started and other widgets from the PUC Home page. 1. Stop the server. ```bash sudo systemctl stop pentaho-server ``` 2. Edit the Home perspective configuration. ```bash sudo nano /opt/pentaho/server/pentaho-server/tomcat/webapps/pentaho/mantle/home/properties/config.properties ``` Add or update: ``` disabled-widgets=getting-started,recents,favorites ``` 3. Start the server and log in to verify. ```bash sudo systemctl start pentaho-server ```Turn off autocomplete on the login page (advanced)
Changing vendor JSPs may be overwritten on upgrade. Prefer SSO or reverse proxy controls. If you must, edit the login JSP. 1. Stop the server. ```bash sudo systemctl stop pentaho-server ``` 2. Edit `PUCLogin.jsp`. ```bash sudo nano /opt/pentaho/server/pentaho-server/tomcat/webapps/pentaho/jsp/PUCLogin.jsp ``` 3. Set autocomplete to off for user/password inputs. ```html ``` 4. Start the server. ```bash sudo systemctl start pentaho-server ```Increase CSV upload limits
Adjust upload limits and (optionally) staging database. 1. Edit `pentaho.xml`. ```bash sudo nano /opt/pentaho/server/pentaho-server/pentaho-solutions/system/pentaho.xml ``` ```xmlReview installation
Accept License Agreement
Keep default installation path
Enter Password: password
Keep it simple
Install Steel Wheels - sampledata
Enter License URL or Activation ID
Pentaho 11
Data Integration - Plugin Manager
Pentaho Server - Plugin Manager
Containers
Docker Container Architecture showing Pentaho Server and PostgreSQL containers
PostgreSQL Database Architecture
| Database | Owner | Purpose & Contents |
|---|---|---|
| jackrabbit | jcr_user | Java Content Repository (JCR) - Stores all Pentaho content including reports, dashboards, data sources, analysis schemas, and user files. This is the primary content storage for the Pentaho repository. |
| quartz | pentaho_user | Quartz Scheduler - Manages all scheduled jobs, triggers, and calendars. Contains tables for job definitions (QRTZ6_JOB_DETAILS), triggers (QRTZ6_TRIGGERS), execution history, and cluster coordination locks. |
| hibernate | hibuser | Hibernate Repository - Hosts security configuration, audit logging, user session data, and contains two additional schemas: pentaho_dilogs (ETL execution logging) and pentaho_operations_mart (analytics data mart). |
Data flow showing HTTP requests and JDBC connections
| Pool Name | Connection Target | Used For |
|---|---|---|
| jdbc/Hibernate | repository:5432/hibernate | Security, Users, Roles |
| jdbc/Quartz | repository:5432/quartz | Job Scheduling |
| jdbc/jackrabbit | repository:5432/jackrabbit | Content Repository |
| jdbc/Audit | repository:5432/hibernate | Audit Logging |
| jdbc/live_logging_info | repository:5432/hibernate | ETL Runtime Logs |
| jdbc/PDI_Operations_Mart | repository:5432/hibernate | Operations Analytics |
| Key | Description |
|---|---|
| postgres_password | PostgreSQL superuser password |
| pentaho_user | Pentaho database username |
| pentaho_password | Pentaho database password |
| jdbc_url | JDBC connection URL |
| File | Purpose |
|---|---|
| docker-compose.yml | Defines all services (pentaho-server, postgres), networks, and volumes |
| docker/Dockerfile | Multi-stage build using debian:trixie-slim with OpenJDK 21 |
| docker-entrypoint.sh | Processes softwareOverride directories at container startup |
| .env | Environment-specific configuration (ports, passwords, memory) |
| deploy.sh | Automated deployment with pre-flight validation checks |
| db_init_postgres/*.sql | PostgreSQL database initialization scripts |
| vault-init.sh | Initializes Vault and stores secrets |
| rotate-secrets.sh | Rotates database passwords securely |
| Variable | Default | Description |
|---|---|---|
| PENTAHO_VERSION | 11.0.0.0-237 | Pentaho Server version |
| PENTAHO_IMAGE_NAME | pentaho/pentaho-server | Docker image name |
| PENTAHO_IMAGE_TAG | 11.0.0.0-237 | Docker image tag |
| POSTGRES_PASSWORD | password | PostgreSQL root password |
| POSTGRES_PORT | 5432 | PostgreSQL exposed port |
| PENTAHO_HTTP_PORT | 8090 | Pentaho HTTP port |
| PENTAHO_HTTPS_PORT | 8443 | Pentaho HTTPS port |
| PENTAHO_MIN_MEMORY | 4096m | JVM minimum heap size |
| PENTAHO_MAX_MEMORY | 8192m | JVM maximum heap size |
| LICENSE_URL | (empty) | EE license server URL |
| VAULT_PORT | 8200 | Vault API port |
Pre-flight checks & Build
Deploy Containers
Helper
Validate Deployment
Enter licensing details
Backup script
K3s & Pentaho Server Architecture
K3s Components
Pentaho Pod
PostgreSQL Pod
PostgreSQL Database Architecture
| Database | Owner | Purpose & Contents |
|---|---|---|
| jackrabbit | jcr_user | Java Content Repository (JCR) - Stores all Pentaho content including reports, dashboards, data sources, analysis schemas, and user files. This is the primary content storage for the Pentaho repository. |
| quartz | pentaho_user | Quartz Scheduler - Manages all scheduled jobs, triggers, and calendars. Contains tables for job definitions (QRTZ6_JOB_DETAILS), triggers (QRTZ6_TRIGGERS), execution history, and cluster coordination locks. |
| hibernate | hibuser | Hibernate Repository - Hosts security configuration, audit logging, user session data, and contains two additional schemas: pentaho_dilogs (ETL execution logging) and pentaho_operations_mart (analytics data mart). |
| Pool Name | Connection Target | Used For |
|---|---|---|
| jdbc/Hibernate | repository:5432/hibernate | Security, Users, Roles |
| jdbc/Quartz | repository:5432/quartz | Job Scheduling |
| jdbc/jackrabbit | repository:5432/jackrabbit | Content Repository |
| jdbc/Audit | repository:5432/hibernate | Audit Logging |
| jdbc/live_logging_info | repository:5432/hibernate | ETL Runtime Logs |
| jdbc/PDI_Operations_Mart | repository:5432/hibernate | Operations Analytics |
verify-k3s.sh
| Aspect | Docker Deployment | K3s Deployment |
|---|---|---|
| Orchestration | Docker Compose | Kubernetes (K3s) |
| Configuration | .env file + docker-compose.yml | Kubernetes manifests (YAML) |
| Secrets | Docker secrets or Vault | Kubernetes Secrets |
| Networking | Docker bridge network | K3s cluster network + Traefik Ingress |
| Storage | Docker volumes | PersistentVolumeClaims (PVCs) |
| Scaling | Manual (docker compose up --scale) | Declarative (replicas in deployment) |
| Health Checks | Docker HEALTHCHECK | Kubernetes readiness/liveness probes |
| Init Scripts | Volume mount to /docker-entrypoint-initdb.d | ConfigMap mounted to PostgreSQL pod |
| Variable | Default | Description |
|---|---|---|
| PENTAHO_VERSION | 11.0.0.0-237 | Pentaho Server version |
| EDITION | ee | Enterprise version |
| INCLUDE_DEMO | 1 | Include demo data |
| IMAGE_TAG | pentaho/pentaho-server:11.0.0.0-237 | Docker image tag |
| PENTAHO_MIN_MEMORY | 4096m | JVM minimum heap size |
| PENTAHO_MAX_MEMORY | 8192m | JVM maximum heap size |
| PENTAHO_DI_JAVA_OPTIONS | "-Dfile.encoding=utf8 -Djava.awt.headless=true" | |
| PENTAHO_IMAGE_NAME | pentaho/pentaho-server | Docker image name |
| TZ | America/NY | Time Zone of server |
| DB_TYPE | postgres | |
| DB_HOST | postgres | |
| DB_PORT | 5432 | PostgreSQL HTTP port |
| PUSH_TO REGISTRY | false | Pushes direct to K3s Regsitry |
| LOAD_INTO_K3S | true | Loads directly to K3s |
| RUN TESTS | true | |
| LICENSE_URL | (empty) | EE license server URL |
| Option (short) | Option (Long) | Description |
|---|---|---|
| -v | --version VERSION | Pentaho version (default: 11.0.0.0-237) |
| -t | --tag TAG | Docker image tag (default: pentaho/pentaho-server:VERSION) |
| -e | --edition EDITION | ee or ce (default: ee) |
| -d | --demo | Include demo content (default: no) |
| -p | --push | Push to registry after build |
| -h | --help | Push to registry after build |
Build & Push Pentaho Server image
Helm Charts
| YAML | Description |
|---|---|
| namespace.yaml | Creates an isolated Kubernetes namespace to contain all Pentaho resources |
| secret.yaml | Stores sensitive database credentials (passwords) encrypted in Kubernetes |
| config-*.yaml | Configures Pentaho environment variables (JVM memory, database settings, paths, timezone) Contains SQL scripts to initialize PostgreSQL databases (jackrabbit, quartz, hibernate) on first startup |
| pvc.yaml | Requests persistent storage volumes for PostgreSQL data and optional Pentaho data/solutions |
| *-deployment.yaml | Deploys Pentaho Business Analytics Server with init container, health probes, and resource limits. Deploys PostgreSQL 15 database server with automatic initialization and persistent storage |
| *-service.yaml | Routes external HTTP/HTTPS traffic to Pentaho Server via Traefik ingress controller. Exposes PostgreSQL port 5432 as a stable DNS endpoint for Pentaho to connect to. |
| ingress.yaml | Routes external HTTP/HTTPS traffic to Pentaho Server via Traefik ingress controller |
Pentaho Server image in K3s repository
Deploy Pentaho Server - Helm Chart
| Aspect | Docker Deployment | K3s Deployment |
|---|---|---|
| Orchestration | Docker Compose | Kubernetes (K3s) |
| Configuration | .env file + docker-compose.yml | Kubernetes manifests (YAML) |
| Secrets | Docker secrets or Vault | Kubernetes Secrets |
| Networking | Docker bridge network | K3s cluster network + Traefik Ingress |
| Storage | Docker volumes | PersistentVolumeClaims (PVCs) |
| Scaling | Manual (docker compose up --scale) | Declarative (replicas in deployment) |
| Health Checks | Docker HEALTHCHECK | Kubernetes readiness/liveness probes |
| Init Scripts | Volume mount to /docker-entrypoint-initdb.d | ConfigMap mounted to PostgreSQL pod |
Deploy Pentaho Server
Validate deployment
Health Check
Pentaho Tiers
Deploy MySQL Docker container
Docker MySQL containers
Relationship Diagram
Check Tables
MySQL
Configure & Test MySQL connection - sampledata
Enable: allowPublicKeyRetrieval
Test connection
Customer Data
SQL query - NYC Customers
General Troubleshooting (click to expand)
**Issue: "permission denied" errors** * Solution: Ensure your user is in the docker group and you've logged out/in or run `newgrp docker` **Issue: Docker service won't start** * Solution: Check logs with `sudo journalctl -u docker.service` **Issue: Cannot connect to Docker daemon** * Solution: Ensure Docker service is running with `sudo systemctl start docker`Pentaho Tiers
Use WSL 2
Add User to docker group
Deploy MySQL
mysql docker containers
Relationship Diagram
Check database & tables
sampledata table info
MySQL
Configure & Test MySQL connection - sampledata
Enable: allowPublicKeyRetrieval
Test connection
Customer Data
SQL query - NYC Customers
Verify K3s
K9s
Verify K3s
Install Helm
Where can I get a copy of the 'workshop files' ?
All the collateral can be found at: \~/Workshop--Installation. You can also copy/fork the Git repository: ``` git clone https://github.com/jporeilly/Workshop--Installation ``` ```git-rebase gh repo clone jporeilly/Workshop--Installation ```I cant write to sampledata database?
For security reasons the privileges from V10 have been removed. You will need to install Docker Desktop and create a MySQL sampledata database container.Why do we need Docker & Docker Compose?
Its easier to manage some of the required applications - databases, Brokers, etc .. - as Docker containers.Where can i download 30-day Pentaho Enterprise Edition?
An 30-day activation code will be emailed to you:VM is unresponsive !
* Refresh the browser session to reconnect. * Try another browser. The recommended browser: Google Chrome. * If you're connecting via a Corporate VPN, then this may cause issues. Contact your IT dept to get the URL 'white' listed.When does the Lab expire ?
The initial duration is 5 days. You will receive an email asking if you wish to extend your time limit.Videos aren't loading?
* Hard refresh your browser. CTRL + F5Is there sound ?
Yes .. There's no sound card attched to the Lab, so you'll need to copy and paste the Lab Guide URL in your host machine browser. 😊Pentaho Virtualization Support Statement
Pentaho supports virtualization technology across the Pentaho Suite. This statement explains what is supported and what may be requested during troubleshooting. #### What is supported Pentaho products that run on: * Supported operating systems * Minimum hardware requirements This applies whether you run Pentaho in a virtual environment or not. #### What you are responsible for You are responsible for failures caused by the hardware layer or operating system layer. This includes misuse of virtualization software. #### What Support may ask you to do Pentaho does not require you to reproduce every issue on physical hardware. Pentaho may ask you to reproduce or diagnose an issue on a native (non-virtual) supported operating system. This request is made only when there is reason to believe virtualization contributes to the issue. #### How virtualization-related issues are handled When a problem may relate to virtualization, investigation is handled as follows: * Pentaho provides standard support for all products. * If an issue occurs in a virtual environment, you may be asked to reproduce it on a physical (non-virtual) server. Pentaho then provides regular support. * You can authorize Pentaho to investigate virtualization-related items at normal time and materials rates. If the issue is virtualization-related, you may request a software change, if a resolution is possible. * If the issue is not virtualization-related, investigation and resolution are covered under regular maintenance. Pentaho is expected to work in virtual environments. #### Performance notes Performance impacts may still occur. These impacts may or may not be caused by virtualization. They may fall outside this support statement.Pentaho’s Global Data Protection & Privacy Policy
During troubleshooting, you may need to share data from your systems. This can include diagnostics, metadata, result sets, and similar artifacts. #### Ways to send data Pentaho Support provides several ways to transmit data, including: * Email * The Customer Portal * Pentaho Content Platform (HCP) #### Before you share Share only what Support requests. Remove or redact sensitive data when possible. See [Hitachi Vantara’s Global Data Protection & Privacy Policy](https://www.hitachivantara.com/en-us/legal) for details on how personal data and information are protected while in our possession.Submit via Portal
Subject Line
Ticket Status
Support Utilities
Pentaho Enterprise components
If your just starting your Pentaho journey, then dive into the components that comprise the Pentaho Enterprise Edition. The core architecture consists of several key layers: the Pentaho Server (also known as the BI Platform), which serves as the central hub for managing users, security, scheduling, and content repository; the Pentaho Data Integration (PDI) engine for ETL processes; and various client tools and interfaces that connect to these services.Linux Installation
The basic installation steps include extracting the archive to a designated directory (commonly `/opt/pentaho`), setting up a supported database like PostgreSQL or MySQL for the Pentaho repository, and running the provided database initialization scripts. The installer requires configuring environment variables, particularly `PENTAHO_HOME` and Java settings, as the platform runs on Java and requires a compatible JDK version.Post Installation Tasks
After the initial setup, administrators need to configure the server properties, including database connections, security settings, and port configurations. The default installation runs Pentaho Server on port 8080 and includes the Pentaho User Console (PUC) for web-based administration and report management.Upgrades & Patches
Pentaho upgrades and patches involve a systematic process to enhance functionality, security, and performance of the business intelligence platform. The upgrade process typically begins with thorough planning, including backing up existing data, configurations, and custom components. Organizations must evaluate their current version, review release notes for new features and breaking changes, and test the upgrade in a development environment before applying it to production systems.Windows Installation
This evaluation wizard setup allows organizations to assess Pentaho's suitability for their business intelligence needs, test integration with existing data sources, and evaluate the user experience before committing to a full commercial license. The installation serves as a sandbox environment for exploring the platform's analytics and reporting capabilities within a Windows infrastructure.Support Utilities
Client / Server
| Port Number | Description |
|---|---|
| 5432 | PostgreSQL Server |
| 8080 | Pentaho Server Tomcat Web Server Startup Port |
| 8012 | Pentaho Server Shutdown Port |
| 9001 | HSQL Server Port |
| 9092 | Embedded H2 Database |
Pentaho Client / Server Architecture
Metadata Editor
Schema Workbench
Aggregation Designer
Analyzer Report
Interactive Report
Dashboard
Carte Cluster
Java versions
| Certified | Supported |
|---|---|
| PostgreSQL 15 | PostgreSQL 14 & 15 |
| MySQL 8.026 | MySQL 8.026 |
| Oracle 23c | Oracle 19c & 23c (including patched versions) |
| MS SQL Server 2019 | Microsoft SQL Server 2017 & 2019 (including patched versions) |
| Maria DB 11.1.2 | Maria DB 11.1.2 |
create server group
server group
Connection details
pgAdmin4 UI
Pentaho Pro Suite
Change from peer to md5
| Script | Database | User / Password |
|---|---|---|
| \i create_jcr_postgresql.sql | Jackrabbit | |
| \i create_quartz_postgresql.sql | Quartz | pentaho_user/password |
| \i create_repository_postgresql.sql | Hibernate | |
| \i pentaho_mart_postgresql.sql | OpsMart | hibuser/password |
| \i pentaho_logging_postgresql.sql | Logging | hibuser/password |
Pentaho Repository databases
| Line # | Section | Check |
|---|---|---|
| line 71/442 | Repository | Filesystem schema: postgresql |
| line 129/442 | Datastore | databaseType: postgresql |
| line 231/442 | Workspace | Filesystem schema: postgresql |
| line 279/442 | Persistence Manager (1) | PersistenceManager schema: postgesql |
| line 347/442 | Versioning | Filesystem schema: postgresql |
| line 398/442 | Persistence Manager (2) | PersistenceManager schema: postgesql |
| line 434/442 | Database Journal | Journal schema: postgresql |
Pentaho User Console
Licensing
License Manager
Trial license - activation code
Analyzer report
Interactive report
Dashboard Designer
Pentaho Pro Suite
Spoon UI
Pentaho Metadata Editor
Schema Workbench
Aggregation Designer
Remove server banner
Most web servers display its version and modules in use by default. Best security practices recommend that you disable this option, since it can be used to find vulnerabilities of your site. 1. Edit: \Starting Tomcat with a Security Manager
Security Manager protects you from an untrusted applet running in your browser, use of a SecurityManager, while running Tomcat can protect your server from trojan servlets, JSPs, JSP beans, and tag libraries or even inadvertent mistakes. ``` start tomcat with –security argumentChange the Web Application Name
These instructions only work on Tomcat servers that are configured to accept context.xml overrides built into deployed .war files. 1. Stop the Pentaho Server. ```bash cd cd /opt/pentaho/server/pentaho-server sudo ./stop-pentaho.sh ``` 2. Edit context.xml. ```bash cd cd /opt/pentaho/server/pentaho-server/tomcat/webapps/pentaho/META-INF sudo nano context.xml ``` 3. Change the context. ```Change Port Number
Pentaho server default port is 8080. 1. Stop the Pentaho Server. ```bash cd cd /opt/pentaho/server/pentaho-server sudo ./stop-pentaho.sh ``` 2. Navigate to. ```bash cd cd /opt/pentaho/server/pentaho-server/tomcat/conf/ sudo nano server.xml ``` 3. Change the port number in the connector from 8080. ```Change SHUTDOWN port and Command
By default, tomcat is configured to be shutdown on 8005 port. Do you know you can shutdown tomcat instance by doing a telnet to IP:port and issuing SHUTDOWN command? ``` # telnet localhost 8005 Trying ::1... telnet: connect to address ::1: Connection refused Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. SHUTDOWN Connection closed by foreign host. # ``` You see having default configuration leads to high-security risk. It’s recommended to change tomcat shutdown port and default command to something unpredictable. 1. Edit Go to $tomcat/conf/server.xml file. ii. Modify server.xml by using vim editor ```Replace default 404, 403, 500 page
Having default page for not found, forbidden, server error exposes Tomcat version and that leads to security risk if you are running with vulnerable version. Let’s look at default 404 page. To mitigate, you can first create a general error page and configure web.xml to redirect to general error page. 1. Go to $tomcat/webapps/$application 2. Create an error.jsp file ```htmlSession Timeout
The session timeout for all web applications must be set to 20 minutes.\ This can be done by editing the file in the $tomcat/conf/web.xml and setting the following configuration option: ```Change the Karaf Startup Timeout Setting
Upon start up, the system waits for Karaf to install all of its features before timing out. If you modify Karaf and it now takes longer to install during start up, you may need to extend the default timeout setting to allow Karaf more time to install. The current default timeout is 2 minutes (120000 milliseconds). You can change this default timeout by editing the server.properties file. 1\. Stop the Pentaho Server. 2\. Navigate to the /pentaho-server/pentaho-solutions/system directory. 3\. Open the server.properties file with any text editor, and search for the karafWaitForBoot parameter. 4\. Uncomment the line containing the parameter and set it to your desired wait time in milliseconds. \# This sets the amount of time the system will wait for karaf to install all of\ \# it’s features before timing out. The default value is 2 minutes but can be\ \# overridden here.\ \#karafWaitForBoot = 120000 5\. Save and close the file. 6\. Restart the Pentaho Server.Remove Sample Data from the Pentaho Server
By default, Pentaho provides a sample data source and a solution directory filled with example content. These samples are provided for evaluation and testing. Once you are ready to move from an evaluation or testing scenario to development or production, you can remove the sample content. Follow the instructions below to completely remove the Pentaho sample data and solutions: 1. Stop the Pentaho Server. ```bash sudo systemctl stop pentaho-server ``` ii. Delete the samples.zip file from the /pentaho-server/pentaho-solutions/system/default-content directory. If you performed a manual WAR build and deployment, then the file path is /pentaho-server/pentaho-solutions/system. iii. Edit the /pentaho/WEB-INF/web.xml file inside of the deployed pentaho.war. As laid down by the Pentaho Plus graphical installer and archive packages, this path should be /pentaho-server/tomcat/webapps/pentaho/WEB-INF/web.xml. If you performed a manual WAR build and deployment, then you must adjust the path to fit your configuration. iv. Remove the hsqldb-databases section from the /pentaho/WEB-INF/web.xml file: v. BEGIN HSQLDB DATABASES ```Disable Home Perspective Widgets
The User Console default Home perspective contains the Getting Started widget, which has easy instructions and tutorials for evaluators. Perform the following steps to hide not only the Getting Started widget, but also other Home perspective widgets. 1. Shut down the Pentaho Server if it is currently running. 1. Choose one of the following options depending on your deployment status: *** * If you have not yet deployed, navigate to: /pentaho-platform/user-console/source/org/pentaho/mantle/home/properties/config.properties file. *** * If you have manually deployed and want to hide widgets later, navigate to: /pentaho-server/tomcat/webapps/pentaho/mantle/home/properties/config.properties file. 3. Find the line that starts with disabled-widgets= and type in the ID of the widget getting-started, as shown in the following example: ``` disabled-widgets=getting-started,recents,favorites ``` 4\. Save and close the file. *** You can also hide the Recents and Favorites widgets using the same method. 1. Locate the /pentaho-server/tomcat/webapps/pentaho/mantle/home directory and open the index.jsp file with any text editor. 2. Find the following line of code and comment it out, then save and close the file. ```javascript ``` 3. Start the Pentaho Server and log in to the User Console.Turn Autocomplete Off for Web App Login Screen
The User Console's sign-in settings have autocomplete turned on by default. Perform the following steps to manually turn off the autocompletion functionality: 1. Stop the Pentaho Server. 2. Modify PUCLogin.jsp. ```bash cd sudo nano /opt/pentaho/server/pentaho-server/tomact/webapps/pentaho/jsp/PUCLogin.jsp ``` 3. Locate the following sections of code and change the autocomplete entry to off, as shown: ``` ``` ``` ``` 3. Save and close the PUCLogin.jsp file. 4. Restart the Pentaho Server.Set System Max Row Limit for Interactive Reports
You can prevent too many resources from hitting your database server at once by setting a system-wide maximum row-limit for Pentaho Interactive Reports. Your users can still define their own design-time row limits in PIR, but they will never be able to go over the maximum number of rows that you have specified while designing their reports. 1. Stop the Pentaho Server. ```bash sudo systemctl stop pentaho-server ``` 2. Edit: /opt/pentaho/server/pentaho-server/pentaho-solutions/system/pentaho-interactive-reporting/settings.xml file. ```bash cd cd /opt/pentaho/server/pentaho-server/pentaho-solutions/system/pentaho-interactive-reporting sudo nano settings.xml ``` 3. Navigate to: \Increase the CSV File Upload Limit
You may find that you need to increase the size of the upload limit for your CSV files. These steps guide you through this process. 1. Go to /pentaho-server/pentaho-solutions/system and open the pentaho.xml file. Edit the XML as needed (sizes are measured in bytes): ```End user license agreement
Default installation path
Pentaho Repository
Keep it simple.
Let me decide for myself
Pentaho Customer Portal
Pentaho Downloads
EE Pentaho Plugins - versions
EE Pentaho 10.2 Plugins
Customer Support Portal
Pentaho Downloads
Packages
Where can I get a copy of the 'workshop files' ?
All the collateral can be found at: \~/Workshop--Installation. You can also copy/fork the Git repository: ``` https://github.com/jporeilly/Workshop--Installation-Linux ``` ```git gh repo clone jporeilly/Workshop--Installation ```I cant write to sampledata database?
For security reasons the privileges from V10 have been removed. You will need to install Docker Desktop and create a MySQL sampledata database container. [Broken mention](broken://pages/nyuNuY53XaIkQsM4MkZX)Why do we need Docker & Docker Compose?
Its easier to manage some of the required applications - databases, Brokers, etc .. - as Docker containers.Where can i download 30-day Pentaho Enterprise Edition?
An 30-day activation code will be emailed to you:VM is unresponsive !
* Refresh the browser session to reconnect. * Try another browser. The recommended browser: Google Chrome. * If you're connecting via a Corporate VPN, then this may cause issues. Contact your IT dept to get the URL 'white' listed.When does the Lab expire ?
The initial duration is 5 days. You will receive an email asking if you wish to extend your time limit.Videos aren't loading?
* Hard refresh your browser. CTRL + F5Is there sound ?
Yes .. There's no sound card attched to the Lab, so you'll need to copy and paste the Lab Guide URL in your host machine browser. 😊Pentaho Virtualization Support Statement
Pentaho supports virtualization technology across the Pentaho Suite. This statement explains what is supported and what may be requested during troubleshooting. #### What is supported Pentaho supports Pentaho products that run on: * Supported operating systems * Minimum hardware requirements This applies whether you run Pentaho in a virtual environment or not. #### What you are responsible for You are responsible for failures caused by the hardware layer or operating system layer. This includes misuse of virtualization software. #### What Support may ask you to do Pentaho does not require you to reproduce every issue on physical hardware. Pentaho may ask you to reproduce or diagnose an issue on a native (non-virtual) supported operating system. This request is made only when there is reason to believe virtualization contributes to the issue. #### How virtualization-related issues are handled When a problem may relate to virtualization, investigation is handled as follows: * Pentaho provides standard support for all products. * If an issue occurs in a virtual environment, you may be asked to reproduce it on a physical (non-virtual) server. Pentaho then provides regular support. * You can authorize Pentaho to investigate virtualization-related items at normal time and materials rates. If the issue is virtualization-related, you may request a software change, if a resolution is possible. * If the issue is not virtualization-related, investigation and resolution are covered under regular maintenance. Pentaho is expected to work in virtual environments. #### Performance notes Performance impacts may still occur. These impacts may or may not be caused by virtualization. They may fall outside this support statement.Pentaho’s Global Data Protection & Privacy Policy
During troubleshooting, you may need to share data from your systems. This can include diagnostics, metadata, result sets, and similar artifacts. #### Ways to send data Pentaho Support provides several ways to transmit data, including: * Email * The Customer Portal * Pentaho Content Platform (HCP) #### Before you share Share only what Support requests. Remove or redact sensitive data when possible. See [Hitachi Vantara’s Global Data Protection & Privacy Policy](https://www.hitachivantara.com/en-us/legal) for details on how personal data and information are protected while in our possession.Submit via Portal
Subject Line
Ticket Status
Support Utilities
Pip[eline Designer
Transformation
Draw a Hop between Nodes
Delete or Disable Hop
Always begin with 'Start' node
| Condition | Description |
|---|---|
| Unconditional | Specifies that the next job entry will be executed regardless of the result of the originating job entry |
| Follow when result is true | Specifies that the next job entry will be executed only when the result of the originating job entry is true; this means a successful execution such as, file found, table found, without error, and so on |
| Follow when result is false | Specifies that the next job entry will only be executed when the result of the originating job entry was false, meaning unsuccessful execution, file not found, table not found, error(s) occurred, and so on |
Jobs
Data Movement
Set Numbr of Copies & Data Movement
Pipeline Designer
Pipeline Designer UI
| Feature | Description |
|---|---|
| Collapse the left menu. | |
| Switch bewteen Design & View mode. | |
| Design Panel. You can also 'Search' for Nodes. | |
| Add a Note | |
| Reset the current Transformation. It will clear all Nodes & connections. Cannot be undone. | |
| RUN the Transformation. The drop-down Run Options enable: Run Configuration, Logging Level, Parameters & Variables | |
| Pause RUN. | |
| Stop RUN. | |
| View the Logging, Preview Data & Step Metrics panel. | |
| View the Kettle Status. | |
| View the Transformation Properties. | |
| Zoom in or Out of the Canvas. | |
| Lock the Canvas. | |
| Switch between Design & XML view. |
Drag & Drop - Generate Rows
Drag & Drop - Dummy
Create a Hop
Configure Generate rows
| Setting | Value |
|---|---|
| Limit | Keep default value of 10 records / rows. |
| Name | message |
| Type | String |
| Value | Hello World |
Configure - Generate rows
Preview data
Add Note
save transformation - gr:hello-world
Run the transformation
Hop information
Save
Pentaho Data Integration Components
Key Concepts & Terminology
Transforming & Enriching Datasets
Scaling out an Enterprise Solution
Pentaho Enterprise
Pentaho Client / Server Architecture
| Execution | Executes ETL jobs and transformations using the Pentaho Data Integration engine |
| Security | Allows you to manage users and roles (default security) or integrate security to your existing security provider such as LDAP or Active Directory |
| Content Management | Provides a centralized repository that allows you to manage your ETL jobs and transformations. This includes full revision history on content and features such as sharing and locking for collaborative development environments. |
| Scheduling & Monitoring | Provides the services allowing you to schedule and monitor activities on the Data Integration Server from within the Spoon design environment (Quartz). |
Carte Server
Welcome page
| OS: Windows / Unix | Action |
|---|---|
| spoon.bat / spoon.sh | Starts Spoon |
| kichen.bat / kitchen.sh | Command Line for Jobs |
| pan.bat / pan.sh | Command Line for Transformations |
| File | Description |
|---|---|
| kettle.properties | main configuration file with global variables |
| shared.xml | list of shared artefacts |
| db.cache | database cache for metadata |
| repositories.xml | list of repositories |
| .spoonrc | settings for the UI |
| .languageChoice | language settings |
kettle.properties
Shared Object - Connection
Welcome screen
kettle options - general
Kettle options - look & feel
Perspectives
Global variables
kettle.properties - Linux
Global variable in Transformation
Steps & Hops = Transformation
copy rows
distribute rows
Steps
parallelism
Changing buffer row set
| PDI Data Type | Java Data Type | Description | Example |
|---|---|---|---|
| BigNumber | BigDecimal | An arbitrary unlimited precision number. | 3.141592653589793238462643383279502884197169399375105820974944 |
| Binary | Byte[] | An array of bytes that contain any type of binary data. | An image file or a compressed file can be stored as Binary data |
| Boolean | Boolean | A boolean value true or false. | A boolean value true or false |
| Date | Date | A date-time value with millisecond precision. | 2023-10-20T10:48:51.123 |
Hierarchical - EE Plugin 9.5+ | BinaryTree | Data items that are related to each other by hierarchical relationships | A family tree |
| Integer | Long | A signed long 64-bit integer. | 42 |
| Internet Address | InetAddress | An Internet Protocol (IP) address. | 192.168.0.1 |
| Number | Double | A double precision floating point value (64bits). | 2.7182818284590452353602874713526624977572470936999 |
| String | String | A variable unlimited length text encoded in UTF-8 (Unicode). | “Hello world!” |
| Timestamp | Timestamp | Allows the specification of fractional seconds to a precision of nanoseconds. | 2023-10-20T10:48:51.123456789 |
Job Entries
hello world.tr
Generate rows
Generate rows - settings
Preview rows
dummy step
Notes
Notes - Style
transformation properties
Run Options
Automatically save transformation
Green ticks indicate successful execution
Logging
Step Metrics
Metrics
Preview data
View
Logging
Change data type
Set row level debugging
Error in step
Logging - error
pdi.log - Linux
Error handling
CSV file input
Hop - Error handling
Correct birthdate format
Errors for incorrectly formatted birthdates
PDI Project
Database Connection
Types of Big Data
Jupyter Notebook
Flat Files
Text files
orders.txt
Add path to file
Text file input - Content
Text File input - Fields
Row flattener
RegEx Evaluation
Replace in String
Select values
Preview data
Survey - Text File Output
Get system info
UDJE
Data grid - text
Data grid - data
Append
Text file input
Text file input - Content
Text file input - Fields
UDJE - concat question numbers
Select values - remove question_num
Append
Text file output - Content
Text file output - Fields
Enter argument
Preview data
Sales & Expenses
Blank template
Blank SourceData
Year
Generate rows - Year
Excel writer - Year
Excel writer - cell
Write Sales
Text file input - sales
Text file input - Content
Block step
Excel writer - Sales
Excel writer - Content
Write Expenses
Text File input - Expenses
Text File input - Content
Text File input - Fields
Block 'Write Sales'
Excel writer - Expenses
Excel writer - Content
Excel Book
X path
Read XML
Get data from XML - file
orders.xml
XML - file
XPath
XML - fields
Preview data
Get XML - URL
Pass URL in data stream field
Read URL
Select XPath
Configure fields
Invalid data type
Logs
Preview Plant Catalog
JSON input
JSON input - file
JSON input - fields
Preview data
No rows returned
Check the **Loop path** first. For this file, it must point to the array: ``` $.document.order[*] ``` If the JSON structure changes, update the loop path.Fields are null
Confirm field paths match the JSON keys. If you loop over `order[*]`, use `productline`, not `$.document.order.productline`.Wrong data types
Use **Get Fields** as a starting point. Then set types explicitly. Example: keep `status` as **String**.RSS
Steel Wheels - sampledata ERP
CRUID
MySQL
Configure & Test MySQL connection - sampledata
Enable: allowPublicKeyRetrieval
Test connection
Customer Data
Sql query - NYC Customers
MySQL - sampledata
Share database connection
Database Explorer - sampledata
SQL
Execute SQL statement
Create databases
Load sales data
CSV file input
Table output - options
Table output - fields
SQL editor
STG_SALES_DATA
Load orders data
Add path to orders.txt
Set Content
Get Fields
Table output - options
Table output -fields
SQL editor
STG_ORDERS_MERGED
Troubleshooting
**“Table already exists”**\ Disable table creation in **Table Output**, or drop the table first. **String truncation / data too long**\ Increase field lengths in **CSV File Input** before you generate SQL. **SQL window is empty**\ Select **Get fields** on the **Database fields** tab first.Read from a database
Table input
Calculate diff days
Number range
Sort rows
Status of 'shipped' orders
Troubleshooting
**Preview shows zero rows**\ Confirm the `WHERE STATUS = 'Shipped'` filter matches your source values. **SQL errors**\ Select **Get SQL select statement** again. Then re-apply your `WHERE` clause. **Date or number conversion issues**\ Fix types in **Select values**. Re-run the preview. **Out of memory during sort**\ Lower the sort size in **Sort rows**, or increase JVM memory.Update employees
Set file path
Text file input - Content
Text file input - Fields
Update fields
Step metrics
Update employees
Troubleshooting
**Step updates 0 rows**\ Confirm `EMPLOYEENUMBER` exists in `EMPLOYEES`. The Update step does not insert. **Updates fail with data type errors**\ Make sure `EMPLOYEE_NUMBER` is numeric. Use **Select values** to cast if needed. **Wrong rows updated**\ Confirm the key mapping is `EMPLOYEENUMBER` (table) = `EMPLOYEE_NUMBER` (stream).Insert / Update
Text File input - File
Text file input - Content
Text File input - Fields
Insert / update options
Step metrics
Insert / Update Employees
Troubleshooting
**All rows updated (no inserts)**\ Those employee numbers already exist in `EMPLOYEES`. **All rows inserted (no updates)**\ Your `EMPLOYEENUMBER` values were not found, or key mapping is wrong. **Duplicate key errors**\ You likely used **Table Output** instead of **Insert/Update**, or you mapped keys incorrectly. **Updates affect too many rows**\ Ensure `EMPLOYEENUMBER` is unique in your table. Fix duplicates before you upsert.Delete
STG_SALES_DATA
STG_SALES_DATA constraint QUANTITYORDERED > 50
CSV File input - PRODUCTLINE list
Set parameters
Get variables
Delete step
STG_SALES_DATA
Troubleshooting
**Nothing was deleted**\ Your criteria did not match any rows. Confirm your comparator and data types. **Too many rows deleted**\ Your comparator is too broad, or you mapped the wrong field names. **Delete step fails with type conversion errors**\ Cast `min_quantityordered` to a number with **Select values**.create sourceDB & targetDB
tr_onboard
CSV File input - customer_data.csv
Preview data
MariaDB: sourceDB connection
| Section Name | Value |
|---|---|
| Connection Name | MariaDB: sourceDB |
| Connection Type | MySQL |
| Host Name | localhost |
| Database Name | sourceDB |
| User name | pentaho_admin |
| Password | password |
create source_customer table
Execute SQL script
RUN Transformation
souurce_customer table
Resolve FirstName
Replace NULL values
Filter: FirstName = Unknown
Lookup FirstName and set as ResolvedFisrtName
Type 1
Type 2
Type 3
Type 4
Type 6
tr_scd
DIM_SCD
Data grid - meta
Data grid - data
Get system info
Data grid - lookup
Configure fields
set lookup mode
Set city to string
Preview data
Type 1
Field mapping and set database strategy
Preview data
DIM_SCD - view data
data Grid - Madrid
Field mapping and set database operation
DIM_SCD update
Data grid - Pariss
Field mapping and set database strategy
Preview data
DIM_SCD - view data
Data grid - edit Paris
Field mapping and set database strategy
DIM_SCD - view data
Preview data
Data grid - edit Paris
Field mapping and set database strategy
DIM_SCD - view data
Field mapping and set database strategy
DIM_SCD - view data
MinIO
| Workshop | Key Skills |
|---|---|
| Sales Dashboard | joins, lookups, aggregations |
| Inventory Reconciliation | XML parsing, outer joins, variance |
| Customer 360 | multi-source, JSONL, calculations |
| Clickstream Funnel | sessionization, pivoting |
| Log Parsing | regex, time-series analysis |
| Data Lake Ingestion | schema normalization, validation |
Sales Dashboard
Text file inputs
Select - sales.csv from VFS connections
Get Fields - Sales
Preview data - Sales
| Setting | Value |
|---|---|
| Step name | Products |
| Filename | pvfs://MinIO/raw-data/csv/products.csv |
| Delimiter | , |
| Head row present | ✅ |
| Format | mixed |
Select - products.csv from VFS connections
Get Fields - Customers
Preview data - Products
| Setting | Value |
|---|---|
| Step name | Customers |
| Filename | pvfs://MinIO/raw-data/csv/customers.csv |
| Delimiter | , |
| Header row present | ✅ |
| Format | mixed |
Select - customers.csv from VFS connections
Get Fields - Customers
Preview data - Customers
Lookups
Product Lookup
Customers Lookup
Lookups - Preview data
Calculator step
| New field | Calculation | Field A | Field B | Value type |
|---|---|---|---|---|
line_total | A * B | quantity | unit_price | Number |
discount_amount | A - B | sale_amount | line_total | Number |
Calculator
Preview data
Formula step
| New Field | Formula |
|---|---|
| customer_full_name | CONCATENATE([first_name];" ";[last_name]) |
| is_high_value | IF([sale_amount]>500;"Yes";"No") |
Formula step
Preview data
Add constants
Add constants
get system info
Get system info
Select values
Select
Preview data
Text File output
MinIO - Dashboard data
Inventory reconciliation
| Setting | Value |
|---|---|
| Step name | Read Warehouse XML |
| File or directory | pvfs://MinIO/raw-data/xml/inventory.xml |
| Loop XPath | /inventory/items/item |
| Encoding | UTF-8 |
| Ignore comments | ✅ |
| Validate XML | No |
| Ignore empty file | ✅ |
Configure XPath
Remap field names & Preview data
Customer 360
Select - sales.csv from VFS connections
Get Fields - Sales
Preview data - Sales
Log Analysis
| Field | Type | Length | Description | Source Mapping |
|---|---|---|---|---|
| product_id | String | 50 | Unique product identifier | CSV: product_id JSON: product_id XML: sku |
| product_name | String | 200 | Product display name | CSV: product_name JSON: product_name XML: name |
| category | String | 100 | Product category | CSV: category JSON: (derived from order type) XML: category |
| price | Number | 15,2 | Unit price in USD | CSV: price JSON: unit_price XML: null (not available) |
| quantity | Integer | 10 | Available stock quantity | CSV: stock_quantity JSON: quantity XML: quantity |
| source_system | String | 10 | Origin system identifier | Constant: 'csv', 'json', or 'xml' |
| ingestion_time | Timestamp | - | When record was ingested | System timestamp |
| Unified field | CSV | JSON | XML | Notes |
|---|---|---|---|---|
| Identifier | product_id | product_id | sku | Same meaning. Different name in XML. |
| Name | product_name | product_name | name | Same meaning. Different name in XML. |
| Category | category | ❌ | category | Missing in JSON. |
| Price | price | unit_price | ❌ | Different name in JSON. Missing in XML. |
| Stock quantity | stock_quantity | quantity | quantity | JSON quantity is not stock. |
Alternative approach (base path + relative field paths)
If your PDI build supports a base “Path” for the JSON Input step, set:\n\n- Base path: `$.data.orders[*].items[*]`\n\nThen set field paths relative to the base:\n\n- `product_id`: `product_id`\n- `product_name`: `product_name`\n- `unit_price`: `unit_price`\n- `quantity`: `quantity`\nSMB Server
SMB Server
Check SMB container.
Shared folders.
Test connection
tr_SMB_File_Retrieval
Content
Get Fields
Preview rows
HCP Solution
Metadata
Snowflake -AWS
Apache Hadoop Architecture
Master / Slave Nodes
Block size
Block Replication
Rack Awareness
MapReduce
Browse file system
DataNode
Jupyter Notebook
Pipeline
sales_analysis.ipynb
sales_analysis
sales_pipeline.ktr
CSV file input
Validation - quantity
Validation - unit_price
Replace in string - standardize
| In stream field | Search | Replace with | Use RegEx |
|---|---|---|---|
| product_category | electronics | Electronics | no |
| product_category | clothing | Clothing | no |
| product_category | home | Home | no |
| product_category | sports | Sports | no |
| product_category | books | Books | no |
Calculator
Formula - profit margin
Text file output
File watcher
RUN the Notebook
Reports
Merge streams
Sorted Merge
Merge streams
Merge Rows (diff)
Merge data streams
Select values
Preview data
Merge row (diff)
Merge rows (diff)
STG_ORDERS_MERGED
Synchronize after merge - Advanced tab
| Option | Description | Default Value |
|---|---|---|
| Operation fieldname | This is a required field. This field is used by the step to obtain an operation flag for the current row. | flagfield |
| Insert when value equal | Specify the value of the Operation fieldname which signifies that anInsert should be carried out. | new |
| Update when value equal | Specify the value of the Operation fieldname which signifies that an Update should be carried out. | changed |
| Delete when value equal | Specify the value of the Operation fieldname which signifies that a Delete should be carried out. | deleted |
| Perform lookup | Performs a lookup when deleting or updating. If the lookup field is not found, then an exception is thrown. This option can be used as an extra check if you wish to check updates/deletes prior to their execution. |
STG_ORDERS_MERGED
Synchronize after merge - FLAG
STG_ORDERS_MERGED - Synchronize
SQL joins
Cross Join
Joins
Database Join
Cross / Cartesian Join
Cross Joins
First names
Middle names
Join rows
UDJE - concat names, initials
Join rows
Select values
UDJE - name
Reservoir sampling
Preview data
Merge Join
ABCD - Data Grid
Red Blue Yellow - data grid
Merge Join
INNER Join
LEFT OUTER Join
RIGHT OUTER Join
FULL OUTER Join
Database Join
Data grid - Meta
Data grid - Data
Database join
Results
Lookup / List tables
Database lookup
Data grid - Meta
Data grid - Data
UDJE - like statement
Dtabase lookup - simple
Logging Results
Database lookup - erorr handling