> For the complete documentation index, see [llms.txt](https://academy.pentaho.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://academy.pentaho.com/solution-architect-path-sandbox/solution-architect/di-developer-practitioner.md).
# DI Developer โ Practitioner
Level 1 of the Solution Architect path: DI Developer โ Practitioner. 37 lessons across 6 modules: build, run and operate Pentaho Data Integration pipelines, ending in a capstone project.
{% hint style="warning" %}
**๐งช SANDBOX โ draft content, built from `developer-di-practitioner` 0.1.27 in Pentaho Content Manager.** Module and lesson names, the reading-or-workshop kinds and the times come straight from each lab's manifest. Items marked REPLACE-ME need an owner's confirmation.
{% endhint %}
The first step on the Solution Architect path: learn Pentaho Data Integration the way a practitioner uses it. By the end you'll build, run and operate real pipelines โ reading and writing files, databases and object storage, enriching and joining the data in flight, and packaging it all as parameterised jobs โ and you'll have delivered a complete daily sales pipeline of your own as the capstone.
{% columns %}
{% column %}
:file-lines:
37 lessons
across 6 modules
{% endcolumn %}
{% column %}
:flask:
34 workshops
hands-on labs
{% endcolumn %}
{% column %}
:clock:
~12h 15m
two to three days, or a week of evenings
{% endcolumn %}
{% endcolumns %}
{% hint style="success" icon="cloud-arrow-up" %}
Every workshop is hands-on and runs on Pentaho's hosting platform. The Lab Guide is embedded in each lab and loads automatically when you open it โ nothing to install.
{% endhint %}
***
Before you start
{% columns %}
{% column %}
{% hint style="success" icon="circle-check" %}
**What you'll need**
Familiarity with the basics of data integration โ tables, files, keys and the idea of a pipeline. No prior Pentaho experience is assumed, and comfort with SQL will make the database and scripting modules go faster. The hosted lab provides Pentaho Data Integration, MySQL and MinIO, all started for you as you sign in.
{% endhint %}
{% endcolumn %}
{% column %}
{% hint style="info" icon="tag" %}
**Built against**
REPLACE-ME โ confirm the Pentaho Data Integration release the hosted labs run, so screens and menus match the guide step for step. The PDI Developer path's Level 1 page cites Enterprise Edition 11.0.0.3-310.
{% endhint %}
{% endcolumn %}
{% endcolumns %}
***
{% hint style="info" icon="trophy" %}
#### What you'll build
A complete Daily Sales Pipeline: a parameterised job that ingests CSV, JSON and master data, validates and enriches the rows, tracks historical changes with a slowly changing dimension, and loads a dimensional model. On the way there you'll have built pipelines against text, JSON, XML and Excel files, a MySQL database and MinIO object storage; joined, looked up and merged data in flight; scripted with Formula and JavaScript; and run it all through jobs with parameters and run configurations โ the skills the Installation and deployment courses that follow take for granted.
{% endhint %}
```mermaid
%%{init: {"theme":"base","themeVariables":{"fontFamily":"Arial, Helvetica, sans-serif","fontSize":"13px","lineColor":"#64748b","clusterBkg":"#fafaf9","clusterBorder":"#d6d3d1","edgeLabelBackground":"#ffffff"},"flowchart":{"curve":"basis","nodeSpacing":28,"rankSpacing":44,"padding":22,"wrappingWidth":600,"htmlLabels":true}}}%%
flowchart TB
classDef input fill:#dbeafe,stroke:#1d4ed8,color:#1e3a8a,text-align:left,rx:6,ry:6
classDef enrich fill:#ede9fe,stroke:#6d28d9,color:#3b0764,text-align:left,rx:6,ry:6
classDef join fill:#ccfbf1,stroke:#0f766e,color:#134e4a,text-align:left,rx:6,ry:6
classDef output fill:#dcfce7,stroke:#15803d,color:#14532d,text-align:left,rx:6,ry:6
classDef check fill:#fef3c7,stroke:#b45309,color:#78350f,text-align:left,rx:6,ry:6
classDef reject fill:#fee2e2,stroke:#b91c1c,color:#7f1d1d,text-align:left,rx:6,ry:6
O["Orders
yesterday's sales, CSV "]:::input
C["Customers
JSON master data "]:::input
P["Products
master data "]:::input
V["Validate
Filter rows: keys present,
values convert "]:::check
R["Rejects
bad rows, logged,
written to a file "]:::reject
ST[("Staging tables
stg_orders, stg_customers,
stg_products, in MySQL ")]:::output
J["Enrich and calculate
Merge join, Stream lookup,
Formula "]:::join
D["SCD Type 2 on regions
Dimension lookup/update "]:::enrich
F[("Dimensional model
fact_sales and dim_region ")]:::output
X["Excel summary
sales by region "]:::output
O --> V
C --> V
P --> V
V -->|"good rows "| ST
V -.->|"bad rows "| R
ST --> J
ST --> D
J --> F
D --> F
F --> X
```
The Daily Sales Pipeline capstone: three sources staged, validated and enriched into a dimensional model and an Excel summary, all run by one job
***
What you'll learn
{% columns %}
{% column %}
* How Pentaho's client tools and server components fit together, and the vocabulary that runs through every lab โ steps, hops, rows, transformations and jobs
* Configure Spoon and the KETTLE variables that drive it, then build a 'Hello World' transformation with logging and error handling
* Read and write text, JSON, XML and Excel files
* Work with relational databases โ connect, read, write, update, insert-update and delete โ and track history with slowly changing dimensions
{% endcolumn %}
{% column %}
* Read from and write to object storage with MinIO
* Shape data in flight with database, merge, XML and row joins, lookups and merge-streams patterns, and script with Formula and Modified JavaScript Value when SQL alone isn't enough
* Move pipelines into production with jobs, order of execution, parameters and variables, metadata-driven table onboarding and run configurations
* Deliver an end-to-end pipeline โ the capstone โ that ingests, validates, enriches and loads a dimensional model
{% endcolumn %}
{% endcolumns %}
***
Course outline
:file-lines: reading ยท :flask: hands-on workshop
{% hint style="info" icon="circle-info" %}
Full lessons unlock once you're enrolled โ this is the outline only. Times are the lab guides' own estimates.
{% endhint %}
| Module | Reading | Workshops | Time |
|---|
| Getting Started | 1 | โ | ~20m |
| Components & Concepts | โ | 5 | ~1h |
| Data Sources | 2 | 13 | ~4h 50m |
| Enriching the Dataset | โ | 9 | ~2h 35m |
| Enterprise Solution | โ | 6 | ~1h 30m |
| Capstone Project | โ | 1 | ~2h |
| Total | 3 | 34 | ~12h 15m |
{% stepper %}
{% step %}
### :rocket: Getting Started
`1 reading ยท ~20m`
How to get the most out of this guide, and how to set your machine up โ whether you are on a lab VM or installing on your own laptop. Five minutes here makes every later lab smoother.
See the full breakdown
| Lesson | Time |
|---|
| :file-lines: | Before You Start | ~20m |
{% endstep %}
{% step %}
### :lightbulb: Components & Concepts
`5 workshops ยท ~1h`
The pieces of Pentaho and the vocabulary that runs through every later lab. Meet the client tools and server components, configure Spoon and the KETTLE variables that drive it, then build a 'Hello World' transformation covering steps, hops, rows and the run/preview cycle โ finishing with logging and error handling.
See the full breakdown
| Lesson | Time |
|---|
| :flask: | Configuring Pentaho Data Integration | ~10m |
| :flask: | Kettle Variables | ~10m |
| :flask: | Hello World | ~15m |
| :flask: | Logging | ~10m |
| :flask: | Error Handling | ~15m |
{% endstep %}
{% step %}
### :database: Data Sources
`2 readings ยท 13 workshops ยท ~4h 50m`
Read and write the formats and systems you'll meet daily โ text, JSON, XML, Excel; relational databases (CRUID + slowly changing dimensions); and the big-data layer: Hadoop, Jupyter, SMB, Snowflake, MinIO.
See the full breakdown
| Lesson | Time |
|---|
| :file-lines: | Review CRUID | ~10m |
| :file-lines: | Review SCDs | ~20m |
| | |
| :flask: | Text File Input | ~20m |
| :flask: | Text File Output | ~25m |
| :flask: | Read JSON | ~15m |
| :flask: | Read XML | ~20m |
| :flask: | Excel Writer | ~30m |
| :flask: | Connecting to Database | ~15m |
| :flask: | Reading from a Database | ~30m |
| :flask: | Writing to a Database | ~20m |
| :flask: | Table Update | ~10m |
| :flask: | Table Insert - Update | ~10m |
| :flask: | Deleting Records | ~15m |
| :flask: | Slowly Changing Dimensions | ~20m |
| :flask: | MinIO | ~30m |
{% endstep %}
{% step %}
### :wand-magic-sparkles: Enriching the Dataset
`9 workshops ยท ~2h 35m`
Shape data inside the pipeline: joins (database, merge, XML, row), lookups, merge-streams patterns, and scripting (Formula, Modified JavaScript Value) when SQL alone isn't enough.
See the full breakdown
| Lesson | Time |
|---|
| :flask: | Merge Streams | ~25m |
| :flask: | Merge Rows (diff) | ~20m |
| :flask: | Database Join | ~10m |
| :flask: | Join Rows | ~25m |
| :flask: | Merge Join | ~10m |
| :flask: | XML Join | ~30m |
| :flask: | Database Lookups | ~15m |
| :flask: | Formula | ~10m |
| :flask: | Modified JavaScript Value | ~10m |
{% endstep %}
{% step %}
### :building: Enterprise Solution
`6 workshops ยท ~1h 30m`
Move from individual transformations to production: jobs, parameters and variables, and scalability (master/slave, partitioning). Get your pipelines ready for scheduling, monitoring, and shipping.
See the full breakdown
| Lesson | Time |
|---|
| :flask: | Hello World (Job) | ~10m |
| :flask: | Order of Execution | ~15m |
| :flask: | Onboarding to a Table | ~25m |
| :flask: | Parameters | ~10m |
| :flask: | Parameters SQL | ~10m |
| :flask: | Run Configurations | ~20m |
{% endstep %}
{% step %}
### :trophy: Capstone Project
`1 workshop ยท ~2h`
Bring it all together: build a parameterised job that ingests CSV, JSON, and master data, validates and enriches the rows, tracks historical changes with an SCD, and loads a dimensional model โ a complete, interview-ready pipeline.
See the full breakdown
| Lesson | Time |
|---|
| :flask: | Daily Sales Pipeline | ~2h |
{% endstep %}
{% endstepper %}
***
Exam & accreditation
{% columns %}
{% column %}
{% hint style="info" icon="clipboard-check" %}
**The course exam**
40 questions drawn at random from a pool of 50, covering the five taught modules โ Getting Started, Components & Concepts, Data Sources, Enriching the Dataset and Enterprise Solution. The pass mark is 80%. It's open-book: browse any module or ask the AI assistant while you work. The Capstone Project is assessed by its own lab checks rather than by the exam.
{% endhint %}
{% endcolumn %}
{% column %}
{% hint style="warning" icon="file-certificate" %}
**Course accreditation**
Finish the lessons and workshops, pass the exam and complete the capstone's checks, and you'll automatically be able to download your course accreditation โ Pentaho Data Integration Developer, Practitioner Level. It states your name, the course and the Pentaho version you studied against, and is valid for two years. The exam is self-paced and not proctored.
This course is the first step on the Certified Solution Architect learning path: DI Developer โ Practitioner, then Installation โ Certified, the Containers, High Availability and Enterprise Security Speciality courses, leading to the Certified Solution Architect credential.
{% endhint %}
{% endcolumn %}
{% endcolumns %}
***
Back to the Solution Architect path
---
# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.
## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.
Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:
```
GET https://academy.pentaho.com/solution-architect-path-sandbox/solution-architect/di-developer-practitioner.md?ask=&goal=
```
`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.
Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.