# Data Integration - [Getting Started](https://academy.pentaho.com/pentaho-data-integration/data-integration/getting-started.md): Get up and running .. - [Components](https://academy.pentaho.com/pentaho-data-integration/data-integration/components.md): Components, User Interface, Configuration options .. - [Configuring PDI UI](https://academy.pentaho.com/pentaho-data-integration/data-integration/components/configuring-pdi-ui.md): PDI UI configuration settings .. - [KETTLE Variables](https://academy.pentaho.com/pentaho-data-integration/data-integration/components/kettle-variables.md): The kettle.properties file contains global variables for KETTLE. - [Concepts & Terminology](https://academy.pentaho.com/pentaho-data-integration/data-integration/concepts-and-terminology.md): Understanding the key concepts & lingo .. - [Hello World](https://academy.pentaho.com/pentaho-data-integration/data-integration/concepts-and-terminology/hello-world.md): Simple transformation to illustrate key concepts .. - [Logging](https://academy.pentaho.com/pentaho-data-integration/data-integration/concepts-and-terminology/logging.md): Set the transformation logging level .. - [Error Handling](https://academy.pentaho.com/pentaho-data-integration/data-integration/concepts-and-terminology/error-handling.md): Handling errors in a transformation .. - [Projects](https://academy.pentaho.com/pentaho-data-integration/data-integration/projects.md): Configure a group of assets into a Project .. - [Project - Sales DWH](https://academy.pentaho.com/pentaho-data-integration/data-integration/projects/project-sales-dwh.md): PDI Lifecycle Management .. - [Data Sources](https://academy.pentaho.com/pentaho-data-integration/data-integration/data-sources.md): Flat files, databases, storage, big data, and notebooks. - [Flat Files](https://academy.pentaho.com/pentaho-data-integration/data-integration/data-sources/flat-files.md): How Data Integration handles Flat files .. - [Text](https://academy.pentaho.com/pentaho-data-integration/data-integration/data-sources/flat-files/text.md): Ingesting Text Files .. - [Text File Input](https://academy.pentaho.com/pentaho-data-integration/data-integration/data-sources/flat-files/text/text-file-input.md): Ingest semi-structured text files into clean rows. - [Text File Output](https://academy.pentaho.com/pentaho-data-integration/data-integration/data-sources/flat-files/text/text-file-output.md): Output text files .. - [Excel](https://academy.pentaho.com/pentaho-data-integration/data-integration/data-sources/flat-files/excel.md): Time for some smoke & mirrors .. - [Excel Writer](https://academy.pentaho.com/pentaho-data-integration/data-integration/data-sources/flat-files/excel/excel-writer.md): Working with Excel .. - [XML](https://academy.pentaho.com/pentaho-data-integration/data-integration/data-sources/flat-files/xml.md): Data exchange & storage .. - [Read XML](https://academy.pentaho.com/pentaho-data-integration/data-integration/data-sources/flat-files/xml/read-xml.md): XML data sources .. - [JSON](https://academy.pentaho.com/pentaho-data-integration/data-integration/data-sources/flat-files/json.md): JavaScript Object Notation .. - [Read JSON](https://academy.pentaho.com/pentaho-data-integration/data-integration/data-sources/flat-files/json/read-json.md): Read JSON objects. - [RSS Feed](https://academy.pentaho.com/pentaho-data-integration/data-integration/data-sources/flat-files/rss-feed.md): RSS reader .. - [Databases](https://academy.pentaho.com/pentaho-data-integration/data-integration/data-sources/databases.md): Steel Wheels .. - [CRUID](https://academy.pentaho.com/pentaho-data-integration/data-integration/data-sources/databases/cruid.md): CRUID database operations are a set of five basic functions that allow us to manipulate data in a persistent storage system, such as a relational database .. - [Database Connections](https://academy.pentaho.com/pentaho-data-integration/data-integration/data-sources/databases/cruid/database-connections.md): Database connections .. - [Create DB table](https://academy.pentaho.com/pentaho-data-integration/data-integration/data-sources/databases/cruid/create-db-table.md): Create tables .. - [Read DB table](https://academy.pentaho.com/pentaho-data-integration/data-integration/data-sources/databases/cruid/read-db-table.md): Read shipped orders from a database table using Table Input. - [Update DB table](https://academy.pentaho.com/pentaho-data-integration/data-integration/data-sources/databases/cruid/update-db-table.md): Update employees in the EMPLOYEES table using the Update step. - [Insert / Update DB](https://academy.pentaho.com/pentaho-data-integration/data-integration/data-sources/databases/cruid/insert-update-db.md): Insert new employees and update existing employees using the Insert/Update step. - [Delete DB table](https://academy.pentaho.com/pentaho-data-integration/data-integration/data-sources/databases/cruid/delete-db-table.md): Delete rows from STG\_SALES\_DATA using transformation-driven criteria. - [Data Cleansing](https://academy.pentaho.com/pentaho-data-integration/data-integration/data-sources/databases/cruid/data-cleansing.md): Traditional data cleansing techiques .. - [SCDs](https://academy.pentaho.com/pentaho-data-integration/data-integration/data-sources/databases/scds.md): Slowly Changing Dimensions .. - [SCDs](https://academy.pentaho.com/pentaho-data-integration/data-integration/data-sources/databases/scds/scds.md): Slowly Changing Dimensions .. - [Storage](https://academy.pentaho.com/pentaho-data-integration/data-integration/data-sources/storage.md) - [MinIO](https://academy.pentaho.com/pentaho-data-integration/data-integration/data-sources/storage/minio.md): Access S3 type Object Store - VFS (Virtual File System) .. - [MinIO](https://academy.pentaho.com/pentaho-data-integration/data-integration/data-sources/storage/minio/minio.md): Hands-on workshops using MinIO as an S3-compatible object store. - [SMB](https://academy.pentaho.com/pentaho-data-integration/data-integration/data-sources/storage/smb.md): File sharing .. - [SMB](https://academy.pentaho.com/pentaho-data-integration/data-integration/data-sources/storage/smb/smb.md): File sharing .. - [Hitachi Content Platform](https://academy.pentaho.com/pentaho-data-integration/data-integration/data-sources/storage/hitachi-content-platform.md): S3 Compatible Object Storage .. - [Big Data](https://academy.pentaho.com/pentaho-data-integration/data-integration/data-sources/big-data.md) - [Snowflake](https://academy.pentaho.com/pentaho-data-integration/data-integration/data-sources/big-data/snowflake.md) - [Apache Hadoop](https://academy.pentaho.com/pentaho-data-integration/data-integration/data-sources/big-data/apache-hadoop.md): Big Data stuff .. - [Apache Hadoop](https://academy.pentaho.com/pentaho-data-integration/data-integration/data-sources/big-data/apache-hadoop/apache-hadoop.md) - [Jupyter Notebook](https://academy.pentaho.com/pentaho-data-integration/data-integration/data-sources/jupyter-notebook.md): Jupyter notebooks are used for data science tasks such as exploratory data analysis (EDA), data cleaning and transformation, data visualization, statistical modeling, machine learning, and so on .. - [PDI to Jupyter Notebook](https://academy.pentaho.com/pentaho-data-integration/data-integration/data-sources/jupyter-notebook/pdi-to-jupyter-notebook.md) - [Enrich Data](https://academy.pentaho.com/pentaho-data-integration/data-integration/enrich-data.md): Enhance the quality of the data .. - [Merge](https://academy.pentaho.com/pentaho-data-integration/data-integration/enrich-data/merge.md): When you merge rows and streams check the number of fields, data types and order. - [Merge Streams](https://academy.pentaho.com/pentaho-data-integration/data-integration/enrich-data/merge/merge-streams.md) - [Merge Rows (diff)](https://academy.pentaho.com/pentaho-data-integration/data-integration/enrich-data/merge/merge-rows-diff.md): Compare merging records .. - [Joins](https://academy.pentaho.com/pentaho-data-integration/data-integration/enrich-data/joins.md): Pentaho Joins .. - [Cross Join](https://academy.pentaho.com/pentaho-data-integration/data-integration/enrich-data/joins/cross-join.md): Good old Cartesian Join .. - [Merge Join](https://academy.pentaho.com/pentaho-data-integration/data-integration/enrich-data/joins/merge-join.md): Standard SQL joins .. - [Database Join](https://academy.pentaho.com/pentaho-data-integration/data-integration/enrich-data/joins/database-join.md): A self join or recursive join .. or is it ? - [XML Join](https://academy.pentaho.com/pentaho-data-integration/data-integration/enrich-data/joins/xml-join.md): Join XML streams .. - [Lookups](https://academy.pentaho.com/pentaho-data-integration/data-integration/enrich-data/lookups.md) - [Database Lookups](https://academy.pentaho.com/pentaho-data-integration/data-integration/enrich-data/lookups/database-lookups.md) - [Scripting](https://academy.pentaho.com/pentaho-data-integration/data-integration/enrich-data/scripting.md) - [Formula](https://academy.pentaho.com/pentaho-data-integration/data-integration/enrich-data/scripting/formula.md) - [Modified JavaScript Value](https://academy.pentaho.com/pentaho-data-integration/data-integration/enrich-data/scripting/modified-javascript-value.md) - [User Defined Java Class](https://academy.pentaho.com/pentaho-data-integration/data-integration/enrich-data/scripting/user-defined-java-class.md) - [Executors](https://academy.pentaho.com/pentaho-data-integration/data-integration/enrich-data/scripting/executors.md) - [Enterprise Solution](https://academy.pentaho.com/pentaho-data-integration/data-integration/enterprise-solution.md): Scale out your Solution .. - [Jobs](https://academy.pentaho.com/pentaho-data-integration/data-integration/enterprise-solution/jobs.md): A process flow that consists of one or more steps that execute tasks such as transformations, scripts, email notifications, file transfers, etc - [Job - Hello World](https://academy.pentaho.com/pentaho-data-integration/data-integration/enterprise-solution/jobs/job-hello-world.md): What's a Pentaho Job .. - [Backward Chaining](https://academy.pentaho.com/pentaho-data-integration/data-integration/enterprise-solution/jobs/backward-chaining.md): Rules when executing in multiple threads .. - [Parallel](https://academy.pentaho.com/pentaho-data-integration/data-integration/enterprise-solution/jobs/parallel.md): Rule when executing in parallel .. - [Parameters & Variables](https://academy.pentaho.com/pentaho-data-integration/data-integration/enterprise-solution/parameters-and-variables.md) - [Parameters](https://academy.pentaho.com/pentaho-data-integration/data-integration/enterprise-solution/parameters-and-variables/parameters.md): Passing parameters .. - [Variables](https://academy.pentaho.com/pentaho-data-integration/data-integration/enterprise-solution/parameters-and-variables/variables.md): Add flexibility / portability with variables .. - [Scalability](https://academy.pentaho.com/pentaho-data-integration/data-integration/enterprise-solution/scalability.md): Static & Dynamic Carte Clusters - [Run Configurations](https://academy.pentaho.com/pentaho-data-integration/data-integration/enterprise-solution/scalability/run-configurations.md): Execute Jobs / Transformations on specific nodes or in a Pentaho Cluster .. - [Partition](https://academy.pentaho.com/pentaho-data-integration/data-integration/enterprise-solution/scalability/partition.md) - [Monitoring & Scheduling](https://academy.pentaho.com/pentaho-data-integration/data-integration/enterprise-solution/monitoring-and-scheduling.md): Monitoring and Scheduling your Jobs & Transformations .. - [Monitoring & Scheduling](https://academy.pentaho.com/pentaho-data-integration/data-integration/enterprise-solution/monitoring-and-scheduling/monitoring-and-scheduling.md): Schedule Pentaho Jobs / Transformations and Monitor results .. - [Logging](https://academy.pentaho.com/pentaho-data-integration/data-integration/enterprise-solution/logging.md) - [Logging](https://academy.pentaho.com/pentaho-data-integration/data-integration/enterprise-solution/logging/logging.md): Implement database logging .. - [Dockmaker](https://academy.pentaho.com/pentaho-data-integration/data-integration/enterprise-solution/dockmaker.md): Pentaho cloud, hybrid-cloud containerized deployments .. - [BA & DI Servers](https://academy.pentaho.com/pentaho-data-integration/data-integration/enterprise-solution/dockmaker/ba-and-di-servers.md): Build & Deploy BA & DI Pentaho Server Containers .. - [Metadata Injection](https://academy.pentaho.com/pentaho-data-integration/data-integration/enterprise-solution/metadata-injection.md): Metadata Injection .. - [MDI](https://academy.pentaho.com/pentaho-data-integration/data-integration/enterprise-solution/metadata-injection/mdi.md): Overview of Metadata Injection .. - [Kettle Plugins](https://academy.pentaho.com/pentaho-data-integration/data-integration/ee-plugins.md): Enterprise Plugins .. - [Hierarchical Data Type](https://academy.pentaho.com/pentaho-data-integration/data-integration/ee-plugins/hierarchical-data-type.md): handling Hierarchical Data Types - JSON & YAML .. - [Kafka](https://academy.pentaho.com/pentaho-data-integration/data-integration/ee-plugins/kafka.md): Streaming platform .. --- # Agent Instructions: Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://academy.pentaho.com/pentaho-data-integration/data-integration.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.