> For the complete documentation index, see [llms.txt](https://academy.pentaho.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://academy.pentaho.com/pentaho-data-integration/data-integration/data-sources.md).

# Data Sources

### Choose a data source

Use this page to orient yourself. Then jump into the specific connector docs:

{% hint style="info" %}

#### What “data source” means in PDI

In practice, a data source is either:

* A file format you parse (CSV, Excel, JSON, XML).
* A service you connect to (a DB, object store, cluster, or API).
  {% endhint %}

{% tabs %}
{% tab title="Flat Files" %}
{% hint style="info" %}

#### Flat files

Use flat files when your data arrives as CSV, TXT, fixed-width, JSON, or XML.

Start here: [Flat Files](/pentaho-data-integration/data-integration/data-sources/flat-files.md).
{% endhint %}

<figure><img src="/files/cbXkAVbVoLXxpwTDHqV6" alt=""><figcaption></figcaption></figure>

{% tabs %}
{% tab title="Structured" %}
{% hint style="info" %}

#### Structured

Structured data uses a predefined model. It is easy to validate and query.

Think tables, rows, and columns. Examples include SQL databases and well-formed CSV files.
{% endhint %}
{% endtab %}

{% tab title="Unstructured" %}
{% hint style="info" %}

#### Unstructured

Unstructured data has no consistent schema. Examples include PDFs, images, video, and free-form text.

You typically need parsing, extraction, or ML to use it.
{% endhint %}
{% endtab %}

{% tab title="Semi-structured" %}
{% hint style="info" %}

#### Semi-structured

Semi-structured data has a loose schema. It uses tags or keys to describe fields and hierarchy.

Common formats are JSON and XML.
{% endhint %}
{% endtab %}

{% tab title="Metadata" %}
{% hint style="info" %}

#### Metadata

Metadata is “data about data”. Examples include headers, schemas, and data dictionaries.
{% endhint %}
{% endtab %}
{% endtabs %}
{% endtab %}

{% tab title="Databases" %}
{% hint style="info" %}

#### Databases

Pentaho connects to databases primarily through JDBC drivers. These drivers are the main interface for database communication.

Start here: [Databases](/pentaho-data-integration/data-integration/data-sources/databases.md).
{% endhint %}

<figure><img src="/files/IL7BZk89nZKZBXrAxInh" alt=""><figcaption><p>Database Connection</p></figcaption></figure>
{% endtab %}

{% tab title="Storage" %}
{% hint style="info" %}

#### Storage

Storage sources are cloud or network repositories. Examples include Amazon S3, Azure Blob Storage, and Google Cloud Storage.

In PDI, you typically connect through VFS. You can read and write across hybrid environments.

Start here: [Storage](/pentaho-data-integration/data-integration/data-sources/storage.md).
{% endhint %}

<figure><img src="/files/ygCVIRdwnpQ4roGXtxdP" alt=""><figcaption></figcaption></figure>
{% endtab %}

{% tab title="Big Data" %}
{% hint style="info" %}

#### Big Data

Big data sources require distributed compute. Common examples are Hadoop (HDFS, Hive, HBase), Spark, NoSQL, and Kafka.

PDI provides specialized steps and adapters for these platforms. This lets you transform data where it lives.

Start here: [Big Data](/pentaho-data-integration/data-integration/data-sources/big-data.md).
{% endhint %}

<figure><img src="/files/2uxLQWlognfwKxCe1BtQ" alt=""><figcaption><p>Types of Big Data</p></figcaption></figure>
{% endtab %}

{% tab title="Jupyter Notebook" %}
{% hint style="info" %}

#### Jupyter Notebook

Jupyter is a web-based notebook for code, visuals, and narrative text. It works well for exploratory analysis and prototyping.

In a PDI workflow, notebooks often handle advanced analysis. PDI handles production orchestration and scheduled pipelines.

Start here: [Jupyter Notebook](/pentaho-data-integration/data-integration/data-sources/jupyter-notebook.md).
{% endhint %}

<figure><img src="/files/WwNybMrPO7WNmGuxpry9" alt=""><figcaption><p>Jupyter Notebook</p></figcaption></figure>
{% endtab %}
{% endtabs %}


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://academy.pentaho.com/pentaho-data-integration/data-integration/data-sources.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.