Glossaries

In the beginning ..

The Fundamental Challenge

Imagine walking into a massive library where every book is written in a different language, with no translations, no common indexing system, and where the same concept has 50 different names depending on who wrote about it. That's essentially what modern enterprise data environments look like without a business glossary.

Pentaho Data Catalog (PDC) is designed to solve the enterprise data discovery and governance challenge, but without glossaries, it's like having a powerful search engine that can only search for technical codes rather than business meanings.

Here's the problem:

x

Translation Layer

Translation Layer

PDC's powerful search and discovery features are only useful if users can find what they're looking for. Glossaries provide the business vocabulary that makes technical assets discoverable by non-technical users.

Without Glossary:

• User searches for "client type" → No results

• User searches for "customer classification" → No results

• User gives up, emails IT, waits 3 days

• IT explains it's called CUST_TYPE_CD

• User forgets by next month, cycle repeats

With Glossary:

• All variations point to single business term: "Customer Type"

• Definition: "Classification of customer as Individual (I) or Store (S)"

• User finds it immediately, understands it, uses it correctly

Semantic Understanding

The Problem:

Consider the term "Status" in AdventureWorks:

• Order Status (1=Processing, 2=Shipped, 3=Delivered)

• Employee Status (A=Active, T=Terminated, L=Leave)

• Payment Status (P=Pending, C=Completed, F=Failed)

• Inventory Status (1=Available, 2=Reserved, 3=Backordered)

Without context, "Status = 1" is meaningless.

How Glossaries Provide Context:

{
  "name": "Order Status",
  "definition": "Current state of customer order in fulfillment process",
  "valid_values": {
    "1": "In Process - Order received, not yet shipped",
    "2": "Approved - Payment verified, ready to ship",
    "3": "Backordered - Awaiting inventory",
    "4": "Rejected - Payment failed or cancelled",
    "5": "Shipped - Left warehouse",
    "6": "Cancelled - Customer cancelled"
  },
  "business_rules": "Orders move from 1→2→5 normally. Status 3 triggers reorder."
}

PDC can now:

• Show users what values mean, not just what they are

• Enable accurate filtering (find all "Shipped" orders, not "Status=5")

• Prevent misinterpretation in reports

• Apply appropriate data quality rules per status type

Lineage & Impact Analysis

The Scenario:

The finance team wants to change how "Revenue" is calculated.

Without Glossary:

• Which of these 47 fields containing "revenue," "sales," or "amount" are affected?

• What reports will break?

• Which dashboards need updating?

• Who needs to be notified?

With Glossary:

The glossary term "Total Revenue" shows:

• Calculation: SubTotal + Tax + Freight

• Source fields: Sales.SalesOrderHeader.SubTotal, TaxAmt, Freight

• Used in: 23 reports, 8 dashboards, 3 ML models

• Stakeholders: Finance, Sales, Operations teams

• Downstream impacts: Commission calculations, quarterly forecasts

PDC uses glossary relationships to:

• Track data lineage from source to consumption

• Perform impact analysis before changes

• Identify all stakeholders affected by modifications

• Ensure changes don't break critical processes

Compliance & Governance

The Challenge:

GDPR auditor asks: "Show me all personal data you collect about EU citizens and prove it's protected."

Without Glossary:

• Manual database scan (500+ tables)

• Guessing which fields contain PII

• No documentation of protection measures

• Weeks of preparation for audit

• High risk of missing critical data

With Compliance Glossary:

SELECT * FROM glossary_terms 
WHERE tags LIKE '%pii%' 
AND tags LIKE '%gdpr%'

Results instantly show:

• All 47 PII fields

• Protection status (encrypted, masked, etc.)

• Retention periods

• Legal basis for processing

• Access controls in place

PDC becomes your compliance command center:

• Automatically identifies sensitive data

• Enforces governance policies

• Generates audit reports

• Tracks consent and retention

• Proves regulatory compliance

Data Quality

The Data Quality Challenge:

How do you know if "Customer Type = X" is an error?

Glossary Enables Quality Rules:

{
  "name": "Customer Type",
  "valid_values": ["I", "S"],
  "quality_rules": {
    "completeness": "NOT NULL",
    "validity": "IN ('I','S')",
    "consistency": "If Store_Name EXISTS, then Customer_Type = 'S'"
  }
}

PDC uses glossary definitions to:

• Automatically generate data quality rules

• Identify invalid values

• Flag inconsistencies

• Track quality metrics over time

• Prevent bad data from entering systems