AutoML

Use PDI + H2O AutoML in Colab to prototype a credit-card fraud model.

Imagine that a direct retailer wants to reduce losses due to orders involving fraudulent use of credit cards. They accept orders via phone and their website, and ship goods directly to the customer.

Basic customer details, such as customer name, date of birth, billing address and preferred shipping address, are stored in a relational database.

Orders, as they come in, are stored in a database. There is also a report of historical instances of fraud contained in a CSV spreadsheet.

Before you start ..

You need Colab access and a working PDI environment. Do the setup first if you have not done it yet.

Complete Prerequisite tasks.
In PDI, you will run autoML.ktr.
- Location: ~/Workshop--Data-Integration/Labs/Module 7 - Machine Learning/AutoML
From that transformation, you will create data/H2O.csv.
- Upload that file to Colab when prompted.

In this workshop, you will:

Prepare data (wrangling).
Create features (feature engineering).
Use H2O AutoML to shortlist candidate models.
Train and evaluate a model in Colab.
Save the best model artifact.

2MB

3MB

270KB

13MB

transaction_details.csv

Open

868KB

transaction_fraud_report.csv

Open

60KB

autoML.ktr

Open

Run through the following steps to determine the best ML model for the dataset:

Data preparation

Use PDI to join customer, transaction, and historical fraud data. Create a single training dataset for AutoML.

Start PDI

cd
cd ~/Pentaho/design-tools/data-integration
./spoon.sh

Open the transformation: autoML.ktr

~/Workshop--Data-Integration/Labs/Module 7 - Machine Learning/AutoML

Browse the various customer data sources:

Customer Data

Where you will find the customer_billing_zip codes, which will be used in feature engineering:

H2O

So, what does the data scientist do at this point?

Typically, they will want to get a feel for the data by examining simple summary statistics and visualizations, followed by applying quick techniques for assessing the relationship between individual attributes (fields) and the target of interest which, in this example, is the reported_as_fraud_historic field.

Following that, if there are attributes that look promising, quick tests with common supervised classification algorithms will be next on the list. This comprises the initial stages of experimental data mining – that is, the process of determining which predictive techniques are going to give the best result for a given problem.

Create the H2O dataset

Run the transformation: autoML.ktr
Preview the results: data/H2O.csv

This will be the dataset used for autoML in Colab.

H2O AutoML trains and ranks multiple models for you. It also builds ensembles and returns a leaderboard, so you can pick a strong baseline quickly.

Open Colab

Connect a runtime

Connect to a hosted runtime.

Upload the notebook

In Colab, select File > Upload notebook.

Upload:

~/Workshop--Data-Integration/Labs/Module 7 - Use Cases/Machine Learning/AutoML/data/credit_card_fraud.ipynb

Upload the dataset

When the notebook prompts for a file upload, upload H2O.csv. Create it by running autoML.ktr in PDI.

AutoML Script

These are the code sections for the Jupyter file: credit_card_fraud.ipynb:

Install the h2o libraries:

# Install H2O
!pip install h2o -q
print("Installation complete.")

Installs H2O.ai's machine learning platform
-q flag = quiet mode (minimal output)

Import libraries:

# Import libraries
import h2o
from h2o.automl import H2OAutoML
import pandas as pd
import numpy as np
from datetime import datetime
import matplotlib.pyplot as plt
import seaborn as sns

# Initialize H2O
h2o.init(max_mem_size='4G', nthreads=-1)
print("\nH2O initialized.")

max_mem_size='4G': Allocates 4GB RAM to H2O cluster
nthreads=-1: Uses all available CPU cores

Upload data:

# Upload data
from google.colab import files
uploaded = files.upload()

Purpose: Interactive file upload widget in Google Colab

Expected file: H2O.csv (semicolon-separated)

Load data:

# Load data
data = pd.read_csv('H2O.csv', sep=';', engine='python', header=None)

data.columns = [
    'first_time_customer', 'order_dollar_amount', 'num_items', 'age',
    'web_order', 'total_transactions', 'hour_of_day',
    'billing_shipping_match', 'fraud'
]

# Convert target to string
data['fraud'] = data['fraud'].astype(str)

print(f"Dataset: {data.shape[0]:,} rows × {data.shape[1]} columns")
print(f"Fraud cases: {(data['fraud']=='1').sum():,} ({(data['fraud']=='1').mean()*100:.2f}%)")

Critical steps:

Custom separator: Uses ; instead of default ,
Manual column naming: Original file has no headers
Target conversion: Converts fraud to string for H2O classification

Feature breakdown:

first_time_customer: Binary indicator (risky for fraud)
order_dollar_amount: Transaction value
num_items: Cart size
age: Customer age
web_order: Online vs in-store
total_transactions: Customer history
hour_of_day: Timing patterns (fraud often occurs at odd hours)
billing_shipping_match: Address mismatch (major fraud signal)
fraud: Target variable (0=legitimate, 1=fraud)

Your upstream transformation may use a different target field name (for example, reported_as_fraud_historic). In this notebook, the target column is named fraud.

Distribution check:

# Quick class distribution check
fraud_dist = data['fraud'].value_counts()

plt.figure(figsize=(10, 4))
fraud_dist.plot(kind='bar', color=['green', 'red'])
plt.title('Class Distribution', fontweight='bold')
plt.xlabel('Class (0=Normal, 1=Fraud)')
plt.ylabel('Count')
plt.xticks([0, 1], ['Normal', 'Fraud'], rotation=0)
for i, v in enumerate(fraud_dist):
    plt.text(i, v + 50, f'{v:,}', ha='center', fontweight='bold')
plt.tight_layout()
plt.show()

Purpose: Reveals class imbalance

Why this matters:

Fraud datasets are typically highly imbalanced (99% legitimate, 1% fraud)
This visualization confirms the need for balance_classes=True later
Shows if we have enough fraud cases to train effectively

Convert to H2O frame:

# Convert to H2O frame
hf = h2o.H2OFrame(data)
hf['fraud'] = hf['fraud'].asfactor()

# Split: 75% train, 25% test
train, test = hf.split_frame(ratios=[0.75], seed=42)

print(f"Training: {train.nrows:,} rows")
print(f"Test:     {test.nrows:,} rows")

# Define features and target
x = train.columns
x.remove('fraud')
y = 'fraud'

Key operations:

H2OFrame conversion: Moves data into H2O's distributed framework
Factor conversion: Tells H2O this is classification (not regression)
75/25 split: Standard training/testing division
Seed=42: Ensures reproducible results - controls randomness
Feature/target separation: Prepares for model training

Why this split?:

75% gives enough data to learn patterns
25% provides robust performance estimates
Random splitting prevents temporal bias

Run Models (Top 5)

# Fast configuration
start_time = datetime.now()

print("\nFAST H2O AutoML")
print("="*70)
print("Configuration:")
print("  Max models: 5 (for speed)")
print("  Max runtime: 15 minutes")
print("  Algorithms: GBM, XGBoost, GLM, DRF, DeepLearning")
print("  Balance classes: True")
print("\nTraining...\n")

aml = H2OAutoML(
    max_models=5,                   # Only 5 models for speed
    max_runtime_secs=900,           # 15 minutes max
    seed=42,
    balance_classes=True,           # Handle imbalanced data
    nfolds=5,                       # 5-fold CV
    sort_metric='AUC',

    # Best algorithms for fraud detection
    include_algos=[
        'GBM',           # Gradient Boosting (usually best)
        'XGBoost',       # Extreme Gradient Boosting
        'GLM',           # Generalized Linear Model (fast)
        'DRF',           # Distributed Random Forest (tree-based)
        'DeepLearning'   # Neural Network
    ],

    verbosity='info'
)

# Train
aml.train(x=x, y=y, training_frame=train)

duration = (datetime.now() - start_time).total_seconds()
print(f"\nTraining completed in {duration/60:.1f} minutes.")
print("="*70)

Configuration explained:

max_models=5: keeps runtime short.
max_runtime_secs=900: hard stop at 15 minutes.
balance_classes=True: important for imbalanced fraud data.
nfolds=5: cross-validation for more stable metrics.
sort_metric='AUC': simple default ranking metric.

Algorithm selection rationale:

GBM (Gradient Boosting Machine)
- Usually #1 performer for fraud
- Handles complex patterns
- Native H2O implementation
XGBoost
- Industry standard for tabular data
- Fast training
- Excellent with imbalanced data
GLM (Generalized Linear Model)
- Fast baseline
- Interpretable coefficients
- Good for linear relationships
DRF (Distributed Random Forest)
- Ensemble of decision trees
- Handles non-linear patterns
- Robust to outliers
DeepLearning
- Neural network
- Captures complex interactions
- May find unexpected patterns

Why no standalone Decision Trees? Ensemble methods (GBM, XGBoost, DRF) combine hundreds of trees, always outperforming single trees.

Model Leaderboard:

# View leaderboard
lb = aml.leaderboard
print("\n ALL MODELS RANKED BY PERFORMANCE")
print("="*70)
print(lb)
print("="*70)
print(f"\n Total models trained: {lb.nrows}")
print(f"Best model: {lb.as_data_frame()['model_id'][0]}")

Output includes:

Model IDs
AUC (Area Under ROC Curve)
AUCPR (Area Under Precision-Recall Curve)
Mean per-class error
RMSE/MSE (less relevant for classification)

Best metric for fraud: AUCPR (handles class imbalance better than AUC)

Get Best Model:

# Get best model
best = aml.leader

print(f"\n BEST MODEL: {best.model_id}")
print(f"Algorithm: {best.algo}")

# Performance on test set
perf = best.model_performance(test)

print("\n TEST SET PERFORMANCE")
print("="*70)
print(f"AUC:       {perf.auc():.4f}")
print(f"Accuracy:  {perf.accuracy()[0][1]:.4f}")
print(f"Precision: {perf.precision()[0][1]:.4f}")
print(f"Recall:    {perf.recall()[0][1]:.4f}")
print(f"F1 Score:  {perf.F1()[0][1]:.4f}")
print("="*70)

Metrics explained for fraud detection:

AUC (0.95+ is excellent): Overall discrimination ability
Accuracy: Misleading with imbalanced data (can be high even missing all fraud)
Precision: Of flagged transactions, what % are actually fraud?
Recall: Of all fraud cases, what % did we catch? (Most critical for fraud)
F1 Score: Balance between precision and recall

Real-world tradeoff:

High recall = catch more fraud (but more false alarms)
High precision = fewer false alarms (but miss more fraud)

Confusion Matrix:

# Confusion matrix
cm = perf.confusion_matrix()
print("\n CONFUSION MATRIX")
print("="*70)
print(cm)
print("="*70)

# Extract values
cm_table = cm.table.as_data_frame()
try:
    tn = int(cm_table.iloc[0, 1])
    fp = int(cm_table.iloc[0, 2])
    fn = int(cm_table.iloc[1, 1])
    tp = int(cm_table.iloc[1, 2])

    print("\n BREAKDOWN")
    print(f"True Negatives:  {tn:>6,}")
    print(f"False Positives: {fp:>6,}")
    print(f"False Negatives: {fn:>6,} (missed fraud)")
    print(f"True Positives:  {tp:>6,}")
    print(f"\nCaught {tp} fraud cases, missed {fn}.")
except:
    print("See confusion matrix table above")

Business impact:

FN (False Negatives): Direct financial loss from undetected fraud
FP (False Positives): Customer friction from declined legitimate transactions
Optimal balance: Depends on fraud costs vs customer experience impact

Feature Importance:

# Top features
varimp = best.varimp(use_pandas=True)
print("\n TOP FEATURES")
print("="*70)
print(varimp.head(10))
print("="*70)

# Plot
best.varimp_plot()

Why this matters:

Explainability: Understand what drives fraud predictions
Feature engineering: Focus efforts on most impactful features
Regulatory compliance: Many industries require model interpretability
Data quality: Verify that sensible features are important

Expected top features:

billing_shipping_match (address verification)
order_dollar_amount (unusually large transactions)
total_transactions (customer history)

Save Model:

model_path = h2o.save_model(model=best, path="./", force=True)
files.download(model_path)

What this saves: an H2O model artifact you can load back into H2O.

Deployment options:

Load in production H2O cluster
Export a MOJO for low-latency scoring (separate step)
Export to PMML if your workflow requires it (separate step)

Summary Report:

# Generate summary
report = f"""
FRAUD DETECTION - QUICK SUMMARY
{'='*60}

Date: {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}

Dataset:
  Total:     {len(data):,}
  Training:  {train.nrows:,}
  Test:      {test.nrows:,}

Model:
  Best:      {best.model_id}
  Algorithm: {best.algo}
  Training:  {duration/60:.1f} minutes

Performance:
  AUC:       {perf.auc():.4f}
  Accuracy:  {perf.accuracy()[0][1]:.4f}
  Precision: {perf.precision()[0][1]:.4f}
  Recall:    {perf.recall()[0][1]:.4f}
  F1:        {perf.F1()[0][1]:.4f}

Model saved: {model_path}

{'='*60}
"""

print(report)

# Save report
with open('fraud_summary.txt', 'w') as f:
    f.write(report)

files.download('fraud_summary.txt')
print("Summary downloaded.")

Purpose: quick documentation for:

Model tracking
Stakeholder reporting
Experiment comparison
Regulatory audit trails

After you validate the notebook, you can move the logic into PDI. Use the AutoML results to decide which models to operationalize.

Python Executor

The Python Executor step lets you run Python code as part of a Pentaho Data Integration (PDI) transformation.

This step is designed to help developers and data scientists focus on Python-based analytics and algorithms while using PDI for common ETL work such as connecting to sources, joining, and filtering.

You can run Python with either:

Row-by-row processing: PDI maps each incoming row to Python variables and runs the script once per row.
All-rows processing: PDI transfers the full dataset at once (for example, into a pandas DataFrame) and runs the script.

This step supports the CPython runtime only.

For further details:

Click the Input tab.
Map PDI fields into Python variables.
Use All rows for DataFrames. It processes the full dataset at once.
Select All rows to process all data at once (for example, when passing a list of dicts).

Option

Description

Available variables

Use the plus sign button to add a Python variable to the input mapping for the script used in the transformation. You can remove the Python variable by clicking the X icon.

Variable name

Enter the name of the Python variable. The list of available variables will update automatically.

Step

Specify the name of the input step to map from. It can be any step in the parent transformation with an outgoing hop connected to the Python Executor step.

Data structure

Specify the data structure from which you want to pull the fields for mapping. You can select one of the following:

· Pandas data frame: the tabular data structure for Python/Pandas.

· NumPy array: the table of values, all the same type, which is indexed by a tuple of positive integers.

· Python List of Dictionaries: each row in the PDI stream becomes a Python dictionary. All the dictionaries are put into a Python list.

In Mapping, configure these fields:

Field Property

Description

Data structure field

The value of the Python data structure field to which you want to map the PDI field.

Data structure type

The value of the data structure type assigned to the data structure field to which you want to map the PDI field.

PDI field

The name of the PDI field which contains the vector data stored in the mapped Python variable.

PDI data type

The value of the data type assigned to the PDI field, such as a date, a number, or a timestamp.

Gradient Boosting Machine

Gradient Boosting Machine (GBM) is an ensemble learning algorithm that builds a series of weak decision trees sequentially, where each new tree focuses on correcting the errors made by the previous ones. The final prediction is a weighted combination of all these trees, resulting in a highly accurate model.

In fraud detection, GBM is particularly effective because it handles the inherent class imbalance (fraud being rare compared to legitimate transactions) and can capture complex, non-linear patterns in transaction data - such as unusual spending amounts, atypical merchant categories, or irregular timing - that simple rule-based systems would miss.

It also provides feature importance rankings, allowing analysts to understand which variables (e.g., transaction velocity, geolocation mismatches, or device fingerprints) are the strongest indicators of fraudulent behavior, making the model both powerful and interpretable for investigation teams.

Gradient Boosting Machine works through an iterative, additive process:

Step 1: Start with a baseline. The algorithm begins with a simple initial prediction, often just the average of the target variable (e.g., "fraud" or "not fraud").

Step 2: Calculate the errors (residuals). It measures how far off that initial prediction is from the actual values. These errors are called residuals.

Step 3: Fit a weak learner to the residuals. A small, shallow decision tree is trained - not on the original target, but on the errors from the previous step. The goal is to learn the pattern in what the model got wrong.

Step 4: Update the prediction. The output of this new tree is added to the existing prediction, scaled down by a learning rate (a small number like 0.1) to prevent overcorrecting. So the updated model becomes:

New Prediction = Old Prediction + (learning rate × New Tree's output)

Step 5: Repeat. Steps 2–4 are repeated for hundreds or thousands of iterations. Each new tree corrects a little more of the remaining error.

The "gradient" in the name comes from the fact that the algorithm uses gradient descent - the same optimization technique used in neural networks - to minimize a loss function (e.g., log-loss for classification). The residuals it fits at each step are actually the negative gradients of that loss function.

Over many rounds, these individually weak trees combine into a very strong predictive model. In fraud detection, this means early trees might learn broad patterns (like "very large transactions are riskier"), while later trees pick up subtle, nuanced signals (like "this card was used in two countries within an hour") that distinguish genuine fraud from normal behavior.

Install GBM

Execute the following script to install GBM.

# Install gbm from R
sudo R -e "install.packages('gbm', repos='https://cran.r-project.org')"

Verify.

R -e "library(gbm); cat('GBM loaded successfully\n')"

Metrics cheat sheet (fraud detection)

Use these metrics to compare models and pick a decision threshold. Fraud data is usually highly imbalanced.

What to look at first

AUCPR: best single-number metric for imbalanced classification.
Recall: how much fraud you catch (minimize false negatives).
Precision: how many alerts are real fraud (minimize false positives).

Use AUC as a sanity check

AUC measures how well the model ranks fraud above legitimate transactions. It can look “good” even when precision is poor at useful thresholds.

Why accuracy is a trap

If fraud is 1% of transactions, a model can be 99% accurate and useless. It can do that by predicting “legitimate” for every row.

Practical selection guide

If fraud losses are expensive, prioritize recall.
If customer friction is expensive, prioritize precision.
Use AUCPR to compare candidates before tuning thresholds.

What to ignore (most of the time)

RMSE/MSE: regression-style errors on probabilities. Not decision-friendly here.

PreviousPrerequisite tasks NextCredit Card

Last updated 4 months ago

Was this helpful?

Good morning

AutoML

Before you start ..

Data preparation

Customer Data

Customer Billing

Customer Transaction

Fraud Details

Feature engineering

H2O

Python Executor

Gradient Boosting Machine

What to look at first

Use AUC as a sanity check

Why accuracy is a trap

Practical selection guide

What to ignore (most of the time)