LLM Integration

Everything hinges on making a clean POST request to Ollama's /api/generate endpoint at http://localhost:11434. This is a local REST API, so there's no external dependency, no API key, and no network latency.

Parameters:

model — Which Ollama model to invoke. The workshops use llama3.2:3b as the default: compact enough to run on CPU-only hardware but capable enough for structured tasks. You can scale up to llama3.1:8b for harder problems at the cost of speed.

stream: false — This is critical for PDI. Streaming sends tokens incrementally, which is great for chat UIs but useless in an ETL row-by-row context. Setting this to false tells Ollama to generate the entire response, then return it as one complete JSON object — which is what PDI needs to write into a field.

format: "json" — Forces the model's output to be valid JSON. Without this, the model might add conversational preamble, markdown fences, or explanations around its JSON answer, all of which break downstream parsing.

keep_alive — This is the single biggest performance optimization for batch processing. By default Ollama unloads the model from memory after each request. Reloading takes 10–30 seconds depending on model size and disk speed. Setting keep_alive: "30m" keeps the model resident in RAM across all rows in a batch run, turning a multi-hour job into a fraction of that time.

temperature: 0.1 — Controls randomness. Values near 0 produce near-deterministic output, meaning the same review will produce the same sentiment classification on repeated runs. This is essential for reproducible ETL — you don't want results changing every time the pipeline runs.

num_predict: 300 — Caps the maximum tokens generated. This prevents runaway generation, protects processing time, and avoids unexpectedly large response payloads overwhelming downstream steps.

When PDI's REST Client step receives the Ollama response, it comes back as a single JSON object with two categories of fields: the content fields that contain the actual result, and the performance/telemetry fields that tell you how the inference went.

The response field is a string containing JSON, not a nested object - which is why the pipeline requires two separate JSON Input steps: one to extract $.response from the Ollama wrapper, and a second to parse the actual model output into structured fields.

Every element of this prompt is doing specific work.

The instruction is a verb, not a question.** "Analyze sentiment" is a direct command. Phrasing it as "Can you tell me the sentiment of this review?" wastes tokens and invites a conversational response rather than structured output.

The data is clearly delimited.** Wrapping the review text in quotes separates it visually and semantically from the instruction. Without this, the model can conflate the instruction with the data, especially for reviews that contain imperative language.

The schema is shown, not described.** Rather than writing "return a JSON object with a field called sentiment that can be positive, negative, or neutral", the prompt shows the exact JSON structure. The model pattern-matches against the example and fills in the values, which is far more reliable than parsing a verbal description of what you want.

Value constraints are embedded inline.** Specifying -1.0 to 1.0 for score and 0-100 for confidence directly in the schema means you don't need a separate instruction section. The model sees the range at the exact point where it needs to apply it.

The Data Quality workshop makes this concrete. Here's the verbose version of a data cleaning prompt:

The verbose version uses roughly 120 tokens. The optimized version uses around 60. On a batch of 1,000 records running on CPU-only hardware at 23 seconds per record, that difference compounds into real time savings. The model doesn't need to be spoken to politely - it needs to be spoken to precisely.

This is the most common failure mode in the workshops and it produces a completely silent error. When you write:

PDI does not substitute the variable. The string ${MODEL_NAME} is sent literally to Ollama, which either returns a 400 Bad Request or — worse — tries to find a model named ${MODEL_NAME} and fails silently. The correct approach is always getVariable():

Even when you set "format": "json" in the Ollama request, some models — particularly on certain prompts — will wrap their JSON in markdown code fences or add a sentence before it:

If your JSON Input step tries to parse the full response field directly, it will fail because the surrounding text makes it invalid JSON. The defensive fix used in the workshops is to scan for the first { and last } in the response string and extract only that substring:

This makes your parser robust to model verbosity regardless of which model or version you're running.

If the model's response hits the num_predict token ceiling mid-generation, Ollama stops and returns whatever was generated so far. The done_reason field will say "length" instead of "stop", but if you're not logging that field, you'll never know.

The symptom is malformed JSON arriving at your parser - the object opens but never closes - which your error handler catches and routes to the failure path. The fix is to monitor eval_count in your output and raise num_predict if it consistently hits the ceiling.

It's tempting to extract everything you might ever want in a single prompt. The NER workshop covers 10 entity types (PERSON, ORGANIZATION, LOCATION, DATE, PRODUCT, MONEY, CONTACT, ID, TECHNOLOGY, POSITION) simultaneously.

This works for NER because entity classification is a well-defined task. But for analytical prompts - asking for sentiment, tone, intent, key phrases, competitive mentions, urgency, and a summary all at once - models tend to fill in fields speculatively rather than accurately. If accuracy matters more than throughput, it's worth splitting complex prompts into focused single-purpose calls, even at the cost of additional API round trips.

When review text or customer data contains quotes, backslashes, or newlines, they can break your JavaScript string concatenation and produce a malformed JSON payload before it even reaches Ollama. A review like "Best product I've ever bought — 5 stars!" is fine, but one containing "He said \"amazing\" and I agree" will break the string if not handled. The fix is to sanitize input text before embedding it in the prompt:

This is especially important for free-text fields pulled from customer-facing systems where you have no control over what users type.