MongoDB Reports
Create Analyzer & Interactive Reports from MongoDB ..
Steel Wheels Inc has customer order information stored in a MongoDB.
Pentaho Data Integration
To make the Use case more interesting, we're going to create a few pipelines in Pentaho Data Integration, that will load files, migrate tables, etc, to a MongoDB running in Docker.
Pentaho Interactive Reporting
The process starts similarly with establishing the MongoDB connection. In Interactive Reporting, you can create reports by dragging and dropping fields from your MongoDB collections directly onto the report canvas. You'll need to first create a metadata layer that translates MongoDB's document structure into a relational-style format that Pentaho can work with. This involves defining how nested documents and arrays should be represented in your reports.
Pentaho Analyzer
You'll need to set up the MongoDB data connection in the Pentaho Server by configuring the MongoDB connector. Once connected, you'll need to create a Mondrian schema that maps your MongoDB collections and fields to a multidimensional model that Analyzer can understand. The schema defines dimensions, measures, and hierarchies based on your MongoDB data structure.
For both reporting tools, it's important to note that performance optimization is crucial when working with MongoDB. This includes creating appropriate indexes in MongoDB to support your reporting queries, and carefully structuring your aggregation pipelines to ensure efficient data retrieval. You may also want to consider using MongoDB's aggregation framework to pre-aggregate data for complex reports.
x
x
For this option, we will guide you through creating a transformation that loads data from different files in your filesystem, and then load them into a MongoDB Collection. Each of these files contains a key that we can use to join data in PDI before we send it to the MongoDB Output step.
Linux
Ensure the Pentaho Server is up and running.
cd
cd /opt/pentaho/server/pentaho-server/
sudo ./start-pentaho.shEnsure Pentaho Data Integration is up and running.
cd
cd ~/Pentaho/design-tools/data-integration
./spoon.shCreate a New Transformation
Select the Design tab in the left-hand-side view.
x
x
x
x
In this transformation you will transfer data from a sample RDBMS to a MongoDB database.
The sample data is called SteelWheels and is available in the Pentaho server, running on the Hypersonic Database Server.
Linux
Ensure the Pentaho Server is up and running. The Hypersonic Database Server - SteelWheels - is an embedded service.
Ensure Pentaho Data Integration is up and running.
Create a New Transformation
The Table Input step reads information from a connected database using SQL statements. Basic SQL statements can be generated automatically by clicking the Get SQL select statement button.
Select the Design tab in the left-hand-side view.
From the Input category folder, find the Table Input step and drag and drop it into the working area in the right-hand-side view.
Double-click on the Table Input step to open the configuration dialog.
Set the Step Name property to Select Customers.
Before we can get any data from the SteelWheels Hypersonic database, we will have to create a JDBC connection to it.
Click on the New button next to the Database Connection pulldown. This will open the Database Connection dialog.
Enter the following details:
You can test the connection by clicking on the Test button at the bottom of the dialog. You should get a message similar to Connection Successful. If not, then you must double-check your connection details.
Click on OK to return to the Table Input step.
Now that we have a valid connection set, we are able to get a list of customers from the SteelWheels database. Copy and paste the following SQL into the query text area:
Click on the Preview button and you will see a table of customer details.
x
This step writes data to a MongoDB collection.
Under the Design tab, from the Big Data category folder, find the MongoDB Output step and drag and drop it into the working area in the right-hand-side view.
Create a Hop from the Table Input step to the MongoDB Output step.
Double-click on the MongoDB Output step.
Enter the following details:
Now, let's define the MongoDB documents structure. Select the Mongo document fields tab.
Click on the Get fields button, and the fields list will be populated with the SteelWheels database fields in the ETL stream.
By default, the column names in the SteelWheels database are in uppercase. In MongoDB, these field names should be in camel case. You can manually edit the names of the MongoDB document paths in this section also. Make sure that the Use Field Name option is set to No for each field, like this:
x
By clicking on Preview document structure, you will see an example of what the document will look like when it is inserted into the MongoDB Customers collection.
Click on the OK button to finish the MongoDB Output configuration.
RUN
x
x
Just for interest let's explore the use of the MongoDB aggregation framework in the MongoDB Input Step. We will create a simple example to get data from a collection and show you how you can take advantage of the MongoDB aggregation framework to prepare data for the PDI stream.
Linux
Ensure the Pentaho Server is up and running. The Hypersonic Database Server - SteelWheels - is an embedded service.
Ensure Pentaho Data Integration is up and running.
Create a New Transformation
Select the Design tab in the left-hand-side view.
From the Big Data category folder, find the MongoDB Input step and drag and drop it into the working area in the right-hand-side view.
Double-click on the step to open the MongoDB Input dialog.
Set the step name to Select 'Baane Mini Imports' Orders.
Select the Input options tab. Click on the Get DBs button and select the SteelWheels option for the Database field. Next, click on Get collections and select the Orders option for the Collection field.
Select the Query tab and then check the Query is aggregation pipeline option. In the text area, write the following aggregation query:
Uncheck the Output single JSON field option.
Select the Fields tab.
Click on the Get Fields button and you will get a list of fields returned by the query.
You can preview your data by clicking on the Preview button.
Click on the OK button to finish the configuration of this step.
x
We want to add a Dummy step to the stream. This step does nothing, but it will allow us to select a step to preview our data. Add the Dummy step from the Flow category to the workspace and name it OUTPUT. 4. Create a hop between the Select 'Baane Mini Imports' Orders step and the OUTPUT step. 5. Select the OUTPUT dummy step and preview the data. 19
x
To 'slice and dice' the dataset we'll need to create OLAP (short for Online Analytical Processing) schemas for Pentaho based on MongoDB.
Pentaho uses the ROLAP (short for Relational Online Analytical Processing) engine, called by Mondrian to convert MDX (short for Multidimensional Expressions) queries into SQL queries.
Let's start with a Mondrian 3.x schema using Schema Workbench. We'll first create a shared dimension, date. A shared dimension can be referenced in different cubes. However, in this particular case, it is not necessary to have a shared dimension because we'll have just one cube..!
x
x
x
x
Linux
Open the Schema Workbench application. With the Schema Workbench opened, let's configure the MongoDB database connection.
In the main menu, select Options -> Connection, and enter the following details:
Click on the Test button and you should get a success message box. Then click on the OK button.
Date Dimension
In the main menu, go to File -> New -> Schema.
x
Select the Schema object and set Orders as the field name.
Right-click on the Schema object and select Add Dimension.
Add a table to the hierarchy by right-clicking and selecting Add Table.
After selecting the table object added, select the -> date option for the name field.
In the default hierarchy (New Hierarchy 0), right-click and select the Add Level.
Let's define the year for this new level, enter the following details:
Add a new level and define the month; enter the following details:
Next, add a new level and define the day:
Select the hierarchy object, remove the default name (New Hierarchy 0), and select date for the primaryKey field.
Next, select the dimension object, and for the name field, set: date.
In the type field, select TimeDimension.
Finally, in the caption field, set Date.
x
x
x
x
x
x
x
x
Was this helpful?