Overview
MongoDB Source Component is SSIS Data Flow Component for retrieving data from MongoDB document database that supports multiple outputs via the composite records pattern.
- Besides the main output that contains the fields from the deduced schema of a collection, there are separate outputs (a.k.a. composite outputs), setup for retrieving the field value for array items (those items are basically nested documents within the main MongoDB document).
- An additional standalone output with a single '$JSON' column enables retrieving the entire document contents as JSON string.
- An error output can be setup to redirect problematic records (in case of a failure processing a MongoDB document field value).
Demonstration
Quick Start
In this section we will show you how to set up a MongoDB Source component.
- In the SSIS Toolbox, locate the MongoDB Source component and drag it onto the Data Flow canvas.
- Double-click on the component on the canvas.
- Once the component editor opens, select a pre-configured MongoDB Connection Manager from the Connection drop-down list or create a new one.
- Select Collection.
- Enter Filter if you need to query a subset of the documents in the MongoDB collection.
- The default values of Batch size and Sample size usually don’t need to be changed.
- Select Timezone - UTC or LocalTime.
- When clicking on Columns tab the component would prepare the outputs and external columns by analyzing the existing data in the collection. Please note that the MongoDB Source can have multiple outputs (see the article about composite records), which columns you can see. The data in these outputs can be processed by downstream transformation and destination components(e.g. multiple OLE DB Destinations can store the data in SQL Server database).
- Click OK to close the component editor.
Congratulations! You have successfully configured the MongoDB Source component.
Parameters
Configuration
Use the parameters below to configure the component.
Select an existing MongoDB connection manager.
The number of documents returned per batch. To let MongoDB engine decide what's the optimal batch size, just leave the default value of 0. To disable batching, use a value of 1.
The MongoDB collection from which documents would be retrieved
The optional filter (i.e. MongoDB query) to apply on the selected source collection. Uses the standard MongoDB JSON syntax for query definitions.
An example of a simple query to find a document by its identifier is:
{ "_id" : ObjectId("5a57388f6f8dff702f254717") }The number of documents to analyze when deducing the schema of the MongoDB collection (i.e. outputs, columns names and column types).
The time zone for dates processing. Allows converting dates to UTC or local time.
Knowledge Base
- Where can I find the documentation for the MongoDB Source?
- MongoDB Source: How to modify the filter property dynamically at runtime
- How to work with composite records handling in a MongoDB source component.
- After modifying the default Data Types in an ETL source, my modifications are lost the next time I open the Source component
What's New
- New: Support reading from views.
- New: Add a custom editor for improved design-time experience.
- New: Improved support for UUID data.
- New: Introduced component.
Related documentation
COZYROC SSIS+ Components Suite is free for testing in your development environment.
A licensed version can be deployed on-premises, on Azure-SSIS IR and on COZYROC Cloud.