Avro Destination

Overview

Avro Destination Component is SSIS Data Flow Component for generating Apache Avro files.

The component metadata is either automatically retrieved from a sample Avro file or can be manually specified in JSON format.
The generated Avro file can contain nested arrays of objects following the composite records pattern), where the fields for the arrays are fed via separate inputs.
The generated Avro content can be written to a file or stored in a variable.

Generate Avro file

In this section we will show you how to set up an Avro Destination component.

In the SSIS Toolbox, locate the COZYROC's Avro Destination component and drag it onto the Data Flow canvas.

Double-click on the component on the canvas.
Once the component editor opens, select the destination where generated data will be stored: then provide Avro sample file or directly write the schema into Schema text editor.

When clicking on Mapping tab the component would prepare the inputs and external columns by analyzing the schema in the Schema text editor. Please note that the Avro Destination can have multiple inputs (see the article about composite records), which columns you can see. The data in these inputs can be processed by upstream transformation and source components (e.g. a Query Transformation can be used to retrieve the necessary data from SQL Server database).
Click OK to close the component editor.

Congratulations! You have successfully configured the Avro Destination component.

Use the parameters below to configure the component.

Destination

Indicates the destination of Avro data. The following options are available:

Value	Description
File	The Avro data will be stored in a file. Select an existing File Connection Manager or create a new one.
Variable	The Avro data will be stored in a variable. Select a variable or create a new one.

Codec

Specify the compression method applied to the destination file. The following options are available:

Value	Description
None	None compression means no compression is applied to the data.
Deflate	Deflate compression uses the DEFLATE algorithm (the same core algorithm behind Gzip) to compress data efficiently. General-purpose compression where a balance between size and compatibility is needed.
Snappy	Snappy compression is a fast algorithm developed by Google, optimized for speed over compression ratio. Used in real-time processing systems where low latency is critical.
Zstd	Zstandard compression is a modern algorithm developed by Facebook, offering a strong balance between speed and compression ratio. Used in systems needing both efficient storage and good performance.

Schema

JSON string representing the schema of the Avro file. For more information visit Components Metadata Schema.

2.3