Overview
Configuration is provided for establishing connections with the Microsoft Azure Queue Storage service. The configuration is used in the REST Connection Manager.
Setup
Press 
icon to get more information about the connection parameters.
Obtain data
Use REST Source component to get data from service resource.
Insert data
Use REST Destination component to insert data into service resource.
Quick Start
In this task, we will show you step-by-step how to create a connection to Azure Queue Storage REST API using COZYROC's REST Connection Manager.
Step 1. Run "Visual Studio" under "Administrator account". Right click on Connection managers Tab and Select New Connection. In Add SSIS Connection Manager scroll down and select Connection managers for REST Connection and click Add.
Step 2. From drop-down menu Configuration select Azure Queue Storage. Set [account] to the desired storage account in the Server host parameter. Then click New in the Token File parameter to create a new token file.
Step 3: In Next Wizard specify your credential values for client_id, client_secret and tenant_ id for the parameters mention in the grid below. The version parameter can be kept as default.
Step 4: Provide Email Address. Then press 'Next' button.
Step 5: Provide Password. Then press 'Next' button.
Step 6: Save generated access token to file. Specify token protection password. Click OK.
Step 7. Click "Test Connection" to test your connection.
Congratulations! You have now established a connection to your Azure Queue Storage instance.
In this task, we will show you step-by-step how to create a connection to Azure Queue Storage REST API using COZYROC's REST Connection Manager.
Step 1. Run "Visual Studio" under "Administrator account". Right click on Connection managers Tab and Select New Connection. In Add SSIS Connection Manager scroll down and select Connection managers for REST Connection and click Add.
Step 2. From drop-down menu Configuration select Azure Queue Storage. Set [account] to the desired storage account in the Server host parameter and and use Toggle button to set Authentication to Password Based
Step 3. In the grid below specify your Storage Account and key and then press Test Connection.
Congratulations! You have now established a connection to your Azure Blob Queue instance.
In this guide, we will show how to read data from the Azure Queue Storage using the COZYROC REST Source component.
Step 1. Select an existing "Data Flow task" or create a new "Data Flow Task".
Step 2. Open "SSIS Toolbox" and drag and drop COZYROC "REST Source" component.
Step 3. Double click on the "REST Source" component and select Azure Queue Storage connection from the combo. Select from available Azure Queue Storage resources listed in the second combo below.
Optionally you can include extra parameters in the grid below if needed.
Step 4. Click on the "Columns tab" and select Azure Queue Storage resource columns you want to read.
Step 5. Click Preview to make sure all necessary resource columns are included.
Bravo! You have now learned how to read data from Azure Queue Storage service.
In this guide, we will show how to write data to the Azure Queue Storage service resource using the COZYROC REST Destination component.
Step 1. Select an existing "Data Flow task" or create a new "Data Flow Task".
Step 2. Open the "SSIS Toolbox" and drag and drop COZYROC "REST Destination" component.
Step 3. Double click on the "REST Destination" component and select Azure Queue Storage connection from the combo. Also, select resource action and Azure Queue Storage resource you want to work on.
Step 4. Click on the "Mapping" tab and map the input columns for the process you are configuring.
If you want to provide input for other resource composite objects, change the input above and map the columns.
Done. You have now learned how to write data into the Azure Queue Storage resource.
Configuration
Base URL address: https://[account].queue.core.windows.net.
- OAuth
-
The authentication uses an authorized token. The token will be refreshed with the following expression:
{{=Date.now() + (response.expires_in - 300) * 1000}}.The authentication has the following user-defined parameters:
- client_id: Required. Identifier of application that is associated with the authentication token.
- client_secret: Specify application client secret. Optional.
- tenant: Required. Specify tenant identifier.
- version:
The following request parameters will be automatically processed during the authentication process:
-
Authorization:
Bearer {{=token.Access}} -
x-ms-version:
{{=token.version}} -
x-ms-date:
{{=new Date().toUTCString()}}
Documentation: https://learn.microsoft.com/en-us/rest/api/authorization/.
- Shared Key
-
The authentication uses a parameters-based authentication.
The authentication has the following user-defined parameters:
- Storage Account: Required. Specify storage account.
- Key: Required. Specify key.
The following request parameters will be automatically processed during the authentication process:
-
Authorization:
{{=azurest.authorizationHeader (request, connection)}} -
x-ms-date:
{{=request.requestToSign.headers['x-ms-date']}} -
x-ms-version:
{{=request.requestToSign.headers['x-ms-version']}}
Documentation: https://learn.microsoft.com/en-us/azure/storage/common/storage-rest-api-auth.
Based on resource template Base.
- [Read] action
-
Endpoint URL address:
/.
The result is extracted from:{{=response.EnumerationResults.Queues.Queue}}.
The result is paginated.The action has the following user-defined parameters:
- timeout: The timeout parameter is expressed in seconds. Optional.
The following request parameters will be automatically processed:
-
comp:
list -
marker:
{{=parameters.iterator}} -
maxresults:
{{=parameters.batchSize}}
Documentation: https://learn.microsoft.com/en-us/rest/api/storageservices/list-queues1.
- [Create] action
-
Endpoint URL address:
/{{=item.Name}}.The action has the following user-defined parameters:
- timeout: The timeout parameter is expressed in seconds. Optional.
The following request parameters will be automatically processed:
-
timeout:
{{=parameters.get('timeout')}}
Documentation: https://learn.microsoft.com/en-us/rest/api/storageservices/create-queue4.
- [Delete] action
-
Endpoint URL address:
/{{=item.Name}}.Documentation: https://learn.microsoft.com/en-us/rest/api/storageservices/delete-queue3.
- Name
A key field. Template: ShortText.
- Metadata
The gathered value is processed with
{{=_.map(value, function(v, k) { return { name: k, value: v }; })}}expression. The value is sent to/{{=item.Name}}address. Field components:- An array.
- Contains the following components: name, value.
- properties
The value is gathered from
/address. The value is sent to/address. Field components:- Contains the following components: StorageServiceProperties.
- ACL
The value is gathered from
/{{=item.Name}}address. The value is sent to/{{=item.Name}}address. Field components:- Contains the following components: SignedIdentifiers.
Based on resource template Base.
- [Read] action
-
Endpoint URL address:
/{{=parameters.queueName}}/messages.
The result is extracted from:{{=utils.ensureArray(response.QueueMessagesList.QueueMessage)}}.
The result is paginated.The action has the following user-defined parameters:
- queueName: Required. Please specify the queue name.
- numofmessages: A nonzero integer value that specifies the number of messages to retrieve from the queue, up to a maximum of 32. Default is one. Optional.
- timeout: Please specify the timeout in seconds. Optional.
Documentation: https://learn.microsoft.com/en-us/rest/api/storageservices/get-messages.
- [Create] action
-
Endpoint URL address:
/{{=parameters.queueName}}/messages.
The action uses POST method.The action has the following user-defined parameters:
- queueName: Required. Please specify the queue name.
The following request parameters will be automatically processed:
-
application/xml:
{{=json2xml({ QueueMessage : item })}} -
visibilitytimeout:
{{=item.VisibilityTimeout}} -
messagettl:
{{=item.MessageTTL}} -
timeout:
{{=parameters.get('timeout')}}
Documentation: https://learn.microsoft.com/en-us/rest/api/storageservices/put-message.
- [Update] action
-
Endpoint URL address:
/{{=parameters.queueName}}/messages/{{=item.MessageId}}.The action has the following user-defined parameters:
- queueName: Required. Please specify the queue name.
The following request parameters will be automatically processed:
-
application/xml:
{{=json2xml({ QueueMessage : item })}} -
visibilitytimeout:
{{=item.VisibilityTimeout}} -
popreceipt:
{{=item.PopReceipt}} -
timeout:
{{=parameters.get('timeout')}}
Documentation: https://learn.microsoft.com/en-us/rest/api/storageservices/update-message.
- [Delete] action
-
Endpoint URL address:
/{{=parameters.queueName}}/messages{{=item.MessageId && '/' + item.MessageId}}.The action has the following user-defined parameters:
- queueName: Required. Please specify the queue name.
The following request parameters will be automatically processed:
-
popreceipt:
{{=item.PopReceipt}} -
timeout:
{{=parameters.get('timeout')}}
Documentation: https://learn.microsoft.com/en-us/rest/api/storageservices/delete-message2 https://learn.microsoft.com/en-us/rest/api/storageservices/clear-messages .
- MessageId
A key field. Data type: DT_GUID
- MessageText
Data type: DT_NTEXT The gathered value is processed with
{{=Base64.decode(value)}}expression. Before setting, the value is processed with{{=Base64.encode(value)}}expression.- PopReceipt
Template: ShortText.
- InsertionTime
Template: DateTime. A read-only field.
- ExpirationTime
Template: DateTime. A read-only field.
- TimeNextVisible
Template: DateTime. A read-only field.
- DequeueCount
Data type: DT_I4 A read-only field.
- VisibilityTimeout
Data type: DT_I4
- MessageTTL
Data type: DT_I4
- [Read] action
-
The following request parameters will be automatically processed:
-
_includeUserParameters:
{{=parameters}}
-
_includeUserParameters:
- [Create] action
-
The action uses PUT method.The following request parameters will be automatically processed:
-
application/xml:
{{=json2xml(item)}} -
_includeUserParameters:
{{=parameters}}
-
application/xml:
- [Update] action
-
The action uses PUT method.The following request parameters will be automatically processed:
-
application/xml:
{{=json2xml(item)}} -
_includeUserParameters:
{{=parameters}}
-
application/xml:
- [Delete] action
-
The action uses DELETE method.
- ShortText
Data type: DT_WSTR ( length 255 )
- LongText
Data type: DT_WSTR ( length 1000 )
- DateTime
Data type: DT_DBTIMESTAMP
- Date
Data type: DT_DBDATE
- RetentionPolicyT
Field components:
- Contains the following components: Enabled, Days.
- MetricT
Field components:
-
Uses template:
ShortText. - Contains the following components: Version, Enabled, IncludeAPIs, RetentionPolicy.
-
Uses template:
- AccessPolicyT
Field components:
-
Uses template:
DateTime. - Contains the following components: Start, Expiry, Permission.
-
Uses template:
- LoggingT
Field components:
-
Uses template:
ShortText. - Contains the following components: Version, Delete, Read, Write, RetentionPolicy.
-
Uses template:
- CoreRuleT
Field components:
-
Uses template:
ShortText. - Contains the following components: AllowedOrigins, AllowedMethods, MaxAgeInSeconds, ExposedHeaders, AllowedHeaders.
-
Uses template:
Knowledge Base
Related scripts
What's New
- New: Introduced connection.
Related documentation
COZYROC SSIS+ Components Suite is free for testing in your development environment.
A licensed version can be deployed on-premises, on Azure-SSIS IR and on COZYROC Cloud.