Overview
Configuration is provided for establishing connections with Microsoft Azure Files service. The configuration is used in the REST Connection Manager.
Setup
Press 
icon to get more information about the connection parameters.
Obtain data
Use REST Source component to get data from service resources.
Insert data
Use REST Destination component to insert data into service resource.
Manage remote files and directories
Use File Transfer Task to manage remote files and directories.
Quick Start
In this task, we will show you step-by-step how to create a connection to Azure Files Service REST API using COZYROC's REST Connection Manager with a Shared Key authentication.
Step 1. Run "Visual Studio" under "Administrator account". Right click on Connection managers Tab and Select New Connection. In Add SSIS Connection Manager scroll down and select Connection managers for REST Connection and click Add.
Step 2. From drop-down menu Configuration select Azure Files. Set [storage_name] to your Storage Account Name in the Server host parameter and then select Shared Key from the Authentication drop-down. Next, provide the storage account name in the Storage Account text box and the shared key value in the Shared Key text box. After that, you can click on Test to verify that connection can be established successfully. Finally, click Ok to save the connection details in the Connection Manager.
In this task, we will show you step-by-step how to create a connection to Azure Files Service REST API using COZYROC's REST Connection Manager with an OAuth 2.0 authentication.
Important Note: Due to Azure File Share policies access to some resources is limited with this type of authentication. Only access to Directories and Files is allowed with OAuth 2.0. For full access to all Azure Files resources you must use Shared Key authentication.
Step 1. Run "Visual Studio" under "Administrator account". Right click on Connection managers Tab and Select New Connection. In Add SSIS Connection Manager scroll down and select Connection managers for REST Connection and click Add.
Step 2. From drop-down menu Configuration select Azure Files. Set [account_name] to your Storage Account Name in the Server host parameter. Then click New in the Token File parameter to create a new token file.
Step 3: Select browser engine to be used during authentication process. If your App is Multi-Tenant provide client_id and leave rest of the parameter as default and, if your App is Single Tenant then provide your app client_id, client_secret and tenant_id. The version can can be default in both scenarios.
Step 4: Provide Email and then press 'Next' button.
Step 5: Provide Password and then press 'Sign in' button.
Step 6: Save generated access token to file. Specify token protection password. Click OK.
Step 7. Click "Ok" to save your connection in Connections Manager.
In this guide, we will show how to read data from the Azure Files Service resource using the COZYROC REST Source component.
Step 1. Select an existing "Data Flow task" or create a new "Data Flow Task".
Step 2. Open "SSIS Toolbox" and drag and drop COZYROC "REST Source" component.
Step 3. Double click on the "REST Source" component and select Azure Table connection from the dropdown box. Select from available Azure Files resources listed in the second dropdown below. Depending on the selected resource you may have to provide additional parameters in the grid below. Optional parameters are also available for selection.
Step 4. Click on the "Columns tab" and select Azure Files resource columns you want to read.
Step 5. Click Preview to make sure all necessary resource columns are included. Click Ok to confirm the configuration.
In this guide, we will show how to write data to the Azure Files Service resources using the COZYROC REST Destination component.
Step 1. Select an existing "Data Flow task" or create a new "Data Flow Task".
Step 2. Open the "SSIS Toolbox" and drag and drop COZYROC "REST Destination" component.
Step 3. Double click on the "REST Destination" component and select Azure Files connection from the dropdown. Also, select resource action and Azure Files resource you want to work with. Depending on the resource and action selected some optional parameters will be available in the grid below.
Step 4. Click on the "Mapping" tab and map the input columns for the process you are configuring. Click "Ok" to confirm the configuration of the REST Destination component.
Configuration
Base URL address: https://[storage_name].file.core.windows.net.
- OAuth
-
The authentication uses an authorized token. The token will be refreshed with the following expression:
{{=Date.now() + (response.expires_in - 300) * 1000}}.The authentication has the following user-defined parameters:
- client_id: Required. Identifier of application that is associated with the authentication token.
- client_secret: Specify application client secret. Optional.
- tenant: Required. Specify tenant identifier.
- version: Required. API version requested. Recommended is '2022-11-02'
The following request parameters will be automatically processed during the authentication process:
-
Authorization:
Bearer {{=token.Access}} -
x-ms-version:
{{=token.version}} -
x-ms-file-request-intent:
backup
Documentation: https://docs.microsoft.com/en-us/powerapps/developer/data-platform/authenticate-oauth.
- Shared Key
-
The authentication uses a parameters-based authentication.
The authentication has the following user-defined parameters:
- Storage Account: Required. Specify storage account.
- Key: Required. Specify key.
The following request parameters will be automatically processed during the authentication process:
-
Authorization:
{{=azurest.authorizationHeader (request, connection, '2022-11-02')}} -
x-ms-date:
{{=request.requestToSign.headers['x-ms-date']}} -
x-ms-version:
{{=request.requestToSign.headers['x-ms-version']}}
Documentation: https://learn.microsoft.com/en-us/azure/storage/common/storage-rest-api-auth.
Based on resource template Base.
- [Read] action
-
Endpoint URL address:
/.
The result is extracted from:{{=[response.StorageServiceProperties]}}.The following request parameters will be automatically processed:
-
comp:
properties -
restype:
service
Documentation: https://learn.microsoft.com/en-us/rest/api/storageservices/get-file-service-properties.
-
comp:
- [Update] action
-
Endpoint URL address:
/.The following request parameters will be automatically processed:
-
comp:
properties -
restype:
service -
application/xml:
{{=json2xml({ StorageServiceProperties : item })}}
Documentation: https://learn.microsoft.com/en-us/rest/api/storageservices/set-file-service-properties.
-
comp:
- HourMetrics
Template: Metrics.
- MinuteMetrics
Template: Metrics.
- Cors
Template: LongText. A composite field.
- ShareDeleteRetentionPolicy
Template: RetentionPolicyBase.
- ProtocolSettings
Field components:
- Contains the following components: SMB.
Based on resource template Pagination.
- [Read] action
-
Endpoint URL address:
/.
The result is extracted from:{{=utils.ensureArray(response.EnumerationResults.Shares.Share)}}.The following request parameters will be automatically processed:
-
comp:
list -
include:
metadata,deleted
Documentation: https://learn.microsoft.com/en-us/rest/api/storageservices/list-shares.
-
comp:
- [Create] action
-
Endpoint URL address:
/{{=item.Name}}.The following request parameters will be automatically processed:
-
restype:
share
Documentation: https://learn.microsoft.com/en-us/rest/api/storageservices/create-share.
-
restype:
- [Update] action
-
Endpoint URL address:
/{{=item.Name}}.The following request parameters will be automatically processed:
-
restype:
share -
comp:
snapshot
Documentation: https://learn.microsoft.com/en-us/rest/api/storageservices/snapshot-share .
-
restype:
- [Delete] action
-
Endpoint URL address:
/{{=item.Name}}.The following request parameters will be automatically processed:
-
restype:
share -
sharesnapshot:
{{=item.sharesnapshot}}
Documentation: https://learn.microsoft.com/en-us/rest/api/storageservices/delete-share.
-
restype:
- Name
A key field. Template: ShortText.
- Version
Template: ShortText.
- Deleted
Data type: DT_BOOL
- Properties
The value is sent to
/{{=item.Name}}address. Field components:-
Uses template:
DateTime. - Contains the following components: Last-Modified, Etag, Quota, DeletedTime, RemainingRetentionDays, LeaseStatus, LeaseState, AccessTier, AccessTierChangeTime, AccessTierTransitionState, ProvisionedIngressMBps, ProvisionedEgressMBps, ProvisionedBandwidthMiBps, DefaultEncryptionScope, DenyEncryptionScopeOverride, EnabledProtocols, RootSquash.
-
Uses template:
- sharesnapshot
Template: DateTime.
- Metadata
Template: MetadataT. The gathered value is processed with
{{=_.map(value, function(v, k) { return { name: k, value: v }; })}}expression. The value is sent to/{{=item.Name}}address.- SignedIdentifiers
The value is gathered from
{{=item.Deleted ? '' : '/' + item.Name}}address. The gathered value is processed with{{=utils.ensureArray(response.SignedIdentifiers.SignedIdentifier)}}expression. The value is sent to/{{=item.Name}}address. Field components:- An array.
- Contains the following components: Id, AccessPolicy.
- Stats
The value is gathered from
/{{=item.Name}}address. The gathered value is processed with{{=response.ShareStats}}expression. Field components:-
Uses template:
BigInt. - Contains the following components: ShareStats.
-
Uses template:
- action
Field components:
- Contains the following components: restore, lease.
Based on resource template Pagination.
- [Read] action
-
Endpoint URL address:
/{{=parameters.full_path}}.
The result is extracted from:{{=utils.ensureArray(response.EnumerationResults.Entries.Directory || []).concat(utils.ensureArray(response.EnumerationResults.Entries.File || []))}}.The action has the following user-defined parameters:
- full_path: Required. Specify the share and directory name.
- prefix: Optional. Filters the results to return only files and directories that have names that begin with the specified prefix.
- sharesnapshot: Optional. The share snapshot parameter is an opaque DateTime value that, when present, specifies the share snapshot to query for the list of files and directories.
- include: Specifies one or more properties to include in the response. Optional.
The following request parameters will be automatically processed:
-
restype:
directory -
comp:
list -
x-ms-file-request-intent:
backup
Documentation: https://learn.microsoft.com/en-us/rest/api/storageservices/list-directories-and-files.
- [Create] action
-
Endpoint URL address:
/{{=item.Name}}.The action has the following user-defined parameters:
- x-ms-file-permission: Optional. This permission is the security descriptor for the directory that's specified in the Security Descriptor Definition Language (SDDL).
- x-ms-file-permission-key: Optional. The key of the permission to be set for the directory.
- x-ms-file-attributes: Optional. The file system attributes to be set on the directory. The default value is Directory.
- x-ms-allow-trailing-dot: Optional. The Boolean value specifies if a trailing dot present in request url should be trimmed or not.
The following request parameters will be automatically processed:
-
restype:
{{=AzureFiles.getResourceSize(item) > 0 ? undefined : 'directory' }} -
x-ms-file-request-intent:
backup -
x-ms-content-length:
{{=AzureFiles.getResourceSize(item) > 0 ? AzureFiles.getResourceSize(item) : undefined }} -
x-ms-type:
{{=AzureFiles.getResourceSize(item) > 0 ? 'file' : undefined }}
Documentation: https://learn.microsoft.com/en-us/rest/api/storageservices/create-directory.
- [Update] action
-
Endpoint URL address:
.Documentation: Placeholder for update to allow child resources to execute custom set methods.
- [Delete] action
-
Endpoint URL address:
/{{=item.Name}}.The action has the following user-defined parameters:
- x-ms-allow-trailing-dot: The Boolean value specifies if a trailing dot present in request url should be trimmed or not. Optional.
- restype: Set to directory if you want to delete directories.
The following request parameters will be automatically processed:
-
x-ms-file-request-intent:
backup -
restype:
{{=parameters.get('restype')}}
Documentation: https://learn.microsoft.com/en-us/rest/api/storageservices/delete-directory https://learn.microsoft.com/en-us/rest/api/storageservices/delete-file2 .
- FileId
Template: ShortText.
- Name
Template: ShortText.
- Properties
The value is sent to
/{{=item.Name}}address. Field components:-
Uses template:
BigInt. - Contains the following components: Content-Length, CreationTime, LastAccessTime, LastWriteTime, ChangeTime, Last-Modified, Etag.
-
Uses template:
- Attributes
Template: ShortText.
- PermissionKey
Template: ShortText.
- DirectoryMetadata
Template: MetadataT. The value is gathered from
/{{=parameters.full_path+'/'+item.Name}}address. The gathered value is processed with{{=AzureFiles.getMetadata(response['@@@headers@@@'])}}expression. The value is sent to/{{=item.Name}}address.- Ranges
The value is gathered from
/{{=parameters.full_path+'/'+item.Name}}address. The gathered value is processed with{{=utils.ensureArray( response.Ranges ? response.Ranges.Range || [] : [] ) }}expression. Field components:- An array.
- Contains the following components: Start, End.
- Metadata
Template: MetadataT. The value is gathered from
/{{=parameters.full_path+'/'+item.Name}}address. The gathered value is processed with{{=AzureFiles.getMetadata(response['@@@headers@@@'])}}expression. The value is sent to/{{=item.Name}}address.- Content
Data type: DT_IMAGE The value is gathered from
/{{=parameters.full_path + '/' + item.Name}}address.- ExtProperties
The value is gathered from
/{{=parameters.full_path + '/' + item.Name}}address. The gathered value is processed with{{=response['@@@headers@@@']}}expression. The value is sent to/{{=item.Name}}address. Field components:-
Uses template:
DateTime. - Contains the following components: Last-Modified, x-ms-type, Content-Length, Content-Type, Content-MD5, Content-Encoding, Content-Language, Cache-Control, Content-Disposition, x-ms-copy-completion-time, x-ms-copy-status-description, x-ms-copy-id, x-ms-copy-progress, x-ms-copy-source, x-ms-copy-status, x-ms-server-encrypted, x-ms-file-permission-key, x-ms-file-attributes, x-ms-file-parent-id, x-ms-lease-duration, x-ms-lease-state, x-ms-lease-status.
-
Uses template:
- action
Field components:
- Contains the following components: rename, copy, putRangeFromUrl.
Based on resource template Pagination.
- [Read] action
-
Endpoint URL address:
/{{=parameters.full_path}}.
The result is extracted from:{{=response.EnumerationResults && response.EnumerationResults.HandleList && response.EnumerationResults.HandleList.Handle && utils.ensureArray(response.EnumerationResults.HandleList.Handle) || []}}.The action has the following user-defined parameters:
- full_path: Required. Specify the full path of the directory (including share name).
- sharesnapshot: Optional. The sharesnapshot parameter is a DateTime value that, when present, specifies the share snapshot to use.
- x-ms-recursive: Optional. A Boolean value that specifies if the operation should also apply to the files and subdirectories of the directory specified in the URI.
- x-ms-allow-trailing-dot: Optional. The Boolean value specifies if a trailing dot present in request url should be trimmed or not.
The following request parameters will be automatically processed:
-
comp:
listhandles -
x-ms-file-request-intent:
backup
Documentation: https://learn.microsoft.com/en-us/rest/api/storageservices/list-handles.
- [Update] action
-
Endpoint URL address:
/{{=item.full_path}}.The action has the following user-defined parameters:
- sharesnapshot: Optional. The sharesnapshot parameter is a DateTime value that, when present, specifies the share snapshot to use.
- x-ms-recursive: Optional. A Boolean value that specifies if the operation should also apply to the files and subdirectories of the directory specified in the URI.
- x-ms-allow-trailing-dot: Optional. The Boolean value specifies if a trailing dot present in request url should be trimmed or not.
The following request parameters will be automatically processed:
-
comp:
forceclosehandles -
x-ms-file-request-intent:
backup -
x-ms-handle-id:
{{=item.HandleId}}
Documentation: https://learn.microsoft.com/en-us/rest/api/storageservices/force-close-handles.
- HandleId
Template: BigInt.
- Path
Template: ShortText.
- ClientIp
Template: ShortText.
- OpenTime
Template: DateTime.
- LastReconnectTime
Template: DateTime.
- FileId
Template: BigInt.
- ParentId
Template: BigInt.
- SessionId
Template: BigInt.
- AccessRightList
Template: LongText. A composite field.
- full_path
Template: ShortText.
- [Read] action
-
The action has the following user-defined parameters:
- timeout: The timeout parameter is expressed in seconds. Optional.
- x-ms-client-request-id: Provides a client-generated, opaque value with a 1-kibibyte (KiB) character limit that's recorded in the logs when logging is configured. Optional.
The following request parameters will be automatically processed:
-
_includeUserParameters:
{{=AzureFiles.getRegularParams(parameters)}} -
_includeUserParameters:
{{=AzureFiles.getHeaderParams(parameters)}}
- [Create] action
-
The action uses PUT method.The action has the following user-defined parameters:
- timeout: The timeout parameter is expressed in seconds. Optional.
- x-ms-client-request-id: Provides a client-generated, opaque value with a 1-kibibyte (KiB) character limit that's recorded in the logs when logging is configured. Optional.
The following request parameters will be automatically processed:
-
_includeUserParameters:
{{=AzureFiles.getRegularParams(parameters)}} -
_includeUserParameters:
{{=AzureFiles.getHeaderParams(parameters)}}
- [Update] action
-
The action uses PUT method.The action has the following user-defined parameters:
- timeout: The timeout parameter is expressed in seconds. Optional.
- x-ms-client-request-id: Provides a client-generated, opaque value with a 1-kibibyte (KiB) character limit that's recorded in the logs when logging is configured. Optional.
The following request parameters will be automatically processed:
-
_includeUserParameters:
{{=AzureFiles.getRegularParams(parameters)}} -
_includeUserParameters:
{{=AzureFiles.getHeaderParams(parameters)}}
- [Delete] action
-
The action uses DELETE method.The action has the following user-defined parameters:
- timeout: The timeout parameter is expressed in seconds. Optional.
- x-ms-client-request-id: Provides a client-generated, opaque value with a 1-kibibyte (KiB) character limit that's recorded in the logs when logging is configured. Optional.
The following request parameters will be automatically processed:
-
_includeUserParameters:
{{=AzureFiles.getRegularParams(parameters)}} -
_includeUserParameters:
{{=AzureFiles.getHeaderParams(parameters)}}
Based on resource template Base.
- [Read] action
-
The result is paginated.The following request parameters will be automatically processed:
-
maxresults:
{{=parameters.batchSize}} -
marker:
{{=parameters.iterator}}
-
maxresults:
- ShortText
Data type: DT_WSTR ( length 255 )
- LongText
Data type: DT_WSTR ( length 1000 )
- DateTime
Data type: DT_DBTIMESTAMP
- Integer
Data type: DT_I4
- BigInt
Data type: DT_I8
- Date
Data type: DT_DBDATE
- Boolean
Data type: DT_BOOL
- RetentionPolicyBase
Field components:
- Contains the following components: Enabled, Days.
- Metrics
Field components:
-
Uses template:
ShortText. - Contains the following components: Version, Enabled, IncludeAPIs, RetentionPolicy.
-
Uses template:
- MetadataT
Field components:
- An array.
- Contains the following components: name, value.
Knowledge Base
Related scripts
What's New
- New: Introduced connection.
Related documentation
COZYROC SSIS+ Components Suite is free for testing in your development environment.
A licensed version can be deployed on-premises, on Azure-SSIS IR and on COZYROC Cloud.