Overview
Configuration is provided for establishing connections with Google Cloud Storage service. The configuration is used in the REST Connection Manager.
Setup
Press 
icon to get more information about the connection parameters.
Obtain data
Use REST Source component to get data from service resource.
Insert data
Use REST Destination component to insert data into service resource.
Manage remote files and directories
Use File Transfer Task to manage remote files and directories.
Quick Start
Step 1. Open the right side panel and drag and drop REST Source component. Double click on it and select GCS connection, select from available resources listed in second drop-down menu (Bucket) and enter values of required parameters in Parameters grid.
Step 2. Check resource metadata by selecting Columns.
Step 3. If necessary, you can configure the Error Output.
Step 4. Click on Preview tab to view the retrieved data.
Step 5. Connect the newly created REST Source component to suitable Destination component and edit mapping.
Step 1. Drag and drop REST Destination component from the SSIS Toolbox. Double click on it and select GCS connection. From the Resource drop-down menu select Bucket and from the Action drop-down menu select Create.
Step 2. Connect suitable Source component to the newly created REST Destination component and map columns.
Step 3. If necessary, you can configure the Error Output.
Configuration
Base URL address: https://storage.googleapis.com/storage/v1.
- OAuth
-
The authentication uses an authorized token. The token will be refreshed with the following expression:
{{=Date.now() + (response.expires_in - 300) * 1000}}.The authentication has the following user-defined parameters:
- Project:
- Scope:
The following request parameters will be automatically processed during the authentication process:
-
Authorization:
Bearer {{=token.Access}}
Documentation: https://cloud.google.com/storage/docs/authentication.
- Service Account
-
The authentication uses a session token.
The authentication has the following user-defined parameters:
- ClientEmail: Required. Specify service account email address.
- KeyFile: Required. Select private key file (p12).
- Project: Required. Specify project id or number.
- Scope: Specify a space-delimited list of the permissions.
- ActOnBehalf: Specify email address of user to act on behalf. Optional.
The following request parameters will be automatically processed during the authentication process:
-
Authorization:
Bearer {{=token.Access}}
Documentation: https://developers.google.com/identity/protocols/oauth2/service-account.
Based on resource template Base.
- [Read] action
-
Endpoint URL address:
/b/{{=parameters.bucket}}/o.The action has the following user-defined parameters:
- bucket: Required. Name of the bucket in which to look for objects.
Documentation: https://cloud.google.com/storage/docs/json_api/v1/objects/list.
- [Create] action
-
Endpoint URL address:
https://storage.googleapis.com/upload/storage/v1/b/{{=item.bucket}}/o.The following request parameters will be automatically processed:
-
uploadType:
multipart -
content:
{{=item.content}}
Documentation: https://cloud.google.com/storage/docs/json_api/v1/objects/insert.
-
uploadType:
- [Update] action
-
Endpoint URL address:
/b/{{=item.bucket}}/o/{{=encodeURIComponent(item.name)}}.Documentation: https://cloud.google.com/storage/docs/json_api/v1/objects/patch.
- [Delete] action
-
Endpoint URL address:
/b/{{=item.bucket}}/o/{{=encodeURIComponent(item.name)}}.Documentation: https://cloud.google.com/storage/docs/json_api/v1/objects/delete.
The resource includes the fields from the Object template.
- content
Data type: DT_IMAGE The value is gathered from
{{= item.size === 0 ? '' : '/b/' + item.bucket + '/o/' + encodeURIComponent(item.name) }}address.
Based on resource template Base.
- [Read] action
-
Endpoint URL address:
/b.The action has the following user-defined parameters:
- project: Required. A valid API project identifier.
The following request parameters will be automatically processed:
-
project:
{{=parameters.project}}
Documentation: https://cloud.google.com/storage/docs/json_api/v1/buckets/list.
- [Create] action
-
Endpoint URL address:
/b.The following request parameters will be automatically processed:
-
project:
{{=item.project}}
Documentation: https://cloud.google.com/storage/docs/json_api/v1/buckets/insert.
-
project:
- [Update] action
-
Endpoint URL address:
/b/{{=item.name}}.Documentation: https://cloud.google.com/storage/docs/json_api/v1/buckets/patch.
- [Delete] action
-
Endpoint URL address:
/b/{{=item.name}}.Documentation: https://cloud.google.com/storage/docs/json_api/v1/buckets/delete.
The resource includes the fields from the Bucket template.
- project
Template: ShortText.
Based on resource template Base.
- [Read] action
-
Endpoint URL address:
/b/{{=parameters.bucket}}/acl.The action has the following user-defined parameters:
- bucket: Required. Name of a bucket.
Documentation: https://cloud.google.com/storage/docs/json_api/v1/bucketAccessControls/list.
- [Create] action
-
Endpoint URL address:
/b/{{=item.bucket}}/acl.Documentation: https://cloud.google.com/storage/docs/json_api/v1/bucketAccessControls/insert.
- [Update] action
-
Endpoint URL address:
/b/{{=item.bucket}}/acl{{=item.entity}}.Documentation: https://cloud.google.com/storage/docs/json_api/v1/bucketAccessControls/patch.
- [Delete] action
-
Endpoint URL address:
/b/{{=item.bucket}}/acl{{=item.entity}}.Documentation: https://cloud.google.com/storage/docs/json_api/v1/bucketAccessControls/delete.
The resource includes the fields from the BucketAccessControl template.
Based on resource template Base.
- [Read] action
-
Endpoint URL address:
/b/{{=parameters.bucket}}/o/{{=encodeURIComponent(parameters.object)}}/acl.The action has the following user-defined parameters:
- bucket: Required. Name of a bucket.
- object: Required. Name of the object.
Documentation: https://cloud.google.com/storage/docs/json_api/v1/objectAccessControls/list.
- [Create] action
-
Endpoint URL address:
/b/{{=item.bucket}}/o/{{=encodeURIComponent(item.object)}}/acl.Documentation: https://cloud.google.com/storage/docs/json_api/v1/objectAccessControls/insert.
- [Update] action
-
Endpoint URL address:
/b/{{=item.bucket}}/o/{{=encodeURIComponent(item.object)}}/acl/{{=item.entity}}.Documentation: https://cloud.google.com/storage/docs/json_api/v1/objectAccessControls/patch.
- [Delete] action
-
Endpoint URL address:
/b/{{=item.bucket}}/o/{{=encodeURIComponent(item.object)}}/acl/{{=item.entity}}.Documentation: https://cloud.google.com/storage/docs/json_api/v1/objectAccessControls/delete.
The resource includes the fields from the ObjectAccessControl template.
Based on resource template Base.
- [Read] action
-
Endpoint URL address:
/b/{{=parameters.bucket}}/defaultObjectAcl.The action has the following user-defined parameters:
- bucket: Required. Name of a bucket.
Documentation: https://cloud.google.com/storage/docs/json_api/v1/defaultObjectAccessControls/list.
- [Create] action
-
Endpoint URL address:
/b/{{=item.bucket}}/defaultObjectAcl.Documentation: https://cloud.google.com/storage/docs/json_api/v1/defaultObjectAccessControls/insert.
- [Update] action
-
Endpoint URL address:
/b/{{=item.bucket}}/defaultObjectAcl/{{=item.entity}}.Documentation: https://cloud.google.com/storage/docs/json_api/v1/defaultObjectAccessControls/patch.
- [Delete] action
-
Endpoint URL address:
/b/{{=item.bucket}}/defaultObjectAcl/{{=item.entity}}.Documentation: https://cloud.google.com/storage/docs/json_api/v1/defaultObjectAccessControls/delete.
The resource includes the fields from the AccessControl template.
- bucket
Template: ShortText.
Based on resource template Base.
- [Read] action
-
Endpoint URL address:
/b/{{=parameters.bucket}}/notificationConfigs.The action has the following user-defined parameters:
- bucket: Required. Name of a bucket.
Documentation: https://cloud.google.com/storage/docs/json_api/v1/notifications/list.
- [Create] action
-
Endpoint URL address:
/b/{{=item.bucket}}/notificationConfigs.Documentation: https://cloud.google.com/storage/docs/json_api/v1/notifications/insert.
- [Delete] action
-
Endpoint URL address:
/b/{{=item.bucket}}/notificationConfigs/{{=item.id}}.Documentation: https://cloud.google.com/storage/docs/json_api/v1/notifications/delete.
The resource includes the fields from the Notification template.
- bucket
Template: ShortText.
- [Read] action
-
The result is extracted from:{{=response.items}}.
The result is paginated.The following request parameters will be automatically processed:
-
pageToken:
{{=parameters.iterator}} -
maxResults:
{{=parameters.batchSize}} -
_includeUserParameters:
{{=parameters}}
-
pageToken:
- [Create] action
-
The action uses POST method.The following request parameters will be automatically processed:
-
_includeUserParameters:
{{=parameters}} -
application/json:
{{=item}}
-
_includeUserParameters:
- [Update] action
-
The action uses PATCH method.The following request parameters will be automatically processed:
-
_includeUserParameters:
{{=parameters}} -
application/json:
{{=item}}
-
_includeUserParameters:
- [Delete] action
-
The action uses DELETE method.
- kind
Template: ShortText. A read-only field.
- etag
Template: ShortText. A read-only field.
- ShortText
Data type: DT_WSTR ( length 255 )
- LongText
Data type: DT_WSTR ( length 1000 )
- DateTime
Data type: DT_DBTIMESTAMP
- Bool
Data type: DT_BOOL
- Long
Data type: DT_I8
- ULong
Data type: DT_UI8
- Int
Data type: DT_I4
- Object
Field components:
-
Uses template:
ShortText. - Contains the following components: id, selfLink, name, bucket, generation, metageneration, contentType, timeCreated, updated, customTime, timeDeleted, temporaryHold, eventBasedHold, retentionExpirationTime, storageClass, timeStorageClassUpdated, size, md5Hash, mediaLink, contentEncoding, contentDisposition, contentLanguage, cacheControl, metadata, acl, owner, crc32c, componentCount, customerEncryption, kmsKeyName.
-
Uses template:
- ObjectAccessControl
Template: AccessControl. Field components:
-
Uses template:
ShortText. - Contains the following components: id, selfLink, bucket, object, generation.
-
Uses template:
- Bucket
Field components:
-
Uses template:
ShortText. - Contains the following components: id, selfLink, projectNumber, name, timeCreated, updated, defaultEventBasedHold, retentionPolicy, metageneration, acl, defaultObjectAcl, iamConfiguration, encryption, owner, location, locationType, rpo, website, logging, versioning, cors, lifecycle, labels, storageClass, billing.
-
Uses template:
- BucketAccessControl
Template: AccessControl. Field components:
-
Uses template:
ShortText. - Contains the following components: id, selfLink, bucket.
-
Uses template:
- AccessControl
Field components:
-
Uses template:
ShortText. - Contains the following components: entity, role, email, entityId, domain, projectTeam.
-
Uses template:
- Notification
Field components:
-
Uses template:
ShortText. - Contains the following components: id, topic, event_types, custom_attributes, payload_format, object_name_prefix, selfLink.
-
Uses template:
Related scripts
What's New
- New: Introduced connection.
Related documentation
COZYROC SSIS+ Components Suite is free for testing in your development environment.
A licensed version can be deployed on-premises, on Azure-SSIS IR and on COZYROC Cloud.