Overview
Configuration is provided for establishing connections with Google Dataplex service. The configuration is used in the REST Connection Manager.
Setup
Press 
icon to get more information about the connection parameters.
Obtain data
Use REST Source component to get data from service resource.
Insert data
Use REST Destination component to insert data into service resource.
Quick Start
In this task, we will show you step-by-step how to create a connection to Google Dataplex REST API using COZYROC's REST Connection Manager.
Step 1. Run "Visual Studio" under "Administrator account". Right click on Connection managers Tab and Select New Connection. In Add SSIS Connection Manager scroll down and select Connection managers for REST Connection and click Add.
Step 2. From drop-down menu Configuration select Google Dataplex. Then click New in the Token File parameter to create a new token file.
Step 3: In the Next Wizard specify your project identifier which can be retrieve from Google Developer Console https://console.cloud.google.com/ and then click on the Next button.
Step 4: In the Next Wizard provide your Email Address and click on the Next button.
Step 5: In the Next Wizard provide your Password and click on the Next button.
Step 6: In the Next wizard click on the continue to retrieve the access and refresh token.
Step 7: Save generated access token to file. Specify token protection password. Click OK.
Step 8. Click "Test Connection" to test your connection.
Congratulations! You have now established a connection to your Google Dataplex instance.
In this guide, we will show how to read data from the Google Dataplex service resource using the COZYROC REST Source component.
Step 1. Select an existing "Data Flow task" or create a new "Data Flow Task".
Step 2. Open "SSIS Toolbox" and drag and drop COZYROC "REST Source" component.
Step 3. Double click on the "REST Source" component and select Google Dataplex connection from the combo. Select from available Google Dataplex resources listed in the second combo below.
Optionally you can include extra parameters in the grid below if needed.
Step 4. Click on the "Columns tab" and select Google Dataplex resource columns you want to read.
Step 5. Click Preview to make sure all necessary resource columns are included.
Bravo! You have now learned how to read data from Google Dataplex service.
In this guide, we will show how to write data to the Google Dataplex service resource using the COZYROC REST Destination component.
Step 1. Select an existing "Data Flow task" or create a new "Data Flow Task".
Step 2. Open the "SSIS Toolbox" and drag and drop COZYROC "REST Destination" component.
Step 3. Double click on the "REST Destination" component and select Google Dataplex connection from the combo. Also, select resource action and Google Dataplex resource you want to work on.
Step 4. Click on the "Mapping" tab and map the input columns for the process you are configuring.
If you want to provide input for other resource composite objects, change the input above and map the columns.
Done. You have now learned how to write data into the Google Dataplex resource.
Configuration
Base URL address: https://dataplex.googleapis.com/v1.
- OAuth
-
The authentication uses an authorized token. The token will be refreshed with the following expression:
{{=Date.now() + (response.expires_in - 300) * 1000}}.The authentication has the following user-defined parameters:
- Project: Required. Please specify the project name
- Scope:
The following request parameters will be automatically processed during the authentication process:
-
Authorization:
Bearer {{=token.Access}}
Documentation: https://developers.google.com/admin-sdk/directory/v1/guides/authorizing.
Based on resource template Dataplex.
- [Read] action
-
Endpoint URL address:
/locations.
The result is extracted from:{{=response.locations}}.Documentation: https://cloud.google.com/dataplex/docs/reference/rest/v1/projects.locations/list.
- [Read single] action
-
Endpoint URL address:
/locations/{{=Dataplex.UI.LocationEditor.getValue(parameters)}}.
The result is extracted from:{{=[response]}}.The action has the following user-defined parameters:
- location: Required. Please specify the location.
The following request parameters will be automatically processed:
-
pageSize:
{{=undefined}}
Documentation: https://cloud.google.com/dataplex/docs/reference/rest/v1/projects.locations/get.
- name
Template: ShortText.
- locationId
Template: ShortText.
- displayName
Template: ShortText.
- labels
Template: LabelMetadataT.
- metadata
Template: LabelMetadataT.
Based on resource template Dataplex.
- [Read] action
-
Endpoint URL address:
/locations/{{=Dataplex.UI.LocationEditor.getValue(parameters)}}/lakes.
The result is extracted from:{{=response.lakes}}.The action has the following user-defined parameters:
- location: Required. Please specify the location.
Documentation: https://cloud.google.com/dataplex/docs/reference/rest/v1/projects.locations.lakes/list.
- [Read single] action
-
Endpoint URL address:
/locations/{{=Dataplex.UI.LocationEditor.getValue(parameters)}}/lakes/{{=Dataplex.UI.LakeEditor.getValue(parameters)}}.
The result is extracted from:{{=[response]}}.The action has the following user-defined parameters:
- location: Required. Please specify the location.
- lake: Required. Please specify the lake identifier.
The following request parameters will be automatically processed:
-
pageSize:
{{=undefined}}
Documentation: https://cloud.google.com/dataplex/docs/reference/rest/v1/projects.locations.lakes/get.
- [Create] action
-
Endpoint URL address:
/locations/{{=item.location}}/lakes.The following request parameters will be automatically processed:
-
lakeId:
{{=item.id}}
Documentation: https://cloud.google.com/dataplex/docs/reference/rest/v1/projects.locations.lakes/create.
-
lakeId:
- [Update] action
-
Endpoint URL address:
/locations/{{=item.location}}/lakes/{{=item.id}}.Documentation: https://cloud.google.com/dataplex/docs/reference/rest/v1/projects.locations.lakes/patch.
- [Delete] action
-
Endpoint URL address:
/locations/{{=item.location}}/lakes/{{=item.id}}.Documentation: https://cloud.google.com/dataplex/docs/reference/rest/v1/projects.locations.lakes/delete.
- id
A key field. Template: ShortText. The gathered value is processed with
{{=item.name.split('lakes/')[1]}}expression.- name
Template: ShortText.
- location
Template: ShortText. The gathered value is processed with
{{=item.name.split('locations/')[1].split('/lakes')[0]}}expression.- displayName
Template: ShortText.
- uid
Template: ShortText.
- createTime
Template: DateTime. A read-only field.
- updateTime
Template: DateTime. A read-only field.
- state
Template: ShortText.
- serviceAccount
Template: ShortText.
- description
Template: LongText.
- metastore
Field components:
-
Uses template:
ShortText. - Contains the following components: service.
-
Uses template:
- assetStatus
Template: AssetStatusT.
- metastoreStatus
Field components:
-
Uses template:
ShortText. - Contains the following components: state, message, updateTime, endpoint.
-
Uses template:
- labels
Template: LabelMetadataT.
- Iam_policy
Template: PolicyT. The value is gathered from
/locations/{{=item.location}}/lakes/{{=item.id}}:getIamPolicyaddress. The gathered value is processed with{{=response}}expression. The value is sent to/locations/{{=item.location}}/lakes/{{=item.id}}:setIamPolicyaddress.
Based on resource template Dataplex.
- [Read] action
-
Endpoint URL address:
/locations/{{=Dataplex.UI.LocationEditor.getValue(parameters)}}/lakes/{{=Dataplex.UI.LakeEditor.getValue(parameters)}}/actions.The action has the following user-defined parameters:
- location: Required. Please specify the location.
- lake: Required. Please specify the lake identifier.
Documentation: https://cloud.google.com/dataplex/docs/reference/rest/v1/projects.locations.lakes.actions/list.
The resource includes the fields from the actionT template.
Based on resource template Dataplex.
- [Read] action
-
Endpoint URL address:
/locations/{{=Dataplex.UI.LocationEditor.getValue(parameters)}}/lakes/{{=Dataplex.UI.LakeEditor.getValue(parameters)}}/content.
The result is extracted from:{{=response.content}}.The action has the following user-defined parameters:
- location: Required. Please specify the location.
- lake: Required. Please specify the lake identifier.
Documentation: https://cloud.google.com/dataplex/docs/reference/rest/v1/projects.locations.lakes.content/list.
- [Read single] action
-
Endpoint URL address:
/locations/{{=Dataplex.UI.LocationEditor.getValue(parameters)}}/lakes/{{=Dataplex.UI.LakeEditor.getValue(parameters)}}/content/{{=parameters.content}}.
The result is extracted from:{{=[response]}}.The action has the following user-defined parameters:
- location: Required. Please specify the location.
- lake: Required. Please specify the lake identifier.
- content: Required. Please specify the content name.
The following request parameters will be automatically processed:
-
pageSize:
{{=undefined}}
Documentation: https://cloud.google.com/dataplex/docs/reference/rest/v1/projects.locations.lakes.content/get.
- [Create] action
-
Endpoint URL address:
/locations/{{=item.location}}/lakes/{{=item.lake}}/content.Documentation: https://cloud.google.com/dataplex/docs/reference/rest/v1/projects.locations.lakes.content/create.
- [Update] action
-
Endpoint URL address:
/locations/{{=item.location}}/lakes/{{=item.lake}}/content/{{=item.id}}.The action has the following user-defined parameters:
- updateMask: Required. Please specify Mask of fields to update.
Documentation: https://cloud.google.com/dataplex/docs/reference/rest/v1/projects.locations.lakes.content/patch.
- [Delete] action
-
Endpoint URL address:
/locations/{{=item.location}}/lakes/{{=item.lake}}/content/{{=item.id}}.Documentation: https://cloud.google.com/dataplex/docs/reference/rest/v1/projects.locations.lakes.content/delete.
The resource includes the fields from the ContentT template.
- Iam_policy
Template: PolicyT. The value is gathered from
/locations/{{=item.location}}/lakes/{{=item.lake}}/content/{{=item.id}}:getIamPolicyaddress. The gathered value is processed with{{=response}}expression. The value is sent to/locations/{{=item.location}}/lakes/{{=item.lake}}/content/{{=item.id}}:setIamPolicyaddress.
Based on resource template Dataplex.
- [Read] action
-
Endpoint URL address:
/locations/{{=Dataplex.UI.LocationEditor.getValue(parameters)}}/lakes/{{=Dataplex.UI.LakeEditor.getValue(parameters)}}/contentitems.
The result is extracted from:{{=response.content}}.The action has the following user-defined parameters:
- location: Required. Please specify the location.
- lake: Required. Please specify the lake identifier.
Documentation: https://cloud.google.com/dataplex/docs/reference/rest/v1/projects.locations.lakes.contentitems/list.
- [Read single] action
-
Endpoint URL address:
/locations/{{=Dataplex.UI.LocationEditor.getValue(parameters)}}/lakes/{{=Dataplex.UI.LakeEditor.getValue(parameters)}}/contentitems/{{=parameters.contentItem}}.
The result is extracted from:{{=[response]}}.The action has the following user-defined parameters:
- location: Required. Please specify the location.
- lake: Required. Please specify the lake identifier.
- contentItem: Required. Please specify the content item name.
The following request parameters will be automatically processed:
-
pageSize:
{{=undefined}}
Documentation: https://cloud.google.com/dataplex/docs/reference/rest/v1/projects.locations.lakes.contentitems/get.
- [Create] action
-
Endpoint URL address:
/locations/{{=item.location}}/lakes/{{=item.lake}}/contentitems.Documentation: https://cloud.google.com/dataplex/docs/reference/rest/v1/projects.locations.lakes.contentitems/create.
- [Update] action
-
Endpoint URL address:
/locations/{{=item.location}}/lakes/{{=item.lake}}/contentitems/{{=item.id}}.The action has the following user-defined parameters:
- updateMask: Required. Please specify the mask of fields to update.
Documentation: https://cloud.google.com/dataplex/docs/reference/rest/v1/projects.locations.lakes.contentitems/patch.
- [Delete] action
-
Endpoint URL address:
/locations/{{=item.location}}/lakes/{{=item.lake}}/contentitems/{{=item.id}}.Documentation: https://cloud.google.com/dataplex/docs/reference/rest/v1/projects.locations.lakes.contentitems/delete.
The resource includes the fields from the ContentT template.
- Iam_policy
Template: PolicyT. The value is gathered from
/locations/{{=item.location}}/lakes/{{=item.lake}}/contentitems/{{=item.id}}:getIamPolicyaddress. The gathered value is processed with{{=response}}expression. The value is sent to/locations/{{=item.location}}/lakes/{{=item.lake}}/contentitems/{{=item.id}}:setIamPolicyaddress.
Based on resource template Dataplex.
- [Read] action
-
Endpoint URL address:
/locations/{{=Dataplex.UI.LocationEditor.getValue(parameters)}}/lakes/{{=Dataplex.UI.LakeEditor.getValue(parameters)}}/environments.
The result is extracted from:{{=response.environments}}.The action has the following user-defined parameters:
- location: Required. Please specify the location.
- lake: Required. Please specify the lake identifier.
Documentation: https://cloud.google.com/dataplex/docs/reference/rest/v1/projects.locations.lakes.environments/list.
- [Read single] action
-
Endpoint URL address:
/locations/{{=Dataplex.UI.LocationEditor.getValue(parameters)}}/lakes/{{=Dataplex.UI.LakeEditor.getValue(parameters)}}/environments/{{=parameters.environment}}.
The result is extracted from:{{=[response]}}.The action has the following user-defined parameters:
- location: Required. Please specify the location.
- lake: Required. Please specify the lake identifier.
- environment: Required. Please specify the environment identifier.
The following request parameters will be automatically processed:
-
pageSize:
{{=undefined}}
Documentation: https://cloud.google.com/dataplex/docs/reference/rest/v1/projects.locations.lakes.environments/get.
- [Create] action
-
Endpoint URL address:
/locations/{{=item.location}}/lakes/{{=item.lake}}/environments.The following request parameters will be automatically processed:
-
environmentId:
{{=item.id}}
Documentation: https://cloud.google.com/dataplex/docs/reference/rest/v1/projects.locations.lakes.environments/create.
-
environmentId:
- [Update] action
-
Endpoint URL address:
/locations/{{=item.location}}/lakes/{{=item.lake}}/environments/{{=item.id}}.
The action uses PATCH method.The action has the following user-defined parameters:
- updateMask: Required. Please specify the mask of fields to update.
Documentation: https://cloud.google.com/dataplex/docs/reference/rest/v1/projects.locations.lakes.environments/patch.
- [Delete] action
-
Endpoint URL address:
/locations/{{=item.location}}/lakes/{{=item.lake}}/environments/{{=item.id}}.Documentation: https://cloud.google.com/dataplex/docs/reference/rest/v1/projects.locations.lakes.environments/delete.
- id
A key field. Template: ShortText. The gathered value is processed with
{{=item.name.split('environments/')[1]}}expression.- name
Template: ShortText.
- location
Template: ShortText. The gathered value is processed with
{{=item.name.split('locations/')[1].split('/lakes')[0]}}expression.- lake
Template: ShortText. The gathered value is processed with
{{=item.name.split('lakes/')[1].split('/environments')[0]}}expression.- displayName
Template: ShortText.
- uid
Template: ShortText.
- createTime
Template: DateTime. A read-only field.
- updateTime
Template: DateTime. A read-only field.
- labels
Template: LabelMetadataT.
- description
Template: LongText.
- state
Template: ShortText.
- infrastructureSpec
Field components:
- Contains the following components: compute, osImage.
- sessionSpec
Field components:
-
Uses template:
ShortText. - Contains the following components: maxIdleDuration, enableFastStartup.
-
Uses template:
- sessionStatus
Field components:
- Contains the following components: active.
- endpoints
Field components:
-
Uses template:
ShortText. - Contains the following components: notebooks, sql.
-
Uses template:
- Iam_policy
Template: PolicyT. The value is gathered from
/locations/{{=item.location}}/lakes/{{=item.lake}}/environments/{{=item.id}}:getIamPolicyaddress. The gathered value is processed with{{=response}}expression. The value is sent to/locations/{{=item.location}}/lakes/{{=item.lake}}/environments/{{=item.id}}:setIamPolicyaddress.- session
The value is gathered from
/locations/{{=item.location}}/lakes/{{=item.lake}}/environments/{{=item.id}}/sessionsaddress. The gathered value is processed with{{=response.sessions}}expression. Field components:- An array.
-
Uses template:
sessionT.
Based on resource template Dataplex.
- [Read] action
-
Endpoint URL address:
/locations/{{=Dataplex.UI.LocationEditor.getValue(parameters)}}/lakes/{{=Dataplex.UI.LakeEditor.getValue(parameters)}}/environments/{{=parameters.environment}}/sessions.
The result is extracted from:{{=response.sessions}}.The action has the following user-defined parameters:
- location: Required. Please specify the location.
- lake: Required. Please specify the lake identifier.
- environment: Required. Please specify the environment identifier.
Documentation: https://cloud.google.com/dataplex/docs/reference/rest/v1/projects.locations.lakes.environments.sessions/list.
The resource includes the fields from the sessionT template.
Based on resource template Dataplex.
- [Read] action
-
Endpoint URL address:
/locations/{{=Dataplex.UI.LocationEditor.getValue(parameters)}}/lakes/{{=Dataplex.UI.LakeEditor.getValue(parameters)}}/tasks.
The result is extracted from:{{=response.tasks}}.The action has the following user-defined parameters:
- location: Required. Please specify the location.
- lake: Required. Please specify the lake identifier.
Documentation: https://cloud.google.com/dataplex/docs/reference/rest/v1/projects.locations.lakes.tasks/list.
- [Read single] action
-
Endpoint URL address:
/locations/{{=Dataplex.UI.LocationEditor.getValue(parameters)}}/lakes/{{=Dataplex.UI.LakeEditor.getValue(parameters)}}/tasks/{{=parameters.task}}.
The result is extracted from:{{=[response]}}.The action has the following user-defined parameters:
- location: Required. Please specify the location.
- lake: Required. Please specify the lake identifier.
- task: Required. Please specify the task.
The following request parameters will be automatically processed:
-
pageSize:
{{=undefined}}
Documentation: https://cloud.google.com/dataplex/docs/reference/rest/v1/projects.locations.lakes.tasks/get.
- [Create] action
-
Endpoint URL address:
/locations/{{=item.location}}/lakes/{{=item.lake}}/tasks.The following request parameters will be automatically processed:
-
taskId:
{{=item.id}}
Documentation: https://cloud.google.com/dataplex/docs/reference/rest/v1/projects.locations.lakes.tasks/create.
-
taskId:
- [Update] action
-
Endpoint URL address:
/locations/{{=item.location}}/lakes/{{=item.lake}}/tasks/{{=item.id}}.Documentation: https://cloud.google.com/dataplex/docs/reference/rest/v1/projects.locations.lakes.tasks/patch.
- [Delete] action
-
Endpoint URL address:
/locations/{{=item.location}}/lakes/{{=item.lake}}/tasks/{{=item.id}}.Documentation: https://cloud.google.com/dataplex/docs/reference/rest/v1/projects.locations.lakes.tasks/delete.
- id
A key field. Template: ShortText. The gathered value is processed with
{{=item.name.split('tasks/')[1]}}expression.- name
Template: ShortText.
- location
Template: ShortText. The gathered value is processed with
{{=item.name.split('locations/')[1].split('/lakes')[0]}}expression.- lake
Template: ShortText. The gathered value is processed with
{{=item.name.split('lakes/')[1].split('/tasks')[0]}}expression.- uid
Template: ShortText.
- createTime
Template: DateTime. A read-only field.
- updateTime
Template: DateTime. A read-only field.
- description
Template: LongText.
- displayName
Template: ShortText.
- state
Template: ShortText.
- labels
Template: LabelMetadataT.
- triggerSpec
Template: TriggerSpectT.
- executionSpec
Template: ExecutionSpecT.
- executionStatus
Template: ExecutionStatusT.
- spark
Template: SparkT.
- Iam_policy
Template: PolicyT. The value is gathered from
/locations/{{=item.location}}/lakes/{{=item.lake}}/tasks/{{=item.id}}:getIamPolicyaddress. The gathered value is processed with{{=response}}expression. The value is sent to/locations/{{=item.location}}/lakes/{{=item.lake}}/tasks/{{=item.id}}:setIamPolicyaddress.
Based on resource template Dataplex.
- [Read] action
-
Endpoint URL address:
/locations/{{=Dataplex.UI.LocationEditor.getValue(parameters)}}/lakes/{{=Dataplex.UI.LakeEditor.getValue(parameters)}}/tasks/{{=parameters.task}}/jobs.
The result is extracted from:{{=response.jobs}}.The action has the following user-defined parameters:
- location: Required. Please specify the location.
- lake: Required. Please specify the lake identifier.
- task: Required. Please specify the task identifier.
Documentation: https://cloud.google.com/dataplex/docs/reference/rest/v1/projects.locations.lakes.tasks.jobs/list.
- [Read single] action
-
Endpoint URL address:
/locations/{{=Dataplex.UI.LocationEditor.getValue(parameters)}}/lakes/{{=Dataplex.UI.LakeEditor.getValue(parameters)}}/tasks/{{=parameters.task}}/jobs/{{=parameters.job}}.
The result is extracted from:{{=[response]}}.The action has the following user-defined parameters:
- location: Required. Please specify the location.
- lake: Required. Please specify the lake identifier.
- task: Required. Please specify the task identifier.
- job: Required. Please specify the job identifier.
The following request parameters will be automatically processed:
-
pageSize:
{{=undefined}}
Documentation: https://cloud.google.com/dataplex/docs/reference/rest/v1/projects.locations.lakes.tasks.jobs/get.
- [Delete] action
-
Endpoint URL address:
/locations/{{=item.location}}/lakes/{{=item.lake}}/tasks/{{=item.task}}/jobs/{{=item.id}}:cancel.
The action uses POST method.Documentation: https://cloud.google.com/dataplex/docs/reference/rest/v1/projects.locations.lakes.tasks.jobs/cancel.
- id
A key field. Template: ShortText. The gathered value is processed with
{{=item.name.split('jobs/')[1]}}expression.- name
Template: ShortText.
- location
Template: ShortText. The gathered value is processed with
{{=item.name.split('locations/')[1].split('/lakes')[0]}}expression.- lake
Template: ShortText. The gathered value is processed with
{{=item.name.split('lakes/')[1].split('/tasks')[0]}}expression.- task
Template: ShortText. The gathered value is processed with
{{=item.name.split('tasks/')[1].split('/jobs')[0]}}expression.- uid
Template: ShortText.
- startTime
Template: DateTime.
- endTime
Template: DateTime.
- state
Template: ShortText.
- retryCount
Data type: DT_I4
- service
Template: ShortText.
- serviceJob
Template: ShortText.
- message
Template: LongText.
Based on resource template Dataplex.
- [Read] action
-
Endpoint URL address:
/locations/{{=Dataplex.UI.LocationEditor.getValue(parameters)}}/lakes/{{=Dataplex.UI.LakeEditor.getValue(parameters)}}/zones.
The result is extracted from:{{=response.zones}}.The action has the following user-defined parameters:
- location: Required. Please specify the location.
- lake: Required. Please specify the lake identifier.
Documentation: https://cloud.google.com/dataplex/docs/reference/rest/v1/projects.locations.lakes.zones/list.
- [Read single] action
-
Endpoint URL address:
/locations/{{=Dataplex.UI.LocationEditor.getValue(parameters)}}/lakes/{{=Dataplex.UI.LakeEditor.getValue(parameters)}}/zones/{{=parameters.zone}}.
The result is extracted from:{{=[response]}}.The action has the following user-defined parameters:
- location: Required. Please specify the location.
- lake: Required. Please specify the lake identifier.
- zone: Required. Please specify the zone identifier.
The following request parameters will be automatically processed:
-
pageSize:
{{=undefined}}
Documentation: https://cloud.google.com/dataplex/docs/reference/rest/v1/projects.locations.lakes.zones/get.
- [Create] action
-
Endpoint URL address:
/locations/{{=item.location}}/lakes/{{=item.lake}}/zones.The following request parameters will be automatically processed:
-
zoneId:
{{=item.id}}
Documentation: https://cloud.google.com/dataplex/docs/reference/rest/v1/projects.locations.lakes.zones/create.
-
zoneId:
- [Update] action
-
Endpoint URL address:
/locations/{{=item.location}}/lakes/{{=item.lake}}/zones/{{=item.id}}.
The action uses PATCH method.The action has the following user-defined parameters:
- updateMask: Required. Please specify the mask of fields to update.
Documentation: https://cloud.google.com/dataplex/docs/reference/rest/v1/projects.locations.lakes.zones/patch.
- [Delete] action
-
Endpoint URL address:
/locations/{{=item.location}}/lakes/{{=item.lake}}/zones/{{=item.id}}.Documentation: https://cloud.google.com/dataplex/docs/reference/rest/v1/projects.locations.lakes.zones/delete.
- id
A key field. Template: ShortText. The gathered value is processed with
{{=item.split('zones/')[1]}}expression.- name
Template: ShortText.
- location
Template: ShortText. The gathered value is processed with
{{=item.name.split('locations/')[1].split('/lakes')[0]}}expression.- lake
Template: ShortText. The gathered value is processed with
{{=item.name.split('lakes/')[1].split('/zones')[0]}}expression.- displayName
Template: ShortText.
- uid
Template: ShortText.
- createTime
Template: DateTime. A read-only field.
- updateTime
Template: DateTime. A read-only field.
- labels
Template: LabelMetadataT.
- description
Template: LongText.
- state
Template: ShortText.
- type
Template: ShortText.
- discoverySpec
Template: DiscoverySpecT.
- resourceSpec
Template: ResourceSpecT.
- assetStatus
Template: AssetStatusT.
- Iam_policy
Template: PolicyT. The value is gathered from
/locations/{{=item.location}}/lakes/{{=item.lake}}/zones/{{=item.id}}:getIamPolicyaddress. The gathered value is processed with{{=response}}expression. The value is sent to/locations/{{=item.location}}/lakes/{{=item.lake}}/zones/{{=item.id}}:setIamPolicyaddress.
Based on resource template Dataplex.
- [Read] action
-
Endpoint URL address:
/locations/{{=Dataplex.UI.LocationEditor.getValue(parameters)}}/lakes/{{=Dataplex.UI.LakeEditor.getValue(parameters)}}/zones/{{=parameters.zone}}/actions.
The result is extracted from:{{=response.actions}}.The action has the following user-defined parameters:
- location: Required. Please specify the location.
- lake: Required. Please specify the lake identifier.
- zone: Required. Please specify the zone identifier.
Documentation: https://cloud.google.com/dataplex/docs/reference/rest/v1/projects.locations.lakes.zones.actions/list.
The resource includes the fields from the actionT template.
Based on resource template Dataplex.
- [Read] action
-
Endpoint URL address:
/locations/{{=Dataplex.UI.LocationEditor.getValue(parameters)}}/lakes/{{=Dataplex.UI.LakeEditor.getValue(parameters)}}/zones/{{=parameters.zone}}/assets.
The result is extracted from:{{=response.assets}}.The action has the following user-defined parameters:
- location: Required. Please specify the location.
- lake: Required. Please specify the lake identifier.
- zone: Required. Please specify the zone.
Documentation: https://cloud.google.com/dataplex/docs/reference/rest/v1/projects.locations.lakes.zones.assets/list.
- [Read single] action
-
Endpoint URL address:
/locations/{{=Dataplex.UI.LocationEditor.getValue(parameters)}}/lakes/{{=Dataplex.UI.LakeEditor.getValue(parameters)}}/zones/{{=parameters.zone}}/assets/{{=parameters.assetId}}.
The result is extracted from:{{=[response]}}.The action has the following user-defined parameters:
- location: Required. Please specify the location.
- lake: Required. Please specify the lake identifier.
- zone: Required. Please specify the zone.
- assetId: Required. Please specify the asset identifier.
The following request parameters will be automatically processed:
-
pageSize:
{{=undefined}}
Documentation: https://cloud.google.com/dataplex/docs/reference/rest/v1/projects.locations.lakes.zones.assets/get.
- [Create] action
-
Endpoint URL address:
/locations/{{=item.location}}/lakes/{{=item.lake}}/zones/{{=item.zone}}/assets.The following request parameters will be automatically processed:
-
assetId:
{{=item.id}}
Documentation: https://cloud.google.com/dataplex/docs/reference/rest/v1/projects.locations.lakes.zones.assets/create.
-
assetId:
- [Update] action
-
Endpoint URL address:
/locations/{{=item.location}}/lakes/{{=item.lake}}/zones/{{=item.zone}}/assets/{{=item.id}}.The action has the following user-defined parameters:
- updateMask: Required. Please specify the mask of fields to update.
Documentation: https://cloud.google.com/dataplex/docs/reference/rest/v1/projects.locations.lakes.zones.assets/patch.
- [Delete] action
-
Endpoint URL address:
/locations/{{=item.location}}/lakes/{{=item.lake}}/zones/{{=item.zone}}/assets/{{=item.id}}.Documentation: https://cloud.google.com/dataplex/docs/reference/rest/v1/projects.locations.lakes.zones.assets/delete.
- id
A key field. Template: ShortText. The gathered value is processed with
{{=item.name.split('assets/')[1]}}expression.- name
Template: ShortText.
- location
Template: ShortText. The gathered value is processed with
{{=item.name.split('locations/')[1].split('/lakes')[0]}}expression.- lake
Template: ShortText. The gathered value is processed with
{{=item.name.split('lakes/')[1].split('/zones')[0]}}expression.- zone
Template: ShortText. The gathered value is processed with
{{=item.name.split('zones/')[1].split('/assets')[0]}}expression.- displayName
Template: ShortText.
- uid
Template: ShortText.
- createTime
Template: DateTime. A read-only field.
- updateTime
Template: DateTime. A read-only field.
- labels
Template: LabelMetadataT.
- description
Template: ShortText.
- state
Template: ShortText.
- resourceSpec
Template: ResourceSpecT.
- resourceStatus
Template: ResourceSecurityT.
- securityStatus
Template: ResourceSecurityT.
- discoverySpec
Template: DiscoverySpecT.
- discoveryStatus
Template: DiscoveryStatusT.
- Iam_policy
Template: PolicyT. The value is gathered from
/locations/{{=item.location}}/lakes/{{=item.lake}}/zones/{{=item.zone}}/assets/{{=item.id}}:getIamPolicyaddress. The gathered value is processed with{{=response}}expression. The value is sent to/locations/{{=item.location}}/lakes/{{=item.lake}}/zones/{{=item.zone}}/assets/{{=item.id}}:setIamPolicyaddress.
Based on resource template Dataplex.
- [Read] action
-
Endpoint URL address:
/locations/{{=Dataplex.UI.LocationEditor.getValue(parameters)}}/lakes/{{=Dataplex.UI.LakeEditor.getValue(parameters)}}/zones/{{=parameters.zone}}/assets/{{=parameters.asset}}/actions.
The result is extracted from:{{=response.actions}}.The action has the following user-defined parameters:
- location: Required. Please specify the location.
- lake: Required. Please specify the lake identifier.
- zone: Required. Please specify the zone.
- asset: Required. Please specify the asset.
Documentation: https://cloud.google.com/dataplex/docs/reference/rest/v1/projects.locations.lakes.zones.assets.actions/list.
The resource includes the fields from the actionT template.
Based on resource template Dataplex.
- [Read] action
-
Endpoint URL address:
/locations/{{=Dataplex.UI.LocationEditor.getValue(parameters)}}/lakes/{{=Dataplex.UI.LakeEditor.getValue(parameters)}}/zones/{{=parameters.zone}}/entities.
The result is extracted from:{{=response.entities}}.The action has the following user-defined parameters:
- location: Required. Please specify the location.
- lake: Required. Please specify the lake identifier.
- zone: Required. Please specify the zone.
- view: Required. Specify the entity view to make a partial list request.
Documentation: https://cloud.google.com/dataplex/docs/reference/rest/v1/projects.locations.lakes.zones.entities/list.
- [Read single] action
-
Endpoint URL address:
/locations/{{=Dataplex.UI.LocationEditor.getValue(parameters)}}/lakes/{{=Dataplex.UI.LakeEditor.getValue(parameters)}}/zones/{{=parameters.zone}}/entities/{{=parameters.entity}}.
The result is extracted from:{{=[response]}}.The action has the following user-defined parameters:
- location: Required. Please specify the location.
- lake: Required. Please specify the lake identifier.
- zone: Required. Please specify the zone.
- entity: Required. Please specify the entity.
The following request parameters will be automatically processed:
-
pageSize:
{{=undefined}}
Documentation: https://cloud.google.com/dataplex/docs/reference/rest/v1/projects.locations.lakes.zones.entities/get.
- [Create] action
-
Endpoint URL address:
/locations/{{=item.location}}/lakes/{{=item.lake}}/zones/{{=item.zone}}/entities.Documentation: https://cloud.google.com/dataplex/docs/reference/rest/v1/projects.locations.lakes.zones.entities/create.
- [Update] action
-
Endpoint URL address:
/locations/{{=item.location}}/lakes/{{=item.lake}}/zones/{{=item.zone}}/entities/{{=item.id}}.Documentation: https://cloud.google.com/dataplex/docs/reference/rest/v1/projects.locations.lakes.zones.entities/update.
- [Delete] action
-
Endpoint URL address:
/locations/{{=item.location}}/lakes/{{=item.lake}}/zones/{{=item.zone}}/entities/{{=item.id}}.The following request parameters will be automatically processed:
-
etag:
{{=item.etag}}
Documentation: https://cloud.google.com/dataplex/docs/reference/rest/v1/projects.locations.lakes.zones.entities/delete.
-
etag:
- id
A key field. Template: ShortText. The gathered value is processed with
{{=item.name.split('entities/')[1]}}expression.- name
Template: ShortText.
- location
Template: ShortText. The gathered value is processed with
{{=item.name.split('locations/')[1].split('/lakes')[0]}}expression.- lake
Template: ShortText. The gathered value is processed with
{{=item.name.split('lakes/')[1].split('/zones')[0]}}expression.- zone
Template: ShortText. The gathered value is processed with
{{=item.name.split('zones/')[1].split('/entities')[0]}}expression.- displayName
Template: ShortText.
- description
Template: LongText.
- createTime
Template: DateTime. A read-only field.
- updateTime
Template: DateTime. A read-only field.
- id
Template: ShortText.
- etag
Template: ShortText.
- type
Template: ShortText.
- asset
Template: ShortText.
- dataPath
Template: ShortText.
- dataPathPattern
Template: ShortText.
- catalogEntry
Template: ShortText.
- system
Template: ShortText.
- format
Field components:
-
Uses template:
ShortText. - Contains the following components: format, compressionFormat, mimeType, csv, json.
-
Uses template:
- compatibility
Field components:
- Contains the following components: hiveMetastore, bigquery.
- schema
Field components:
-
Uses template:
ShortText. - Contains the following components: userManaged, fields, partitionFields, partitionStyle.
-
Uses template:
Based on resource template Dataplex.
- [Read] action
-
Endpoint URL address:
/locations/{{=Dataplex.UI.LocationEditor.getValue(parameters)}}/lakes/{{=Dataplex.UI.LakeEditor.getValue(parameters)}}/zones/{{=parameters.zone}}/entities/{{=parameters.entity}}/partitions.The action has the following user-defined parameters:
- location: Required. Please specify the location.
- lake: Required. Please specify the lake identifier.
- zone: Required. Please specify the zone.
- entity: Required. Please specify the entity.
Documentation: https://cloud.google.com/dataplex/docs/reference/rest/v1/projects.locations.lakes.zones.entities.partitions/list.
- [Read single] action
-
Endpoint URL address:
/locations/{{=Dataplex.UI.LocationEditor.getValue(parameters)}}/lakes/{{=Dataplex.UI.LakeEditor.getValue(parameters)}}/zones/{{=parameters.zone}}/entities/{{=parameters.entity}}/partitions/{{=parameters.partition}}.
The result is extracted from:{{=[response]}}.The action has the following user-defined parameters:
- location: Required. Please specify the location.
- lake: Required. Please specify the lake identifier.
- zone: Required. Please specify the zone identifier.
- entity: Required. Please specify the entity identifier.
- partition: Required. Please specify the partition identifier.
The following request parameters will be automatically processed:
-
pageSize:
{{=undefined}}
Documentation: https://cloud.google.com/dataplex/docs/reference/rest/v1/projects.locations.lakes.zones.entities.partitions/get.
- [Create] action
-
Endpoint URL address:
/locations/{{=item.location}}/lakes/{{=item.lake}}/zones/{{=item.zone}}/entities/{{=item.entity}}/partitions.Documentation: https://cloud.google.com/dataplex/docs/reference/rest/v1/projects.locations.lakes.zones.entities.partitions/create.
- [Delete] action
-
Endpoint URL address:
/locations/{{=item.location}}/lakes/{{=item.lake}}/zones/{{=item.zone}}/entities/{{=item.entity}}/partititions/{{=item.id}}.Documentation: https://cloud.google.com/dataplex/docs/reference/rest/v1/projects.locations.lakes.zones.entities.partitions/delete.
- id
A key field. Template: ShortText. The gathered value is processed with
{{=item.name.split('partitions/')[1]}}expression.- name
Template: ShortText.
- location
Template: ShortText. The gathered value is processed with
{{=item.name.split('locations/')[1].split('/lakes')[0]}}expression.- lake
Template: ShortText. The gathered value is processed with
{{=item.name.split('lakes/')[1].split('/zones')[0]}}expression.- zone
Template: ShortText. The gathered value is processed with
{{=item.name.split('zones/')[1].split('/entities')[0]}}expression.- entity
Template: ShortText. The gathered value is processed with
{{=item.name.split('entities/')[1].split('/partitions')[0]}}expression.- values
Template: LongText. A composite field.
- etag
Template: ShortText.
Based on resource template Dataplex.
- [Read] action
-
Endpoint URL address:
/locations/{{=Dataplex.UI.LocationEditor.getValue(parameters)}}/operations.
The result is extracted from:{{=response.operations}}.The action has the following user-defined parameters:
- location: Required. Please specify the location.
Documentation: https://cloud.google.com/dataplex/docs/reference/rest/v1/projects.locations.operations/list.
- [Read single] action
-
Endpoint URL address:
/locations/{{=Dataplex.UI.LocationEditor.getValue(parameters)}}/operations/{{=parameters.operation}}.
The result is extracted from:{{=[response]}}.The action has the following user-defined parameters:
- location: Required. Please specify the location.
- operation: Required. Please specify the operation.
The following request parameters will be automatically processed:
-
pageSize:
{{=undefined}}
Documentation: https://cloud.google.com/dataplex/docs/reference/rest/v1/projects.locations.operations/get.
- [Delete] action
-
Endpoint URL address:
/locations/{{=item.location}}/operations/{{=item.id}}.Documentation: https://cloud.google.com/dataplex/docs/reference/rest/v1/projects.locations.operations/delete.
- id
A key field. Template: ShortText. The gathered value is processed with
{{=item.name.split('operations/')[1]}}expression.- name
Template: ShortText.
- location
Template: ShortText. The gathered value is processed with
{{=item.name.split('locations/')[1].split('/operations')[0]}}expression.- metadata
Field components:
-
Uses template:
ShortText. - Contains the following components: @type, createTime, endTime, target, verb, requestedCancellation, apiVersion.
-
Uses template:
- done
Data type: DT_BOOL
- response
Field components:
-
Uses template:
ShortText. - Contains the following components: @type, name, uid, createTime, updateTime, description, displayName, state, triggerSpec, executionStatus, executionSpec, resourceSpec, resourceStatus, securityStatus, discoveryStatus, spark.
-
Uses template:
- [Read] action
-
The result is paginated.The following request parameters will be automatically processed:
-
pageToken:
{{=parameters.iterator}} -
pageSize:
{{=parameters.batchSize}} -
_includeUserParameters:
{{=parameters}}
-
pageToken:
- [Create] action
-
The action uses POST method.The following request parameters will be automatically processed:
-
_includeUserParameters:
{{=parameters}} -
application/json:
{{=item}}
-
_includeUserParameters:
- [Update] action
-
The action uses PATCH method.The following request parameters will be automatically processed:
-
_includeUserParameters:
{{=parameters}} -
application/json:
{{=item}}
-
_includeUserParameters:
- [Delete] action
-
The action uses DELETE method.
Based on resource template Base.
- ShortText
Data type: DT_WSTR ( length 255 )
- LongText
Data type: DT_WSTR ( length 1000 )
- DateTime
Data type: DT_DBTIMESTAMP
- Date
Data type: DT_DBDATE
- ConditionT
Field components:
-
Uses template:
ShortText. - Contains the following components: expression, title, description, location.
-
Uses template:
- AutoLogConfigT
Field components:
-
Uses template:
ShortText. - Contains the following components: logType, exemptedMembers.
-
Uses template:
- BindingT
Field components:
-
Uses template:
ShortText. - Contains the following components: role, members, condition.
-
Uses template:
- AuditConfigT
Field components:
-
Uses template:
ShortText. - Contains the following components: service, auditLogConfigs.
-
Uses template:
- actionT
Field components:
-
Uses template:
ShortText. - Contains the following components: category, issue, detectTime, name, lake, zone, asset, dataLocations, invalidDataFormat, incompatibleDataSchema, invalidDataPartition, missingData, missingResource, unauthorizedResource, failedSecurityPolicyApply, invalidDataOrganization.
-
Uses template:
- LabelMetadataT
The gathered value is processed with
{{=_.map(value, function(v, k) { return { name: k, value: v }; })}}expression. Field components:- An array.
- Contains the following components: name, value.
- PolicyT
Field components:
-
Uses template:
ShortText. - Contains the following components: bindings, auditConfigs, etag, version, updateMask, name.
-
Uses template:
- ContentT
Field components:
-
Uses template:
ShortText. - Contains the following components: id, name, location, lake, uid, path, createTime, updateTime, labels, description, dataText, sqlScript, notebook.
-
Uses template:
- sessionT
Field components:
-
Uses template:
ShortText. - Contains the following components: name, userId, createTime, state.
-
Uses template:
- TriggerSpectT
Field components:
-
Uses template:
ShortText. - Contains the following components: type, startTime, disabled, maxRetries, schedule.
-
Uses template:
- ExecutionSpecT
Field components:
-
Uses template:
ShortText. - Contains the following components: args, serviceAccount, project.
-
Uses template:
- ExecutionStatusT
Field components:
-
Uses template:
DateTime. - Contains the following components: updateTime, latestJob.
-
Uses template:
- ResourceSpecT
Field components:
-
Uses template:
ShortText. - Contains the following components: locationType, name, type.
-
Uses template:
- ResourceSecurityT
Field components:
-
Uses template:
ShortText. - Contains the following components: state, message, updateTime.
-
Uses template:
- DiscoveryStatusT
Field components:
-
Uses template:
ShortText. - Contains the following components: state, message, updateTime, lastRunTime, stats.
-
Uses template:
- DiscoverySpecT
Field components:
-
Uses template:
LongText. - Contains the following components: enabled, includePatterns, excludePatterns, csvOptions, jsonOptions, schedule.
-
Uses template:
- AssetStatusT
Field components:
-
Uses template:
DateTime. - Contains the following components: updateTime, activeAssets, securityPolicyApplyingAssets.
-
Uses template:
- VpcNetworkT
Field components:
-
Uses template:
LongText. - Contains the following components: networkTags, network, subNetwork.
-
Uses template:
- PropertyT
Field components:
- Contains the following components: properties.
- ContainerImageT
Field components:
-
Uses template:
ShortText. - Contains the following components: image, javaJars, pythonPackages, properties.
-
Uses template:
- SparkT
Field components:
-
Uses template:
LongText. - Contains the following components: fileUris, archiveUris, infrastructureSpec, mainJarFileUri, mainClass, pythonScriptFile, sqlScriptFile, sqlScript.
-
Uses template:
- CSVT
Field components:
-
Uses template:
ShortText. - Contains the following components: encoding, headerRows, delimiter, quote, disableTypeInference.
-
Uses template:
Knowledge Base
What's New
- New: Introduced connection.
Related documentation
COZYROC SSIS+ Components Suite is free for testing in your development environment.
A licensed version can be deployed on-premises, on Azure-SSIS IR and on COZYROC Cloud.