Integration model
Metadata extraction
The methods offered by the Redshift JDBC driver are used for metadata extraction, through which schema and table definitions are accessed.
It extracts the following attributes, which must be named the same in the attribute_definition table, field name, for them to appear in the template.
-
catalog with the catalog value in the database.
-
schema with the schema value in the database.
-
physicalName and name with the same value, the table name
-
path with the concatenation of the catalog, schema and table values.
-
infrastructure with the selected value.
-
technology with the selected value.
-
zone with the selected value.
It will also send the following attributes related to the dataset_fields of the requested resource:
-
name with the value of the corresponding field.
-
physicalName with the value of the corresponding field
-
defaultValue with the default value defined for the corresponding field.
-
fieldDataType with the data type defined for the corresponding field.
-
length with the size of the corresponding field.
-
incrementalField indicating whether it is an incremental field.
-
position position occupied by the corresponding field.
-
precision with the precision value of the corresponding field.
-
nullable indicating whether the corresponding field is nullable.
-
pk indicating whether the field is a pk.
-
description with the corresponding value for the field.
In both cases, extra attributes are retrieved (depending on the driver and the specific metadata of the table) that the driver provides for both the table and the columns.
The attributes to be created in Anjana must have the following types:
|
Attribute name |
Attribute type |
|
Catalog |
INPUT_TEXT |
|
Schema |
INPUT_TEXT |
|
PhysicalName |
INPUT_TEXT |
|
Path |
INPUT_TEXT |
|
Infrastructure |
SELECT |
|
Technology |
SELECT |
|
Zone |
SELECT |
|
Name |
INPUT_TEXT |
|
DefaulValue |
INPUT_TEXT |
|
FieldDataType |
INPUT_TEXT |
|
Length |
INPUT_NUMBER |
|
IncrementalField |
INPUT_CHECKBOX |
|
Position |
INPUT_NUMBER |
|
Precision |
INPUT_NUMBER |
|
Nullable |
INPUT_CHECKBOX |
|
Pk |
INPUT_CHECKBOX |
|
Description |
ENRICHED_TEXT_AREA_INTERNATIONAL |
The plugin is able to perform metadata extraction on the following types of elements:
Database table
Data sampling
Using the Redshift JDBC driver, a simple SELECT query is executed to access a limited number of table elements to retrieve a sample of the stored data. Additionally, the values of sensitive fields are replaced with asterisks.
Required credentials
Metadata extraction
User with the necessary permissions/roles to perform SELECT on the catalogs, schemas and tables to be governed.
Tables with special characters in the name
This technology allows characters such as “/” in the name; if they are being used, the path-separator must be configured with a character other than “/”. See File extraction for more details.
Data sampling
User with the necessary permissions/roles to perform SELECT on the tables to be governed.