Integration model
Metadata extraction
To extract metadata from an object, the methods provided by the Redshift JDBC driver are used, through which access to the definition of schemas and tables is obtained.
It extracts the following attributes, which must be named the same way in the attribute_definition table, name field, so that they appear in the template.
-
catalog with the catalog value in the database.
-
schema with the schema value in the database.
-
physicalName and name with the same value, the table name
-
path with the concatenation of the catalog, schema and table values.
-
infrastructure with the selected value.
-
technology with the selected value.
-
zone with the selected value.
It will also send us the following attributes relating to the dataset_fields of the requested resource:
-
name with the value of the corresponding field.
-
physicalName with the value of the corresponding field
-
defaultValue with the default value defined for the corresponding field.
-
fieldDataType with the data type defined for the corresponding field.
-
length with the size of the corresponding field.
-
incrementalField indicating whether it is an incremental field.
-
position the position occupied by the corresponding field.
-
precision with the precision value of the corresponding field.
-
nullable indicating whether the corresponding field is nullable.
-
pk indicating whether the field is a pk.
-
description with the corresponding value for the field.
In both cases, extra attributes are retrieved (depending on the driver and the specific metadata of the table) that the driver provides about both the table and the columns.
The attributes to be created in Anjana must have the following types:
|
Attribute name |
Attribute type |
|
Catalog |
INPUT_TEXT |
|
Schema |
INPUT_TEXT |
|
PhysicalName |
INPUT_TEXT |
|
Path |
INPUT_TEXT |
|
Infrastructure |
SELECT |
|
Technology |
SELECT |
|
Zone |
SELECT |
|
Name |
INPUT_TEXT |
|
DefaulValue |
INPUT_TEXT |
|
FieldDataType |
INPUT_TEXT |
|
Length |
INPUT_NUMBER |
|
IncrementalField |
INPUT_CHECKBOX |
|
Position |
INPUT_NUMBER |
|
Precision |
INPUT_NUMBER |
|
Nullable |
INPUT_CHECKBOX |
|
Pk |
INPUT_CHECKBOX |
|
Description |
ENRICHED_TEXT_AREA_INTERNATIONAL |
The plugin is able to extract metadata from the following types of elements:
Database table
Data sampling
Using the Redshift JDBC driver, a simple SELECT query is run to access a limited number of records in the table in order to retrieve a sample of the stored data. Additionally, the values of sensitive fields are replaced with asterisks.
Required credentials
Metadata extraction
A user with the permissions/roles needed to run SELECT on the catalogs, schemas and tables to be governed.
Tables with characters in the name
This technology allows characters such as "/" in the name; if these are being used, the path-separator must be configured with a character other than "/". See File extraction for more details.
Data sampling
A user with the permissions/roles needed to run SELECT on the tables to be governed.