Snowflake

Integration model

Metadata extraction

To extract metadata for an object, the methods provided by the Snowflake JDBC driver are used, through which the definition of schemas and tables is accessed.

It extracts the following attributes, which must be named the same in the attribute_definition table, name field, in order to appear in the template.

• catalog with the catalog value in the database.

• schema with the schema value in the database.

• physicalName and name with the same value, the table name

• path with the concatenation of the catalog, schema and table values.

• infrastructure with the selected value.

• technology with the selected value.

• zone with the selected value.

• tags for tags explicitly created on the asset

• inheritedTags for tags inherited from ancestor assets (Account > Database > Schema)

It will also send us the following attributes related to the dataset_fields of the requested resource:

• name with the value of the corresponding field.

• physicalName with the value of the corresponding field

• defaultValue with the default value defined for the corresponding field.

• fieldDataType with the data type defined for the corresponding field.

• length with the size of the corresponding field.

• incrementalField indicating whether it is an incremental field.

• position the position occupied by the corresponding field.

• precision with the precision value of the corresponding field.

• nullable indicating whether the corresponding field is nullable.

• pk indicating whether the field is a pk.

• description with the corresponding value for the field.

• tags for tags explicitly created on the asset

• inheritedTags for tags inherited from ancestor assets (Account > Database > Schema > Table/View)

In both cases, extra attributes are retrieved (depending on the driver and the specific metadata of the table) that the driver provides about both the table and the columns.

The attributes to be created in Anjana must have the following types:

The plugin is able to extract metadata from the following types of elements:

• Database tables

• Views

Data sampling

Using the Snowflake JDBC driver, a simple SELECT query is executed to access a limited number of elements in the table in order to retrieve a sample of the stored data. Additionally, the values of sensitive fields are replaced with asterisks.

Configuration

Connection format

The format of the connection URL is jdbc:snowflake://<orgname>-<account>.snowflakecomputing.com/?db=<database>&warehouse=<warehouse>&JDBC_QUERY_RESULT_FORMAT=JSON

Permissions

User or role with USAGE permissions on the database and schema where the tables/views to be governed are located.

User or role with REFERENCE permissions on the tables or views from which metadata is to be extracted.

User or role with SELECT permissions on the tables or views for which a data sample is to be obtained.

Metadata extraction

Below is the Snowflake documentation showing the different fields extracted from the tables *:

https://docs.snowflake.com/en/developer-guide/jdbc/jdbc-api#object-databasemetadata

Tables with special characters in the name