Lineage

By clicking on the Lineage icon, the user accesses the full lineage functionality, which gives them the ability to graphically explore the relationships between the different assets to understand the data lifecycle from its sources to its final destinations.

Lineage exploration starts from a pivot element from which, interactively, relationships can be expanded.

The pivot element can be an entity, a relationship, a user, or an organizational unit:

• Entity: navigation allows discovering the assets related to the entity, the organizational unit it belongs to, its owners and the users with access to its data (adherence) if it is a governed structure.

• Relationship: navigation allows discovering the assets related to each other and the other relationships they have.

• User: navigation allows discovering the assets to which the user has access (adherence) or those they own.

• Organizational unit: navigation allows discovering the objects and owners of its domain.

The information displayed for each graph node is:

• object subtype icon

• name

• status icon and node border in the same color (if it is an entity)

Additionally, each node can have buttons at the top, bottom, right, or left in order to expand or collapse its relationships with other nodes.

And the relationships that can exist between them are represented as follows:

• parenthood: upward or downward, or as aggregation.

An aggregation relationship is represented by a node that contains its child nodes. In the following figure, the DSA Sales Analytics contains the Monthly Sales dataset, and in turn the Monthly Sales dataset contains its dataset_fields (e.g. product_ID).

However, the same entities and relationships are represented below without being aggregation relationships.

• input and/or output: to the left or right. In the following figure, the dataset is an output of the ETL instance Sales Results and an input of Data cleaning.

When accessing the lineage screen, the user must determine the pivot element from the form.

To do so, the user must select one of the object types that can act as pivot element as explained above (entity, relationship, organizational unit, or user) and then choose the element. The element search component offers autocomplete functionality and allows using filters to locate entities and relationships.

Once the pivot element is selected, the layer configured as the default in the layer selector is automatically loaded.

Layers allow lineage exploration from different perspectives, facilitating understanding and increasing the usability of the graph for the various stakeholders, since each layer's configuration includes the set of entities and relationships it will contain, as well as which relationships are considered aggregation relationships within it.

Anjana provides a Base Layer by default, which contains all entity and relationship types.

Thus, when choosing a configured layer, the entity and relationship filters with the subtypes defined for that layer are automatically loaded into the lineage.

These filters are used to narrow down the set of entities or relationships to display in the graph based on the entity subtype, relationship subtype, or object status.

The lineage form also includes a checkbox that filters by active object statuses, hiding expired and disabled ones.

The Autofocus option makes it easier for the user to visualize the graph by automatically centering the nodes within it.

Enable movement allows the user to organize the graph nodes by placing them in the desired position.

All changes made in this panel (layer change, entity filters, relationship filters, status filters, autofocus and movement) will be applied to the lineage when the user clicks the Apply button, except for changes to the pivot element, which will always update the graph automatically.

It is important to note that any change to the pivot element, filters, lineage layers, etc. will reset the graph, keeping only the pivot element. For this reason, Anjana asks the user to confirm the change.

Finally, in the lineage form the “Export lineage as image” button allows downloading the graph currently visible on screen in .png format. This feature is especially useful for sharing, documenting the represented information without relying on direct access to the application, or for saving a snapshot of the current state before making changes.

To check what elements the layer contains, the user can click the legend button and the entities and relationships corresponding to the layer selected in the layer selector will automatically be displayed.

As can be seen in the legend, the example layer aggregates the STRUCTURE-type relationship between a DATASET and its DATASET_FIELDS, and the INSTANCE_PROCESS-type relationship between a process and its instances. Therefore, the lineage is represented as follows:

Whereas, in a different layer with a different configuration, the same nodes are displayed like this:

If the user wants to search for a specific object within the deployed lineage, they can open the search tool and perform the search within the graph.

If, as in the case of the image, there are two objects whose name matches the search, clicking on one of them will center the graph on that object and highlight it in blue.

Within the graph, the user can click on any of the objects (entity or relationship) to view its related information in the Selected Node panel and progressively display the information flow.

If at any point the user wants to change the pivot element to continue exploring from that new object, they can do so from here, where they can also navigate to the specific lineage of the selected object.

Repeated nodes in the lineage

When accessing the lineage of an entity (pivot element) that is the target of a relationship configured as an aggregation relationship in the layer being displayed, the entity will be shown contained within the source entity. If this entity is part of a set of aggregations, the entire family will be shown expanded.

If the pivot element is the target of more than one aggregation relationship, meaning it has multiple parents with an aggregation relationship, the element will appear repeated as many times as it has parents.

In the following figure, one can observe how the “Monthly Sales Report” (pivot element) is included in 4 DSAs and, therefore, appears repeated in the graph once alongside each of its parents.

However, if the same Report is reached by expanding the lineage from another entity (in the case of the following image, from the dataset “Gross Margin”), the Report's parents are not shown by default.

In this last case, it will be necessary to expand the Report's parents to show them in the lineage and, again, the node will appear repeated since it is the target of multiple aggregation relationships within the selected layer.

On the other hand, when a node appears repeated in the lineage and also contains its own children, each of those nodes can be expanded independently.

In the following figure, one can observe that the dataset “Gross Margin” is contained in two DSAs and also has several dataset fields. The user can expand the fields separately in each repetition of the dataset.

Just as they can be expanded individually, they can also be collapsed individually.

Different nodes in the same layer

It is possible that different nodes are displayed in the lineage depending on the selected pivot element. This is because, by default, if the pivot element is a relationship, the source and target entities are shown even without passing the entity filter of the chosen layer; however, when choosing the source or target as the pivot, the other entity may not pass the entity subtype filter and, therefore, may not appear in the lineage graph. An example is presented below.

In this figure, the pivot element of the lineage is the relationship between the Report Behaviour Analysis and the Term Active Customer. This is why both entities appear in the graph.

However, if the Behaviour Analysis Report is chosen as the pivot element, the Term Active Customer is not shown because the entity filter of this layer does not include Terms.

The same can happen with relationships or node statuses that do not pass the corresponding filter.